03/31/2025 02:15:40.329 7fdb946b6748 [contrail-vrouter-nodemgr] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 03/31/2025 02:15:40.329 7fdb946b6748 [contrail-vrouter-nodemgr] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 03/31/2025 02:15:40.329 7fdb946b6748 [contrail-vrouter-nodemgr] [INFO]: SANDESH: Flow Logging: [None] -> [False] 03/31/2025 02:15:40.377 7fdb9227d048 [contrail-vrouter-nodemgr] [ERROR]: Cannot write http_port 8102 to /tmp/contrail-vrouter-nodemgr.0.http_port 03/31/2025 02:15:40.378 7fdb9227d048 [contrail-vrouter-nodemgr] [ERROR]: Starting Introspect on HTTP Port 8102 03/31/2025 02:15:40.398 7fdb92282978 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvStart] in state[none] 03/31/2025 02:15:40.409 7fdb92282978 [contrail-vrouter-nodemgr] [DEBUG]: Add uve in the [ObjectVRouter:NodeStatusUVE] map 03/31/2025 02:15:40.410 7fdb92282978 [contrail-vrouter-nodemgr] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 03/31/2025 02:15:40.410 7fdb92282978 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 03/31/2025 02:15:40.410 7fdb92282978 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:15:40.410 7fdb92282978 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 03/31/2025 02:15:40.420 7fdb92282978 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:15:40.421 7fdb92282978 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:15:40.422 7fdb92282978 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387340410167 collector_name = collector_ip = collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 03/31/2025 02:15:40.423 7fdb92282978 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:15:40.974 7fdb946b6748 [contrail-vrouter-nodemgr] [DEBUG]: DBG: get_current_processes: '[{'name': 'contrail-vrouter-agent', 'group': 'contrail-vrouter-agent', 'pid': 29957862423847692625607712081126286978877682937110320820489569397903755537530, 'start': '1743387332000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-vrouter-nodemgr', 'group': 'contrail-vrouter-nodemgr', 'pid': 52917628320319147002431172831533834767079181290470762700178333566896946927038, 'start': '1743387329000000', 'statename': 'PROCESS_STATE_RUNNING'}]' 03/31/2025 02:15:41.188 7fdb9227d630 [contrail-vrouter-nodemgr] [DEBUG]: send sync_uve in the [ObjectVRouter:NodeStatusUVE] map 03/31/2025 02:15:41.287 7fdb946b6748 [contrail-vrouter-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-vrouter-nodemgr', _source='cn-jenkins-deploy-platform-juju-os-494-1', _node_type='Compute', _instance_id='0', _timestamp=1743387341287117, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-494-1', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-vrouter-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectVRouter'), _seqnum=0) 03/31/2025 02:15:41.287 7fdb946b6748 [contrail-vrouter-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-vrouter-nodemgr', _source='cn-jenkins-deploy-platform-juju-os-494-1', _node_type='Compute', _instance_id='0', _timestamp=1743387341287648, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-494-1', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='contrail-vrouter-agent', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1743387332000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-vrouter-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1743387329000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "nightly", "build-time" : "2025-03-31 00:53:57.142496", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-8423-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectVRouter'), _seqnum=0) 03/31/2025 02:15:41.291 7fdb91f3dd68 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:15:41.292 7fdb91f3dd68 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 build_info = {"build-info" : [{"build-version" : "nightly", "build-time" : "2025-03-31 00:53:57.142496", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-8423-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]} system_cpu_info = << num_socket = 1 num_cpu = 8 num_core_per_socket = 8 num_thread_per_core = 1 >> running_package_version = nightly-nightly.el7 installed_package_version = nightly-nightly.el7 >> 03/31/2025 02:15:41.292 7fdb91f3dd68 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:15:41.292 7fdb91f3dd68 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:15:41.292 7fdb91f3dd68 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 03/31/2025 02:15:41.292 7fdb91f3dd68 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:15:41.292 7fdb91f3dd68 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:15:41.292 7fdb91f3dd68 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 deleted = False process_info = [ << process_name = contrail-vrouter-agent process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1743387332000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-vrouter-nodemgr process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1743387329000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, ] build_info = {"build-info" : [{"build-version" : "nightly", "build-time" : "2025-03-31 00:53:57.142496", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-8423-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]} >> 03/31/2025 02:15:41.292 7fdb91f3dd68 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:15:44.355 7fdb946b6748 [contrail-vrouter-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-vrouter-nodemgr', _source='cn-jenkins-deploy-platform-juju-os-494-1', _node_type='Compute', _instance_id='0', _timestamp=1743387344354955, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-494-1', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='contrail-vrouter-agent', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1743387332000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-vrouter-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1743387329000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "nightly", "build-time" : "2025-03-31 00:53:57.142496", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-8423-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectVRouter'), _seqnum=0) 03/31/2025 02:15:44.355 7fdb946b6748 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-494-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=50846644, partition_space_available_1k=30243840, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=[], build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=145264, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=65164, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=14963440, free=9758880, buffers=837612, cached=7105180, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=11.52, five_min_avg=5.31, fifteen_min_avg=3.03, cpu_share=0, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 03/31/2025 02:15:44.355 7fdb9164bd68 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:15:44.355 7fdb9164bd68 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 deleted = False process_info = [ << process_name = contrail-vrouter-agent process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1743387332000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-vrouter-nodemgr process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1743387329000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, ] build_info = {"build-info" : [{"build-version" : "nightly", "build-time" : "2025-03-31 00:53:57.142496", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-8423-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]} >> 03/31/2025 02:15:44.355 7fdb9164bd68 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:15:44.355 7fdb9164bd68 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:15:44.356 7fdb9164bd68 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 50846644 partition_space_available_1k = 30243840 percentage_partition_space_used = 63 >> } all_core_file_list = [ ] process_mem_cpu_usage = { contrail-vrouter-agent : << mem_virt = 145264 cpu_share = 0 mem_res = 0 >>contrail-vrouter-nodemgr : << mem_virt = 65164 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665112 used = 14963440 free = 9758880 buffers = 837612 cached = 7105180 node_type = vrouter >> system_cpu_usage = << one_min_avg = 11.52 five_min_avg = 5.31 fifteen_min_avg = 3.03 cpu_share = 0 node_type = vrouter >> >> 03/31/2025 02:15:44.356 7fdb9164bd68 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:15:44.405 7fdb9164bd68 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:15:49.412 7fdb9164bd68 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:15:49.413 7fdb9164bd68 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:15:49.413 7fdb9164bd68 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:15:49.413 7fdb9164bd68 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:15:49.413 7fdb9164bd68 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:15:49.413 7fdb9164bd68 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:15:49.414 7fdb9164bd68 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:15:49.414 7fdb9164bd68 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:15:49.414 7fdb9164bd68 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:15:49.414 7fdb9164bd68 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:15:49.414 7fdb9164bd68 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:15:49.414 7fdb9164bd68 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387340410167 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 03/31/2025 02:15:49.414 7fdb9164bd68 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:15:53.417 7fdb9164bd68 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:15:58.420 7fdb9164bd68 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:15:58.420 7fdb9164bd68 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:15:58.420 7fdb9164bd68 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:15:58.421 7fdb9164bd68 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:15:58.421 7fdb9164bd68 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:15:58.421 7fdb9164bd68 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:15:58.421 7fdb9164bd68 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:15:58.421 7fdb9164bd68 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:15:58.421 7fdb9164bd68 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:15:58.421 7fdb9164bd68 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:15:58.421 7fdb9164bd68 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:15:58.422 7fdb9164bd68 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387340410167 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 03/31/2025 02:15:58.422 7fdb9164bd68 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:16:02.425 7fdb91e558d0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:16:02.538 7fdb90a8db70 [contrail-vrouter-nodemgr] [DEBUG]: send sync_uve in the [ObjectVRouter:NodeStatusUVE] map 03/31/2025 02:16:07.431 7fdb91e558d0 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:16:07.432 7fdb91e558d0 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:16:07.432 7fdb91e558d0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:16:07.432 7fdb91e558d0 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:16:07.432 7fdb91e558d0 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:16:07.432 7fdb91e558d0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:16:07.433 7fdb91e558d0 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:16:07.433 7fdb91e558d0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:16:07.433 7fdb91e558d0 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:16:07.433 7fdb91e558d0 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:16:07.433 7fdb91e558d0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:16:07.434 7fdb91e558d0 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387340410167 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 03/31/2025 02:16:07.434 7fdb91e558d0 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:16:11.436 7fdb91e558d0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:16:13.108 7fdb9a57d048 [contrail-vrouter-nodemgr] [DEBUG]: send sync_uve in the [ObjectVRouter:NodeStatusUVE] map 03/31/2025 02:16:16.441 7fdb91e558d0 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:16:16.441 7fdb91e558d0 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:16:16.441 7fdb91e558d0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:16:16.441 7fdb91e558d0 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:16:16.441 7fdb91e558d0 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:16:16.442 7fdb91e558d0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:16:16.442 7fdb91e558d0 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:16:16.442 7fdb91e558d0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:16:16.442 7fdb91e558d0 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:16:16.442 7fdb91e558d0 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:16:16.442 7fdb91e558d0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:16:16.442 7fdb91e558d0 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387340410167 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 03/31/2025 02:16:16.443 7fdb91e558d0 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:16:20.445 7fdb92282f60 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:16:25.451 7fdb92282f60 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:16:25.452 7fdb92282f60 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:16:25.452 7fdb92282f60 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:16:25.452 7fdb92282f60 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:16:25.452 7fdb92282f60 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:16:25.452 7fdb92282f60 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:16:25.452 7fdb92282f60 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:16:25.454 7fdb92282f60 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:16:25.454 7fdb92282f60 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:16:25.454 7fdb92282f60 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:16:25.454 7fdb92282f60 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:16:25.454 7fdb92282f60 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387340410167 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 03/31/2025 02:16:25.454 7fdb92282f60 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:16:29.454 7fdb91e552e8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:16:34.461 7fdb91e552e8 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:16:34.461 7fdb91e552e8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:16:34.461 7fdb91e552e8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:16:34.461 7fdb91e552e8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:16:34.461 7fdb91e552e8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:16:34.461 7fdb91e552e8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:16:34.462 7fdb91e552e8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:16:34.462 7fdb91e552e8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:16:34.462 7fdb91e552e8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:16:34.462 7fdb91e552e8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:16:34.462 7fdb91e552e8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:16:34.462 7fdb91e552e8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387340410167 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 03/31/2025 02:16:34.462 7fdb91e552e8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:16:38.464 7fdb91e552e8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:16:43.468 7fdb91e552e8 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:16:43.468 7fdb91e552e8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:16:43.468 7fdb91e552e8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:16:43.468 7fdb91e552e8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:16:43.468 7fdb91e552e8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:16:43.468 7fdb91e552e8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:16:43.469 7fdb91e552e8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:16:43.469 7fdb91e552e8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:16:43.469 7fdb91e552e8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:16:43.469 7fdb91e552e8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:16:43.469 7fdb91e552e8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:16:43.469 7fdb91e552e8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387340410167 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 03/31/2025 02:16:43.469 7fdb91e552e8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:16:44.325 7fdb946b6748 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-494-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=50846308, partition_space_available_1k=30244176, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=147076, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=65168, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=14660644, free=9700084, buffers=841292, cached=7463092, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=6.33, five_min_avg=5.08, fifteen_min_avg=3.1, cpu_share=4.12, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 03/31/2025 02:16:44.325 7fdb92297a20 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:16:44.326 7fdb92297a20 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 50846308 partition_space_available_1k = 30244176 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-vrouter-agent : << mem_virt = 147076 cpu_share = 0 mem_res = 0 >>contrail-vrouter-nodemgr : << mem_virt = 65168 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665112 used = 14660644 free = 9700084 buffers = 841292 cached = 7463092 node_type = vrouter >> system_cpu_usage = << one_min_avg = 6.33 five_min_avg = 5.08 fifteen_min_avg = 3.1 cpu_share = 4.12 node_type = vrouter >> >> 03/31/2025 02:16:44.326 7fdb92297a20 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:16:47.471 7fdb92297a20 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:16:52.479 7fdb92297a20 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:16:52.479 7fdb92297a20 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:16:52.479 7fdb92297a20 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:16:52.479 7fdb92297a20 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:16:52.479 7fdb92297a20 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:16:52.479 7fdb92297a20 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:16:52.480 7fdb92297a20 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:16:52.480 7fdb92297a20 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:16:52.480 7fdb92297a20 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:16:52.480 7fdb92297a20 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:16:52.480 7fdb92297a20 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:16:52.480 7fdb92297a20 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387340410167 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 03/31/2025 02:16:52.480 7fdb92297a20 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:16:56.482 7fdb92297a20 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:17:00.721 7fdb94354828 [contrail-vrouter-nodemgr] [DEBUG]: send sync_uve in the [ObjectVRouter:NodeStatusUVE] map 03/31/2025 02:17:01.485 7fdb92297a20 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:17:01.485 7fdb92297a20 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:17:01.485 7fdb92297a20 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:17:01.486 7fdb92297a20 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:17:01.486 7fdb92297a20 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:17:01.486 7fdb92297a20 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:17:01.486 7fdb92297a20 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:17:01.486 7fdb92297a20 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:17:01.486 7fdb92297a20 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:17:01.486 7fdb92297a20 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:17:01.486 7fdb92297a20 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:17:01.486 7fdb92297a20 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387340410167 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 03/31/2025 02:17:01.487 7fdb92297a20 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:17:05.489 7fdb92297a20 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:17:10.496 7fdb92297a20 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:17:10.496 7fdb92297a20 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:17:10.496 7fdb92297a20 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:17:10.497 7fdb92297a20 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:17:10.497 7fdb92297a20 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:17:10.497 7fdb92297a20 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:17:10.497 7fdb92297a20 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:17:10.497 7fdb92297a20 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:17:10.497 7fdb92297a20 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:17:10.497 7fdb92297a20 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:17:10.497 7fdb92297a20 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:17:10.498 7fdb92297a20 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387340410167 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 03/31/2025 02:17:10.498 7fdb92297a20 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:17:14.497 7fdb9164b8d0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:17:19.503 7fdb9164b8d0 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:17:19.503 7fdb9164b8d0 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:17:19.504 7fdb9164b8d0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:17:19.504 7fdb9164b8d0 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:17:19.504 7fdb9164b8d0 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:17:19.504 7fdb9164b8d0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:17:19.504 7fdb9164b8d0 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:17:19.505 7fdb9164b8d0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:17:19.505 7fdb9164b8d0 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:17:19.505 7fdb9164b8d0 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:17:19.505 7fdb9164b8d0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:17:19.505 7fdb9164b8d0 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387340410167 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 03/31/2025 02:17:19.505 7fdb9164b8d0 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:17:23.507 7fdb9164b8d0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:17:28.509 7fdb9164b8d0 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:17:28.510 7fdb9164b8d0 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:17:28.510 7fdb9164b8d0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:17:28.510 7fdb9164b8d0 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:17:28.510 7fdb9164b8d0 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:17:28.510 7fdb9164b8d0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:17:28.511 7fdb9164b8d0 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:17:28.511 7fdb9164b8d0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:17:28.511 7fdb9164b8d0 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:17:28.511 7fdb9164b8d0 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:17:28.511 7fdb9164b8d0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:17:28.511 7fdb9164b8d0 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387340410167 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 03/31/2025 02:17:28.511 7fdb9164b8d0 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:17:32.513 7fdb9165b978 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:17:37.520 7fdb9165b978 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:17:37.520 7fdb9165b978 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:17:37.520 7fdb9165b978 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:17:37.520 7fdb9165b978 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:17:37.520 7fdb9165b978 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:17:37.520 7fdb9165b978 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:17:37.521 7fdb9165b978 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:17:37.521 7fdb9165b978 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:17:37.521 7fdb9165b978 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:17:37.521 7fdb9165b978 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:17:37.521 7fdb9165b978 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:17:37.521 7fdb9165b978 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387340410167 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 03/31/2025 02:17:37.521 7fdb9165b978 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:17:41.520 7fdb9165b978 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:17:45.067 7fdb946b6748 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-494-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=50856856, partition_space_available_1k=30233628, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=148192, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=65216, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=14610808, free=9721820, buffers=842948, cached=7489536, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=3.76, five_min_avg=4.56, fifteen_min_avg=3.05, cpu_share=2.38, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 03/31/2025 02:17:46.525 7fdb9165b978 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:17:46.525 7fdb9165b978 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:17:46.525 7fdb9165b978 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:17:46.525 7fdb9165b978 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:17:46.526 7fdb9165b978 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:17:46.526 7fdb9165b978 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:17:46.526 7fdb9165b978 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 50856856 partition_space_available_1k = 30233628 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-vrouter-agent : << mem_virt = 148192 cpu_share = 0 mem_res = 0 >>contrail-vrouter-nodemgr : << mem_virt = 65216 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665112 used = 14610808 free = 9721820 buffers = 842948 cached = 7489536 node_type = vrouter >> system_cpu_usage = << one_min_avg = 3.76 five_min_avg = 4.56 fifteen_min_avg = 3.05 cpu_share = 2.38 node_type = vrouter >> >> 03/31/2025 02:17:46.526 7fdb9165b978 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:17:46.526 7fdb9165b978 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:17:46.526 7fdb9165b978 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:17:46.526 7fdb9165b978 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:17:46.527 7fdb9165b978 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:17:46.527 7fdb9165b978 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:17:46.527 7fdb9165b978 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:17:46.527 7fdb9165b978 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387340410167 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 03/31/2025 02:17:46.527 7fdb9165b978 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:17:50.529 7fdb9165b978 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:17:55.535 7fdb9165b978 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:17:55.536 7fdb9165b978 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:17:55.536 7fdb9165b978 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:17:55.536 7fdb9165b978 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:17:55.536 7fdb9165b978 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:17:55.536 7fdb9165b978 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:17:55.537 7fdb9165b978 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:17:55.537 7fdb9165b978 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:17:55.537 7fdb9165b978 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:17:55.537 7fdb9165b978 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:17:55.537 7fdb9165b978 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:17:55.537 7fdb9165b978 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387340410167 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 03/31/2025 02:17:55.537 7fdb9165b978 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:17:59.537 7fdb8beb66d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:18:04.543 7fdb8beb66d8 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:18:04.543 7fdb8beb66d8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:18:04.543 7fdb8beb66d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:18:04.543 7fdb8beb66d8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:18:04.543 7fdb8beb66d8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:18:04.544 7fdb8beb66d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:18:04.544 7fdb8beb66d8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:18:04.544 7fdb8beb66d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:18:04.544 7fdb8beb66d8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:18:04.544 7fdb8beb66d8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:18:04.544 7fdb8beb66d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:18:04.544 7fdb8beb66d8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387340410167 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 03/31/2025 02:18:04.544 7fdb8beb66d8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:18:08.547 7fdb8beb66d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:18:13.550 7fdb8beb66d8 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:18:13.550 7fdb8beb66d8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:18:13.551 7fdb8beb66d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:18:13.551 7fdb8beb66d8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:18:13.551 7fdb8beb66d8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:18:13.551 7fdb8beb66d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:18:13.551 7fdb8beb66d8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:18:13.551 7fdb8beb66d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:18:13.552 7fdb8beb66d8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:18:13.552 7fdb8beb66d8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:18:13.552 7fdb8beb66d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:18:13.552 7fdb8beb66d8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387340410167 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 03/31/2025 02:18:13.552 7fdb8beb66d8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:18:17.554 7fdb8beb66d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:18:22.561 7fdb8beb66d8 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:18:22.561 7fdb8beb66d8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:18:22.561 7fdb8beb66d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:18:22.561 7fdb8beb66d8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:18:22.561 7fdb8beb66d8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:18:22.562 7fdb8beb66d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:18:22.562 7fdb8beb66d8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:18:22.562 7fdb8beb66d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:18:22.562 7fdb8beb66d8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:18:22.562 7fdb8beb66d8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:18:22.562 7fdb8beb66d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:18:22.562 7fdb8beb66d8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387340410167 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 03/31/2025 02:18:22.562 7fdb8beb66d8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:18:26.565 7fdb8beb66d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:18:31.570 7fdb8beb66d8 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:18:31.571 7fdb8beb66d8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:18:31.571 7fdb8beb66d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:18:31.571 7fdb8beb66d8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:18:31.571 7fdb8beb66d8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:18:31.571 7fdb8beb66d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:18:31.571 7fdb8beb66d8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:18:31.572 7fdb8beb66d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:18:31.572 7fdb8beb66d8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:18:31.572 7fdb8beb66d8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:18:31.572 7fdb8beb66d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:18:31.572 7fdb8beb66d8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387340410167 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 03/31/2025 02:18:31.572 7fdb8beb66d8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:18:35.574 7fdb8beb66d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:18:40.584 7fdb8beb66d8 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:18:40.585 7fdb8beb66d8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:18:40.585 7fdb8beb66d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:18:40.585 7fdb8beb66d8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:18:40.585 7fdb8beb66d8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:18:40.585 7fdb8beb66d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:18:40.585 7fdb8beb66d8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:18:40.585 7fdb8beb66d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:18:40.586 7fdb8beb66d8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:18:40.586 7fdb8beb66d8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:18:40.586 7fdb8beb66d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:18:40.586 7fdb8beb66d8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387340410167 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 03/31/2025 02:18:40.586 7fdb8beb66d8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:18:44.586 7fdb91650390 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:18:45.329 7fdb946b6748 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-494-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=50860164, partition_space_available_1k=30230320, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=149464, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=65240, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=14628568, free=9698808, buffers=842984, cached=7494752, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=2.44, five_min_avg=4.03, fifteen_min_avg=2.96, cpu_share=2.25, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 03/31/2025 02:18:49.592 7fdb91650390 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:18:49.593 7fdb91650390 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:18:49.593 7fdb91650390 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:18:49.593 7fdb91650390 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:18:49.593 7fdb91650390 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:18:49.593 7fdb91650390 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:18:49.593 7fdb91650390 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 50860164 partition_space_available_1k = 30230320 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-vrouter-agent : << mem_virt = 149464 cpu_share = 0 mem_res = 0 >>contrail-vrouter-nodemgr : << mem_virt = 65240 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665112 used = 14628568 free = 9698808 buffers = 842984 cached = 7494752 node_type = vrouter >> system_cpu_usage = << one_min_avg = 2.44 five_min_avg = 4.03 fifteen_min_avg = 2.96 cpu_share = 2.25 node_type = vrouter >> >> 03/31/2025 02:18:49.593 7fdb91650390 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:18:49.593 7fdb91650390 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:18:49.594 7fdb91650390 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:18:49.594 7fdb91650390 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:18:49.594 7fdb91650390 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:18:49.594 7fdb91650390 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:18:49.594 7fdb91650390 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:18:49.594 7fdb91650390 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387340410167 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 03/31/2025 02:18:49.595 7fdb91650390 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:18:53.597 7fdb91e82048 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:18:58.599 7fdb91e82048 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:18:58.599 7fdb91e82048 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:18:58.599 7fdb91e82048 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:18:58.599 7fdb91e82048 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:18:58.599 7fdb91e82048 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:18:58.599 7fdb91e82048 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:18:58.600 7fdb91e82048 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:18:58.600 7fdb91e82048 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:18:58.600 7fdb91e82048 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:18:58.600 7fdb91e82048 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:18:58.600 7fdb91e82048 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:18:58.600 7fdb91e82048 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387340410167 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 03/31/2025 02:18:58.600 7fdb91e82048 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:19:02.601 7fdb91650828 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:19:07.606 7fdb91650828 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:19:07.606 7fdb91650828 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:19:07.606 7fdb91650828 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:19:07.606 7fdb91650828 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:19:07.607 7fdb91650828 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:19:07.607 7fdb91650828 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:19:07.607 7fdb91650828 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:19:07.607 7fdb91650828 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:19:07.607 7fdb91650828 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:19:07.607 7fdb91650828 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:19:07.607 7fdb91650828 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:19:07.607 7fdb91650828 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387340410167 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 03/31/2025 02:19:07.607 7fdb91650828 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:19:10.469 7fdb91650828 [contrail-vrouter-nodemgr] [DEBUG]: send sync_uve in the [ObjectVRouter:NodeStatusUVE] map 03/31/2025 02:19:11.608 7fdb91650828 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:19:16.613 7fdb91650828 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:19:16.613 7fdb91650828 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:19:16.613 7fdb91650828 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:19:16.613 7fdb91650828 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:19:16.613 7fdb91650828 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:19:16.613 7fdb91650828 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:19:16.614 7fdb91650828 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:19:16.614 7fdb91650828 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:19:16.614 7fdb91650828 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:19:16.614 7fdb91650828 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:19:16.614 7fdb91650828 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:19:16.614 7fdb91650828 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387340410167 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 03/31/2025 02:19:16.614 7fdb91650828 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:19:19.246 7fdb9164b588 [contrail-vrouter-nodemgr] [DEBUG]: send sync_uve in the [ObjectVRouter:NodeStatusUVE] map 03/31/2025 02:19:20.613 7fdb9164b588 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:19:25.619 7fdb9164b588 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:19:25.619 7fdb9164b588 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:19:25.619 7fdb9164b588 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:19:25.619 7fdb9164b588 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:19:25.620 7fdb9164b588 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:19:25.620 7fdb9164b588 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:19:25.620 7fdb9164b588 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:19:25.620 7fdb9164b588 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:19:25.620 7fdb9164b588 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:19:25.620 7fdb9164b588 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:19:25.621 7fdb9164b588 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:19:25.621 7fdb9164b588 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387340410167 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 03/31/2025 02:19:25.621 7fdb9164b588 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:19:29.620 7fdb91652390 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:19:30.832 7fdb946a7b70 [contrail-vrouter-nodemgr] [DEBUG]: send sync_uve in the [ObjectVRouter:NodeStatusUVE] map 03/31/2025 02:19:34.626 7fdb91652390 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:19:34.626 7fdb91652390 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:19:34.626 7fdb91652390 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:19:34.626 7fdb91652390 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:19:34.626 7fdb91652390 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:19:34.626 7fdb91652390 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:19:34.627 7fdb91652390 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:19:34.627 7fdb91652390 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:19:34.627 7fdb91652390 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:19:34.627 7fdb91652390 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:19:34.627 7fdb91652390 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:19:34.627 7fdb91652390 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387340410167 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 03/31/2025 02:19:34.627 7fdb91652390 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:19:38.631 7fdb91652390 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:19:41.430 7fdb946b6748 [contrail-vrouter-nodemgr] [WARNING]: Exception in run_periodically job: 404 Client Error: Not Found ("No such container: 423b8c9ba3cc30831528ef21450f56452d10f2a1bd5472ffeb8ff46573b76c7a") 03/31/2025 02:19:47.869 7ff6f2605648 [contrail-vrouter-nodemgr] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 03/31/2025 02:19:47.870 7ff6f2605648 [contrail-vrouter-nodemgr] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 03/31/2025 02:19:47.870 7ff6f2605648 [contrail-vrouter-nodemgr] [INFO]: SANDESH: Flow Logging: [None] -> [False] 03/31/2025 02:19:47.888 7ff6f01f6048 [contrail-vrouter-nodemgr] [ERROR]: Cannot write http_port 8102 to /tmp/contrail-vrouter-nodemgr.0.http_port 03/31/2025 02:19:47.888 7ff6f01f6048 [contrail-vrouter-nodemgr] [ERROR]: Starting Introspect on HTTP Port 8102 03/31/2025 02:19:47.889 7ff6f01fe978 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvStart] in state[none] 03/31/2025 02:19:47.890 7ff6f01fe978 [contrail-vrouter-nodemgr] [DEBUG]: Add uve in the [ObjectVRouter:NodeStatusUVE] map 03/31/2025 02:19:47.890 7ff6f01fe978 [contrail-vrouter-nodemgr] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 03/31/2025 02:19:47.891 7ff6f01fe978 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 03/31/2025 02:19:47.891 7ff6f01fe978 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:19:47.891 7ff6f01fe978 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 03/31/2025 02:19:47.891 7ff6f01fe978 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:19:47.891 7ff6f01fe978 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:19:47.891 7ff6f01fe978 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 03/31/2025 02:19:47.892 7ff6f01fe978 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:19:48.108 7ff6f2605648 [contrail-vrouter-nodemgr] [DEBUG]: DBG: get_current_processes: '[{'name': 'contrail-vrouter-agent', 'group': 'contrail-vrouter-agent', 'pid': 64036605438499028842560858000628884640009772655538780962095384382818285862660, 'start': '1743387577000000', 'statename': 'PROCESS_STATE_RUNNING'}, {'name': 'contrail-vrouter-nodemgr', 'group': 'contrail-vrouter-nodemgr', 'pid': 94414443188488351063217311149720870966631556393091168532605946104646101478171, 'start': '1743387585000000', 'statename': 'PROCESS_STATE_RUNNING'}]' 03/31/2025 02:19:48.276 7ff6f2605648 [contrail-vrouter-nodemgr] [INFO]: send_nodemgr_process_status: Sending UVE:NodeStatusUVE(_context='', _scope='', _module='contrail-vrouter-nodemgr', _source='cn-jenkins-deploy-platform-juju-os-494-1', _node_type='Compute', _instance_id='0', _timestamp=1743387588275958, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-494-1', deleted=None, status=None, process_status=[ProcessStatus(module_id='contrail-vrouter-nodemgr', instance_id='0', state='Non-Functional', connection_infos=dict_values([ConnectionInfo(type='Collector', name='Collector', server_addrs=[''], status='Down', description='none to Idle on EvStart')]), flag_infos=None, description='Collector:Collector[none to Idle on EvStart] connection down')], process_info=None, disk_usage_info=None, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectVRouter'), _seqnum=0) 03/31/2025 02:19:48.276 7ff6f2605648 [contrail-vrouter-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-vrouter-nodemgr', _source='cn-jenkins-deploy-platform-juju-os-494-1', _node_type='Compute', _instance_id='0', _timestamp=1743387588276470, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-494-1', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='contrail-vrouter-agent', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1743387577000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-vrouter-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1743387585000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "nightly", "build-time" : "2025-03-31 00:53:57.142496", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-8423-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectVRouter'), _seqnum=0) 03/31/2025 02:19:48.280 7ff6eeb45438 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:19:48.281 7ff6eeb45438 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 build_info = {"build-info" : [{"build-version" : "nightly", "build-time" : "2025-03-31 00:53:57.142496", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-8423-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]} system_cpu_info = << num_socket = 1 num_cpu = 8 num_core_per_socket = 8 num_thread_per_core = 1 >> running_package_version = nightly-nightly.el7 installed_package_version = nightly-nightly.el7 >> 03/31/2025 02:19:48.281 7ff6eeb45438 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:19:48.281 7ff6eeb45438 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:19:48.285 7ff6eeb45438 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 03/31/2025 02:19:48.285 7ff6eeb45438 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:19:48.285 7ff6eeb45438 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:19:48.285 7ff6eeb45438 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 deleted = False process_info = [ << process_name = contrail-vrouter-agent process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1743387577000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-vrouter-nodemgr process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1743387585000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, ] build_info = {"build-info" : [{"build-version" : "nightly", "build-time" : "2025-03-31 00:53:57.142496", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-8423-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]} >> 03/31/2025 02:19:48.285 7ff6eeb45438 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:19:51.893 7ff6ee9b7438 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:19:52.160 7ff6f2605648 [contrail-vrouter-nodemgr] [INFO]: send_process_state_db: Sending UVE: NodeStatusUVE(_context='', _scope='', _module='contrail-vrouter-nodemgr', _source='cn-jenkins-deploy-platform-juju-os-494-1', _node_type='Compute', _instance_id='0', _timestamp=1743387592160552, _versionsig=3672106175, _type=6, _hints=1, _client_context='', _client=None, _role=0, _logger=None, _level=2147483647, _category='', _send_queue_enabled=True, _http_server=None, _connect_to_collector=True, _disable_sending_object_logs=False, _disable_sending_all_messages=False, _more=False, data=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-494-1', deleted=False, status=None, process_status=None, process_info=[ProcessInfo(process_name='contrail-vrouter-agent', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1743387577000000', last_stop_time='', last_exit_time='', core_file_list=[]), ProcessInfo(process_name='contrail-vrouter-nodemgr', process_state='PROCESS_STATE_RUNNING', start_count=1, stop_count=0, exit_count=0, last_start_time='1743387585000000', last_stop_time='', last_exit_time='', core_file_list=[])], disk_usage_info=None, description=None, all_core_file_list=None, build_info='{"build-info" : [{"build-version" : "nightly", "build-time" : "2025-03-31 00:53:57.142496", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-8423-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]}', process_mem_cpu_usage=None, system_cpu_info=None, system_mem_usage=None, system_cpu_usage=None, running_package_version=None, installed_package_version=None, _table='ObjectVRouter'), _seqnum=0) 03/31/2025 02:19:52.161 7ff6f2605648 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-494-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=50866464, partition_space_available_1k=30224020, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=[], build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=110336, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50424, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=14781804, free=9468864, buffers=843168, cached=7571276, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=7.06, five_min_avg=4.74, fifteen_min_avg=3.26, cpu_share=0, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 03/31/2025 02:19:56.900 7ff6ee9b7438 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:19:56.900 7ff6ee9b7438 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:19:56.900 7ff6ee9b7438 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:19:56.900 7ff6ee9b7438 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:19:56.900 7ff6ee9b7438 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:19:56.900 7ff6ee9b7438 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:19:56.901 7ff6ee9b7438 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 deleted = False process_info = [ << process_name = contrail-vrouter-agent process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1743387577000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, << process_name = contrail-vrouter-nodemgr process_state = PROCESS_STATE_RUNNING start_count = 1 stop_count = 0 exit_count = 0 last_start_time = 1743387585000000 last_stop_time = last_exit_time = core_file_list = [ ] >>, ] build_info = {"build-info" : [{"build-version" : "nightly", "build-time" : "2025-03-31 00:53:57.142496", "build-user" : "root", "build-hostname" : "jenkins-build-prepare-sandbox-8423-1.novalocal", "build-id" : "nightly", "build-number" : "nightly.el7"}]} >> 03/31/2025 02:19:56.901 7ff6ee9b7438 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:19:56.901 7ff6ee9b7438 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:19:56.901 7ff6ee9b7438 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 50866464 partition_space_available_1k = 30224020 percentage_partition_space_used = 63 >> } all_core_file_list = [ ] process_mem_cpu_usage = { contrail-vrouter-agent : << mem_virt = 110336 cpu_share = 0 mem_res = 0 >>contrail-vrouter-nodemgr : << mem_virt = 50424 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665112 used = 14781804 free = 9468864 buffers = 843168 cached = 7571276 node_type = vrouter >> system_cpu_usage = << one_min_avg = 7.06 five_min_avg = 4.74 fifteen_min_avg = 3.26 cpu_share = 0 node_type = vrouter >> >> 03/31/2025 02:19:56.901 7ff6ee9b7438 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:19:56.901 7ff6ee9b7438 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:19:56.902 7ff6ee9b7438 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:19:56.902 7ff6ee9b7438 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:19:56.902 7ff6ee9b7438 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:19:56.902 7ff6ee9b7438 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:19:56.902 7ff6ee9b7438 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:19:56.902 7ff6ee9b7438 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:19:56.902 7ff6ee9b7438 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:20:00.904 7ff6f8283240 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:20:05.909 7ff6f8283240 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:20:05.909 7ff6f8283240 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:20:05.909 7ff6f8283240 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:20:05.909 7ff6f8283240 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:20:05.909 7ff6f8283240 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:20:05.909 7ff6f8283240 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:20:05.910 7ff6f8283240 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:20:05.910 7ff6f8283240 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:20:05.910 7ff6f8283240 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:20:05.910 7ff6f8283240 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:20:05.910 7ff6f8283240 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:20:05.911 7ff6f8283240 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:20:05.911 7ff6f8283240 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:20:09.913 7ff6f8283240 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:20:14.919 7ff6f8283240 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:20:14.919 7ff6f8283240 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:20:14.919 7ff6f8283240 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:20:14.919 7ff6f8283240 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:20:14.919 7ff6f8283240 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:20:14.919 7ff6f8283240 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:20:14.920 7ff6f8283240 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:20:14.920 7ff6f8283240 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:20:14.920 7ff6f8283240 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:20:14.920 7ff6f8283240 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:20:14.920 7ff6f8283240 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:20:14.920 7ff6f8283240 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:20:14.920 7ff6f8283240 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:20:18.919 7ff6f23d0b70 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:20:23.925 7ff6f23d0b70 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:20:23.925 7ff6f23d0b70 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:20:23.925 7ff6f23d0b70 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:20:23.925 7ff6f23d0b70 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:20:23.925 7ff6f23d0b70 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:20:23.925 7ff6f23d0b70 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:20:23.926 7ff6f23d0b70 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:20:23.926 7ff6f23d0b70 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:20:23.926 7ff6f23d0b70 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:20:23.926 7ff6f23d0b70 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:20:23.926 7ff6f23d0b70 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:20:23.927 7ff6f23d0b70 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:20:23.927 7ff6f23d0b70 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:20:27.929 7ff6f23d0b70 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:20:32.935 7ff6f23d0b70 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:20:32.935 7ff6f23d0b70 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:20:32.935 7ff6f23d0b70 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:20:32.935 7ff6f23d0b70 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:20:32.935 7ff6f23d0b70 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:20:32.936 7ff6f23d0b70 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:20:32.936 7ff6f23d0b70 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:20:32.936 7ff6f23d0b70 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:20:32.936 7ff6f23d0b70 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:20:32.936 7ff6f23d0b70 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:20:32.936 7ff6f23d0b70 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:20:32.936 7ff6f23d0b70 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:20:32.936 7ff6f23d0b70 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:20:36.937 7ff6efe0b978 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:20:41.943 7ff6efe0b978 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:20:41.943 7ff6efe0b978 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:20:41.944 7ff6efe0b978 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:20:41.944 7ff6efe0b978 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:20:41.944 7ff6efe0b978 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:20:41.944 7ff6efe0b978 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:20:41.944 7ff6efe0b978 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:20:41.945 7ff6efe0b978 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:20:41.945 7ff6efe0b978 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:20:41.945 7ff6efe0b978 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:20:41.945 7ff6efe0b978 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:20:41.945 7ff6efe0b978 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:20:41.945 7ff6efe0b978 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:20:45.947 7ff6efe0b978 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:20:50.950 7ff6efe0b978 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:20:50.950 7ff6efe0b978 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:20:50.951 7ff6efe0b978 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:20:50.951 7ff6efe0b978 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:20:50.951 7ff6efe0b978 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:20:50.951 7ff6efe0b978 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:20:50.952 7ff6efe0b978 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:20:50.952 7ff6efe0b978 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:20:50.952 7ff6efe0b978 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:20:50.952 7ff6efe0b978 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:20:50.952 7ff6efe0b978 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:20:50.952 7ff6efe0b978 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:20:50.952 7ff6efe0b978 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:20:51.309 7ff6f2605648 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-494-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=50876616, partition_space_available_1k=30213868, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=111988, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50416, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=14658684, free=9486768, buffers=843180, cached=7676480, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=5.08, five_min_avg=4.57, fifteen_min_avg=3.29, cpu_share=3.88, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 03/31/2025 02:20:51.310 7ff6efe0b198 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:20:51.310 7ff6efe0b198 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 50876616 partition_space_available_1k = 30213868 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-vrouter-agent : << mem_virt = 111988 cpu_share = 0 mem_res = 0 >>contrail-vrouter-nodemgr : << mem_virt = 50416 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665112 used = 14658684 free = 9486768 buffers = 843180 cached = 7676480 node_type = vrouter >> system_cpu_usage = << one_min_avg = 5.08 five_min_avg = 4.57 fifteen_min_avg = 3.29 cpu_share = 3.88 node_type = vrouter >> >> 03/31/2025 02:20:51.310 7ff6efe0b198 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:20:54.954 7ff6efe0b198 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:20:59.961 7ff6efe0b198 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:20:59.961 7ff6efe0b198 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:20:59.961 7ff6efe0b198 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:20:59.961 7ff6efe0b198 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:20:59.961 7ff6efe0b198 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:20:59.961 7ff6efe0b198 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:20:59.962 7ff6efe0b198 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:20:59.962 7ff6efe0b198 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:20:59.962 7ff6efe0b198 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:20:59.962 7ff6efe0b198 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:20:59.962 7ff6efe0b198 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:20:59.962 7ff6efe0b198 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:20:59.962 7ff6efe0b198 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:21:03.965 7ff6efe0b198 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:21:08.971 7ff6efe0b198 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:21:08.972 7ff6efe0b198 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:21:08.972 7ff6efe0b198 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:21:08.972 7ff6efe0b198 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:21:08.972 7ff6efe0b198 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:21:08.972 7ff6efe0b198 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:21:08.973 7ff6efe0b198 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:21:08.973 7ff6efe0b198 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:21:08.973 7ff6efe0b198 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:21:08.973 7ff6efe0b198 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:21:08.973 7ff6efe0b198 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:21:08.973 7ff6efe0b198 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:21:08.973 7ff6efe0b198 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:21:12.974 7ff6f2f9bac8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:21:17.980 7ff6f2f9bac8 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:21:17.981 7ff6f2f9bac8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:21:17.981 7ff6f2f9bac8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:21:17.981 7ff6f2f9bac8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:21:17.981 7ff6f2f9bac8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:21:17.981 7ff6f2f9bac8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:21:17.982 7ff6f2f9bac8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:21:17.982 7ff6f2f9bac8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:21:17.982 7ff6f2f9bac8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:21:17.982 7ff6f2f9bac8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:21:17.982 7ff6f2f9bac8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:21:17.982 7ff6f2f9bac8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:21:17.982 7ff6f2f9bac8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:21:21.984 7ff6efe190f0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:21:26.989 7ff6efe190f0 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:21:26.989 7ff6efe190f0 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:21:26.989 7ff6efe190f0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:21:26.989 7ff6efe190f0 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:21:26.989 7ff6efe190f0 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:21:26.989 7ff6efe190f0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:21:26.990 7ff6efe190f0 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:21:26.990 7ff6efe190f0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:21:26.990 7ff6efe190f0 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:21:26.990 7ff6efe190f0 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:21:26.990 7ff6efe190f0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:21:26.990 7ff6efe190f0 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:21:26.990 7ff6efe190f0 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:21:30.993 7ff6efe190f0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:21:35.995 7ff6efe190f0 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:21:35.996 7ff6efe190f0 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:21:35.996 7ff6efe190f0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:21:35.996 7ff6efe190f0 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:21:35.996 7ff6efe190f0 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:21:35.996 7ff6efe190f0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:21:35.996 7ff6efe190f0 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:21:35.997 7ff6efe190f0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:21:35.997 7ff6efe190f0 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:21:35.997 7ff6efe190f0 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:21:35.997 7ff6efe190f0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:21:35.997 7ff6efe190f0 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:21:35.997 7ff6efe190f0 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:21:39.999 7ff6ee9c6588 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:21:45.005 7ff6ee9c6588 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:21:45.005 7ff6ee9c6588 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:21:45.005 7ff6ee9c6588 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:21:45.005 7ff6ee9c6588 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:21:45.005 7ff6ee9c6588 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:21:45.005 7ff6ee9c6588 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:21:45.006 7ff6ee9c6588 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:21:45.006 7ff6ee9c6588 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:21:45.006 7ff6ee9c6588 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:21:45.006 7ff6ee9c6588 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:21:45.006 7ff6ee9c6588 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:21:45.006 7ff6ee9c6588 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:21:45.007 7ff6ee9c6588 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:21:49.006 7ff6ee9c6588 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:21:51.308 7ff6f2605648 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-494-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=50880244, partition_space_available_1k=30210240, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=113064, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50468, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=14667084, free=9473372, buffers=843180, cached=7681476, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=3.75, five_min_avg=4.28, fifteen_min_avg=3.27, cpu_share=2.38, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 03/31/2025 02:21:54.010 7ff6ee9c6588 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:21:54.010 7ff6ee9c6588 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:21:54.010 7ff6ee9c6588 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:21:54.011 7ff6ee9c6588 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:21:54.011 7ff6ee9c6588 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:21:54.011 7ff6ee9c6588 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:21:54.011 7ff6ee9c6588 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 50880244 partition_space_available_1k = 30210240 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-vrouter-agent : << mem_virt = 113064 cpu_share = 0 mem_res = 0 >>contrail-vrouter-nodemgr : << mem_virt = 50468 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665112 used = 14667084 free = 9473372 buffers = 843180 cached = 7681476 node_type = vrouter >> system_cpu_usage = << one_min_avg = 3.75 five_min_avg = 4.28 fifteen_min_avg = 3.27 cpu_share = 2.38 node_type = vrouter >> >> 03/31/2025 02:21:54.011 7ff6ee9c6588 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:21:54.011 7ff6ee9c6588 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:21:54.011 7ff6ee9c6588 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:21:54.012 7ff6ee9c6588 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:21:54.012 7ff6ee9c6588 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:21:54.012 7ff6ee9c6588 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:21:54.012 7ff6ee9c6588 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:21:54.012 7ff6ee9c6588 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:21:54.012 7ff6ee9c6588 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:21:58.013 7ff6f23d0780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:22:03.019 7ff6f23d0780 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:22:03.019 7ff6f23d0780 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:22:03.019 7ff6f23d0780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:22:03.019 7ff6f23d0780 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:22:03.019 7ff6f23d0780 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:22:03.019 7ff6f23d0780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:22:03.020 7ff6f23d0780 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:22:03.020 7ff6f23d0780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:22:03.020 7ff6f23d0780 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:22:03.020 7ff6f23d0780 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:22:03.020 7ff6f23d0780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:22:03.020 7ff6f23d0780 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:22:03.020 7ff6f23d0780 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:22:07.021 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:22:12.025 7ff6efecbeb8 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:22:12.025 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:22:12.025 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:22:12.025 7ff6efecbeb8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:22:12.025 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:22:12.025 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:22:12.026 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:22:12.026 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:22:12.026 7ff6efecbeb8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:22:12.026 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:22:12.026 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:22:12.026 7ff6efecbeb8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:22:12.026 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:22:16.029 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:22:21.033 7ff6efecbeb8 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:22:21.033 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:22:21.033 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:22:21.033 7ff6efecbeb8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:22:21.033 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:22:21.033 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:22:21.034 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:22:21.034 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:22:21.034 7ff6efecbeb8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:22:21.034 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:22:21.034 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:22:21.034 7ff6efecbeb8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:22:21.034 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:22:25.033 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:22:30.037 7ff6efecbeb8 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:22:30.037 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:22:30.037 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:22:30.037 7ff6efecbeb8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:22:30.037 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:22:30.037 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:22:30.038 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:22:30.038 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:22:30.038 7ff6efecbeb8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:22:30.038 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:22:30.038 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:22:30.038 7ff6efecbeb8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:22:30.038 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:22:34.041 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:22:39.047 7ff6efecbeb8 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:22:39.047 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:22:39.047 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:22:39.047 7ff6efecbeb8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:22:39.047 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:22:39.048 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:22:39.048 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:22:39.048 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:22:39.048 7ff6efecbeb8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:22:39.048 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:22:39.048 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:22:39.048 7ff6efecbeb8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:22:39.048 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:22:43.052 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:22:48.058 7ff6efecbeb8 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:22:48.058 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:22:48.058 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:22:48.058 7ff6efecbeb8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:22:48.058 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:22:48.059 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:22:48.059 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:22:48.059 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:22:48.059 7ff6efecbeb8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:22:48.059 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:22:48.059 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:22:48.059 7ff6efecbeb8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:22:48.059 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:22:51.314 7ff6f2605648 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-494-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=50884264, partition_space_available_1k=30206220, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=114200, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50468, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=14666800, free=9469064, buffers=843184, cached=7686064, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=2.47, five_min_avg=3.79, fifteen_min_avg=3.16, cpu_share=2.38, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 03/31/2025 02:22:51.314 7ff6f941a0f0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:22:51.314 7ff6f941a0f0 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 50884264 partition_space_available_1k = 30206220 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-vrouter-agent : << mem_virt = 114200 cpu_share = 0 mem_res = 0 >>contrail-vrouter-nodemgr : << mem_virt = 50468 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665112 used = 14666800 free = 9469064 buffers = 843184 cached = 7686064 node_type = vrouter >> system_cpu_usage = << one_min_avg = 2.47 five_min_avg = 3.79 fifteen_min_avg = 3.16 cpu_share = 2.38 node_type = vrouter >> >> 03/31/2025 02:22:51.314 7ff6f941a0f0 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:22:52.059 7ff6f941a0f0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:22:57.064 7ff6f941a0f0 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:22:57.065 7ff6f941a0f0 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:22:57.065 7ff6f941a0f0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:22:57.065 7ff6f941a0f0 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:22:57.065 7ff6f941a0f0 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:22:57.065 7ff6f941a0f0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:22:57.065 7ff6f941a0f0 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:22:57.066 7ff6f941a0f0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:22:57.066 7ff6f941a0f0 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:22:57.066 7ff6f941a0f0 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:22:57.066 7ff6f941a0f0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:22:57.066 7ff6f941a0f0 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:22:57.066 7ff6f941a0f0 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:23:01.066 7ff6f941a0f0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:23:06.070 7ff6f941a0f0 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:23:06.070 7ff6f941a0f0 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:23:06.070 7ff6f941a0f0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:23:06.071 7ff6f941a0f0 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:23:06.071 7ff6f941a0f0 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:23:06.071 7ff6f941a0f0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:23:06.071 7ff6f941a0f0 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:23:06.071 7ff6f941a0f0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:23:06.071 7ff6f941a0f0 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:23:06.071 7ff6f941a0f0 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:23:06.071 7ff6f941a0f0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:23:06.072 7ff6f941a0f0 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:23:06.072 7ff6f941a0f0 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:23:10.073 7ff6f941a0f0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:23:15.077 7ff6f941a0f0 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:23:15.077 7ff6f941a0f0 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:23:15.077 7ff6f941a0f0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:23:15.077 7ff6f941a0f0 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:23:15.077 7ff6f941a0f0 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:23:15.077 7ff6f941a0f0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:23:15.078 7ff6f941a0f0 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:23:15.078 7ff6f941a0f0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:23:15.078 7ff6f941a0f0 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:23:15.078 7ff6f941a0f0 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:23:15.078 7ff6f941a0f0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:23:15.078 7ff6f941a0f0 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:23:15.078 7ff6f941a0f0 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:23:19.082 7ff6f941a0f0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:23:24.087 7ff6f941a0f0 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:23:24.087 7ff6f941a0f0 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:23:24.087 7ff6f941a0f0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:23:24.088 7ff6f941a0f0 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:23:24.088 7ff6f941a0f0 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:23:24.088 7ff6f941a0f0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:23:24.088 7ff6f941a0f0 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:23:24.088 7ff6f941a0f0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:23:24.088 7ff6f941a0f0 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:23:24.088 7ff6f941a0f0 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:23:24.088 7ff6f941a0f0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:23:24.089 7ff6f941a0f0 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:23:24.089 7ff6f941a0f0 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:23:28.089 7ff6f941a0f0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:23:33.093 7ff6f941a0f0 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:23:33.093 7ff6f941a0f0 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:23:33.093 7ff6f941a0f0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:23:33.093 7ff6f941a0f0 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:23:33.093 7ff6f941a0f0 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:23:33.094 7ff6f941a0f0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:23:33.094 7ff6f941a0f0 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:23:33.094 7ff6f941a0f0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:23:33.094 7ff6f941a0f0 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:23:33.094 7ff6f941a0f0 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:23:33.094 7ff6f941a0f0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:23:33.094 7ff6f941a0f0 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:23:33.095 7ff6f941a0f0 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:23:37.094 7ff6efe29b70 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:23:40.459 7ff6eeb2c978 [contrail-vrouter-nodemgr] [DEBUG]: send sync_uve in the [ObjectVRouter:NodeStatusUVE] map 03/31/2025 02:23:42.098 7ff6efe29b70 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:23:42.098 7ff6efe29b70 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:23:42.098 7ff6efe29b70 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:23:42.098 7ff6efe29b70 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:23:42.098 7ff6efe29b70 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:23:42.098 7ff6efe29b70 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:23:42.099 7ff6efe29b70 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:23:42.099 7ff6efe29b70 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:23:42.099 7ff6efe29b70 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:23:42.099 7ff6efe29b70 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:23:42.099 7ff6efe29b70 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:23:42.099 7ff6efe29b70 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:23:42.099 7ff6efe29b70 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:23:46.102 7ff6efe29b70 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:23:49.070 7ff6efeb6d68 [contrail-vrouter-nodemgr] [DEBUG]: send sync_uve in the [ObjectVRouter:NodeStatusUVE] map 03/31/2025 02:23:51.104 7ff6efe29b70 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:23:51.104 7ff6efe29b70 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:23:51.105 7ff6efe29b70 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:23:51.105 7ff6efe29b70 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:23:51.105 7ff6efe29b70 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:23:51.106 7ff6efe29b70 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:23:51.106 7ff6efe29b70 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:23:51.107 7ff6efe29b70 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:23:51.107 7ff6efe29b70 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:23:51.107 7ff6efe29b70 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:23:51.107 7ff6efe29b70 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:23:51.107 7ff6efe29b70 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:23:51.108 7ff6efe29b70 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:23:51.314 7ff6f2605648 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-494-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=50888916, partition_space_available_1k=30201568, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=115280, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50488, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=14725472, free=9401916, buffers=843236, cached=7694488, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=2.28, five_min_avg=3.45, fifteen_min_avg=3.08, cpu_share=2.5, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 03/31/2025 02:23:51.314 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:23:51.314 7ff6efeb6828 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 50888916 partition_space_available_1k = 30201568 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-vrouter-agent : << mem_virt = 115280 cpu_share = 0 mem_res = 0 >>contrail-vrouter-nodemgr : << mem_virt = 50488 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665112 used = 14725472 free = 9401916 buffers = 843236 cached = 7694488 node_type = vrouter >> system_cpu_usage = << one_min_avg = 2.28 five_min_avg = 3.45 fifteen_min_avg = 3.08 cpu_share = 2.5 node_type = vrouter >> >> 03/31/2025 02:23:51.314 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:23:55.108 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:24:00.114 7ff6efeb6828 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:24:00.114 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:24:00.114 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:24:00.114 7ff6efeb6828 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:24:00.115 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:24:00.115 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:24:00.115 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:24:00.115 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:24:00.115 7ff6efeb6828 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:24:00.115 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:24:00.115 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:24:00.115 7ff6efeb6828 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:24:00.115 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:24:04.118 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:24:04.575 7ff6f01f6390 [contrail-vrouter-nodemgr] [DEBUG]: send sync_uve in the [ObjectVRouter:NodeStatusUVE] map 03/31/2025 02:24:09.123 7ff6efeb6828 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:24:09.123 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:24:09.123 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:24:09.123 7ff6efeb6828 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:24:09.123 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:24:09.124 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:24:09.124 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:24:09.124 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:24:09.124 7ff6efeb6828 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:24:09.124 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:24:09.124 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:24:09.124 7ff6efeb6828 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:24:09.124 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:24:13.127 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:24:14.149 7ff6eeb1aa20 [contrail-vrouter-nodemgr] [DEBUG]: send sync_uve in the [ObjectVRouter:NodeStatusUVE] map 03/31/2025 02:24:18.131 7ff6efeb6828 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:24:18.131 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:24:18.131 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:24:18.131 7ff6efeb6828 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:24:18.131 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:24:18.132 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:24:18.132 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:24:18.132 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:24:18.132 7ff6efeb6828 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:24:18.132 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:24:18.132 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:24:18.133 7ff6efeb6828 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:24:18.133 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:24:22.133 7ff6f3706f60 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:24:27.139 7ff6f3706f60 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:24:27.140 7ff6f3706f60 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:24:27.140 7ff6f3706f60 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:24:27.140 7ff6f3706f60 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:24:27.140 7ff6f3706f60 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:24:27.140 7ff6f3706f60 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:24:27.140 7ff6f3706f60 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:24:27.141 7ff6f3706f60 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:24:27.141 7ff6f3706f60 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:24:27.141 7ff6f3706f60 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:24:27.141 7ff6f3706f60 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:24:27.141 7ff6f3706f60 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:24:27.141 7ff6f3706f60 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:24:31.143 7ff6f3706f60 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:24:36.145 7ff6f3706f60 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:24:36.146 7ff6f3706f60 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:24:36.146 7ff6f3706f60 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:24:36.146 7ff6f3706f60 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:24:36.146 7ff6f3706f60 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:24:36.146 7ff6f3706f60 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:24:36.146 7ff6f3706f60 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:24:36.146 7ff6f3706f60 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:24:36.147 7ff6f3706f60 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:24:36.147 7ff6f3706f60 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:24:36.147 7ff6f3706f60 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:24:36.147 7ff6f3706f60 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:24:36.147 7ff6f3706f60 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:24:40.149 7ff6f3706f60 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:24:45.154 7ff6f3706f60 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:24:45.154 7ff6f3706f60 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:24:45.154 7ff6f3706f60 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:24:45.154 7ff6f3706f60 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:24:45.154 7ff6f3706f60 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:24:45.154 7ff6f3706f60 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:24:45.155 7ff6f3706f60 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:24:45.155 7ff6f3706f60 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:24:45.155 7ff6f3706f60 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:24:45.155 7ff6f3706f60 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:24:45.155 7ff6f3706f60 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:24:45.155 7ff6f3706f60 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:24:45.155 7ff6f3706f60 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:24:49.156 7ff6ee9a7198 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:24:51.324 7ff6f2605648 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-494-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=50897636, partition_space_available_1k=30192848, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=116420, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50552, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=14687160, free=9425660, buffers=843244, cached=7709048, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=4.75, five_min_avg=4.28, fifteen_min_avg=3.41, cpu_share=4.38, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 03/31/2025 02:24:54.161 7ff6ee9a7198 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:24:54.161 7ff6ee9a7198 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:24:54.161 7ff6ee9a7198 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:24:54.161 7ff6ee9a7198 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:24:54.161 7ff6ee9a7198 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:24:54.161 7ff6ee9a7198 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:24:54.161 7ff6ee9a7198 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 50897636 partition_space_available_1k = 30192848 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-vrouter-agent : << mem_virt = 116420 cpu_share = 0 mem_res = 0 >>contrail-vrouter-nodemgr : << mem_virt = 50552 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665112 used = 14687160 free = 9425660 buffers = 843244 cached = 7709048 node_type = vrouter >> system_cpu_usage = << one_min_avg = 4.75 five_min_avg = 4.28 fifteen_min_avg = 3.41 cpu_share = 4.38 node_type = vrouter >> >> 03/31/2025 02:24:54.161 7ff6ee9a7198 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:24:54.162 7ff6ee9a7198 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:24:54.162 7ff6ee9a7198 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:24:54.162 7ff6ee9a7198 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:24:54.162 7ff6ee9a7198 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:24:54.162 7ff6ee9a7198 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:24:54.162 7ff6ee9a7198 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:24:54.162 7ff6ee9a7198 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:24:54.163 7ff6ee9a7198 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:24:58.165 7ff6ee9a7198 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:25:03.170 7ff6ee9a7198 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:25:03.171 7ff6ee9a7198 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:25:03.171 7ff6ee9a7198 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:25:03.171 7ff6ee9a7198 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:25:03.171 7ff6ee9a7198 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:25:03.171 7ff6ee9a7198 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:25:03.171 7ff6ee9a7198 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:25:03.171 7ff6ee9a7198 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:25:03.172 7ff6ee9a7198 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:25:03.172 7ff6ee9a7198 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:25:03.172 7ff6ee9a7198 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:25:03.172 7ff6ee9a7198 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:25:03.172 7ff6ee9a7198 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:25:07.171 7ff6f20ab2e8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:25:12.177 7ff6f20ab2e8 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:25:12.177 7ff6f20ab2e8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:25:12.177 7ff6f20ab2e8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:25:12.177 7ff6f20ab2e8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:25:12.177 7ff6f20ab2e8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:25:12.177 7ff6f20ab2e8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:25:12.178 7ff6f20ab2e8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:25:12.178 7ff6f20ab2e8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:25:12.178 7ff6f20ab2e8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:25:12.178 7ff6f20ab2e8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:25:12.178 7ff6f20ab2e8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:25:12.178 7ff6f20ab2e8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:25:12.178 7ff6f20ab2e8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:25:16.181 7ff6f20ab2e8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:25:21.186 7ff6f20ab2e8 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:25:21.186 7ff6f20ab2e8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:25:21.186 7ff6f20ab2e8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:25:21.186 7ff6f20ab2e8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:25:21.187 7ff6f20ab2e8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:25:21.187 7ff6f20ab2e8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:25:21.187 7ff6f20ab2e8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:25:21.187 7ff6f20ab2e8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:25:21.187 7ff6f20ab2e8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:25:21.187 7ff6f20ab2e8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:25:21.188 7ff6f20ab2e8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:25:21.188 7ff6f20ab2e8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:25:21.188 7ff6f20ab2e8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:25:25.190 7ff6f01f66d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:25:30.196 7ff6f01f66d8 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:25:30.196 7ff6f01f66d8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:25:30.196 7ff6f01f66d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:25:30.196 7ff6f01f66d8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:25:30.196 7ff6f01f66d8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:25:30.197 7ff6f01f66d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:25:30.197 7ff6f01f66d8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:25:30.197 7ff6f01f66d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:25:30.197 7ff6f01f66d8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:25:30.197 7ff6f01f66d8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:25:30.197 7ff6f01f66d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:25:30.197 7ff6f01f66d8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:25:30.198 7ff6f01f66d8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:25:34.201 7ff6f01f66d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:25:39.205 7ff6f01f66d8 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:25:39.205 7ff6f01f66d8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:25:39.205 7ff6f01f66d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:25:39.205 7ff6f01f66d8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:25:39.205 7ff6f01f66d8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:25:39.205 7ff6f01f66d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:25:39.206 7ff6f01f66d8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:25:39.206 7ff6f01f66d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:25:39.206 7ff6f01f66d8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:25:39.206 7ff6f01f66d8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:25:39.206 7ff6f01f66d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:25:39.206 7ff6f01f66d8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:25:39.206 7ff6f01f66d8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:25:43.209 7ff6ee99e8d0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:25:48.215 7ff6ee99e8d0 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:25:48.216 7ff6ee99e8d0 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:25:48.216 7ff6ee99e8d0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:25:48.216 7ff6ee99e8d0 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:25:48.216 7ff6ee99e8d0 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:25:48.216 7ff6ee99e8d0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:25:48.216 7ff6ee99e8d0 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:25:48.216 7ff6ee99e8d0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:25:48.217 7ff6ee99e8d0 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:25:48.217 7ff6ee99e8d0 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:25:48.217 7ff6ee99e8d0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:25:48.217 7ff6ee99e8d0 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:25:48.217 7ff6ee99e8d0 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:25:51.319 7ff6f2605648 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-494-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=50901240, partition_space_available_1k=30189244, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=116540, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50536, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=14732680, free=9375164, buffers=843244, cached=7714024, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=3.52, five_min_avg=3.96, fifteen_min_avg=3.35, cpu_share=2.62, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 03/31/2025 02:25:51.319 7ff6efe0fb70 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:25:51.320 7ff6efe0fb70 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 50901240 partition_space_available_1k = 30189244 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-vrouter-agent : << mem_virt = 116540 cpu_share = 0 mem_res = 0 >>contrail-vrouter-nodemgr : << mem_virt = 50536 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665112 used = 14732680 free = 9375164 buffers = 843244 cached = 7714024 node_type = vrouter >> system_cpu_usage = << one_min_avg = 3.52 five_min_avg = 3.96 fifteen_min_avg = 3.35 cpu_share = 2.62 node_type = vrouter >> >> 03/31/2025 02:25:51.320 7ff6efe0fb70 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:25:52.216 7ff6f01fef60 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:25:57.223 7ff6f01fef60 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:25:57.223 7ff6f01fef60 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:25:57.223 7ff6f01fef60 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:25:57.223 7ff6f01fef60 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:25:57.223 7ff6f01fef60 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:25:57.224 7ff6f01fef60 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:25:57.224 7ff6f01fef60 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:25:57.224 7ff6f01fef60 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:25:57.224 7ff6f01fef60 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:25:57.224 7ff6f01fef60 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:25:57.224 7ff6f01fef60 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:25:57.224 7ff6f01fef60 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:25:57.225 7ff6f01fef60 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:26:01.225 7ff6eeb1a048 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:26:06.231 7ff6eeb1a048 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:26:06.231 7ff6eeb1a048 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:26:06.231 7ff6eeb1a048 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:26:06.231 7ff6eeb1a048 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:26:06.231 7ff6eeb1a048 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:26:06.231 7ff6eeb1a048 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:26:06.232 7ff6eeb1a048 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:26:06.232 7ff6eeb1a048 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:26:06.232 7ff6eeb1a048 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:26:06.232 7ff6eeb1a048 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:26:06.232 7ff6eeb1a048 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:26:06.232 7ff6eeb1a048 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:26:06.232 7ff6eeb1a048 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:26:10.234 7ff6f01f1e10 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:26:15.239 7ff6f01f1e10 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:26:15.240 7ff6f01f1e10 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:26:15.240 7ff6f01f1e10 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:26:15.240 7ff6f01f1e10 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:26:15.240 7ff6f01f1e10 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:26:15.240 7ff6f01f1e10 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:26:15.241 7ff6f01f1e10 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:26:15.241 7ff6f01f1e10 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:26:15.241 7ff6f01f1e10 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:26:15.241 7ff6f01f1e10 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:26:15.241 7ff6f01f1e10 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:26:15.241 7ff6f01f1e10 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:26:15.241 7ff6f01f1e10 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:26:19.241 7ff6f01f1e10 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:26:24.243 7ff6f01f1e10 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:26:24.243 7ff6f01f1e10 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:26:24.244 7ff6f01f1e10 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:26:24.244 7ff6f01f1e10 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:26:24.244 7ff6f01f1e10 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:26:24.244 7ff6f01f1e10 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:26:24.244 7ff6f01f1e10 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:26:24.244 7ff6f01f1e10 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:26:24.244 7ff6f01f1e10 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:26:24.245 7ff6f01f1e10 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:26:24.245 7ff6f01f1e10 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:26:24.245 7ff6f01f1e10 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:26:24.245 7ff6f01f1e10 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:26:28.245 7ff6f01f1e10 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:26:33.250 7ff6f01f1e10 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:26:33.251 7ff6f01f1e10 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:26:33.251 7ff6f01f1e10 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:26:33.251 7ff6f01f1e10 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:26:33.251 7ff6f01f1e10 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:26:33.251 7ff6f01f1e10 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:26:33.251 7ff6f01f1e10 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:26:33.252 7ff6f01f1e10 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:26:33.252 7ff6f01f1e10 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:26:33.252 7ff6f01f1e10 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:26:33.252 7ff6f01f1e10 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:26:33.252 7ff6f01f1e10 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:26:33.252 7ff6f01f1e10 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:26:37.251 7ff6f01f6780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:26:42.257 7ff6f01f6780 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:26:42.257 7ff6f01f6780 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:26:42.257 7ff6f01f6780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:26:42.257 7ff6f01f6780 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:26:42.257 7ff6f01f6780 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:26:42.257 7ff6f01f6780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:26:42.258 7ff6f01f6780 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:26:42.258 7ff6f01f6780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:26:42.258 7ff6f01f6780 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:26:42.258 7ff6f01f6780 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:26:42.258 7ff6f01f6780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:26:42.258 7ff6f01f6780 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:26:42.258 7ff6f01f6780 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:26:46.261 7ff6f01f6780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:26:51.264 7ff6f01f6780 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:26:51.265 7ff6f01f6780 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:26:51.265 7ff6f01f6780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:26:51.265 7ff6f01f6780 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:26:51.265 7ff6f01f6780 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:26:51.265 7ff6f01f6780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:26:51.266 7ff6f01f6780 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:26:51.266 7ff6f01f6780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:26:51.266 7ff6f01f6780 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:26:51.266 7ff6f01f6780 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:26:51.266 7ff6f01f6780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:26:51.266 7ff6f01f6780 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:26:51.266 7ff6f01f6780 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:26:51.320 7ff6f2605648 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-494-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=50905888, partition_space_available_1k=30184596, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=116680, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50560, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=14682168, free=9485636, buffers=843248, cached=7654060, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=2.65, five_min_avg=3.65, fifteen_min_avg=3.29, cpu_share=2.75, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 03/31/2025 02:26:51.321 7ff6eeb44048 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:26:51.321 7ff6eeb44048 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 50905888 partition_space_available_1k = 30184596 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-vrouter-agent : << mem_virt = 116680 cpu_share = 0 mem_res = 0 >>contrail-vrouter-nodemgr : << mem_virt = 50560 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665112 used = 14682168 free = 9485636 buffers = 843248 cached = 7654060 node_type = vrouter >> system_cpu_usage = << one_min_avg = 2.65 five_min_avg = 3.65 fifteen_min_avg = 3.29 cpu_share = 2.75 node_type = vrouter >> >> 03/31/2025 02:26:51.321 7ff6eeb44048 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:26:55.267 7ff6f2fa6630 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:27:00.273 7ff6f2fa6630 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:27:00.273 7ff6f2fa6630 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:27:00.274 7ff6f2fa6630 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:27:00.274 7ff6f2fa6630 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:27:00.274 7ff6f2fa6630 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:27:00.274 7ff6f2fa6630 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:27:00.274 7ff6f2fa6630 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:27:00.275 7ff6f2fa6630 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:27:00.275 7ff6f2fa6630 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:27:00.275 7ff6f2fa6630 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:27:00.275 7ff6f2fa6630 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:27:00.275 7ff6f2fa6630 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:27:00.275 7ff6f2fa6630 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:27:04.277 7ff6f2fa6630 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:27:09.280 7ff6f2fa6630 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:27:09.281 7ff6f2fa6630 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:27:09.281 7ff6f2fa6630 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:27:09.281 7ff6f2fa6630 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:27:09.281 7ff6f2fa6630 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:27:09.281 7ff6f2fa6630 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:27:09.281 7ff6f2fa6630 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:27:09.282 7ff6f2fa6630 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:27:09.282 7ff6f2fa6630 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:27:09.282 7ff6f2fa6630 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:27:09.282 7ff6f2fa6630 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:27:09.282 7ff6f2fa6630 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:27:09.282 7ff6f2fa6630 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:27:13.285 7ff6f2fa6630 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:27:18.290 7ff6f2fa6630 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:27:18.290 7ff6f2fa6630 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:27:18.290 7ff6f2fa6630 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:27:18.291 7ff6f2fa6630 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:27:18.291 7ff6f2fa6630 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:27:18.291 7ff6f2fa6630 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:27:18.291 7ff6f2fa6630 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:27:18.291 7ff6f2fa6630 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:27:18.291 7ff6f2fa6630 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:27:18.291 7ff6f2fa6630 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:27:18.292 7ff6f2fa6630 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:27:18.292 7ff6f2fa6630 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:27:18.292 7ff6f2fa6630 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:27:22.295 7ff6f2fa6630 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:27:27.301 7ff6f2fa6630 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:27:27.301 7ff6f2fa6630 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:27:27.301 7ff6f2fa6630 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:27:27.301 7ff6f2fa6630 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:27:27.301 7ff6f2fa6630 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:27:27.301 7ff6f2fa6630 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:27:27.302 7ff6f2fa6630 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:27:27.302 7ff6f2fa6630 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:27:27.302 7ff6f2fa6630 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:27:27.302 7ff6f2fa6630 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:27:27.302 7ff6f2fa6630 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:27:27.302 7ff6f2fa6630 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:27:27.302 7ff6f2fa6630 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:27:31.304 7ff6f2fa6630 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:27:36.310 7ff6f2fa6630 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:27:36.310 7ff6f2fa6630 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:27:36.311 7ff6f2fa6630 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:27:36.311 7ff6f2fa6630 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:27:36.311 7ff6f2fa6630 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:27:36.311 7ff6f2fa6630 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:27:36.311 7ff6f2fa6630 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:27:36.311 7ff6f2fa6630 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:27:36.311 7ff6f2fa6630 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:27:36.311 7ff6f2fa6630 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:27:36.312 7ff6f2fa6630 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:27:36.312 7ff6f2fa6630 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:27:36.312 7ff6f2fa6630 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:27:40.313 7ff6f23d0780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:27:45.317 7ff6f23d0780 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:27:45.318 7ff6f23d0780 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:27:45.318 7ff6f23d0780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:27:45.318 7ff6f23d0780 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:27:45.318 7ff6f23d0780 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:27:45.318 7ff6f23d0780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:27:45.318 7ff6f23d0780 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:27:45.319 7ff6f23d0780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:27:45.319 7ff6f23d0780 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:27:45.319 7ff6f23d0780 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:27:45.319 7ff6f23d0780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:27:45.319 7ff6f23d0780 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:27:45.319 7ff6f23d0780 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:27:49.318 7ff6ee9a7cc0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:27:51.324 7ff6f2605648 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-494-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=50909292, partition_space_available_1k=30181192, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=116876, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50616, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=14674020, free=9489752, buffers=843248, cached=7658092, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=2.64, five_min_avg=3.43, fifteen_min_avg=3.23, cpu_share=2.62, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 03/31/2025 02:27:54.321 7ff6ee9a7cc0 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:27:54.321 7ff6ee9a7cc0 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:27:54.321 7ff6ee9a7cc0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:27:54.321 7ff6ee9a7cc0 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:27:54.322 7ff6ee9a7cc0 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:27:54.322 7ff6ee9a7cc0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:27:54.322 7ff6ee9a7cc0 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 50909292 partition_space_available_1k = 30181192 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-vrouter-agent : << mem_virt = 116876 cpu_share = 0 mem_res = 0 >>contrail-vrouter-nodemgr : << mem_virt = 50616 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665112 used = 14674020 free = 9489752 buffers = 843248 cached = 7658092 node_type = vrouter >> system_cpu_usage = << one_min_avg = 2.64 five_min_avg = 3.43 fifteen_min_avg = 3.23 cpu_share = 2.62 node_type = vrouter >> >> 03/31/2025 02:27:54.322 7ff6ee9a7cc0 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:27:54.322 7ff6ee9a7cc0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:27:54.322 7ff6ee9a7cc0 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:27:54.322 7ff6ee9a7cc0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:27:54.323 7ff6ee9a7cc0 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:27:54.323 7ff6ee9a7cc0 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:27:54.323 7ff6ee9a7cc0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:27:54.323 7ff6ee9a7cc0 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:27:54.323 7ff6ee9a7cc0 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:27:58.325 7ff6f01fe978 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:28:02.861 7ff6efe0b588 [contrail-vrouter-nodemgr] [DEBUG]: send sync_uve in the [ObjectVRouter:NodeStatusUVE] map 03/31/2025 02:28:03.327 7ff6f01fe978 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:28:03.328 7ff6f01fe978 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:28:03.328 7ff6f01fe978 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:28:03.328 7ff6f01fe978 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:28:03.328 7ff6f01fe978 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:28:03.328 7ff6f01fe978 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:28:03.329 7ff6f01fe978 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:28:03.329 7ff6f01fe978 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:28:03.329 7ff6f01fe978 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:28:03.329 7ff6f01fe978 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:28:03.329 7ff6f01fe978 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:28:03.329 7ff6f01fe978 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:28:03.329 7ff6f01fe978 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:28:07.332 7ff6f01fe978 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:28:11.364 7ff6f3706f60 [contrail-vrouter-nodemgr] [DEBUG]: send sync_uve in the [ObjectVRouter:NodeStatusUVE] map 03/31/2025 02:28:12.335 7ff6f01fe978 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:28:12.335 7ff6f01fe978 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:28:12.335 7ff6f01fe978 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:28:12.335 7ff6f01fe978 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:28:12.335 7ff6f01fe978 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:28:12.336 7ff6f01fe978 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:28:12.336 7ff6f01fe978 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:28:12.336 7ff6f01fe978 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:28:12.336 7ff6f01fe978 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:28:12.337 7ff6f01fe978 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:28:12.337 7ff6f01fe978 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:28:12.337 7ff6f01fe978 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:28:12.337 7ff6f01fe978 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:28:16.339 7ff6f01fe978 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:28:21.345 7ff6f01fe978 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:28:21.345 7ff6f01fe978 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:28:21.345 7ff6f01fe978 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:28:21.346 7ff6f01fe978 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:28:21.346 7ff6f01fe978 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:28:21.346 7ff6f01fe978 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:28:21.346 7ff6f01fe978 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:28:21.346 7ff6f01fe978 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:28:21.346 7ff6f01fe978 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:28:21.346 7ff6f01fe978 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:28:21.346 7ff6f01fe978 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:28:21.347 7ff6f01fe978 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:28:21.347 7ff6f01fe978 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:28:22.468 7ff6f01fe978 [contrail-vrouter-nodemgr] [DEBUG]: send sync_uve in the [ObjectVRouter:NodeStatusUVE] map 03/31/2025 02:28:25.348 7ff6eeb2ec18 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:28:30.354 7ff6eeb2ec18 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:28:30.355 7ff6eeb2ec18 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:28:30.355 7ff6eeb2ec18 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:28:30.355 7ff6eeb2ec18 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:28:30.355 7ff6eeb2ec18 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:28:30.355 7ff6eeb2ec18 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:28:30.356 7ff6eeb2ec18 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:28:30.356 7ff6eeb2ec18 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:28:30.356 7ff6eeb2ec18 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:28:30.356 7ff6eeb2ec18 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:28:30.356 7ff6eeb2ec18 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:28:30.356 7ff6eeb2ec18 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:28:30.356 7ff6eeb2ec18 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:28:32.030 7ff6eeb2ec18 [contrail-vrouter-nodemgr] [DEBUG]: send sync_uve in the [ObjectVRouter:NodeStatusUVE] map 03/31/2025 02:28:34.357 7ff6eeb2ec18 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:28:39.359 7ff6eeb2ec18 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:28:39.359 7ff6eeb2ec18 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:28:39.360 7ff6eeb2ec18 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:28:39.360 7ff6eeb2ec18 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:28:39.360 7ff6eeb2ec18 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:28:39.360 7ff6eeb2ec18 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:28:39.360 7ff6eeb2ec18 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:28:39.360 7ff6eeb2ec18 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:28:39.360 7ff6eeb2ec18 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:28:39.360 7ff6eeb2ec18 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:28:39.361 7ff6eeb2ec18 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:28:39.361 7ff6eeb2ec18 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:28:39.361 7ff6eeb2ec18 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:28:43.364 7ff6eeb1ab70 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:28:48.366 7ff6eeb1ab70 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:28:48.366 7ff6eeb1ab70 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:28:48.366 7ff6eeb1ab70 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:28:48.367 7ff6eeb1ab70 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:28:48.367 7ff6eeb1ab70 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:28:48.367 7ff6eeb1ab70 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:28:48.367 7ff6eeb1ab70 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:28:48.367 7ff6eeb1ab70 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:28:48.367 7ff6eeb1ab70 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:28:48.368 7ff6eeb1ab70 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:28:48.368 7ff6eeb1ab70 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:28:48.368 7ff6eeb1ab70 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:28:48.368 7ff6eeb1ab70 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:28:51.329 7ff6f2605648 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-494-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=50913256, partition_space_available_1k=30177228, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=117100, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50604, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=14685128, free=9473796, buffers=843248, cached=7662940, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=2.02, five_min_avg=3.13, fifteen_min_avg=3.14, cpu_share=3.12, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 03/31/2025 02:28:51.330 7ff6f1f4fa20 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:28:51.330 7ff6f1f4fa20 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 50913256 partition_space_available_1k = 30177228 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-vrouter-agent : << mem_virt = 117100 cpu_share = 0 mem_res = 0 >>contrail-vrouter-nodemgr : << mem_virt = 50604 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665112 used = 14685128 free = 9473796 buffers = 843248 cached = 7662940 node_type = vrouter >> system_cpu_usage = << one_min_avg = 2.02 five_min_avg = 3.13 fifteen_min_avg = 3.14 cpu_share = 3.12 node_type = vrouter >> >> 03/31/2025 02:28:51.330 7ff6f1f4fa20 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:28:52.367 7ff6f1f4fa20 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:28:57.372 7ff6f1f4fa20 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:28:57.372 7ff6f1f4fa20 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:28:57.372 7ff6f1f4fa20 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:28:57.372 7ff6f1f4fa20 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:28:57.372 7ff6f1f4fa20 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:28:57.372 7ff6f1f4fa20 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:28:57.373 7ff6f1f4fa20 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:28:57.373 7ff6f1f4fa20 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:28:57.373 7ff6f1f4fa20 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:28:57.373 7ff6f1f4fa20 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:28:57.373 7ff6f1f4fa20 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:28:57.373 7ff6f1f4fa20 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:28:57.373 7ff6f1f4fa20 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:29:01.375 7ff6f1f4fa20 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:29:06.379 7ff6f1f4fa20 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:29:06.379 7ff6f1f4fa20 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:29:06.379 7ff6f1f4fa20 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:29:06.380 7ff6f1f4fa20 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:29:06.380 7ff6f1f4fa20 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:29:06.380 7ff6f1f4fa20 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:29:06.380 7ff6f1f4fa20 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:29:06.380 7ff6f1f4fa20 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:29:06.380 7ff6f1f4fa20 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:29:06.380 7ff6f1f4fa20 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:29:06.381 7ff6f1f4fa20 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:29:06.381 7ff6f1f4fa20 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:29:06.381 7ff6f1f4fa20 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:29:10.381 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:29:15.385 7ff6efecbeb8 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:29:15.386 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:29:15.386 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:29:15.386 7ff6efecbeb8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:29:15.386 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:29:15.386 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:29:15.386 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:29:15.386 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:29:15.387 7ff6efecbeb8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:29:15.387 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:29:15.387 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:29:15.387 7ff6efecbeb8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:29:15.387 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:29:19.390 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:29:24.391 7ff6efecbeb8 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:29:24.392 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:29:24.392 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:29:24.392 7ff6efecbeb8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:29:24.392 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:29:24.392 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:29:24.392 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:29:24.393 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:29:24.393 7ff6efecbeb8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:29:24.393 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:29:24.393 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:29:24.393 7ff6efecbeb8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:29:24.393 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:29:28.396 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:29:33.403 7ff6efecbeb8 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:29:33.403 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:29:33.403 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:29:33.403 7ff6efecbeb8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:29:33.403 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:29:33.403 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:29:33.404 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:29:33.404 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:29:33.404 7ff6efecbeb8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:29:33.404 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:29:33.404 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:29:33.404 7ff6efecbeb8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:29:33.404 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:29:37.406 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:29:42.411 7ff6efecbeb8 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:29:42.411 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:29:42.411 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:29:42.411 7ff6efecbeb8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:29:42.412 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:29:42.412 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:29:42.412 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:29:42.412 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:29:42.412 7ff6efecbeb8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:29:42.412 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:29:42.412 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:29:42.413 7ff6efecbeb8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:29:42.413 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:29:46.414 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:29:51.424 7ff6efecbeb8 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:29:51.425 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:29:51.425 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:29:51.425 7ff6efecbeb8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:29:51.425 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:29:51.426 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:29:51.426 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:29:51.426 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:29:51.427 7ff6efecbeb8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:29:51.427 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:29:51.427 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:29:51.427 7ff6efecbeb8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:29:51.427 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:29:51.442 7ff6f2605648 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-494-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=50917700, partition_space_available_1k=30172784, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=117272, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50640, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=15085216, free=9060100, buffers=843248, cached=7676548, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=3.37, five_min_avg=3.2, fifteen_min_avg=3.16, cpu_share=3.38, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 03/31/2025 02:29:51.444 7ff6f3706f60 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:29:51.444 7ff6f3706f60 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 50917700 partition_space_available_1k = 30172784 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-vrouter-agent : << mem_virt = 117272 cpu_share = 0 mem_res = 0 >>contrail-vrouter-nodemgr : << mem_virt = 50640 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665112 used = 15085216 free = 9060100 buffers = 843248 cached = 7676548 node_type = vrouter >> system_cpu_usage = << one_min_avg = 3.37 five_min_avg = 3.2 fifteen_min_avg = 3.16 cpu_share = 3.38 node_type = vrouter >> >> 03/31/2025 02:29:51.445 7ff6f3706f60 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:29:55.436 7ff6f01f6780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:30:00.442 7ff6f01f6780 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:30:00.442 7ff6f01f6780 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:30:00.442 7ff6f01f6780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:30:00.442 7ff6f01f6780 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:30:00.442 7ff6f01f6780 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:30:00.442 7ff6f01f6780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:30:00.443 7ff6f01f6780 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:30:00.443 7ff6f01f6780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:30:00.443 7ff6f01f6780 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:30:00.443 7ff6f01f6780 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:30:00.443 7ff6f01f6780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:30:00.443 7ff6f01f6780 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:30:00.443 7ff6f01f6780 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:30:04.446 7ff6f01f6780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:30:09.448 7ff6f01f6780 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:30:09.448 7ff6f01f6780 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:30:09.448 7ff6f01f6780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:30:09.448 7ff6f01f6780 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:30:09.449 7ff6f01f6780 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:30:09.449 7ff6f01f6780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:30:09.449 7ff6f01f6780 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:30:09.449 7ff6f01f6780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:30:09.449 7ff6f01f6780 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:30:09.450 7ff6f01f6780 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:30:09.450 7ff6f01f6780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:30:09.450 7ff6f01f6780 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:30:09.450 7ff6f01f6780 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:30:13.452 7ff6eddfa6d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:30:18.457 7ff6eddfa6d8 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:30:18.457 7ff6eddfa6d8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:30:18.457 7ff6eddfa6d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:30:18.457 7ff6eddfa6d8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:30:18.457 7ff6eddfa6d8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:30:18.457 7ff6eddfa6d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:30:18.458 7ff6eddfa6d8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:30:18.458 7ff6eddfa6d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:30:18.458 7ff6eddfa6d8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:30:18.458 7ff6eddfa6d8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:30:18.458 7ff6eddfa6d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:30:18.458 7ff6eddfa6d8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:30:18.458 7ff6eddfa6d8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:30:22.461 7ff6eddfa6d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:30:27.466 7ff6eddfa6d8 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:30:27.466 7ff6eddfa6d8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:30:27.466 7ff6eddfa6d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:30:27.467 7ff6eddfa6d8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:30:27.467 7ff6eddfa6d8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:30:27.467 7ff6eddfa6d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:30:27.467 7ff6eddfa6d8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:30:27.468 7ff6eddfa6d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:30:27.468 7ff6eddfa6d8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:30:27.468 7ff6eddfa6d8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:30:27.468 7ff6eddfa6d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:30:27.468 7ff6eddfa6d8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:30:27.468 7ff6eddfa6d8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:30:31.470 7ff6eddfa6d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:30:36.476 7ff6eddfa6d8 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:30:36.477 7ff6eddfa6d8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:30:36.477 7ff6eddfa6d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:30:36.477 7ff6eddfa6d8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:30:36.477 7ff6eddfa6d8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:30:36.477 7ff6eddfa6d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:30:36.477 7ff6eddfa6d8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:30:36.477 7ff6eddfa6d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:30:36.478 7ff6eddfa6d8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:30:36.478 7ff6eddfa6d8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:30:36.478 7ff6eddfa6d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:30:36.478 7ff6eddfa6d8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:30:36.478 7ff6eddfa6d8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:30:40.477 7ff6eeb388d0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:30:45.484 7ff6eeb388d0 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:30:45.484 7ff6eeb388d0 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:30:45.484 7ff6eeb388d0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:30:45.484 7ff6eeb388d0 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:30:45.484 7ff6eeb388d0 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:30:45.484 7ff6eeb388d0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:30:45.485 7ff6eeb388d0 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:30:45.485 7ff6eeb388d0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:30:45.485 7ff6eeb388d0 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:30:45.485 7ff6eeb388d0 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:30:45.485 7ff6eeb388d0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:30:45.485 7ff6eeb388d0 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:30:45.485 7ff6eeb388d0 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:30:49.484 7ff6efe2df60 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:30:51.331 7ff6f2605648 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-494-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=50927532, partition_space_available_1k=30162952, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=117376, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50684, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=14704152, free=9430340, buffers=843272, cached=7687348, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=2.69, five_min_avg=3.08, fifteen_min_avg=3.12, cpu_share=3.5, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 03/31/2025 02:30:54.487 7ff6efe2df60 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:30:54.487 7ff6efe2df60 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:30:54.487 7ff6efe2df60 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:30:54.487 7ff6efe2df60 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:30:54.487 7ff6efe2df60 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:30:54.487 7ff6efe2df60 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:30:54.488 7ff6efe2df60 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 50927532 partition_space_available_1k = 30162952 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-vrouter-agent : << mem_virt = 117376 cpu_share = 0 mem_res = 0 >>contrail-vrouter-nodemgr : << mem_virt = 50684 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665112 used = 14704152 free = 9430340 buffers = 843272 cached = 7687348 node_type = vrouter >> system_cpu_usage = << one_min_avg = 2.69 five_min_avg = 3.08 fifteen_min_avg = 3.12 cpu_share = 3.5 node_type = vrouter >> >> 03/31/2025 02:30:54.488 7ff6efe2df60 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:30:54.488 7ff6efe2df60 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:30:54.488 7ff6efe2df60 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:30:54.488 7ff6efe2df60 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:30:54.489 7ff6efe2df60 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:30:54.489 7ff6efe2df60 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:30:54.489 7ff6efe2df60 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:30:54.489 7ff6efe2df60 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:30:54.489 7ff6efe2df60 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:30:58.490 7ff6f25de780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:31:03.493 7ff6f25de780 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:31:03.493 7ff6f25de780 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:31:03.493 7ff6f25de780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:31:03.493 7ff6f25de780 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:31:03.493 7ff6f25de780 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:31:03.493 7ff6f25de780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:31:03.494 7ff6f25de780 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:31:03.494 7ff6f25de780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:31:03.494 7ff6f25de780 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:31:03.494 7ff6f25de780 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:31:03.494 7ff6f25de780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:31:03.494 7ff6f25de780 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:31:03.494 7ff6f25de780 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:31:07.497 7ff6f25de780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:31:12.499 7ff6f25de780 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:31:12.499 7ff6f25de780 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:31:12.499 7ff6f25de780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:31:12.500 7ff6f25de780 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:31:12.500 7ff6f25de780 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:31:12.500 7ff6f25de780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:31:12.501 7ff6f25de780 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:31:12.501 7ff6f25de780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:31:12.501 7ff6f25de780 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:31:12.501 7ff6f25de780 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:31:12.501 7ff6f25de780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:31:12.501 7ff6f25de780 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:31:12.501 7ff6f25de780 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:31:16.502 7ff6f25de780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:31:21.507 7ff6f25de780 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:31:21.508 7ff6f25de780 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:31:21.508 7ff6f25de780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:31:21.508 7ff6f25de780 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:31:21.508 7ff6f25de780 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:31:21.508 7ff6f25de780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:31:21.508 7ff6f25de780 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:31:21.508 7ff6f25de780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:31:21.508 7ff6f25de780 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:31:21.509 7ff6f25de780 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:31:21.509 7ff6f25de780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:31:21.509 7ff6f25de780 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:31:21.509 7ff6f25de780 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:31:25.508 7ff6f25d2198 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:31:30.515 7ff6f25d2198 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:31:30.515 7ff6f25d2198 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:31:30.515 7ff6f25d2198 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:31:30.515 7ff6f25d2198 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:31:30.515 7ff6f25d2198 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:31:30.515 7ff6f25d2198 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:31:30.516 7ff6f25d2198 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:31:30.516 7ff6f25d2198 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:31:30.516 7ff6f25d2198 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:31:30.516 7ff6f25d2198 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:31:30.516 7ff6f25d2198 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:31:30.516 7ff6f25d2198 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:31:30.516 7ff6f25d2198 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:31:34.519 7ff6f25d2198 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:31:39.525 7ff6f25d2198 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:31:39.525 7ff6f25d2198 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:31:39.525 7ff6f25d2198 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:31:39.526 7ff6f25d2198 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:31:39.526 7ff6f25d2198 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:31:39.526 7ff6f25d2198 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:31:39.526 7ff6f25d2198 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:31:39.526 7ff6f25d2198 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:31:39.526 7ff6f25d2198 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:31:39.526 7ff6f25d2198 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:31:39.526 7ff6f25d2198 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:31:39.526 7ff6f25d2198 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:31:39.527 7ff6f25d2198 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:31:43.530 7ff6f020ff60 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:31:48.531 7ff6f020ff60 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:31:48.532 7ff6f020ff60 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:31:48.533 7ff6f020ff60 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:31:48.533 7ff6f020ff60 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:31:48.533 7ff6f020ff60 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:31:48.533 7ff6f020ff60 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:31:48.534 7ff6f020ff60 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:31:48.534 7ff6f020ff60 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:31:48.535 7ff6f020ff60 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:31:48.535 7ff6f020ff60 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:31:48.535 7ff6f020ff60 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:31:48.535 7ff6f020ff60 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:31:48.535 7ff6f020ff60 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:31:51.336 7ff6f2605648 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-494-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=50930876, partition_space_available_1k=30159608, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=117504, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50672, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=14709864, free=9420560, buffers=843288, cached=7691400, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=1.9, five_min_avg=2.8, fifteen_min_avg=3.02, cpu_share=2.5, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 03/31/2025 02:31:51.336 7ff6f23d0780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:31:51.336 7ff6f23d0780 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 50930876 partition_space_available_1k = 30159608 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-vrouter-agent : << mem_virt = 117504 cpu_share = 0 mem_res = 0 >>contrail-vrouter-nodemgr : << mem_virt = 50672 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665112 used = 14709864 free = 9420560 buffers = 843288 cached = 7691400 node_type = vrouter >> system_cpu_usage = << one_min_avg = 1.9 five_min_avg = 2.8 fifteen_min_avg = 3.02 cpu_share = 2.5 node_type = vrouter >> >> 03/31/2025 02:31:51.337 7ff6f23d0780 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:31:52.533 7ff6f23d0780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:31:57.536 7ff6f23d0780 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:31:57.537 7ff6f23d0780 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:31:57.537 7ff6f23d0780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:31:57.537 7ff6f23d0780 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:31:57.537 7ff6f23d0780 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:31:57.537 7ff6f23d0780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:31:57.538 7ff6f23d0780 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:31:57.538 7ff6f23d0780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:31:57.538 7ff6f23d0780 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:31:57.538 7ff6f23d0780 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:31:57.538 7ff6f23d0780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:31:57.538 7ff6f23d0780 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:31:57.538 7ff6f23d0780 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:32:01.540 7ff6f23d0780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:32:06.545 7ff6f23d0780 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:32:06.545 7ff6f23d0780 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:32:06.545 7ff6f23d0780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:32:06.545 7ff6f23d0780 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:32:06.545 7ff6f23d0780 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:32:06.546 7ff6f23d0780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:32:06.546 7ff6f23d0780 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:32:06.546 7ff6f23d0780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:32:06.546 7ff6f23d0780 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:32:06.546 7ff6f23d0780 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:32:06.546 7ff6f23d0780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:32:06.546 7ff6f23d0780 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:32:06.546 7ff6f23d0780 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:32:10.546 7ff6f1fe6eb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:32:15.550 7ff6f1fe6eb8 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:32:15.550 7ff6f1fe6eb8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:32:15.551 7ff6f1fe6eb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:32:15.551 7ff6f1fe6eb8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:32:15.551 7ff6f1fe6eb8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:32:15.551 7ff6f1fe6eb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:32:15.551 7ff6f1fe6eb8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:32:15.551 7ff6f1fe6eb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:32:15.552 7ff6f1fe6eb8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:32:15.552 7ff6f1fe6eb8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:32:15.552 7ff6f1fe6eb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:32:15.552 7ff6f1fe6eb8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:32:15.552 7ff6f1fe6eb8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:32:19.554 7ff6f1fe6eb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:32:24.558 7ff6f1fe6eb8 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:32:24.559 7ff6f1fe6eb8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:32:24.559 7ff6f1fe6eb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:32:24.559 7ff6f1fe6eb8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:32:24.559 7ff6f1fe6eb8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:32:24.559 7ff6f1fe6eb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:32:24.559 7ff6f1fe6eb8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:32:24.559 7ff6f1fe6eb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:32:24.560 7ff6f1fe6eb8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:32:24.560 7ff6f1fe6eb8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:32:24.560 7ff6f1fe6eb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:32:24.560 7ff6f1fe6eb8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:32:24.560 7ff6f1fe6eb8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:32:28.563 7ff6f01f1e10 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:32:33.569 7ff6f01f1e10 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:32:33.569 7ff6f01f1e10 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:32:33.569 7ff6f01f1e10 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:32:33.569 7ff6f01f1e10 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:32:33.569 7ff6f01f1e10 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:32:33.569 7ff6f01f1e10 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:32:33.570 7ff6f01f1e10 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:32:33.570 7ff6f01f1e10 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:32:33.570 7ff6f01f1e10 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:32:33.570 7ff6f01f1e10 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:32:33.570 7ff6f01f1e10 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:32:33.570 7ff6f01f1e10 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:32:33.570 7ff6f01f1e10 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:32:37.573 7ff6f01f1e10 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:32:42.576 7ff6f01f1e10 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:32:42.577 7ff6f01f1e10 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:32:42.577 7ff6f01f1e10 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:32:42.577 7ff6f01f1e10 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:32:42.577 7ff6f01f1e10 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:32:42.578 7ff6f01f1e10 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:32:42.579 7ff6f01f1e10 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:32:42.579 7ff6f01f1e10 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:32:42.579 7ff6f01f1e10 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:32:42.579 7ff6f01f1e10 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:32:42.579 7ff6f01f1e10 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:32:42.580 7ff6f01f1e10 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:32:42.580 7ff6f01f1e10 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:32:46.580 7ff6f01f1e10 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:32:51.337 7ff6f2605648 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-494-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=50934540, partition_space_available_1k=30155944, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=117628, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50728, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=14696540, free=9427496, buffers=843296, cached=7697780, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=2.21, five_min_avg=2.75, fifteen_min_avg=2.99, cpu_share=2.5, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 03/31/2025 02:32:51.581 7ff6f01f1e10 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:32:51.582 7ff6f01f1e10 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:32:51.582 7ff6f01f1e10 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:32:51.582 7ff6f01f1e10 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:32:51.582 7ff6f01f1e10 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:32:51.582 7ff6f01f1e10 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:32:51.582 7ff6f01f1e10 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 50934540 partition_space_available_1k = 30155944 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-vrouter-agent : << mem_virt = 117628 cpu_share = 0 mem_res = 0 >>contrail-vrouter-nodemgr : << mem_virt = 50728 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665112 used = 14696540 free = 9427496 buffers = 843296 cached = 7697780 node_type = vrouter >> system_cpu_usage = << one_min_avg = 2.21 five_min_avg = 2.75 fifteen_min_avg = 2.99 cpu_share = 2.5 node_type = vrouter >> >> 03/31/2025 02:32:51.583 7ff6f01f1e10 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:32:51.583 7ff6f01f1e10 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:32:51.583 7ff6f01f1e10 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:32:51.583 7ff6f01f1e10 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:32:51.583 7ff6f01f1e10 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:32:51.583 7ff6f01f1e10 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:32:51.583 7ff6f01f1e10 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:32:51.584 7ff6f01f1e10 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:32:51.584 7ff6f01f1e10 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:32:55.582 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:33:00.588 7ff6efeb6828 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:33:00.588 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:33:00.589 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:33:00.589 7ff6efeb6828 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:33:00.589 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:33:00.589 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:33:00.589 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:33:00.589 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:33:00.589 7ff6efeb6828 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:33:00.589 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:33:00.589 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:33:00.590 7ff6efeb6828 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:33:00.590 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:33:04.593 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:33:09.599 7ff6efeb6828 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:33:09.599 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:33:09.599 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:33:09.599 7ff6efeb6828 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:33:09.600 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:33:09.600 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:33:09.600 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:33:09.600 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:33:09.600 7ff6efeb6828 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:33:09.600 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:33:09.600 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:33:09.601 7ff6efeb6828 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:33:09.601 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:33:13.601 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:33:18.607 7ff6efeb6828 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:33:18.607 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:33:18.607 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:33:18.608 7ff6efeb6828 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:33:18.608 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:33:18.608 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:33:18.608 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:33:18.608 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:33:18.608 7ff6efeb6828 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:33:18.608 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:33:18.608 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:33:18.609 7ff6efeb6828 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:33:18.609 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:33:21.226 7ff6efeb6828 [contrail-vrouter-nodemgr] [DEBUG]: send sync_uve in the [ObjectVRouter:NodeStatusUVE] map 03/31/2025 02:33:22.608 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:33:27.611 7ff6efeb6828 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:33:27.612 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:33:27.612 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:33:27.612 7ff6efeb6828 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:33:27.612 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:33:27.612 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:33:27.613 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:33:27.613 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:33:27.613 7ff6efeb6828 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:33:27.613 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:33:27.613 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:33:27.613 7ff6efeb6828 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:33:27.613 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:33:31.615 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:33:35.251 7ff6efe170f0 [contrail-vrouter-nodemgr] [DEBUG]: send sync_uve in the [ObjectVRouter:NodeStatusUVE] map 03/31/2025 02:33:36.618 7ff6efeb6828 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:33:36.618 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:33:36.618 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:33:36.618 7ff6efeb6828 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:33:36.618 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:33:36.618 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:33:36.619 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:33:36.619 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:33:36.619 7ff6efeb6828 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:33:36.619 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:33:36.619 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:33:36.619 7ff6efeb6828 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:33:36.619 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:33:40.621 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:33:43.843 7ff6f20ab2e8 [contrail-vrouter-nodemgr] [DEBUG]: send sync_uve in the [ObjectVRouter:NodeStatusUVE] map 03/31/2025 02:33:45.625 7ff6efeb6828 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:33:45.625 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:33:45.625 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:33:45.625 7ff6efeb6828 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:33:45.625 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:33:45.625 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:33:45.626 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:33:45.626 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:33:45.626 7ff6efeb6828 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:33:45.626 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:33:45.626 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:33:45.626 7ff6efeb6828 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:33:45.626 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:33:49.626 7ff6eeb1a198 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:33:51.342 7ff6f2605648 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-494-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=50942816, partition_space_available_1k=30147668, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=117820, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50748, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=14712372, free=9400700, buffers=843560, cached=7708480, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=1.96, five_min_avg=2.57, fifteen_min_avg=2.91, cpu_share=2.62, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 03/31/2025 02:33:54.630 7ff6eeb1a198 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:33:54.630 7ff6eeb1a198 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:33:54.630 7ff6eeb1a198 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:33:54.631 7ff6eeb1a198 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:33:54.631 7ff6eeb1a198 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:33:54.631 7ff6eeb1a198 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:33:54.631 7ff6eeb1a198 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 50942816 partition_space_available_1k = 30147668 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-vrouter-agent : << mem_virt = 117820 cpu_share = 0 mem_res = 0 >>contrail-vrouter-nodemgr : << mem_virt = 50748 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665112 used = 14712372 free = 9400700 buffers = 843560 cached = 7708480 node_type = vrouter >> system_cpu_usage = << one_min_avg = 1.96 five_min_avg = 2.57 fifteen_min_avg = 2.91 cpu_share = 2.62 node_type = vrouter >> >> 03/31/2025 02:33:54.631 7ff6eeb1a198 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:33:54.631 7ff6eeb1a198 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:33:54.631 7ff6eeb1a198 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:33:54.632 7ff6eeb1a198 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:33:54.632 7ff6eeb1a198 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:33:54.632 7ff6eeb1a198 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:33:54.632 7ff6eeb1a198 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:33:54.632 7ff6eeb1a198 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:33:54.632 7ff6eeb1a198 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:33:54.868 7ff6eeb1a198 [contrail-vrouter-nodemgr] [DEBUG]: send sync_uve in the [ObjectVRouter:NodeStatusUVE] map 03/31/2025 02:33:58.633 7ff6efe17e10 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:34:03.639 7ff6efe17e10 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:34:03.639 7ff6efe17e10 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:34:03.640 7ff6efe17e10 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:34:03.640 7ff6efe17e10 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:34:03.640 7ff6efe17e10 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:34:03.640 7ff6efe17e10 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:34:03.641 7ff6efe17e10 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:34:03.641 7ff6efe17e10 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:34:03.641 7ff6efe17e10 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:34:03.641 7ff6efe17e10 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:34:03.641 7ff6efe17e10 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:34:03.641 7ff6efe17e10 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:34:03.641 7ff6efe17e10 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:34:07.641 7ff6efe17e10 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:34:12.643 7ff6efe17e10 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:34:12.644 7ff6efe17e10 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:34:12.644 7ff6efe17e10 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:34:12.644 7ff6efe17e10 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:34:12.644 7ff6efe17e10 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:34:12.644 7ff6efe17e10 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:34:12.645 7ff6efe17e10 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:34:12.645 7ff6efe17e10 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:34:12.645 7ff6efe17e10 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:34:12.645 7ff6efe17e10 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:34:12.645 7ff6efe17e10 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:34:12.645 7ff6efe17e10 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:34:12.645 7ff6efe17e10 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:34:16.645 7ff6eeb2e240 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:34:21.651 7ff6eeb2e240 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:34:21.651 7ff6eeb2e240 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:34:21.651 7ff6eeb2e240 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:34:21.651 7ff6eeb2e240 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:34:21.651 7ff6eeb2e240 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:34:21.652 7ff6eeb2e240 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:34:21.652 7ff6eeb2e240 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:34:21.652 7ff6eeb2e240 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:34:21.652 7ff6eeb2e240 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:34:21.652 7ff6eeb2e240 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:34:21.652 7ff6eeb2e240 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:34:21.652 7ff6eeb2e240 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:34:21.652 7ff6eeb2e240 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:34:25.655 7ff6eeb2e240 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:34:30.661 7ff6eeb2e240 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:34:30.661 7ff6eeb2e240 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:34:30.661 7ff6eeb2e240 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:34:30.661 7ff6eeb2e240 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:34:30.661 7ff6eeb2e240 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:34:30.661 7ff6eeb2e240 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:34:30.662 7ff6eeb2e240 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:34:30.662 7ff6eeb2e240 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:34:30.662 7ff6eeb2e240 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:34:30.662 7ff6eeb2e240 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:34:30.662 7ff6eeb2e240 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:34:30.662 7ff6eeb2e240 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:34:30.662 7ff6eeb2e240 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:34:34.665 7ff6efe19eb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:34:39.676 7ff6efe19eb8 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:34:39.676 7ff6efe19eb8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:34:39.676 7ff6efe19eb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:34:39.676 7ff6efe19eb8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:34:39.676 7ff6efe19eb8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:34:39.676 7ff6efe19eb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:34:39.677 7ff6efe19eb8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:34:39.677 7ff6efe19eb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:34:39.677 7ff6efe19eb8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:34:39.677 7ff6efe19eb8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:34:39.677 7ff6efe19eb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:34:39.677 7ff6efe19eb8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:34:39.677 7ff6efe19eb8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:34:43.677 7ff6f01fe4e0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:34:48.680 7ff6f01fe4e0 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:34:48.680 7ff6f01fe4e0 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:34:48.680 7ff6f01fe4e0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:34:48.681 7ff6f01fe4e0 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:34:48.681 7ff6f01fe4e0 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:34:48.681 7ff6f01fe4e0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:34:48.681 7ff6f01fe4e0 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:34:48.681 7ff6f01fe4e0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:34:48.682 7ff6f01fe4e0 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:34:48.682 7ff6f01fe4e0 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:34:48.682 7ff6f01fe4e0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:34:48.682 7ff6f01fe4e0 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:34:48.682 7ff6f01fe4e0 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:34:51.353 7ff6f2605648 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-494-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=50948172, partition_space_available_1k=30142312, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=117972, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50744, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=14721040, free=9373208, buffers=843568, cached=7727296, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=4.82, five_min_avg=3.24, fifteen_min_avg=3.12, cpu_share=4.38, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 03/31/2025 02:34:51.353 7ff6ee99eac8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:34:51.354 7ff6ee99eac8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 50948172 partition_space_available_1k = 30142312 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-vrouter-agent : << mem_virt = 117972 cpu_share = 0 mem_res = 0 >>contrail-vrouter-nodemgr : << mem_virt = 50744 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665112 used = 14721040 free = 9373208 buffers = 843568 cached = 7727296 node_type = vrouter >> system_cpu_usage = << one_min_avg = 4.82 five_min_avg = 3.24 fifteen_min_avg = 3.12 cpu_share = 4.38 node_type = vrouter >> >> 03/31/2025 02:34:51.354 7ff6ee99eac8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:34:52.681 7ff6ee99eac8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:34:57.684 7ff6ee99eac8 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:34:57.684 7ff6ee99eac8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:34:57.684 7ff6ee99eac8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:34:57.684 7ff6ee99eac8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:34:57.684 7ff6ee99eac8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:34:57.684 7ff6ee99eac8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:34:57.685 7ff6ee99eac8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:34:57.685 7ff6ee99eac8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:34:57.685 7ff6ee99eac8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:34:57.685 7ff6ee99eac8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:34:57.685 7ff6ee99eac8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:34:57.686 7ff6ee99eac8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:34:57.686 7ff6ee99eac8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:35:01.688 7ff6f1fe6eb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:35:06.693 7ff6f1fe6eb8 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:35:06.693 7ff6f1fe6eb8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:35:06.693 7ff6f1fe6eb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:35:06.693 7ff6f1fe6eb8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:35:06.693 7ff6f1fe6eb8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:35:06.693 7ff6f1fe6eb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:35:06.694 7ff6f1fe6eb8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:35:06.694 7ff6f1fe6eb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:35:06.694 7ff6f1fe6eb8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:35:06.694 7ff6f1fe6eb8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:35:06.694 7ff6f1fe6eb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:35:06.694 7ff6f1fe6eb8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:35:06.694 7ff6f1fe6eb8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:35:10.696 7ff6f1fe6eb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:35:15.702 7ff6f1fe6eb8 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:35:15.702 7ff6f1fe6eb8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:35:15.702 7ff6f1fe6eb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:35:15.702 7ff6f1fe6eb8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:35:15.702 7ff6f1fe6eb8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:35:15.702 7ff6f1fe6eb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:35:15.703 7ff6f1fe6eb8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:35:15.703 7ff6f1fe6eb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:35:15.703 7ff6f1fe6eb8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:35:15.703 7ff6f1fe6eb8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:35:15.703 7ff6f1fe6eb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:35:15.703 7ff6f1fe6eb8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:35:15.703 7ff6f1fe6eb8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:35:19.705 7ff6f1fe6eb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:35:24.711 7ff6f1fe6eb8 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:35:24.712 7ff6f1fe6eb8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:35:24.712 7ff6f1fe6eb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:35:24.712 7ff6f1fe6eb8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:35:24.712 7ff6f1fe6eb8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:35:24.712 7ff6f1fe6eb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:35:24.712 7ff6f1fe6eb8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:35:24.713 7ff6f1fe6eb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:35:24.713 7ff6f1fe6eb8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:35:24.713 7ff6f1fe6eb8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:35:24.713 7ff6f1fe6eb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:35:24.713 7ff6f1fe6eb8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:35:24.713 7ff6f1fe6eb8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:35:28.712 7ff6f25b7630 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:35:33.718 7ff6f25b7630 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:35:33.718 7ff6f25b7630 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:35:33.719 7ff6f25b7630 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:35:33.719 7ff6f25b7630 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:35:33.719 7ff6f25b7630 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:35:33.719 7ff6f25b7630 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:35:33.719 7ff6f25b7630 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:35:33.719 7ff6f25b7630 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:35:33.719 7ff6f25b7630 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:35:33.720 7ff6f25b7630 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:35:33.720 7ff6f25b7630 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:35:33.720 7ff6f25b7630 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:35:33.720 7ff6f25b7630 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:35:37.722 7ff6f25b7630 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:35:42.725 7ff6f25b7630 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:35:42.725 7ff6f25b7630 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:35:42.725 7ff6f25b7630 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:35:42.725 7ff6f25b7630 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:35:42.725 7ff6f25b7630 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:35:42.725 7ff6f25b7630 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:35:42.726 7ff6f25b7630 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:35:42.726 7ff6f25b7630 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:35:42.726 7ff6f25b7630 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:35:42.726 7ff6f25b7630 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:35:42.726 7ff6f25b7630 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:35:42.726 7ff6f25b7630 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:35:42.726 7ff6f25b7630 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:35:46.729 7ff6f25b7630 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:35:51.348 7ff6f2605648 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-494-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=50953452, partition_space_available_1k=30137032, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=118124, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50796, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=14732880, free=9335224, buffers=844576, cached=7752432, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=4.2, five_min_avg=3.31, fifteen_min_avg=3.15, cpu_share=2.5, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 03/31/2025 02:35:51.731 7ff6f25b7630 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:35:51.731 7ff6f25b7630 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:35:51.731 7ff6f25b7630 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:35:51.731 7ff6f25b7630 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:35:51.731 7ff6f25b7630 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:35:51.731 7ff6f25b7630 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:35:51.732 7ff6f25b7630 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 50953452 partition_space_available_1k = 30137032 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-vrouter-agent : << mem_virt = 118124 cpu_share = 0 mem_res = 0 >>contrail-vrouter-nodemgr : << mem_virt = 50796 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665112 used = 14732880 free = 9335224 buffers = 844576 cached = 7752432 node_type = vrouter >> system_cpu_usage = << one_min_avg = 4.2 five_min_avg = 3.31 fifteen_min_avg = 3.15 cpu_share = 2.5 node_type = vrouter >> >> 03/31/2025 02:35:51.732 7ff6f25b7630 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:35:51.732 7ff6f25b7630 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:35:51.732 7ff6f25b7630 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:35:51.732 7ff6f25b7630 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:35:51.733 7ff6f25b7630 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:35:51.733 7ff6f25b7630 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:35:51.733 7ff6f25b7630 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:35:51.733 7ff6f25b7630 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:35:51.733 7ff6f25b7630 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:35:55.734 7ff6f25b7630 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:36:00.738 7ff6f25b7630 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:36:00.738 7ff6f25b7630 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:36:00.739 7ff6f25b7630 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:36:00.739 7ff6f25b7630 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:36:00.739 7ff6f25b7630 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:36:00.739 7ff6f25b7630 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:36:00.739 7ff6f25b7630 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:36:00.739 7ff6f25b7630 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:36:00.740 7ff6f25b7630 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:36:00.740 7ff6f25b7630 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:36:00.740 7ff6f25b7630 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:36:00.740 7ff6f25b7630 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:36:00.740 7ff6f25b7630 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:36:04.743 7ff6f25b7630 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:36:09.749 7ff6f25b7630 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:36:09.749 7ff6f25b7630 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:36:09.749 7ff6f25b7630 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:36:09.750 7ff6f25b7630 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:36:09.750 7ff6f25b7630 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:36:09.750 7ff6f25b7630 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:36:09.750 7ff6f25b7630 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:36:09.750 7ff6f25b7630 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:36:09.750 7ff6f25b7630 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:36:09.750 7ff6f25b7630 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:36:09.751 7ff6f25b7630 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:36:09.751 7ff6f25b7630 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:36:09.751 7ff6f25b7630 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:36:13.751 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:36:18.757 7ff6efecbeb8 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:36:18.757 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:36:18.757 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:36:18.757 7ff6efecbeb8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:36:18.757 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:36:18.757 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:36:18.758 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:36:18.758 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:36:18.758 7ff6efecbeb8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:36:18.758 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:36:18.758 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:36:18.758 7ff6efecbeb8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:36:18.758 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:36:22.761 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:36:27.762 7ff6efecbeb8 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:36:27.763 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:36:27.763 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:36:27.763 7ff6efecbeb8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:36:27.763 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:36:27.763 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:36:27.764 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:36:27.764 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:36:27.764 7ff6efecbeb8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:36:27.764 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:36:27.764 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:36:27.764 7ff6efecbeb8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:36:27.764 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:36:31.767 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:36:36.772 7ff6efecbeb8 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:36:36.773 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:36:36.773 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:36:36.773 7ff6efecbeb8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:36:36.773 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:36:36.773 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:36:36.774 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:36:36.774 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:36:36.774 7ff6efecbeb8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:36:36.774 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:36:36.774 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:36:36.774 7ff6efecbeb8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:36:36.774 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:36:40.776 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:36:45.781 7ff6efecbeb8 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:36:45.781 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:36:45.781 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:36:45.781 7ff6efecbeb8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:36:45.781 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:36:45.781 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:36:45.782 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:36:45.782 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:36:45.782 7ff6efecbeb8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:36:45.782 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:36:45.782 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:36:45.782 7ff6efecbeb8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:36:45.782 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:36:49.782 7ff6efe19978 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:36:51.354 7ff6f2605648 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-494-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=50957816, partition_space_available_1k=30132668, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=118256, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50820, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=14750824, free=9310500, buffers=844584, cached=7759204, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=2.55, five_min_avg=3.05, fifteen_min_avg=3.07, cpu_share=2.5, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 03/31/2025 02:36:54.788 7ff6efe19978 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:36:54.788 7ff6efe19978 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:36:54.788 7ff6efe19978 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:36:54.788 7ff6efe19978 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:36:54.788 7ff6efe19978 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:36:54.788 7ff6efe19978 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:36:54.789 7ff6efe19978 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 50957816 partition_space_available_1k = 30132668 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-vrouter-agent : << mem_virt = 118256 cpu_share = 0 mem_res = 0 >>contrail-vrouter-nodemgr : << mem_virt = 50820 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665112 used = 14750824 free = 9310500 buffers = 844584 cached = 7759204 node_type = vrouter >> system_cpu_usage = << one_min_avg = 2.55 five_min_avg = 3.05 fifteen_min_avg = 3.07 cpu_share = 2.5 node_type = vrouter >> >> 03/31/2025 02:36:54.789 7ff6efe19978 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:36:54.789 7ff6efe19978 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:36:54.789 7ff6efe19978 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:36:54.789 7ff6efe19978 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:36:54.789 7ff6efe19978 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:36:54.789 7ff6efe19978 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:36:54.790 7ff6efe19978 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:36:54.790 7ff6efe19978 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:36:54.790 7ff6efe19978 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:36:58.789 7ff6eeb2eac8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:37:03.795 7ff6eeb2eac8 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:37:03.795 7ff6eeb2eac8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:37:03.796 7ff6eeb2eac8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:37:03.796 7ff6eeb2eac8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:37:03.796 7ff6eeb2eac8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:37:03.796 7ff6eeb2eac8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:37:03.796 7ff6eeb2eac8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:37:03.796 7ff6eeb2eac8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:37:03.796 7ff6eeb2eac8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:37:03.796 7ff6eeb2eac8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:37:03.797 7ff6eeb2eac8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:37:03.797 7ff6eeb2eac8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:37:03.797 7ff6eeb2eac8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:37:07.800 7ff6eeb2eac8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:37:12.805 7ff6eeb2eac8 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:37:12.805 7ff6eeb2eac8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:37:12.805 7ff6eeb2eac8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:37:12.805 7ff6eeb2eac8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:37:12.805 7ff6eeb2eac8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:37:12.805 7ff6eeb2eac8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:37:12.806 7ff6eeb2eac8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:37:12.806 7ff6eeb2eac8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:37:12.806 7ff6eeb2eac8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:37:12.806 7ff6eeb2eac8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:37:12.806 7ff6eeb2eac8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:37:12.806 7ff6eeb2eac8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:37:12.806 7ff6eeb2eac8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:37:16.809 7ff6eeb2eac8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:37:21.814 7ff6eeb2eac8 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:37:21.814 7ff6eeb2eac8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:37:21.814 7ff6eeb2eac8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:37:21.814 7ff6eeb2eac8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:37:21.814 7ff6eeb2eac8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:37:21.814 7ff6eeb2eac8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:37:21.815 7ff6eeb2eac8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:37:21.815 7ff6eeb2eac8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:37:21.815 7ff6eeb2eac8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:37:21.815 7ff6eeb2eac8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:37:21.815 7ff6eeb2eac8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:37:21.815 7ff6eeb2eac8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:37:21.815 7ff6eeb2eac8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:37:25.817 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:37:30.821 7ff6efecbeb8 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:37:30.821 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:37:30.821 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:37:30.821 7ff6efecbeb8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:37:30.821 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:37:30.821 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:37:30.822 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:37:30.822 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:37:30.822 7ff6efecbeb8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:37:30.822 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:37:30.822 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:37:30.822 7ff6efecbeb8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:37:30.822 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:37:34.825 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:37:39.832 7ff6efecbeb8 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:37:39.832 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:37:39.832 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:37:39.832 7ff6efecbeb8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:37:39.832 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:37:39.832 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:37:39.833 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:37:39.833 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:37:39.833 7ff6efecbeb8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:37:39.833 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:37:39.833 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:37:39.833 7ff6efecbeb8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:37:39.833 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:37:43.832 7ff6efe0f2e8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:37:43.907 7ff6ee99e0f0 [contrail-vrouter-nodemgr] [DEBUG]: send sync_uve in the [ObjectVRouter:NodeStatusUVE] map 03/31/2025 02:37:48.834 7ff6efe0f2e8 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:37:48.835 7ff6efe0f2e8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:37:48.835 7ff6efe0f2e8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:37:48.835 7ff6efe0f2e8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:37:48.835 7ff6efe0f2e8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:37:48.835 7ff6efe0f2e8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:37:48.836 7ff6efe0f2e8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:37:48.836 7ff6efe0f2e8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:37:48.836 7ff6efe0f2e8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:37:48.836 7ff6efe0f2e8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:37:48.836 7ff6efe0f2e8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:37:48.836 7ff6efe0f2e8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:37:48.836 7ff6efe0f2e8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:37:51.357 7ff6f2605648 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-494-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=50965896, partition_space_available_1k=30124588, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=118404, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50812, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=14758500, free=9297704, buffers=844592, cached=7764316, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=1.87, five_min_avg=2.75, fifteen_min_avg=2.96, cpu_share=2.5, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 03/31/2025 02:37:51.357 7ff6efe0fa20 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:37:51.358 7ff6efe0fa20 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 50965896 partition_space_available_1k = 30124588 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-vrouter-agent : << mem_virt = 118404 cpu_share = 0 mem_res = 0 >>contrail-vrouter-nodemgr : << mem_virt = 50812 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665112 used = 14758500 free = 9297704 buffers = 844592 cached = 7764316 node_type = vrouter >> system_cpu_usage = << one_min_avg = 1.87 five_min_avg = 2.75 fifteen_min_avg = 2.96 cpu_share = 2.5 node_type = vrouter >> >> 03/31/2025 02:37:51.358 7ff6efe0fa20 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:37:52.836 7ff6efe0fa20 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:37:57.843 7ff6efe0fa20 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:37:57.843 7ff6efe0fa20 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:37:57.843 7ff6efe0fa20 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:37:57.843 7ff6efe0fa20 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:37:57.843 7ff6efe0fa20 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:37:57.843 7ff6efe0fa20 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:37:57.844 7ff6efe0fa20 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:37:57.844 7ff6efe0fa20 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:37:57.844 7ff6efe0fa20 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:37:57.844 7ff6efe0fa20 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:37:57.844 7ff6efe0fa20 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:37:57.844 7ff6efe0fa20 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:37:57.844 7ff6efe0fa20 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:38:01.845 7ff6f2f9bac8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:38:06.851 7ff6f2f9bac8 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:38:06.851 7ff6f2f9bac8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:38:06.851 7ff6f2f9bac8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:38:06.851 7ff6f2f9bac8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:38:06.851 7ff6f2f9bac8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:38:06.852 7ff6f2f9bac8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:38:06.852 7ff6f2f9bac8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:38:06.852 7ff6f2f9bac8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:38:06.852 7ff6f2f9bac8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:38:06.852 7ff6f2f9bac8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:38:06.852 7ff6f2f9bac8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:38:06.852 7ff6f2f9bac8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:38:06.852 7ff6f2f9bac8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:38:10.855 7ff6f2f9bac8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:38:15.858 7ff6f2f9bac8 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:38:15.858 7ff6f2f9bac8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:38:15.858 7ff6f2f9bac8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:38:15.858 7ff6f2f9bac8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:38:15.859 7ff6f2f9bac8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:38:15.859 7ff6f2f9bac8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:38:15.859 7ff6f2f9bac8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:38:15.859 7ff6f2f9bac8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:38:15.859 7ff6f2f9bac8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:38:15.859 7ff6f2f9bac8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:38:15.859 7ff6f2f9bac8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:38:15.859 7ff6f2f9bac8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:38:15.860 7ff6f2f9bac8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:38:19.861 7ff6f2f9bac8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:38:24.868 7ff6f2f9bac8 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:38:24.868 7ff6f2f9bac8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:38:24.868 7ff6f2f9bac8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:38:24.868 7ff6f2f9bac8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:38:24.868 7ff6f2f9bac8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:38:24.869 7ff6f2f9bac8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:38:24.869 7ff6f2f9bac8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:38:24.869 7ff6f2f9bac8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:38:24.869 7ff6f2f9bac8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:38:24.869 7ff6f2f9bac8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:38:24.869 7ff6f2f9bac8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:38:24.869 7ff6f2f9bac8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:38:24.869 7ff6f2f9bac8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:38:28.869 7ff6f1f34390 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:38:33.874 7ff6f1f34390 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:38:33.875 7ff6f1f34390 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:38:33.875 7ff6f1f34390 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:38:33.875 7ff6f1f34390 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:38:33.875 7ff6f1f34390 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:38:33.875 7ff6f1f34390 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:38:33.875 7ff6f1f34390 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:38:33.875 7ff6f1f34390 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:38:33.876 7ff6f1f34390 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:38:33.876 7ff6f1f34390 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:38:33.876 7ff6f1f34390 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:38:33.876 7ff6f1f34390 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:38:33.876 7ff6f1f34390 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:38:37.877 7ff6f1f34390 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:38:42.883 7ff6f1f34390 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:38:42.884 7ff6f1f34390 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:38:42.884 7ff6f1f34390 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:38:42.884 7ff6f1f34390 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:38:42.884 7ff6f1f34390 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:38:42.884 7ff6f1f34390 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:38:42.885 7ff6f1f34390 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:38:42.885 7ff6f1f34390 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:38:42.885 7ff6f1f34390 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:38:42.885 7ff6f1f34390 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:38:42.885 7ff6f1f34390 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:38:42.885 7ff6f1f34390 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:38:42.885 7ff6f1f34390 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:38:46.885 7ff6efd4b588 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:38:51.888 7ff6efd4b588 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:38:51.889 7ff6efd4b588 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:38:51.889 7ff6efd4b588 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:38:51.889 7ff6efd4b588 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:38:51.889 7ff6efd4b588 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:38:51.889 7ff6efd4b588 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:38:51.890 7ff6efd4b588 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:38:51.890 7ff6efd4b588 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:38:51.890 7ff6efd4b588 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:38:51.890 7ff6efd4b588 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:38:51.890 7ff6efd4b588 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:38:51.890 7ff6efd4b588 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:38:51.890 7ff6efd4b588 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:38:52.232 7ff6f2605648 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-494-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=50967988, partition_space_available_1k=30122496, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=118540, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50832, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=14852172, free=9190284, buffers=844592, cached=7778064, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=5.66, five_min_avg=3.43, fifteen_min_avg=3.17, cpu_share=3.75, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 03/31/2025 02:38:52.233 7ff6eeb1a048 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:38:52.233 7ff6eeb1a048 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 50967988 partition_space_available_1k = 30122496 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-vrouter-agent : << mem_virt = 118540 cpu_share = 0 mem_res = 0 >>contrail-vrouter-nodemgr : << mem_virt = 50832 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665112 used = 14852172 free = 9190284 buffers = 844592 cached = 7778064 node_type = vrouter >> system_cpu_usage = << one_min_avg = 5.66 five_min_avg = 3.43 fifteen_min_avg = 3.17 cpu_share = 3.75 node_type = vrouter >> >> 03/31/2025 02:38:52.234 7ff6eeb1a048 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:38:55.893 7ff6eeb1a048 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:39:00.896 7ff6eeb1a048 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:39:00.896 7ff6eeb1a048 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:39:00.896 7ff6eeb1a048 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:39:00.896 7ff6eeb1a048 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:39:00.896 7ff6eeb1a048 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:39:00.896 7ff6eeb1a048 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:39:00.897 7ff6eeb1a048 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:39:00.897 7ff6eeb1a048 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:39:00.897 7ff6eeb1a048 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:39:00.897 7ff6eeb1a048 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:39:00.897 7ff6eeb1a048 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:39:00.897 7ff6eeb1a048 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:39:00.897 7ff6eeb1a048 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:39:04.899 7ff6eeb1a048 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:39:08.088 7ff6efe3f588 [contrail-vrouter-nodemgr] [DEBUG]: send sync_uve in the [ObjectVRouter:NodeStatusUVE] map 03/31/2025 02:39:09.903 7ff6eeb1a048 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:39:09.903 7ff6eeb1a048 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:39:09.903 7ff6eeb1a048 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:39:09.903 7ff6eeb1a048 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:39:09.904 7ff6eeb1a048 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:39:09.904 7ff6eeb1a048 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:39:09.905 7ff6eeb1a048 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:39:09.905 7ff6eeb1a048 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:39:09.905 7ff6eeb1a048 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:39:09.905 7ff6eeb1a048 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:39:09.905 7ff6eeb1a048 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:39:09.905 7ff6eeb1a048 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:39:09.905 7ff6eeb1a048 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:39:13.905 7ff6eeb1a048 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:39:16.736 7ff6f2638780 [contrail-vrouter-nodemgr] [DEBUG]: send sync_uve in the [ObjectVRouter:NodeStatusUVE] map 03/31/2025 02:39:18.909 7ff6eeb1a048 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:39:18.909 7ff6eeb1a048 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:39:18.909 7ff6eeb1a048 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:39:18.909 7ff6eeb1a048 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:39:18.909 7ff6eeb1a048 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:39:18.909 7ff6eeb1a048 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:39:18.910 7ff6eeb1a048 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:39:18.910 7ff6eeb1a048 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:39:18.911 7ff6eeb1a048 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:39:18.912 7ff6eeb1a048 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:39:18.912 7ff6eeb1a048 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:39:18.912 7ff6eeb1a048 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:39:18.912 7ff6eeb1a048 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:39:22.913 7ff6eeb1a048 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:39:27.917 7ff6eeb1a048 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:39:27.917 7ff6eeb1a048 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:39:27.917 7ff6eeb1a048 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:39:27.917 7ff6eeb1a048 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:39:27.917 7ff6eeb1a048 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:39:27.917 7ff6eeb1a048 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:39:27.918 7ff6eeb1a048 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:39:27.918 7ff6eeb1a048 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:39:27.918 7ff6eeb1a048 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:39:27.918 7ff6eeb1a048 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:39:27.918 7ff6eeb1a048 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:39:27.919 7ff6eeb1a048 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:39:27.919 7ff6eeb1a048 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:39:28.037 7ff6eeb1a048 [contrail-vrouter-nodemgr] [DEBUG]: send sync_uve in the [ObjectVRouter:NodeStatusUVE] map 03/31/2025 02:39:31.917 7ff6eeb45c18 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:39:36.923 7ff6eeb45c18 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:39:36.924 7ff6eeb45c18 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:39:36.924 7ff6eeb45c18 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:39:36.924 7ff6eeb45c18 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:39:36.924 7ff6eeb45c18 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:39:36.924 7ff6eeb45c18 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:39:36.924 7ff6eeb45c18 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:39:36.924 7ff6eeb45c18 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:39:36.925 7ff6eeb45c18 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:39:36.925 7ff6eeb45c18 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:39:36.925 7ff6eeb45c18 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:39:36.925 7ff6eeb45c18 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:39:36.925 7ff6eeb45c18 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:39:40.927 7ff6eeb45c18 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:39:45.930 7ff6eeb45c18 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:39:45.931 7ff6eeb45c18 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:39:45.931 7ff6eeb45c18 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:39:45.931 7ff6eeb45c18 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:39:45.931 7ff6eeb45c18 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:39:45.931 7ff6eeb45c18 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:39:45.931 7ff6eeb45c18 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:39:45.931 7ff6eeb45c18 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:39:45.932 7ff6eeb45c18 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:39:45.932 7ff6eeb45c18 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:39:45.932 7ff6eeb45c18 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:39:45.932 7ff6eeb45c18 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:39:45.932 7ff6eeb45c18 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:39:49.931 7ff6eeb45c18 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:39:51.364 7ff6f2605648 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-494-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=50974516, partition_space_available_1k=30115968, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=118792, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50884, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=14811308, free=9222504, buffers=844596, cached=7786704, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=3.49, five_min_avg=3.26, fifteen_min_avg=3.13, cpu_share=2.5, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 03/31/2025 02:39:54.935 7ff6eeb45c18 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:39:54.935 7ff6eeb45c18 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:39:54.935 7ff6eeb45c18 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:39:54.936 7ff6eeb45c18 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:39:54.936 7ff6eeb45c18 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:39:54.936 7ff6eeb45c18 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:39:54.936 7ff6eeb45c18 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 50974516 partition_space_available_1k = 30115968 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-vrouter-agent : << mem_virt = 118792 cpu_share = 0 mem_res = 0 >>contrail-vrouter-nodemgr : << mem_virt = 50884 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665112 used = 14811308 free = 9222504 buffers = 844596 cached = 7786704 node_type = vrouter >> system_cpu_usage = << one_min_avg = 3.49 five_min_avg = 3.26 fifteen_min_avg = 3.13 cpu_share = 2.5 node_type = vrouter >> >> 03/31/2025 02:39:54.936 7ff6eeb45c18 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:39:54.936 7ff6eeb45c18 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:39:54.936 7ff6eeb45c18 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:39:54.937 7ff6eeb45c18 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:39:54.937 7ff6eeb45c18 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:39:54.937 7ff6eeb45c18 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:39:54.937 7ff6eeb45c18 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:39:54.937 7ff6eeb45c18 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:39:54.937 7ff6eeb45c18 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:39:58.938 7ff6eeb45c18 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:40:03.942 7ff6eeb45c18 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:40:03.942 7ff6eeb45c18 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:40:03.942 7ff6eeb45c18 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:40:03.942 7ff6eeb45c18 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:40:03.943 7ff6eeb45c18 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:40:03.943 7ff6eeb45c18 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:40:03.944 7ff6eeb45c18 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:40:03.944 7ff6eeb45c18 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:40:03.944 7ff6eeb45c18 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:40:03.944 7ff6eeb45c18 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:40:03.944 7ff6eeb45c18 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:40:03.945 7ff6eeb45c18 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:40:03.945 7ff6eeb45c18 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:40:07.946 7ff6ee99e978 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:40:12.952 7ff6ee99e978 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:40:12.952 7ff6ee99e978 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:40:12.952 7ff6ee99e978 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:40:12.953 7ff6ee99e978 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:40:12.953 7ff6ee99e978 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:40:12.953 7ff6ee99e978 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:40:12.953 7ff6ee99e978 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:40:12.953 7ff6ee99e978 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:40:12.953 7ff6ee99e978 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:40:12.954 7ff6ee99e978 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:40:12.954 7ff6ee99e978 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:40:12.954 7ff6ee99e978 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:40:12.954 7ff6ee99e978 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:40:16.954 7ff6f23d0780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:40:21.959 7ff6f23d0780 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:40:21.959 7ff6f23d0780 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:40:21.959 7ff6f23d0780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:40:21.960 7ff6f23d0780 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:40:21.960 7ff6f23d0780 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:40:21.960 7ff6f23d0780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:40:21.960 7ff6f23d0780 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:40:21.960 7ff6f23d0780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:40:21.960 7ff6f23d0780 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:40:21.960 7ff6f23d0780 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:40:21.961 7ff6f23d0780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:40:21.961 7ff6f23d0780 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:40:21.962 7ff6f23d0780 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:40:25.962 7ff6f23d0780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:40:30.964 7ff6f23d0780 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:40:30.965 7ff6f23d0780 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:40:30.965 7ff6f23d0780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:40:30.965 7ff6f23d0780 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:40:30.965 7ff6f23d0780 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:40:30.965 7ff6f23d0780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:40:30.966 7ff6f23d0780 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:40:30.966 7ff6f23d0780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:40:30.966 7ff6f23d0780 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:40:30.966 7ff6f23d0780 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:40:30.966 7ff6f23d0780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:40:30.966 7ff6f23d0780 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:40:30.966 7ff6f23d0780 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:40:34.968 7ff6f23d0780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:40:39.975 7ff6f23d0780 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:40:39.975 7ff6f23d0780 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:40:39.975 7ff6f23d0780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:40:39.975 7ff6f23d0780 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:40:39.975 7ff6f23d0780 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:40:39.975 7ff6f23d0780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:40:39.976 7ff6f23d0780 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:40:39.976 7ff6f23d0780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:40:39.976 7ff6f23d0780 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:40:39.976 7ff6f23d0780 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:40:39.976 7ff6f23d0780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:40:39.976 7ff6f23d0780 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:40:39.976 7ff6f23d0780 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:40:43.979 7ff6f23d0780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:40:48.982 7ff6f23d0780 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:40:48.982 7ff6f23d0780 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:40:48.982 7ff6f23d0780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:40:48.982 7ff6f23d0780 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:40:48.982 7ff6f23d0780 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:40:48.983 7ff6f23d0780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:40:48.983 7ff6f23d0780 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:40:48.983 7ff6f23d0780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:40:48.983 7ff6f23d0780 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:40:48.983 7ff6f23d0780 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:40:48.983 7ff6f23d0780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:40:48.984 7ff6f23d0780 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:40:48.984 7ff6f23d0780 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:40:51.359 7ff6f2605648 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-494-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=50974944, partition_space_available_1k=30115540, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=118960, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50880, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=14840664, free=9190668, buffers=844596, cached=7789184, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=2.45, five_min_avg=3.0, fifteen_min_avg=3.05, cpu_share=2.25, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 03/31/2025 02:40:51.359 7ff6ee99ea20 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:40:51.359 7ff6ee99ea20 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 50974944 partition_space_available_1k = 30115540 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-vrouter-agent : << mem_virt = 118960 cpu_share = 0 mem_res = 0 >>contrail-vrouter-nodemgr : << mem_virt = 50880 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665112 used = 14840664 free = 9190668 buffers = 844596 cached = 7789184 node_type = vrouter >> system_cpu_usage = << one_min_avg = 2.45 five_min_avg = 3.0 fifteen_min_avg = 3.05 cpu_share = 2.25 node_type = vrouter >> >> 03/31/2025 02:40:51.359 7ff6ee99ea20 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:40:52.982 7ff6ee99ea20 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:40:57.989 7ff6ee99ea20 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:40:57.989 7ff6ee99ea20 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:40:57.989 7ff6ee99ea20 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:40:57.989 7ff6ee99ea20 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:40:57.989 7ff6ee99ea20 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:40:57.990 7ff6ee99ea20 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:40:57.990 7ff6ee99ea20 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:40:57.990 7ff6ee99ea20 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:40:57.990 7ff6ee99ea20 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:40:57.990 7ff6ee99ea20 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:40:57.990 7ff6ee99ea20 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:40:57.990 7ff6ee99ea20 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:40:57.990 7ff6ee99ea20 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:41:01.990 7ff6f1fad438 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:41:06.994 7ff6f1fad438 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:41:06.995 7ff6f1fad438 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:41:06.995 7ff6f1fad438 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:41:06.995 7ff6f1fad438 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:41:06.995 7ff6f1fad438 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:41:06.995 7ff6f1fad438 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:41:06.996 7ff6f1fad438 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:41:06.996 7ff6f1fad438 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:41:06.996 7ff6f1fad438 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:41:06.996 7ff6f1fad438 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:41:06.996 7ff6f1fad438 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:41:06.996 7ff6f1fad438 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:41:06.996 7ff6f1fad438 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:41:10.997 7ff6f1fad438 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:41:15.998 7ff6f1fad438 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:41:15.999 7ff6f1fad438 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:41:15.999 7ff6f1fad438 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:41:15.999 7ff6f1fad438 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:41:15.999 7ff6f1fad438 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:41:15.999 7ff6f1fad438 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:41:16.000 7ff6f1fad438 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:41:16.000 7ff6f1fad438 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:41:16.000 7ff6f1fad438 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:41:16.000 7ff6f1fad438 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:41:16.000 7ff6f1fad438 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:41:16.000 7ff6f1fad438 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:41:16.000 7ff6f1fad438 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:41:20.002 7ff6f1fad438 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:41:25.008 7ff6f1fad438 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:41:25.009 7ff6f1fad438 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:41:25.009 7ff6f1fad438 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:41:25.009 7ff6f1fad438 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:41:25.009 7ff6f1fad438 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:41:25.009 7ff6f1fad438 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:41:25.010 7ff6f1fad438 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:41:25.010 7ff6f1fad438 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:41:25.010 7ff6f1fad438 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:41:25.010 7ff6f1fad438 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:41:25.010 7ff6f1fad438 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:41:25.010 7ff6f1fad438 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:41:25.010 7ff6f1fad438 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:41:29.013 7ff6f1fad438 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:41:34.017 7ff6f1fad438 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:41:34.018 7ff6f1fad438 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:41:34.018 7ff6f1fad438 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:41:34.018 7ff6f1fad438 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:41:34.018 7ff6f1fad438 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:41:34.018 7ff6f1fad438 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:41:34.018 7ff6f1fad438 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:41:34.018 7ff6f1fad438 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:41:34.019 7ff6f1fad438 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:41:34.019 7ff6f1fad438 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:41:34.019 7ff6f1fad438 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:41:34.019 7ff6f1fad438 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:41:34.019 7ff6f1fad438 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:41:38.021 7ff6f1fad438 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:41:43.027 7ff6f1fad438 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:41:43.027 7ff6f1fad438 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:41:43.027 7ff6f1fad438 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:41:43.027 7ff6f1fad438 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:41:43.027 7ff6f1fad438 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:41:43.027 7ff6f1fad438 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:41:43.028 7ff6f1fad438 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:41:43.028 7ff6f1fad438 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:41:43.028 7ff6f1fad438 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:41:43.028 7ff6f1fad438 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:41:43.028 7ff6f1fad438 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:41:43.028 7ff6f1fad438 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:41:43.028 7ff6f1fad438 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:41:47.028 7ff6efe02b70 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:41:51.361 7ff6f2605648 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-494-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=50978596, partition_space_available_1k=30111888, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=119084, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50936, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=14810432, free=9213772, buffers=844596, cached=7796312, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=2.56, five_min_avg=2.94, fifteen_min_avg=3.02, cpu_share=2.0, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 03/31/2025 02:41:52.030 7ff6efe02b70 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:41:52.031 7ff6efe02b70 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:41:52.031 7ff6efe02b70 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:41:52.031 7ff6efe02b70 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:41:52.031 7ff6efe02b70 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:41:52.031 7ff6efe02b70 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:41:52.031 7ff6efe02b70 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 50978596 partition_space_available_1k = 30111888 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-vrouter-agent : << mem_virt = 119084 cpu_share = 0 mem_res = 0 >>contrail-vrouter-nodemgr : << mem_virt = 50936 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665112 used = 14810432 free = 9213772 buffers = 844596 cached = 7796312 node_type = vrouter >> system_cpu_usage = << one_min_avg = 2.56 five_min_avg = 2.94 fifteen_min_avg = 3.02 cpu_share = 2.0 node_type = vrouter >> >> 03/31/2025 02:41:52.031 7ff6efe02b70 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:41:52.031 7ff6efe02b70 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:41:52.032 7ff6efe02b70 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:41:52.032 7ff6efe02b70 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:41:52.032 7ff6efe02b70 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:41:52.032 7ff6efe02b70 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:41:52.032 7ff6efe02b70 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:41:52.032 7ff6efe02b70 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:41:52.032 7ff6efe02b70 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:41:56.033 7ff6f01f6390 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:42:01.038 7ff6f01f6390 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:42:01.038 7ff6f01f6390 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:42:01.038 7ff6f01f6390 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:42:01.038 7ff6f01f6390 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:42:01.038 7ff6f01f6390 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:42:01.038 7ff6f01f6390 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:42:01.039 7ff6f01f6390 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:42:01.039 7ff6f01f6390 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:42:01.039 7ff6f01f6390 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:42:01.039 7ff6f01f6390 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:42:01.039 7ff6f01f6390 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:42:01.040 7ff6f01f6390 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:42:01.040 7ff6f01f6390 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:42:05.041 7ff6ee98d240 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:42:05.200 7ff6efe19390 [contrail-vrouter-nodemgr] [DEBUG]: send sync_uve in the [ObjectVRouter:NodeStatusUVE] map 03/31/2025 02:42:10.047 7ff6ee98d240 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:42:10.047 7ff6ee98d240 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:42:10.047 7ff6ee98d240 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:42:10.048 7ff6ee98d240 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:42:10.048 7ff6ee98d240 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:42:10.048 7ff6ee98d240 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:42:10.048 7ff6ee98d240 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:42:10.048 7ff6ee98d240 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:42:10.048 7ff6ee98d240 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:42:10.049 7ff6ee98d240 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:42:10.049 7ff6ee98d240 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:42:10.049 7ff6ee98d240 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:42:10.049 7ff6ee98d240 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:42:14.051 7ff6ee98d240 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:42:19.055 7ff6ee98d240 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:42:19.055 7ff6ee98d240 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:42:19.055 7ff6ee98d240 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:42:19.055 7ff6ee98d240 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:42:19.055 7ff6ee98d240 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:42:19.056 7ff6ee98d240 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:42:19.056 7ff6ee98d240 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:42:19.056 7ff6ee98d240 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:42:19.056 7ff6ee98d240 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:42:19.056 7ff6ee98d240 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:42:19.056 7ff6ee98d240 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:42:19.056 7ff6ee98d240 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:42:19.056 7ff6ee98d240 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:42:23.059 7ff6f84d0048 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:42:28.066 7ff6f84d0048 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:42:28.066 7ff6f84d0048 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:42:28.066 7ff6f84d0048 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:42:28.066 7ff6f84d0048 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:42:28.066 7ff6f84d0048 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:42:28.066 7ff6f84d0048 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:42:28.067 7ff6f84d0048 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:42:28.067 7ff6f84d0048 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:42:28.067 7ff6f84d0048 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:42:28.067 7ff6f84d0048 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:42:28.067 7ff6f84d0048 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:42:28.067 7ff6f84d0048 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:42:28.067 7ff6f84d0048 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:42:32.066 7ff6f01f1e10 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:42:37.072 7ff6f01f1e10 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:42:37.072 7ff6f01f1e10 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:42:37.072 7ff6f01f1e10 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:42:37.072 7ff6f01f1e10 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:42:37.072 7ff6f01f1e10 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:42:37.072 7ff6f01f1e10 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:42:37.073 7ff6f01f1e10 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:42:37.073 7ff6f01f1e10 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:42:37.073 7ff6f01f1e10 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:42:37.073 7ff6f01f1e10 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:42:37.073 7ff6f01f1e10 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:42:37.073 7ff6f01f1e10 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:42:37.073 7ff6f01f1e10 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:42:41.077 7ff6f01f1e10 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:42:46.083 7ff6f01f1e10 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:42:46.083 7ff6f01f1e10 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:42:46.083 7ff6f01f1e10 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:42:46.084 7ff6f01f1e10 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:42:46.084 7ff6f01f1e10 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:42:46.084 7ff6f01f1e10 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:42:46.084 7ff6f01f1e10 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:42:46.084 7ff6f01f1e10 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:42:46.084 7ff6f01f1e10 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:42:46.084 7ff6f01f1e10 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:42:46.085 7ff6f01f1e10 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:42:46.085 7ff6f01f1e10 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:42:46.085 7ff6f01f1e10 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:42:50.085 7ff6efe0f0f0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:42:51.362 7ff6f2605648 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-494-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=50982620, partition_space_available_1k=30107864, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=119220, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50956, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=14805116, free=9212828, buffers=844596, cached=7802572, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=2.31, five_min_avg=2.81, fifteen_min_avg=2.97, cpu_share=2.25, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 03/31/2025 02:42:55.091 7ff6efe0f0f0 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:42:55.091 7ff6efe0f0f0 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:42:55.091 7ff6efe0f0f0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:42:55.092 7ff6efe0f0f0 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:42:55.092 7ff6efe0f0f0 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:42:55.092 7ff6efe0f0f0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:42:55.092 7ff6efe0f0f0 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 50982620 partition_space_available_1k = 30107864 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-vrouter-agent : << mem_virt = 119220 cpu_share = 0 mem_res = 0 >>contrail-vrouter-nodemgr : << mem_virt = 50956 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665112 used = 14805116 free = 9212828 buffers = 844596 cached = 7802572 node_type = vrouter >> system_cpu_usage = << one_min_avg = 2.31 five_min_avg = 2.81 fifteen_min_avg = 2.97 cpu_share = 2.25 node_type = vrouter >> >> 03/31/2025 02:42:55.092 7ff6efe0f0f0 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:42:55.092 7ff6efe0f0f0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:42:55.092 7ff6efe0f0f0 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:42:55.093 7ff6efe0f0f0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:42:55.093 7ff6efe0f0f0 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:42:55.093 7ff6efe0f0f0 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:42:55.093 7ff6efe0f0f0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:42:55.093 7ff6efe0f0f0 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:42:55.093 7ff6efe0f0f0 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:42:59.092 7ff6eeb1a240 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:43:04.096 7ff6eeb1a240 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:43:04.096 7ff6eeb1a240 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:43:04.096 7ff6eeb1a240 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:43:04.096 7ff6eeb1a240 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:43:04.096 7ff6eeb1a240 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:43:04.096 7ff6eeb1a240 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:43:04.097 7ff6eeb1a240 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:43:04.097 7ff6eeb1a240 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:43:04.097 7ff6eeb1a240 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:43:04.097 7ff6eeb1a240 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:43:04.097 7ff6eeb1a240 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:43:04.097 7ff6eeb1a240 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:43:04.097 7ff6eeb1a240 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:43:08.099 7ff6efe02630 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:43:13.105 7ff6efe02630 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:43:13.105 7ff6efe02630 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:43:13.105 7ff6efe02630 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:43:13.105 7ff6efe02630 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:43:13.105 7ff6efe02630 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:43:13.105 7ff6efe02630 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:43:13.106 7ff6efe02630 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:43:13.106 7ff6efe02630 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:43:13.106 7ff6efe02630 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:43:13.106 7ff6efe02630 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:43:13.106 7ff6efe02630 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:43:13.106 7ff6efe02630 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:43:13.106 7ff6efe02630 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:43:17.105 7ff6ee98dac8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:43:22.111 7ff6ee98dac8 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:43:22.112 7ff6ee98dac8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:43:22.112 7ff6ee98dac8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:43:22.112 7ff6ee98dac8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:43:22.112 7ff6ee98dac8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:43:22.112 7ff6ee98dac8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:43:22.113 7ff6ee98dac8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:43:22.113 7ff6ee98dac8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:43:22.113 7ff6ee98dac8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:43:22.113 7ff6ee98dac8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:43:22.113 7ff6ee98dac8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:43:22.113 7ff6ee98dac8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:43:22.113 7ff6ee98dac8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:43:26.114 7ff6ee98dac8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:43:31.121 7ff6ee98dac8 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:43:31.121 7ff6ee98dac8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:43:31.121 7ff6ee98dac8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:43:31.121 7ff6ee98dac8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:43:31.121 7ff6ee98dac8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:43:31.122 7ff6ee98dac8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:43:31.122 7ff6ee98dac8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:43:31.122 7ff6ee98dac8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:43:31.122 7ff6ee98dac8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:43:31.122 7ff6ee98dac8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:43:31.123 7ff6ee98dac8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:43:31.123 7ff6ee98dac8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:43:31.123 7ff6ee98dac8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:43:35.125 7ff6eeb2c630 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:43:40.129 7ff6eeb2c630 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:43:40.129 7ff6eeb2c630 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:43:40.129 7ff6eeb2c630 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:43:40.129 7ff6eeb2c630 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:43:40.129 7ff6eeb2c630 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:43:40.129 7ff6eeb2c630 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:43:40.130 7ff6eeb2c630 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:43:40.130 7ff6eeb2c630 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:43:40.130 7ff6eeb2c630 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:43:40.130 7ff6eeb2c630 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:43:40.130 7ff6eeb2c630 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:43:40.130 7ff6eeb2c630 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:43:40.130 7ff6eeb2c630 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:43:44.133 7ff6eeb2c630 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:43:49.135 7ff6eeb2c630 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:43:49.135 7ff6eeb2c630 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:43:49.135 7ff6eeb2c630 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:43:49.135 7ff6eeb2c630 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:43:49.135 7ff6eeb2c630 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:43:49.135 7ff6eeb2c630 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:43:49.136 7ff6eeb2c630 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:43:49.136 7ff6eeb2c630 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:43:49.136 7ff6eeb2c630 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:43:49.136 7ff6eeb2c630 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:43:49.136 7ff6eeb2c630 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:43:49.137 7ff6eeb2c630 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:43:49.137 7ff6eeb2c630 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:43:51.362 7ff6f2605648 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-494-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=50986144, partition_space_available_1k=30104340, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=119312, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=50948, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=14814860, free=9199028, buffers=844596, cached=7806628, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=2.18, five_min_avg=2.69, fifteen_min_avg=2.92, cpu_share=1.88, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 03/31/2025 02:43:51.363 7ff6ee9b78d0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:43:51.363 7ff6ee9b78d0 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 50986144 partition_space_available_1k = 30104340 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-vrouter-agent : << mem_virt = 119312 cpu_share = 0 mem_res = 0 >>contrail-vrouter-nodemgr : << mem_virt = 50948 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665112 used = 14814860 free = 9199028 buffers = 844596 cached = 7806628 node_type = vrouter >> system_cpu_usage = << one_min_avg = 2.18 five_min_avg = 2.69 fifteen_min_avg = 2.92 cpu_share = 1.88 node_type = vrouter >> >> 03/31/2025 02:43:51.363 7ff6ee9b78d0 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:43:53.137 7ff6ee9b78d0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:43:58.143 7ff6ee9b78d0 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:43:58.144 7ff6ee9b78d0 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:43:58.144 7ff6ee9b78d0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:43:58.144 7ff6ee9b78d0 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:43:58.144 7ff6ee9b78d0 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:43:58.144 7ff6ee9b78d0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:43:58.144 7ff6ee9b78d0 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:43:58.144 7ff6ee9b78d0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:43:58.145 7ff6ee9b78d0 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:43:58.145 7ff6ee9b78d0 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:43:58.145 7ff6ee9b78d0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:43:58.145 7ff6ee9b78d0 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:43:58.145 7ff6ee9b78d0 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:44:02.148 7ff6ee9b78d0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:44:07.154 7ff6ee9b78d0 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:44:07.155 7ff6ee9b78d0 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:44:07.155 7ff6ee9b78d0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:44:07.155 7ff6ee9b78d0 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:44:07.155 7ff6ee9b78d0 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:44:07.155 7ff6ee9b78d0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:44:07.155 7ff6ee9b78d0 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:44:07.156 7ff6ee9b78d0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:44:07.156 7ff6ee9b78d0 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:44:07.156 7ff6ee9b78d0 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:44:07.156 7ff6ee9b78d0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:44:07.156 7ff6ee9b78d0 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:44:07.156 7ff6ee9b78d0 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:44:11.158 7ff6f911fe10 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:44:15.049 7ff6ee9974e0 [contrail-vrouter-nodemgr] [DEBUG]: send sync_uve in the [ObjectVRouter:NodeStatusUVE] map 03/31/2025 02:44:16.161 7ff6f911fe10 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:44:16.161 7ff6f911fe10 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:44:16.161 7ff6f911fe10 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:44:16.161 7ff6f911fe10 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:44:16.161 7ff6f911fe10 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:44:16.162 7ff6f911fe10 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:44:16.162 7ff6f911fe10 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:44:16.162 7ff6f911fe10 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:44:16.162 7ff6f911fe10 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:44:16.162 7ff6f911fe10 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:44:16.163 7ff6f911fe10 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:44:16.163 7ff6f911fe10 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:44:16.163 7ff6f911fe10 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:44:20.164 7ff6f25de4e0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:44:23.697 7ff6f2139588 [contrail-vrouter-nodemgr] [DEBUG]: send sync_uve in the [ObjectVRouter:NodeStatusUVE] map 03/31/2025 02:44:25.167 7ff6f25de4e0 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:44:25.168 7ff6f25de4e0 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:44:25.168 7ff6f25de4e0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:44:25.168 7ff6f25de4e0 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:44:25.168 7ff6f25de4e0 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:44:25.168 7ff6f25de4e0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:44:25.169 7ff6f25de4e0 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:44:25.169 7ff6f25de4e0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:44:25.169 7ff6f25de4e0 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:44:25.169 7ff6f25de4e0 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:44:25.169 7ff6f25de4e0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:44:25.169 7ff6f25de4e0 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:44:25.169 7ff6f25de4e0 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:44:29.171 7ff6f25de4e0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:44:34.176 7ff6f25de4e0 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:44:34.177 7ff6f25de4e0 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:44:34.177 7ff6f25de4e0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:44:34.177 7ff6f25de4e0 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:44:34.177 7ff6f25de4e0 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:44:34.177 7ff6f25de4e0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:44:34.177 7ff6f25de4e0 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:44:34.177 7ff6f25de4e0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:44:34.178 7ff6f25de4e0 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:44:34.178 7ff6f25de4e0 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:44:34.178 7ff6f25de4e0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:44:34.178 7ff6f25de4e0 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:44:34.178 7ff6f25de4e0 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:44:37.883 7ff6f25de4e0 [contrail-vrouter-nodemgr] [DEBUG]: send sync_uve in the [ObjectVRouter:NodeStatusUVE] map 03/31/2025 02:44:38.177 7ff6efd4b048 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:44:43.181 7ff6efd4b048 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:44:43.181 7ff6efd4b048 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:44:43.181 7ff6efd4b048 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:44:43.181 7ff6efd4b048 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:44:43.181 7ff6efd4b048 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:44:43.181 7ff6efd4b048 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:44:43.182 7ff6efd4b048 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:44:43.182 7ff6efd4b048 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:44:43.182 7ff6efd4b048 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:44:43.182 7ff6efd4b048 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:44:43.182 7ff6efd4b048 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:44:43.182 7ff6efd4b048 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:44:43.182 7ff6efd4b048 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:44:47.185 7ff6efd4b048 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:44:51.374 7ff6f2605648 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-494-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=50991092, partition_space_available_1k=30099392, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=119436, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51004, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=14930148, free=9070540, buffers=844596, cached=7819828, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=5.86, five_min_avg=3.6, fifteen_min_avg=3.21, cpu_share=4.0, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 03/31/2025 02:44:52.187 7ff6efd4b048 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:44:52.187 7ff6efd4b048 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:44:52.187 7ff6efd4b048 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:44:52.188 7ff6efd4b048 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:44:52.188 7ff6efd4b048 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:44:52.188 7ff6efd4b048 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:44:52.188 7ff6efd4b048 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 50991092 partition_space_available_1k = 30099392 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-vrouter-agent : << mem_virt = 119436 cpu_share = 0 mem_res = 0 >>contrail-vrouter-nodemgr : << mem_virt = 51004 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665112 used = 14930148 free = 9070540 buffers = 844596 cached = 7819828 node_type = vrouter >> system_cpu_usage = << one_min_avg = 5.86 five_min_avg = 3.6 fifteen_min_avg = 3.21 cpu_share = 4.0 node_type = vrouter >> >> 03/31/2025 02:44:52.188 7ff6efd4b048 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:44:52.188 7ff6efd4b048 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:44:52.189 7ff6efd4b048 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:44:52.189 7ff6efd4b048 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:44:52.189 7ff6efd4b048 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:44:52.189 7ff6efd4b048 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:44:52.189 7ff6efd4b048 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:44:52.189 7ff6efd4b048 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:44:52.189 7ff6efd4b048 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:44:56.191 7ff6ee98cf60 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:45:01.197 7ff6ee98cf60 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:45:01.197 7ff6ee98cf60 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:45:01.197 7ff6ee98cf60 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:45:01.197 7ff6ee98cf60 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:45:01.197 7ff6ee98cf60 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:45:01.198 7ff6ee98cf60 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:45:01.198 7ff6ee98cf60 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:45:01.198 7ff6ee98cf60 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:45:01.198 7ff6ee98cf60 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:45:01.198 7ff6ee98cf60 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:45:01.198 7ff6ee98cf60 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:45:01.198 7ff6ee98cf60 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:45:01.198 7ff6ee98cf60 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:45:05.200 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:45:10.205 7ff6efeb6828 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:45:10.205 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:45:10.205 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:45:10.205 7ff6efeb6828 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:45:10.205 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:45:10.206 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:45:10.206 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:45:10.206 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:45:10.206 7ff6efeb6828 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:45:10.206 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:45:10.206 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:45:10.206 7ff6efeb6828 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:45:10.206 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:45:14.209 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:45:19.212 7ff6efeb6828 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:45:19.212 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:45:19.212 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:45:19.212 7ff6efeb6828 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:45:19.212 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:45:19.213 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:45:19.213 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:45:19.213 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:45:19.213 7ff6efeb6828 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:45:19.213 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:45:19.213 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:45:19.213 7ff6efeb6828 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:45:19.213 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:45:23.217 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:45:28.223 7ff6efeb6828 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:45:28.223 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:45:28.224 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:45:28.224 7ff6efeb6828 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:45:28.224 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:45:28.224 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:45:28.224 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:45:28.224 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:45:28.225 7ff6efeb6828 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:45:28.225 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:45:28.225 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:45:28.225 7ff6efeb6828 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:45:28.225 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:45:32.227 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:45:37.233 7ff6efeb6828 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:45:37.233 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:45:37.233 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:45:37.233 7ff6efeb6828 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:45:37.233 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:45:37.234 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:45:37.234 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:45:37.234 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:45:37.234 7ff6efeb6828 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:45:37.234 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:45:37.234 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:45:37.234 7ff6efeb6828 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:45:37.234 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:45:41.237 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:45:46.243 7ff6efeb6828 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:45:46.243 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:45:46.243 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:45:46.244 7ff6efeb6828 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:45:46.244 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:45:46.244 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:45:46.244 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:45:46.244 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:45:46.244 7ff6efeb6828 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:45:46.244 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:45:46.244 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:45:46.245 7ff6efeb6828 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:45:46.245 7ff6efeb6828 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:45:50.244 7ff6efe36ac8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:45:51.363 7ff6f2605648 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-494-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=50995632, partition_space_available_1k=30094852, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=119520, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51020, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=14926516, free=9068948, buffers=844596, cached=7825052, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=3.48, five_min_avg=3.32, fifteen_min_avg=3.14, cpu_share=2.38, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 03/31/2025 02:45:55.249 7ff6efe36ac8 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:45:55.249 7ff6efe36ac8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:45:55.249 7ff6efe36ac8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:45:55.249 7ff6efe36ac8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:45:55.249 7ff6efe36ac8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:45:55.249 7ff6efe36ac8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:45:55.250 7ff6efe36ac8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 50995632 partition_space_available_1k = 30094852 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-vrouter-agent : << mem_virt = 119520 cpu_share = 0 mem_res = 0 >>contrail-vrouter-nodemgr : << mem_virt = 51020 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665112 used = 14926516 free = 9068948 buffers = 844596 cached = 7825052 node_type = vrouter >> system_cpu_usage = << one_min_avg = 3.48 five_min_avg = 3.32 fifteen_min_avg = 3.14 cpu_share = 2.38 node_type = vrouter >> >> 03/31/2025 02:45:55.250 7ff6efe36ac8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:45:55.250 7ff6efe36ac8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:45:55.250 7ff6efe36ac8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:45:55.250 7ff6efe36ac8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:45:55.250 7ff6efe36ac8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:45:55.250 7ff6efe36ac8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:45:55.250 7ff6efe36ac8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:45:55.251 7ff6efe36ac8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:45:55.251 7ff6efe36ac8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:45:59.253 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:46:04.255 7ff6efecbeb8 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:46:04.255 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:46:04.256 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:46:04.256 7ff6efecbeb8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:46:04.256 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:46:04.256 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:46:04.257 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:46:04.257 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:46:04.257 7ff6efecbeb8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:46:04.257 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:46:04.257 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:46:04.257 7ff6efecbeb8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:46:04.257 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:46:08.258 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:46:13.262 7ff6efecbeb8 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:46:13.263 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:46:13.263 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:46:13.263 7ff6efecbeb8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:46:13.263 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:46:13.263 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:46:13.263 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:46:13.263 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:46:13.264 7ff6efecbeb8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:46:13.264 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:46:13.264 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:46:13.264 7ff6efecbeb8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:46:13.264 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:46:17.267 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:46:22.272 7ff6efecbeb8 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:46:22.273 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:46:22.273 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:46:22.273 7ff6efecbeb8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:46:22.273 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:46:22.273 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:46:22.273 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:46:22.274 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:46:22.274 7ff6efecbeb8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:46:22.274 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:46:22.274 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:46:22.274 7ff6efecbeb8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:46:22.274 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:46:26.277 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:46:31.282 7ff6efecbeb8 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:46:31.283 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:46:31.283 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:46:31.283 7ff6efecbeb8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:46:31.283 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:46:31.283 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:46:31.284 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:46:31.284 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:46:31.284 7ff6efecbeb8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:46:31.284 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:46:31.284 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:46:31.284 7ff6efecbeb8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:46:31.284 7ff6efecbeb8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:46:35.285 7ff6f01feb70 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:46:40.291 7ff6f01feb70 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:46:40.291 7ff6f01feb70 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:46:40.291 7ff6f01feb70 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:46:40.292 7ff6f01feb70 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:46:40.292 7ff6f01feb70 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:46:40.292 7ff6f01feb70 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:46:40.292 7ff6f01feb70 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:46:40.293 7ff6f01feb70 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:46:40.293 7ff6f01feb70 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:46:40.293 7ff6f01feb70 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:46:40.293 7ff6f01feb70 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:46:40.293 7ff6f01feb70 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:46:40.293 7ff6f01feb70 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:46:44.296 7ff6f01feb70 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:46:46.916 7ff6eeb44f60 [contrail-vrouter-nodemgr] [DEBUG]: send sync_uve in the [ObjectVRouter:NodeStatusUVE] map 03/31/2025 02:46:49.298 7ff6f01feb70 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:46:49.298 7ff6f01feb70 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:46:49.298 7ff6f01feb70 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:46:49.298 7ff6f01feb70 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:46:49.298 7ff6f01feb70 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:46:49.298 7ff6f01feb70 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:46:49.299 7ff6f01feb70 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:46:49.299 7ff6f01feb70 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:46:49.299 7ff6f01feb70 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:46:49.299 7ff6f01feb70 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:46:49.299 7ff6f01feb70 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:46:49.299 7ff6f01feb70 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:46:49.300 7ff6f01feb70 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:46:51.372 7ff6f2605648 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-494-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51004884, partition_space_available_1k=30085600, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=119636, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51012, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=15033516, free=8951096, buffers=844596, cached=7835904, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=2.91, five_min_avg=3.19, fifteen_min_avg=3.11, cpu_share=2.5, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 03/31/2025 02:46:51.372 7ff6f01feb70 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:46:51.373 7ff6f01feb70 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51004884 partition_space_available_1k = 30085600 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-vrouter-agent : << mem_virt = 119636 cpu_share = 0 mem_res = 0 >>contrail-vrouter-nodemgr : << mem_virt = 51012 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665112 used = 15033516 free = 8951096 buffers = 844596 cached = 7835904 node_type = vrouter >> system_cpu_usage = << one_min_avg = 2.91 five_min_avg = 3.19 fifteen_min_avg = 3.11 cpu_share = 2.5 node_type = vrouter >> >> 03/31/2025 02:46:51.373 7ff6f01feb70 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:46:53.300 7ff6f01feb70 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:46:58.306 7ff6f01feb70 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:46:58.306 7ff6f01feb70 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:46:58.307 7ff6f01feb70 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:46:58.307 7ff6f01feb70 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:46:58.307 7ff6f01feb70 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:46:58.307 7ff6f01feb70 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:46:58.307 7ff6f01feb70 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:46:58.307 7ff6f01feb70 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:46:58.307 7ff6f01feb70 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:46:58.307 7ff6f01feb70 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:46:58.308 7ff6f01feb70 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:46:58.308 7ff6f01feb70 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:46:58.308 7ff6f01feb70 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:47:02.308 7ff6f01feb70 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:47:07.312 7ff6f01feb70 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:47:07.312 7ff6f01feb70 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:47:07.312 7ff6f01feb70 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:47:07.313 7ff6f01feb70 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:47:07.313 7ff6f01feb70 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:47:07.313 7ff6f01feb70 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:47:07.313 7ff6f01feb70 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:47:07.313 7ff6f01feb70 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:47:07.313 7ff6f01feb70 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:47:07.314 7ff6f01feb70 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:47:07.314 7ff6f01feb70 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:47:07.314 7ff6f01feb70 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:47:07.314 7ff6f01feb70 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:47:11.315 7ff6f01feb70 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:47:16.321 7ff6f01feb70 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:47:16.322 7ff6f01feb70 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:47:16.322 7ff6f01feb70 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:47:16.322 7ff6f01feb70 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:47:16.322 7ff6f01feb70 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:47:16.322 7ff6f01feb70 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:47:16.322 7ff6f01feb70 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:47:16.322 7ff6f01feb70 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:47:16.323 7ff6f01feb70 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:47:16.323 7ff6f01feb70 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:47:16.323 7ff6f01feb70 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:47:16.323 7ff6f01feb70 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:47:16.323 7ff6f01feb70 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:47:20.323 7ff6f202c780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:47:25.329 7ff6f202c780 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:47:25.330 7ff6f202c780 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:47:25.330 7ff6f202c780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:47:25.330 7ff6f202c780 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:47:25.330 7ff6f202c780 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:47:25.330 7ff6f202c780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:47:25.330 7ff6f202c780 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:47:25.331 7ff6f202c780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:47:25.331 7ff6f202c780 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:47:25.331 7ff6f202c780 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:47:25.331 7ff6f202c780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:47:25.331 7ff6f202c780 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:47:25.331 7ff6f202c780 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:47:29.333 7ff6f202c780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:47:34.334 7ff6f202c780 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:47:34.334 7ff6f202c780 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:47:34.334 7ff6f202c780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:47:34.335 7ff6f202c780 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:47:34.335 7ff6f202c780 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:47:34.335 7ff6f202c780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:47:34.335 7ff6f202c780 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:47:34.335 7ff6f202c780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:47:34.336 7ff6f202c780 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:47:34.336 7ff6f202c780 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:47:34.336 7ff6f202c780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:47:34.336 7ff6f202c780 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:47:34.336 7ff6f202c780 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:47:38.337 7ff6f202c780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:47:43.343 7ff6f202c780 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:47:43.344 7ff6f202c780 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:47:43.344 7ff6f202c780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:47:43.344 7ff6f202c780 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:47:43.344 7ff6f202c780 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:47:43.344 7ff6f202c780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:47:43.344 7ff6f202c780 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:47:43.345 7ff6f202c780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:47:43.345 7ff6f202c780 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:47:43.345 7ff6f202c780 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:47:43.345 7ff6f202c780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:47:43.345 7ff6f202c780 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:47:43.345 7ff6f202c780 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:47:47.347 7ff6f202c780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:47:51.369 7ff6f2605648 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-494-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=50999408, partition_space_available_1k=30091076, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=119720, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51064, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=14991268, free=8997660, buffers=844596, cached=7831588, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=1.89, five_min_avg=2.83, fifteen_min_avg=2.99, cpu_share=2.38, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 03/31/2025 02:47:52.349 7ff6f202c780 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:47:52.349 7ff6f202c780 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:47:52.349 7ff6f202c780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:47:52.349 7ff6f202c780 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:47:52.349 7ff6f202c780 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:47:52.349 7ff6f202c780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:47:52.350 7ff6f202c780 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 50999408 partition_space_available_1k = 30091076 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-vrouter-agent : << mem_virt = 119720 cpu_share = 0 mem_res = 0 >>contrail-vrouter-nodemgr : << mem_virt = 51064 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665112 used = 14991268 free = 8997660 buffers = 844596 cached = 7831588 node_type = vrouter >> system_cpu_usage = << one_min_avg = 1.89 five_min_avg = 2.83 fifteen_min_avg = 2.99 cpu_share = 2.38 node_type = vrouter >> >> 03/31/2025 02:47:52.350 7ff6f202c780 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:47:52.350 7ff6f202c780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:47:52.350 7ff6f202c780 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:47:52.350 7ff6f202c780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:47:52.350 7ff6f202c780 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:47:52.350 7ff6f202c780 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:47:52.351 7ff6f202c780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:47:52.351 7ff6f202c780 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:47:52.351 7ff6f202c780 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:47:56.353 7ff6f202c780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:48:01.359 7ff6f202c780 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:48:01.359 7ff6f202c780 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:48:01.359 7ff6f202c780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:48:01.359 7ff6f202c780 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:48:01.359 7ff6f202c780 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:48:01.359 7ff6f202c780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:48:01.360 7ff6f202c780 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:48:01.360 7ff6f202c780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:48:01.360 7ff6f202c780 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:48:01.360 7ff6f202c780 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:48:01.360 7ff6f202c780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:48:01.360 7ff6f202c780 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:48:01.360 7ff6f202c780 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:48:05.360 7ff6f23b7ac8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:48:10.364 7ff6f23b7ac8 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:48:10.364 7ff6f23b7ac8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:48:10.364 7ff6f23b7ac8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:48:10.365 7ff6f23b7ac8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:48:10.365 7ff6f23b7ac8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:48:10.365 7ff6f23b7ac8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:48:10.365 7ff6f23b7ac8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:48:10.365 7ff6f23b7ac8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:48:10.365 7ff6f23b7ac8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:48:10.365 7ff6f23b7ac8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:48:10.365 7ff6f23b7ac8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:48:10.366 7ff6f23b7ac8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:48:10.366 7ff6f23b7ac8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:48:14.369 7ff6f23b7ac8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:48:19.374 7ff6f23b7ac8 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:48:19.374 7ff6f23b7ac8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:48:19.374 7ff6f23b7ac8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:48:19.374 7ff6f23b7ac8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:48:19.374 7ff6f23b7ac8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:48:19.374 7ff6f23b7ac8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:48:19.375 7ff6f23b7ac8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:48:19.375 7ff6f23b7ac8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:48:19.375 7ff6f23b7ac8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:48:19.375 7ff6f23b7ac8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:48:19.375 7ff6f23b7ac8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:48:19.375 7ff6f23b7ac8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:48:19.375 7ff6f23b7ac8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:48:23.377 7ff6f23b7ac8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:48:28.383 7ff6f23b7ac8 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:48:28.383 7ff6f23b7ac8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:48:28.383 7ff6f23b7ac8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:48:28.383 7ff6f23b7ac8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:48:28.383 7ff6f23b7ac8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:48:28.383 7ff6f23b7ac8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:48:28.384 7ff6f23b7ac8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:48:28.384 7ff6f23b7ac8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:48:28.384 7ff6f23b7ac8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:48:28.384 7ff6f23b7ac8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:48:28.384 7ff6f23b7ac8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:48:28.384 7ff6f23b7ac8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:48:28.384 7ff6f23b7ac8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:48:32.386 7ff6f23b7ac8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:48:37.390 7ff6f23b7ac8 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:48:37.390 7ff6f23b7ac8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:48:37.390 7ff6f23b7ac8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:48:37.391 7ff6f23b7ac8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:48:37.391 7ff6f23b7ac8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:48:37.391 7ff6f23b7ac8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:48:37.391 7ff6f23b7ac8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:48:37.391 7ff6f23b7ac8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:48:37.392 7ff6f23b7ac8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:48:37.392 7ff6f23b7ac8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:48:37.392 7ff6f23b7ac8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:48:37.392 7ff6f23b7ac8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:48:37.392 7ff6f23b7ac8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:48:41.393 7ff6f23b7ac8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:48:46.399 7ff6f23b7ac8 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:48:46.400 7ff6f23b7ac8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:48:46.400 7ff6f23b7ac8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:48:46.400 7ff6f23b7ac8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:48:46.400 7ff6f23b7ac8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:48:46.400 7ff6f23b7ac8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:48:46.401 7ff6f23b7ac8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:48:46.401 7ff6f23b7ac8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:48:46.401 7ff6f23b7ac8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:48:46.401 7ff6f23b7ac8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:48:46.401 7ff6f23b7ac8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:48:46.401 7ff6f23b7ac8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:48:46.401 7ff6f23b7ac8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:48:50.400 7ff6ee99ec18 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:48:51.371 7ff6f2605648 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-494-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51003056, partition_space_available_1k=30087428, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=119816, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51092, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=15007316, free=8976724, buffers=844596, cached=7836476, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=2.05, five_min_avg=2.7, fifteen_min_avg=2.93, cpu_share=2.38, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 03/31/2025 02:48:55.406 7ff6ee99ec18 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:48:55.406 7ff6ee99ec18 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:48:55.407 7ff6ee99ec18 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:48:55.407 7ff6ee99ec18 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:48:55.407 7ff6ee99ec18 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:48:55.407 7ff6ee99ec18 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:48:55.407 7ff6ee99ec18 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51003056 partition_space_available_1k = 30087428 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-vrouter-agent : << mem_virt = 119816 cpu_share = 0 mem_res = 0 >>contrail-vrouter-nodemgr : << mem_virt = 51092 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665112 used = 15007316 free = 8976724 buffers = 844596 cached = 7836476 node_type = vrouter >> system_cpu_usage = << one_min_avg = 2.05 five_min_avg = 2.7 fifteen_min_avg = 2.93 cpu_share = 2.38 node_type = vrouter >> >> 03/31/2025 02:48:55.407 7ff6ee99ec18 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:48:55.407 7ff6ee99ec18 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:48:55.408 7ff6ee99ec18 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:48:55.408 7ff6ee99ec18 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:48:55.408 7ff6ee99ec18 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:48:55.408 7ff6ee99ec18 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:48:55.408 7ff6ee99ec18 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:48:55.408 7ff6ee99ec18 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:48:55.408 7ff6ee99ec18 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:48:59.155 7ff6efe02c18 [contrail-vrouter-nodemgr] [DEBUG]: send sync_uve in the [ObjectVRouter:NodeStatusUVE] map 03/31/2025 02:48:59.407 7ff6efe02c18 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:49:04.413 7ff6efe02c18 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:49:04.413 7ff6efe02c18 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:49:04.413 7ff6efe02c18 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:49:04.413 7ff6efe02c18 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:49:04.414 7ff6efe02c18 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:49:04.414 7ff6efe02c18 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:49:04.414 7ff6efe02c18 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:49:04.414 7ff6efe02c18 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:49:04.414 7ff6efe02c18 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:49:04.414 7ff6efe02c18 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:49:04.414 7ff6efe02c18 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:49:04.414 7ff6efe02c18 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:49:04.415 7ff6efe02c18 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:49:07.787 7ff6efe3fcc0 [contrail-vrouter-nodemgr] [DEBUG]: send sync_uve in the [ObjectVRouter:NodeStatusUVE] map 03/31/2025 02:49:08.414 7ff6efe3fcc0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:49:13.419 7ff6efe3fcc0 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:49:13.419 7ff6efe3fcc0 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:49:13.419 7ff6efe3fcc0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:49:13.419 7ff6efe3fcc0 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:49:13.419 7ff6efe3fcc0 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:49:13.419 7ff6efe3fcc0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:49:13.420 7ff6efe3fcc0 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:49:13.420 7ff6efe3fcc0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:49:13.420 7ff6efe3fcc0 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:49:13.420 7ff6efe3fcc0 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:49:13.420 7ff6efe3fcc0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:49:13.420 7ff6efe3fcc0 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:49:13.420 7ff6efe3fcc0 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:49:17.424 7ff6efe3fcc0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:49:19.002 7ff6eeb44b70 [contrail-vrouter-nodemgr] [DEBUG]: send sync_uve in the [ObjectVRouter:NodeStatusUVE] map 03/31/2025 02:49:22.427 7ff6efe3fcc0 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:49:22.428 7ff6efe3fcc0 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:49:22.428 7ff6efe3fcc0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:49:22.428 7ff6efe3fcc0 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:49:22.428 7ff6efe3fcc0 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:49:22.428 7ff6efe3fcc0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:49:22.429 7ff6efe3fcc0 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:49:22.429 7ff6efe3fcc0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:49:22.429 7ff6efe3fcc0 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:49:22.429 7ff6efe3fcc0 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:49:22.429 7ff6efe3fcc0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:49:22.429 7ff6efe3fcc0 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:49:22.429 7ff6efe3fcc0 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:49:26.431 7ff6efe3fcc0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:49:31.437 7ff6efe3fcc0 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:49:31.437 7ff6efe3fcc0 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:49:31.438 7ff6efe3fcc0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:49:31.438 7ff6efe3fcc0 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:49:31.438 7ff6efe3fcc0 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:49:31.438 7ff6efe3fcc0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:49:31.438 7ff6efe3fcc0 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:49:31.438 7ff6efe3fcc0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:49:31.438 7ff6efe3fcc0 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:49:31.438 7ff6efe3fcc0 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:49:31.439 7ff6efe3fcc0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:49:31.439 7ff6efe3fcc0 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:49:31.439 7ff6efe3fcc0 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:49:35.438 7ff6ee9c08d0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:49:40.444 7ff6ee9c08d0 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:49:40.445 7ff6ee9c08d0 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:49:40.445 7ff6ee9c08d0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:49:40.445 7ff6ee9c08d0 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:49:40.445 7ff6ee9c08d0 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:49:40.445 7ff6ee9c08d0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:49:40.445 7ff6ee9c08d0 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:49:40.445 7ff6ee9c08d0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:49:40.446 7ff6ee9c08d0 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:49:40.446 7ff6ee9c08d0 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:49:40.446 7ff6ee9c08d0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:49:40.446 7ff6ee9c08d0 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:49:40.446 7ff6ee9c08d0 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:49:44.448 7ff6ee9c08d0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:49:49.449 7ff6ee9c08d0 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:49:49.450 7ff6ee9c08d0 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:49:49.450 7ff6ee9c08d0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:49:49.450 7ff6ee9c08d0 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:49:49.450 7ff6ee9c08d0 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:49:49.450 7ff6ee9c08d0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:49:49.451 7ff6ee9c08d0 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:49:49.451 7ff6ee9c08d0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:49:49.451 7ff6ee9c08d0 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:49:49.451 7ff6ee9c08d0 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:49:49.451 7ff6ee9c08d0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:49:49.451 7ff6ee9c08d0 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:49:49.451 7ff6ee9c08d0 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:49:51.382 7ff6f2605648 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-494-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51002352, partition_space_available_1k=30088132, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=119984, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51092, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=15006736, free=8975492, buffers=844604, cached=7838280, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=2.72, five_min_avg=2.73, fifteen_min_avg=2.93, cpu_share=2.5, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 03/31/2025 02:49:51.382 7ff6efd4b780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:49:51.383 7ff6efd4b780 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51002352 partition_space_available_1k = 30088132 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-vrouter-agent : << mem_virt = 119984 cpu_share = 0 mem_res = 0 >>contrail-vrouter-nodemgr : << mem_virt = 51092 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665112 used = 15006736 free = 8975492 buffers = 844604 cached = 7838280 node_type = vrouter >> system_cpu_usage = << one_min_avg = 2.72 five_min_avg = 2.73 fifteen_min_avg = 2.93 cpu_share = 2.5 node_type = vrouter >> >> 03/31/2025 02:49:51.383 7ff6efd4b780 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:49:53.452 7ff6efd4b780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:49:58.458 7ff6efd4b780 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:49:58.458 7ff6efd4b780 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:49:58.459 7ff6efd4b780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:49:58.459 7ff6efd4b780 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:49:58.459 7ff6efd4b780 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:49:58.459 7ff6efd4b780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:49:58.459 7ff6efd4b780 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:49:58.459 7ff6efd4b780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:49:58.460 7ff6efd4b780 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:49:58.460 7ff6efd4b780 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:49:58.460 7ff6efd4b780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:49:58.460 7ff6efd4b780 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:49:58.460 7ff6efd4b780 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:50:02.461 7ff6efd4b780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:50:07.463 7ff6efd4b780 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:50:07.464 7ff6efd4b780 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:50:07.464 7ff6efd4b780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:50:07.464 7ff6efd4b780 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:50:07.464 7ff6efd4b780 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:50:07.464 7ff6efd4b780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:50:07.465 7ff6efd4b780 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:50:07.465 7ff6efd4b780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:50:07.465 7ff6efd4b780 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:50:07.465 7ff6efd4b780 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:50:07.465 7ff6efd4b780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:50:07.465 7ff6efd4b780 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:50:07.465 7ff6efd4b780 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:50:11.468 7ff6f22cb828 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:50:16.474 7ff6f22cb828 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:50:16.474 7ff6f22cb828 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:50:16.474 7ff6f22cb828 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:50:16.475 7ff6f22cb828 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:50:16.475 7ff6f22cb828 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:50:16.475 7ff6f22cb828 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:50:16.475 7ff6f22cb828 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:50:16.475 7ff6f22cb828 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:50:16.475 7ff6f22cb828 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:50:16.475 7ff6f22cb828 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:50:16.475 7ff6f22cb828 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:50:16.476 7ff6f22cb828 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:50:16.476 7ff6f22cb828 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:50:20.477 7ff6f22cb828 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:50:25.484 7ff6f22cb828 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:50:25.484 7ff6f22cb828 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:50:25.484 7ff6f22cb828 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:50:25.484 7ff6f22cb828 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:50:25.484 7ff6f22cb828 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:50:25.484 7ff6f22cb828 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:50:25.485 7ff6f22cb828 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:50:25.485 7ff6f22cb828 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:50:25.485 7ff6f22cb828 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:50:25.485 7ff6f22cb828 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:50:25.485 7ff6f22cb828 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:50:25.485 7ff6f22cb828 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:50:25.485 7ff6f22cb828 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:50:29.488 7ff6f22cb828 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:50:34.493 7ff6f22cb828 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:50:34.494 7ff6f22cb828 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:50:34.494 7ff6f22cb828 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:50:34.494 7ff6f22cb828 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:50:34.494 7ff6f22cb828 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:50:34.494 7ff6f22cb828 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:50:34.495 7ff6f22cb828 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:50:34.495 7ff6f22cb828 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:50:34.495 7ff6f22cb828 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:50:34.495 7ff6f22cb828 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:50:34.495 7ff6f22cb828 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:50:34.495 7ff6f22cb828 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:50:34.495 7ff6f22cb828 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:50:38.496 7ff6efe19b70 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:50:43.501 7ff6efe19b70 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:50:43.501 7ff6efe19b70 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:50:43.501 7ff6efe19b70 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:50:43.502 7ff6efe19b70 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:50:43.502 7ff6efe19b70 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:50:43.502 7ff6efe19b70 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:50:43.502 7ff6efe19b70 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:50:43.502 7ff6efe19b70 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:50:43.502 7ff6efe19b70 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:50:43.502 7ff6efe19b70 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:50:43.502 7ff6efe19b70 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:50:43.503 7ff6efe19b70 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:50:43.503 7ff6efe19b70 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:50:47.505 7ff6efe19b70 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:50:47.979 7ff6eeb44cc0 [contrail-vrouter-nodemgr] [DEBUG]: send sync_uve in the [ObjectVRouter:NodeStatusUVE] map 03/31/2025 02:50:51.379 7ff6f2605648 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-494-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51017276, partition_space_available_1k=30073208, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=120080, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51136, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=15098224, free=8861928, buffers=844612, cached=7860348, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=3.85, five_min_avg=3.07, fifteen_min_avg=3.03, cpu_share=4.12, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 03/31/2025 02:50:52.508 7ff6efe19b70 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:50:52.509 7ff6efe19b70 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:50:52.509 7ff6efe19b70 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:50:52.509 7ff6efe19b70 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:50:52.509 7ff6efe19b70 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:50:52.509 7ff6efe19b70 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:50:52.509 7ff6efe19b70 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51017276 partition_space_available_1k = 30073208 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-vrouter-agent : << mem_virt = 120080 cpu_share = 0 mem_res = 0 >>contrail-vrouter-nodemgr : << mem_virt = 51136 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665112 used = 15098224 free = 8861928 buffers = 844612 cached = 7860348 node_type = vrouter >> system_cpu_usage = << one_min_avg = 3.85 five_min_avg = 3.07 fifteen_min_avg = 3.03 cpu_share = 4.12 node_type = vrouter >> >> 03/31/2025 02:50:52.509 7ff6efe19b70 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:50:52.509 7ff6efe19b70 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:50:52.510 7ff6efe19b70 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:50:52.510 7ff6efe19b70 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:50:52.510 7ff6efe19b70 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:50:52.511 7ff6efe19b70 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:50:52.511 7ff6efe19b70 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:50:52.511 7ff6efe19b70 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:50:52.511 7ff6efe19b70 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:50:56.513 7ff6f2638780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:51:01.517 7ff6f2638780 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:51:01.517 7ff6f2638780 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:51:01.517 7ff6f2638780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:51:01.517 7ff6f2638780 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:51:01.517 7ff6f2638780 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:51:01.517 7ff6f2638780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:51:01.518 7ff6f2638780 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:51:01.518 7ff6f2638780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:51:01.518 7ff6f2638780 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:51:01.518 7ff6f2638780 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:51:01.518 7ff6f2638780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:51:01.518 7ff6f2638780 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:51:01.518 7ff6f2638780 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:51:05.521 7ff6f2638780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:51:10.527 7ff6f2638780 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:51:10.527 7ff6f2638780 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:51:10.527 7ff6f2638780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:51:10.527 7ff6f2638780 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:51:10.527 7ff6f2638780 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:51:10.527 7ff6f2638780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:51:10.528 7ff6f2638780 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:51:10.528 7ff6f2638780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:51:10.528 7ff6f2638780 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:51:10.528 7ff6f2638780 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:51:10.528 7ff6f2638780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:51:10.528 7ff6f2638780 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:51:10.528 7ff6f2638780 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:51:14.527 7ff6f2638780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:51:19.532 7ff6f2638780 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:51:19.532 7ff6f2638780 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:51:19.532 7ff6f2638780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:51:19.533 7ff6f2638780 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:51:19.533 7ff6f2638780 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:51:19.533 7ff6f2638780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:51:19.533 7ff6f2638780 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:51:19.533 7ff6f2638780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:51:19.533 7ff6f2638780 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:51:19.533 7ff6f2638780 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:51:19.534 7ff6f2638780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:51:19.534 7ff6f2638780 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:51:19.534 7ff6f2638780 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:51:23.535 7ff6efe02048 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:51:28.541 7ff6efe02048 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:51:28.541 7ff6efe02048 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:51:28.541 7ff6efe02048 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:51:28.541 7ff6efe02048 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:51:28.541 7ff6efe02048 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:51:28.542 7ff6efe02048 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:51:28.542 7ff6efe02048 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:51:28.542 7ff6efe02048 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:51:28.542 7ff6efe02048 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:51:28.542 7ff6efe02048 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:51:28.542 7ff6efe02048 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:51:28.542 7ff6efe02048 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:51:28.542 7ff6efe02048 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:51:32.546 7ff6efe02048 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:51:37.549 7ff6efe02048 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:51:37.549 7ff6efe02048 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:51:37.549 7ff6efe02048 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:51:37.549 7ff6efe02048 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:51:37.549 7ff6efe02048 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:51:37.549 7ff6efe02048 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:51:37.550 7ff6efe02048 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:51:37.550 7ff6efe02048 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:51:37.550 7ff6efe02048 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:51:37.550 7ff6efe02048 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:51:37.550 7ff6efe02048 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:51:37.550 7ff6efe02048 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:51:37.550 7ff6efe02048 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:51:41.553 7ff6efe02048 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:51:46.559 7ff6efe02048 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:51:46.559 7ff6efe02048 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:51:46.559 7ff6efe02048 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:51:46.559 7ff6efe02048 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:51:46.559 7ff6efe02048 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:51:46.559 7ff6efe02048 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:51:46.560 7ff6efe02048 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:51:46.560 7ff6efe02048 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:51:46.560 7ff6efe02048 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:51:46.560 7ff6efe02048 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:51:46.560 7ff6efe02048 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:51:46.560 7ff6efe02048 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:51:46.560 7ff6efe02048 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:51:50.560 7ff6ee9b7048 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:51:51.377 7ff6f2605648 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-494-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51011576, partition_space_available_1k=30078908, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=120196, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51156, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=15092132, free=8871268, buffers=844612, cached=7857100, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=2.5, five_min_avg=2.8, fifteen_min_avg=2.94, cpu_share=2.62, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 03/31/2025 02:51:55.566 7ff6ee9b7048 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:51:55.566 7ff6ee9b7048 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:51:55.566 7ff6ee9b7048 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:51:55.566 7ff6ee9b7048 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:51:55.566 7ff6ee9b7048 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:51:55.567 7ff6ee9b7048 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:51:55.567 7ff6ee9b7048 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51011576 partition_space_available_1k = 30078908 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-vrouter-agent : << mem_virt = 120196 cpu_share = 0 mem_res = 0 >>contrail-vrouter-nodemgr : << mem_virt = 51156 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665112 used = 15092132 free = 8871268 buffers = 844612 cached = 7857100 node_type = vrouter >> system_cpu_usage = << one_min_avg = 2.5 five_min_avg = 2.8 fifteen_min_avg = 2.94 cpu_share = 2.62 node_type = vrouter >> >> 03/31/2025 02:51:55.567 7ff6ee9b7048 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:51:55.567 7ff6ee9b7048 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:51:55.567 7ff6ee9b7048 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:51:55.567 7ff6ee9b7048 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:51:55.568 7ff6ee9b7048 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:51:55.568 7ff6ee9b7048 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:51:55.568 7ff6ee9b7048 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:51:55.568 7ff6ee9b7048 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:51:55.568 7ff6ee9b7048 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:51:59.570 7ff6f202c780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:52:04.577 7ff6f202c780 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:52:04.577 7ff6f202c780 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:52:04.577 7ff6f202c780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:52:04.577 7ff6f202c780 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:52:04.577 7ff6f202c780 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:52:04.577 7ff6f202c780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:52:04.578 7ff6f202c780 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:52:04.578 7ff6f202c780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:52:04.578 7ff6f202c780 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:52:04.578 7ff6f202c780 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:52:04.578 7ff6f202c780 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:52:04.578 7ff6f202c780 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:52:04.578 7ff6f202c780 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:52:08.578 7ff6f01fe978 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:52:13.585 7ff6f01fe978 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:52:13.585 7ff6f01fe978 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:52:13.585 7ff6f01fe978 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:52:13.586 7ff6f01fe978 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:52:13.586 7ff6f01fe978 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:52:13.586 7ff6f01fe978 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:52:13.587 7ff6f01fe978 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:52:13.587 7ff6f01fe978 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:52:13.587 7ff6f01fe978 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:52:13.587 7ff6f01fe978 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:52:13.588 7ff6f01fe978 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:52:13.588 7ff6f01fe978 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:52:13.588 7ff6f01fe978 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:52:17.589 7ff6f01fe978 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:52:22.591 7ff6f01fe978 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:52:22.592 7ff6f01fe978 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:52:22.592 7ff6f01fe978 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:52:22.592 7ff6f01fe978 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:52:22.592 7ff6f01fe978 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:52:22.592 7ff6f01fe978 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:52:22.593 7ff6f01fe978 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:52:22.593 7ff6f01fe978 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:52:22.593 7ff6f01fe978 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:52:22.594 7ff6f01fe978 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:52:22.594 7ff6f01fe978 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:52:22.594 7ff6f01fe978 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:52:22.594 7ff6f01fe978 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:52:26.595 7ff6f01fe978 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:52:31.601 7ff6f01fe978 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:52:31.601 7ff6f01fe978 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:52:31.601 7ff6f01fe978 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:52:31.601 7ff6f01fe978 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:52:31.601 7ff6f01fe978 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:52:31.602 7ff6f01fe978 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:52:31.602 7ff6f01fe978 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:52:31.602 7ff6f01fe978 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:52:31.602 7ff6f01fe978 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:52:31.603 7ff6f01fe978 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:52:31.603 7ff6f01fe978 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:52:31.603 7ff6f01fe978 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:52:31.603 7ff6f01fe978 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:52:35.605 7ff6f01fe978 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:52:40.609 7ff6f01fe978 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:52:40.609 7ff6f01fe978 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:52:40.609 7ff6f01fe978 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:52:40.609 7ff6f01fe978 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:52:40.610 7ff6f01fe978 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:52:40.610 7ff6f01fe978 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:52:40.610 7ff6f01fe978 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:52:40.610 7ff6f01fe978 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:52:40.610 7ff6f01fe978 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:52:40.610 7ff6f01fe978 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:52:40.610 7ff6f01fe978 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:52:40.610 7ff6f01fe978 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:52:40.610 7ff6f01fe978 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:52:44.613 7ff6f01fe978 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:52:49.615 7ff6f01fe978 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:52:49.616 7ff6f01fe978 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:52:49.616 7ff6f01fe978 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:52:49.616 7ff6f01fe978 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:52:49.616 7ff6f01fe978 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:52:49.616 7ff6f01fe978 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:52:49.617 7ff6f01fe978 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:52:49.617 7ff6f01fe978 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:52:49.617 7ff6f01fe978 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:52:49.617 7ff6f01fe978 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:52:49.617 7ff6f01fe978 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:52:49.617 7ff6f01fe978 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:52:49.618 7ff6f01fe978 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:52:51.383 7ff6f2605648 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-494-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51016272, partition_space_available_1k=30074212, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=120308, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51152, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=15016444, free=8939616, buffers=844612, cached=7864440, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=1.81, five_min_avg=2.56, fifteen_min_avg=2.85, cpu_share=2.62, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 03/31/2025 02:52:51.383 7ff6ee9b7828 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:52:51.384 7ff6ee9b7828 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51016272 partition_space_available_1k = 30074212 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-vrouter-agent : << mem_virt = 120308 cpu_share = 0 mem_res = 0 >>contrail-vrouter-nodemgr : << mem_virt = 51152 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665112 used = 15016444 free = 8939616 buffers = 844612 cached = 7864440 node_type = vrouter >> system_cpu_usage = << one_min_avg = 1.81 five_min_avg = 2.56 fifteen_min_avg = 2.85 cpu_share = 2.62 node_type = vrouter >> >> 03/31/2025 02:52:51.384 7ff6ee9b7828 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:52:53.617 7ff6ee9b76d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:52:58.624 7ff6ee9b76d8 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:52:58.624 7ff6ee9b76d8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:52:58.624 7ff6ee9b76d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:52:58.625 7ff6ee9b76d8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:52:58.625 7ff6ee9b76d8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:52:58.625 7ff6ee9b76d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:52:58.625 7ff6ee9b76d8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:52:58.625 7ff6ee9b76d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:52:58.625 7ff6ee9b76d8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:52:58.626 7ff6ee9b76d8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:52:58.626 7ff6ee9b76d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:52:58.626 7ff6ee9b76d8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:52:58.626 7ff6ee9b76d8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:53:02.626 7ff6ee9b76d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:53:07.628 7ff6ee9b76d8 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:53:07.629 7ff6ee9b76d8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:53:07.629 7ff6ee9b76d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:53:07.629 7ff6ee9b76d8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:53:07.629 7ff6ee9b76d8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:53:07.629 7ff6ee9b76d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:53:07.630 7ff6ee9b76d8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:53:07.630 7ff6ee9b76d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:53:07.630 7ff6ee9b76d8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:53:07.630 7ff6ee9b76d8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:53:07.630 7ff6ee9b76d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:53:07.630 7ff6ee9b76d8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:53:07.630 7ff6ee9b76d8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:53:11.633 7ff6ee9b76d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:53:16.637 7ff6ee9b76d8 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:53:16.637 7ff6ee9b76d8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:53:16.637 7ff6ee9b76d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:53:16.637 7ff6ee9b76d8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:53:16.637 7ff6ee9b76d8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:53:16.637 7ff6ee9b76d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:53:16.638 7ff6ee9b76d8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:53:16.638 7ff6ee9b76d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:53:16.638 7ff6ee9b76d8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:53:16.638 7ff6ee9b76d8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:53:16.638 7ff6ee9b76d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:53:16.638 7ff6ee9b76d8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:53:16.638 7ff6ee9b76d8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:53:20.640 7ff6ee9b76d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:53:25.645 7ff6ee9b76d8 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:53:25.645 7ff6ee9b76d8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:53:25.646 7ff6ee9b76d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:53:25.646 7ff6ee9b76d8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:53:25.646 7ff6ee9b76d8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:53:25.646 7ff6ee9b76d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:53:25.647 7ff6ee9b76d8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:53:25.647 7ff6ee9b76d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:53:25.647 7ff6ee9b76d8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:53:25.647 7ff6ee9b76d8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:53:25.647 7ff6ee9b76d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:53:25.648 7ff6ee9b76d8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:53:25.648 7ff6ee9b76d8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:53:29.649 7ff6ee9b76d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:53:34.655 7ff6ee9b76d8 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:53:34.656 7ff6ee9b76d8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:53:34.656 7ff6ee9b76d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:53:34.656 7ff6ee9b76d8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:53:34.656 7ff6ee9b76d8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:53:34.656 7ff6ee9b76d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:53:34.657 7ff6ee9b76d8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:53:34.657 7ff6ee9b76d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:53:34.657 7ff6ee9b76d8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:53:34.658 7ff6ee9b76d8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:53:34.658 7ff6ee9b76d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:53:34.658 7ff6ee9b76d8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:53:34.658 7ff6ee9b76d8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:53:38.657 7ff6efe29e10 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:53:43.663 7ff6efe29e10 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:53:43.663 7ff6efe29e10 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:53:43.663 7ff6efe29e10 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:53:43.663 7ff6efe29e10 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:53:43.663 7ff6efe29e10 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:53:43.663 7ff6efe29e10 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:53:43.664 7ff6efe29e10 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:53:43.664 7ff6efe29e10 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:53:43.664 7ff6efe29e10 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:53:43.664 7ff6efe29e10 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:53:43.664 7ff6efe29e10 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:53:43.664 7ff6efe29e10 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:53:43.664 7ff6efe29e10 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:53:47.668 7ff6efe29e10 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:53:51.384 7ff6f2605648 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-494-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51015180, partition_space_available_1k=30075304, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=120388, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51208, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=15030564, free=8926104, buffers=844612, cached=7863832, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=1.8, five_min_avg=2.42, fifteen_min_avg=2.78, cpu_share=2.5, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 03/31/2025 02:53:52.671 7ff6efe29e10 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:53:52.672 7ff6efe29e10 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:53:52.672 7ff6efe29e10 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:53:52.672 7ff6efe29e10 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:53:52.672 7ff6efe29e10 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:53:52.672 7ff6efe29e10 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:53:52.672 7ff6efe29e10 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51015180 partition_space_available_1k = 30075304 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-vrouter-agent : << mem_virt = 120388 cpu_share = 0 mem_res = 0 >>contrail-vrouter-nodemgr : << mem_virt = 51208 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665112 used = 15030564 free = 8926104 buffers = 844612 cached = 7863832 node_type = vrouter >> system_cpu_usage = << one_min_avg = 1.8 five_min_avg = 2.42 fifteen_min_avg = 2.78 cpu_share = 2.5 node_type = vrouter >> >> 03/31/2025 02:53:52.672 7ff6efe29e10 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:53:52.672 7ff6efe29e10 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:53:52.673 7ff6efe29e10 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:53:52.673 7ff6efe29e10 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:53:52.673 7ff6efe29e10 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:53:52.673 7ff6efe29e10 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:53:52.673 7ff6efe29e10 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:53:52.673 7ff6efe29e10 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:53:52.673 7ff6efe29e10 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:53:56.673 7ff6f1fe66d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:54:01.679 7ff6f1fe66d8 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:54:01.679 7ff6f1fe66d8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:54:01.679 7ff6f1fe66d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:54:01.679 7ff6f1fe66d8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:54:01.679 7ff6f1fe66d8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:54:01.679 7ff6f1fe66d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:54:01.680 7ff6f1fe66d8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:54:01.680 7ff6f1fe66d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:54:01.680 7ff6f1fe66d8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:54:01.680 7ff6f1fe66d8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:54:01.680 7ff6f1fe66d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:54:01.680 7ff6f1fe66d8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:54:01.680 7ff6f1fe66d8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:54:05.684 7ff6f1fe66d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:54:10.688 7ff6f1fe66d8 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:54:10.688 7ff6f1fe66d8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:54:10.688 7ff6f1fe66d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:54:10.688 7ff6f1fe66d8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:54:10.688 7ff6f1fe66d8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:54:10.689 7ff6f1fe66d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:54:10.689 7ff6f1fe66d8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:54:10.689 7ff6f1fe66d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:54:10.689 7ff6f1fe66d8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:54:10.689 7ff6f1fe66d8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:54:10.689 7ff6f1fe66d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:54:10.689 7ff6f1fe66d8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:54:10.690 7ff6f1fe66d8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:54:12.110 7ff6f1fe66d8 [contrail-vrouter-nodemgr] [DEBUG]: send sync_uve in the [ObjectVRouter:NodeStatusUVE] map 03/31/2025 02:54:14.690 7ff6f1fe66d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:54:19.697 7ff6f1fe66d8 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:54:19.697 7ff6f1fe66d8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:54:19.697 7ff6f1fe66d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:54:19.697 7ff6f1fe66d8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:54:19.697 7ff6f1fe66d8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:54:19.697 7ff6f1fe66d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:54:19.698 7ff6f1fe66d8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:54:19.698 7ff6f1fe66d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:54:19.698 7ff6f1fe66d8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:54:19.698 7ff6f1fe66d8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:54:19.698 7ff6f1fe66d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:54:19.698 7ff6f1fe66d8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:54:19.698 7ff6f1fe66d8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:54:20.722 7ff6f1fe66d8 [contrail-vrouter-nodemgr] [DEBUG]: send sync_uve in the [ObjectVRouter:NodeStatusUVE] map 03/31/2025 02:54:23.698 7ff6efe02a20 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:54:28.705 7ff6efe02a20 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:54:28.706 7ff6efe02a20 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:54:28.706 7ff6efe02a20 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:54:28.706 7ff6efe02a20 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:54:28.706 7ff6efe02a20 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:54:28.706 7ff6efe02a20 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:54:28.707 7ff6efe02a20 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:54:28.707 7ff6efe02a20 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:54:28.707 7ff6efe02a20 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:54:28.707 7ff6efe02a20 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:54:28.707 7ff6efe02a20 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:54:28.707 7ff6efe02a20 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:54:28.707 7ff6efe02a20 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:54:31.926 7ff6efe02a20 [contrail-vrouter-nodemgr] [DEBUG]: send sync_uve in the [ObjectVRouter:NodeStatusUVE] map 03/31/2025 02:54:32.705 7ff6efe02a20 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:54:37.711 7ff6efe02a20 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:54:37.712 7ff6efe02a20 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:54:37.712 7ff6efe02a20 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:54:37.712 7ff6efe02a20 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:54:37.712 7ff6efe02a20 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:54:37.712 7ff6efe02a20 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:54:37.712 7ff6efe02a20 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:54:37.712 7ff6efe02a20 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:54:37.713 7ff6efe02a20 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:54:37.713 7ff6efe02a20 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:54:37.713 7ff6efe02a20 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:54:37.713 7ff6efe02a20 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:54:37.713 7ff6efe02a20 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:54:41.715 7ff6efe17c18 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:54:46.722 7ff6efe17c18 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:54:46.722 7ff6efe17c18 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:54:46.722 7ff6efe17c18 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:54:46.722 7ff6efe17c18 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:54:46.722 7ff6efe17c18 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:54:46.722 7ff6efe17c18 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:54:46.723 7ff6efe17c18 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:54:46.723 7ff6efe17c18 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:54:46.723 7ff6efe17c18 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:54:46.723 7ff6efe17c18 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:54:46.723 7ff6efe17c18 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:54:46.723 7ff6efe17c18 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:54:46.723 7ff6efe17c18 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:54:50.723 7ff6efe29d68 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:54:51.395 7ff6f2605648 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-494-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51019524, partition_space_available_1k=30070960, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=120488, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51228, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=15027532, free=8923364, buffers=844612, cached=7869604, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=1.98, five_min_avg=2.35, fifteen_min_avg=2.73, cpu_share=2.75, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 03/31/2025 02:54:55.727 7ff6efe29d68 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:54:55.728 7ff6efe29d68 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:54:55.728 7ff6efe29d68 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:54:55.728 7ff6efe29d68 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:54:55.728 7ff6efe29d68 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:54:55.728 7ff6efe29d68 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:54:55.728 7ff6efe29d68 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51019524 partition_space_available_1k = 30070960 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-vrouter-agent : << mem_virt = 120488 cpu_share = 0 mem_res = 0 >>contrail-vrouter-nodemgr : << mem_virt = 51228 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665112 used = 15027532 free = 8923364 buffers = 844612 cached = 7869604 node_type = vrouter >> system_cpu_usage = << one_min_avg = 1.98 five_min_avg = 2.35 fifteen_min_avg = 2.73 cpu_share = 2.75 node_type = vrouter >> >> 03/31/2025 02:54:55.728 7ff6efe29d68 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:54:55.728 7ff6efe29d68 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:54:55.729 7ff6efe29d68 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:54:55.729 7ff6efe29d68 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:54:55.729 7ff6efe29d68 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:54:55.729 7ff6efe29d68 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:54:55.729 7ff6efe29d68 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:54:55.729 7ff6efe29d68 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:54:55.730 7ff6efe29d68 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:54:59.731 7ff6ee997c18 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:55:04.737 7ff6ee997c18 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:55:04.737 7ff6ee997c18 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:55:04.738 7ff6ee997c18 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:55:04.738 7ff6ee997c18 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:55:04.738 7ff6ee997c18 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:55:04.738 7ff6ee997c18 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:55:04.739 7ff6ee997c18 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:55:04.739 7ff6ee997c18 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:55:04.739 7ff6ee997c18 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:55:04.739 7ff6ee997c18 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:55:04.739 7ff6ee997c18 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:55:04.739 7ff6ee997c18 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:55:04.739 7ff6ee997c18 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:55:08.738 7ff6eeb452e8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:55:13.744 7ff6eeb452e8 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:55:13.745 7ff6eeb452e8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:55:13.745 7ff6eeb452e8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:55:13.745 7ff6eeb452e8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:55:13.745 7ff6eeb452e8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:55:13.745 7ff6eeb452e8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:55:13.746 7ff6eeb452e8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:55:13.746 7ff6eeb452e8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:55:13.746 7ff6eeb452e8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:55:13.746 7ff6eeb452e8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:55:13.746 7ff6eeb452e8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:55:13.746 7ff6eeb452e8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:55:13.746 7ff6eeb452e8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:55:17.748 7ff6eeb452e8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:55:22.754 7ff6eeb452e8 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:55:22.754 7ff6eeb452e8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:55:22.754 7ff6eeb452e8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:55:22.755 7ff6eeb452e8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:55:22.755 7ff6eeb452e8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:55:22.755 7ff6eeb452e8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:55:22.755 7ff6eeb452e8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:55:22.755 7ff6eeb452e8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:55:22.755 7ff6eeb452e8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:55:22.755 7ff6eeb452e8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:55:22.756 7ff6eeb452e8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:55:22.756 7ff6eeb452e8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:55:22.756 7ff6eeb452e8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:55:26.757 7ff6efe19eb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:55:31.763 7ff6efe19eb8 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:55:31.763 7ff6efe19eb8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:55:31.764 7ff6efe19eb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:55:31.764 7ff6efe19eb8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:55:31.764 7ff6efe19eb8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:55:31.764 7ff6efe19eb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:55:31.764 7ff6efe19eb8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:55:31.764 7ff6efe19eb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:55:31.764 7ff6efe19eb8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:55:31.765 7ff6efe19eb8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:55:31.765 7ff6efe19eb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:55:31.765 7ff6efe19eb8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:55:31.765 7ff6efe19eb8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:55:35.766 7ff6efe19eb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:55:40.769 7ff6efe19eb8 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:55:40.770 7ff6efe19eb8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:55:40.770 7ff6efe19eb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:55:40.770 7ff6efe19eb8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:55:40.770 7ff6efe19eb8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:55:40.770 7ff6efe19eb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:55:40.770 7ff6efe19eb8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:55:40.770 7ff6efe19eb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:55:40.774 7ff6efe19eb8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:55:40.774 7ff6efe19eb8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:55:40.774 7ff6efe19eb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:55:40.775 7ff6efe19eb8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:55:40.775 7ff6efe19eb8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:55:44.774 7ff6efe19eb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:55:49.777 7ff6efe19eb8 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:55:49.778 7ff6efe19eb8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:55:49.778 7ff6efe19eb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:55:49.778 7ff6efe19eb8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:55:49.778 7ff6efe19eb8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:55:49.778 7ff6efe19eb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:55:49.779 7ff6efe19eb8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:55:49.779 7ff6efe19eb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:55:49.779 7ff6efe19eb8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:55:49.779 7ff6efe19eb8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:55:49.779 7ff6efe19eb8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:55:49.779 7ff6efe19eb8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:55:49.779 7ff6efe19eb8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:55:52.094 7ff6f2605648 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-494-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51027536, partition_space_available_1k=30062948, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=120572, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51220, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=15137372, free=8791780, buffers=844620, cached=7891340, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=7.16, five_min_avg=3.45, fifteen_min_avg=3.06, cpu_share=4.0, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 03/31/2025 02:55:52.094 7ff6eeb2cf60 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:55:52.095 7ff6eeb2cf60 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51027536 partition_space_available_1k = 30062948 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-vrouter-agent : << mem_virt = 120572 cpu_share = 0 mem_res = 0 >>contrail-vrouter-nodemgr : << mem_virt = 51220 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665112 used = 15137372 free = 8791780 buffers = 844620 cached = 7891340 node_type = vrouter >> system_cpu_usage = << one_min_avg = 7.16 five_min_avg = 3.45 fifteen_min_avg = 3.06 cpu_share = 4.0 node_type = vrouter >> >> 03/31/2025 02:55:52.095 7ff6eeb2cf60 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:55:53.782 7ff6efe02ac8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:55:58.790 7ff6efe02ac8 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:55:58.790 7ff6efe02ac8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:55:58.790 7ff6efe02ac8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:55:58.790 7ff6efe02ac8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:55:58.791 7ff6efe02ac8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:55:58.791 7ff6efe02ac8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:55:58.791 7ff6efe02ac8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:55:58.791 7ff6efe02ac8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:55:58.791 7ff6efe02ac8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:55:58.791 7ff6efe02ac8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:55:58.791 7ff6efe02ac8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:55:58.792 7ff6efe02ac8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:55:58.792 7ff6efe02ac8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:56:02.792 7ff6ee98d240 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:56:07.798 7ff6ee98d240 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:56:07.799 7ff6ee98d240 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:56:07.799 7ff6ee98d240 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:56:07.799 7ff6ee98d240 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:56:07.799 7ff6ee98d240 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:56:07.799 7ff6ee98d240 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:56:07.799 7ff6ee98d240 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:56:07.800 7ff6ee98d240 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:56:07.800 7ff6ee98d240 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:56:07.800 7ff6ee98d240 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:56:07.800 7ff6ee98d240 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:56:07.800 7ff6ee98d240 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:56:07.800 7ff6ee98d240 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:56:11.801 7ff6efe326d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:56:16.805 7ff6efe326d8 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:56:16.805 7ff6efe326d8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:56:16.805 7ff6efe326d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:56:16.805 7ff6efe326d8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:56:16.805 7ff6efe326d8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:56:16.805 7ff6efe326d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:56:16.806 7ff6efe326d8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:56:16.806 7ff6efe326d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:56:16.806 7ff6efe326d8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:56:16.806 7ff6efe326d8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:56:16.806 7ff6efe326d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:56:16.806 7ff6efe326d8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:56:16.806 7ff6efe326d8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:56:20.809 7ff6efe326d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:56:25.812 7ff6efe326d8 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:56:25.812 7ff6efe326d8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:56:25.812 7ff6efe326d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:56:25.812 7ff6efe326d8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:56:25.812 7ff6efe326d8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:56:25.812 7ff6efe326d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:56:25.813 7ff6efe326d8 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:56:25.813 7ff6efe326d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:56:25.813 7ff6efe326d8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:56:25.813 7ff6efe326d8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:56:25.813 7ff6efe326d8 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:56:25.813 7ff6efe326d8 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:56:25.814 7ff6efe326d8 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:56:29.815 7ff6ee98d4e0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:56:34.821 7ff6ee98d4e0 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:56:34.821 7ff6ee98d4e0 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:56:34.821 7ff6ee98d4e0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:56:34.821 7ff6ee98d4e0 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:56:34.821 7ff6ee98d4e0 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:56:34.821 7ff6ee98d4e0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:56:34.822 7ff6ee98d4e0 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:56:34.822 7ff6ee98d4e0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:56:34.822 7ff6ee98d4e0 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:56:34.822 7ff6ee98d4e0 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:56:34.822 7ff6ee98d4e0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:56:34.822 7ff6ee98d4e0 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:56:34.822 7ff6ee98d4e0 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:56:37.355 7ff6ee98d4e0 [contrail-vrouter-nodemgr] [DEBUG]: send sync_uve in the [ObjectVRouter:NodeStatusUVE] map 03/31/2025 02:56:38.822 7ff6ee98d4e0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:56:43.826 7ff6ee98d4e0 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:56:43.827 7ff6ee98d4e0 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:56:43.827 7ff6ee98d4e0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:56:43.827 7ff6ee98d4e0 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:56:43.827 7ff6ee98d4e0 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:56:43.827 7ff6ee98d4e0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:56:43.828 7ff6ee98d4e0 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:56:43.828 7ff6ee98d4e0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:56:43.828 7ff6ee98d4e0 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:56:43.828 7ff6ee98d4e0 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:56:43.828 7ff6ee98d4e0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:56:43.828 7ff6ee98d4e0 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:56:43.828 7ff6ee98d4e0 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:56:47.829 7ff6f22cb0f0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:56:51.390 7ff6f2605648 [contrail-vrouter-nodemgr] [DEBUG]: DBG: event_tick_60: node_status=NodeStatus(name='cn-jenkins-deploy-platform-juju-os-494-1', deleted=None, status=None, process_status=None, process_info=None, disk_usage_info={'/dev/vda1': DiskPartitionUsageStats(partition_type='ext4', partition_name=None, partition_space_used_1k=51028452, partition_space_available_1k=30062032, percentage_partition_space_used=63, _LinuxSysData__key='/dev/vda1')}, description=None, all_core_file_list=None, build_info=None, process_mem_cpu_usage={'contrail-vrouter-agent': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=120668, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-agent'), 'contrail-vrouter-nodemgr': ProcessCpuInfo(module_id=None, inst_id=None, mem_virt=51276, cpu_share=0, mem_res=0, _EventManager__key='contrail-vrouter-nodemgr')}, system_cpu_info=None, system_mem_usage=SysMemInfo(total=32665112, used=15069336, free=8857700, buffers=844620, cached=7893456, node_type='vrouter'), system_cpu_usage=SysCpuInfo(one_min_avg=3.67, five_min_avg=3.09, fifteen_min_avg=2.96, cpu_share=2.25, node_type='vrouter'), running_package_version=None, installed_package_version=None, _table='ObjectVRouter') 03/31/2025 02:56:52.832 7ff6f22cb0f0 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:56:52.833 7ff6f22cb0f0 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:56:52.833 7ff6f22cb0f0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:56:52.833 7ff6f22cb0f0 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:56:52.833 7ff6f22cb0f0 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:56:52.833 7ff6f22cb0f0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:56:52.833 7ff6f22cb0f0 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 disk_usage_info = { /dev/vda1 : << partition_type = ext4 partition_space_used_1k = 51028452 partition_space_available_1k = 30062032 percentage_partition_space_used = 63 >> } process_mem_cpu_usage = { contrail-vrouter-agent : << mem_virt = 120668 cpu_share = 0 mem_res = 0 >>contrail-vrouter-nodemgr : << mem_virt = 51276 cpu_share = 0 mem_res = 0 >> } system_mem_usage = << total = 32665112 used = 15069336 free = 8857700 buffers = 844620 cached = 7893456 node_type = vrouter >> system_cpu_usage = << one_min_avg = 3.67 five_min_avg = 3.09 fifteen_min_avg = 2.96 cpu_share = 2.25 node_type = vrouter >> >> 03/31/2025 02:56:52.833 7ff6f22cb0f0 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:56:52.833 7ff6f22cb0f0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:56:52.834 7ff6f22cb0f0 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:56:52.834 7ff6f22cb0f0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:56:52.834 7ff6f22cb0f0 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:56:52.834 7ff6f22cb0f0 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:56:52.834 7ff6f22cb0f0 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:56:52.834 7ff6f22cb0f0 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:56:52.834 7ff6f22cb0f0 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:56:56.835 7ff6f25d2198 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:57:01.841 7ff6f25d2198 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:57:01.842 7ff6f25d2198 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:57:01.842 7ff6f25d2198 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:57:01.842 7ff6f25d2198 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:57:01.842 7ff6f25d2198 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:57:01.842 7ff6f25d2198 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:57:01.842 7ff6f25d2198 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:57:01.842 7ff6f25d2198 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:57:01.842 7ff6f25d2198 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:57:01.843 7ff6f25d2198 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:57:01.843 7ff6f25d2198 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:57:01.843 7ff6f25d2198 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:57:01.843 7ff6f25d2198 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:57:05.846 7ff6f25d2198 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:57:10.849 7ff6f25d2198 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:57:10.850 7ff6f25d2198 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:57:10.850 7ff6f25d2198 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:57:10.850 7ff6f25d2198 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:57:10.850 7ff6f25d2198 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:57:10.850 7ff6f25d2198 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:57:10.851 7ff6f25d2198 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:57:10.851 7ff6f25d2198 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:57:10.851 7ff6f25d2198 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:57:10.851 7ff6f25d2198 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:57:10.851 7ff6f25d2198 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:57:10.851 7ff6f25d2198 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:57:10.851 7ff6f25d2198 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:57:14.853 7ff6f25d2198 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:57:19.857 7ff6f25d2198 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:57:19.857 7ff6f25d2198 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:57:19.857 7ff6f25d2198 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:57:19.858 7ff6f25d2198 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:57:19.858 7ff6f25d2198 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:57:19.858 7ff6f25d2198 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:57:19.858 7ff6f25d2198 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:57:19.858 7ff6f25d2198 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:57:19.858 7ff6f25d2198 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:57:19.858 7ff6f25d2198 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:57:19.858 7ff6f25d2198 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:57:19.859 7ff6f25d2198 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:57:19.859 7ff6f25d2198 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:57:23.862 7ff6f25d2198 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:57:28.867 7ff6f25d2198 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:57:28.867 7ff6f25d2198 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:57:28.867 7ff6f25d2198 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:57:28.868 7ff6f25d2198 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:57:28.868 7ff6f25d2198 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:57:28.868 7ff6f25d2198 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:57:28.868 7ff6f25d2198 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:57:28.868 7ff6f25d2198 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:57:28.868 7ff6f25d2198 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:57:28.868 7ff6f25d2198 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:57:28.868 7ff6f25d2198 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:57:28.869 7ff6f25d2198 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:57:28.869 7ff6f25d2198 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:57:32.254 7ff6f25d2198 [contrail-vrouter-nodemgr] [DEBUG]: send sync_uve in the [ObjectVRouter:NodeStatusUVE] map 03/31/2025 02:57:32.868 7ff6f25d2198 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 03/31/2025 02:57:37.873 7ff6f25d2198 [contrail-vrouter-nodemgr] [ERROR]: Session Event: TCP Connect Fail 03/31/2025 02:57:37.873 7ff6f25d2198 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 03/31/2025 02:57:37.873 7ff6f25d2198 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:57:37.873 7ff6f25d2198 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 03/31/2025 02:57:37.874 7ff6f25d2198 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 03/31/2025 02:57:37.874 7ff6f25d2198 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 03/31/2025 02:57:37.874 7ff6f25d2198 [contrail-vrouter-nodemgr] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 03/31/2025 02:57:37.874 7ff6f25d2198 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:57:37.874 7ff6f25d2198 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-os-494-1 process_status = [ << module_id = contrail-vrouter-nodemgr instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.17:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 03/31/2025 02:57:37.874 7ff6f25d2198 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:57:37.874 7ff6f25d2198 [contrail-vrouter-nodemgr] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 03/31/2025 02:57:37.874 7ff6f25d2198 [contrail-vrouter-nodemgr] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-os-494-1:Compute:contrail-vrouter-nodemgr:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8102 start_time = 1743387587890833 collector_name = collector_ip = 10.0.0.17:8086 collector_list = [ 10.0.0.17:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 03/31/2025 02:57:37.875 7ff6f25d2198 [contrail-vrouter-nodemgr] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]