2026-03-19 Thu 05:36:02:748.247 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: http-server-port 8091 2026-03-19 Thu 05:36:02:748.365 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: Max-tasks 16 2026-03-19 Thu 05:36:02:748.374 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: Max-slice 100 2026-03-19 Thu 05:36:02:748.380 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: Collectors 10.0.0.38:8086 2026-03-19 Thu 05:36:02:748.385 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: Collectors 10.0.0.50:8086 2026-03-19 Thu 05:36:02:748.390 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: Collectors 10.0.0.33:8086 2026-03-19 Thu 05:36:02:748.414 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: SANDESH: ROLE : Generator 2026-03-19 Thu 05:36:02:748.420 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: SANDESH: MODULE : contrail-query-engine 2026-03-19 Thu 05:36:02:748.426 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: SANDESH: SOURCE : cn-jenkins-deploy-platform-ansible-os-5495-1. 2026-03-19 Thu 05:36:02:748.431 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: SANDESH: NODE TYPE : Database 2026-03-19 Thu 05:36:02:748.436 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: SANDESH: INSTANCE ID : 0 2026-03-19 Thu 05:36:02:748.441 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: SANDESH: HTTP SERVER PORT : 8091 2026-03-19 Thu 05:36:02:748.447 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: SANDESH: System Log Send Rate Limit: 0 -> 0 2026-03-19 Thu 05:36:02:749.104 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: HTTP Introspect Init 2026-03-19 Thu 05:36:02:749.221 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: TCP [SYS_DEBUG]: TcpServerMessageLog: Server 0.0.0.0:8091 Initialization complete src/contrail-common/io/tcp_server.cc 137 2026-03-19 Thu 05:36:02:749.287 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: Sandesh Http Server Port: 8091 2026-03-19 Thu 05:36:02:749.329 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: SANDESH: NOT Writing http_port 8091TO : /tmp/contrail-query-engine.0.http_port 2026-03-19 Thu 05:36:02:749.339 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: SANDESH: CONNECT TO COLLECTOR: 1 2026-03-19 Thu 05:36:02:749.419 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: Idle 2026-03-19 Thu 05:36:02:749.588 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: SANDESH: No Client: 1773898562749509 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1.:Database:contrail-query-engine:0 client_info= [ status = Idle successful_connections = 0 pid = 1 http_port = 8091 start_time = 1773898562749429 collector_name = collector_ip = 0.0.0.0:0 collector_list= [ [ ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 0, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 0, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-03-19 Thu 05:36:02:749.799 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: SANDESH: Logging: DISABLED -> ENABLED 2026-03-19 Thu 05:36:02:749.819 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: SANDESH: Logging: LEVEL: [ INVALID ] -> [ SYS_DEBUG ] log4level: [ NOTSET ] -> [ DEBUG ] 2026-03-19 Thu 05:36:02:749.833 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: Cassandra Servers: 10.0.0.38:9042 10.0.0.50:9042 10.0.0.33:9042 2026-03-19 Thu 05:36:02:749.945 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642059306752, Pid 1]: Processing scm::EvStart in state Idle 2026-03-19 Thu 05:36:02:750.043 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642059306752, Pid 1]: Disconnect 2026-03-19 Thu 05:36:02:750.197 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642059306752, Pid 1]: Processing scm::EvCollectorUpdate in state Disconnect 2026-03-19 Thu 05:36:02:752.488 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642059306752, Pid 1]: Connect : Start Connect timer 10.0.0.38:8086 2026-03-19 Thu 05:36:02:752.630 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642059306752, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-03-19 Thu 05:36:02:752.663 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642059306752, Pid 1]: SANDESH: Send FAILED: 1773898562750031 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Idle : EvStart -> Disconnect ], ] ] flag_infos= [ [ ] ] description = Number of connections:1, Expected:3 Missing: Database,Redis-Query:Query ], ] ] ] 2026-03-19 Thu 05:36:02:752.678 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642059306752, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-03-19 Thu 05:36:02:766.090 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642059306752, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-03-19 Thu 05:36:02:766.692 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642059306752, Pid 1]: SANDESH: Send FAILED: 1773898562750135 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1.:Database:contrail-query-engine:0 client_info= [ status = Disconnect successful_connections = 0 pid = 1 http_port = 8091 start_time = 1773898562749429 collector_name = collector_ip = 0.0.0.0:0 collector_list= [ [ ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 0, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-03-19 Thu 05:36:02:766.759 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642059306752, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-03-19 Thu 05:36:02:766.784 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642059306752, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-03-19 Thu 05:36:02:766.933 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642059306752, Pid 1]: SANDESH: Send FAILED: 1773898562750322 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1.:Database:contrail-query-engine:0 client_info= [ status = Disconnect successful_connections = 0 pid = 1 http_port = 8091 start_time = 1773898562749429 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 0, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-03-19 Thu 05:36:02:766.954 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642059306752, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-03-19 Thu 05:36:02:766.981 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642059306752, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-03-19 Thu 05:36:02:767.000 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642059306752, Pid 1]: SANDESH: Send FAILED: 1773898562750356 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Disconnect : EvCollectorUpdate -> Connect ], ] ] flag_infos= [ [ ] ] description = Number of connections:1, Expected:3 Missing: Database,Redis-Query:Query ], ] ] ] 2026-03-19 Thu 05:36:02:767.007 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642059306752, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-03-19 Thu 05:36:02:767.018 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642059306752, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-03-19 Thu 05:36:02:767.095 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642059306752, Pid 1]: SANDESH: Send FAILED: 1773898562752607 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1.:Database:contrail-query-engine:0 client_info= [ status = Connect successful_connections = 0 pid = 1 http_port = 8091 start_time = 1773898562749429 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 0, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-03-19 Thu 05:36:02:767.123 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642059306752, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-03-19 Thu 05:36:02:779.330 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: Database initialization failed src/contrail-analytics/contrail-query-engine/query.cc 997 2026-03-19 Thu 05:36:02:779.367 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: Older table exists. will query both the tables src/contrail-analytics/contrail-query-engine/query.cc 1043 2026-03-19 Thu 05:36:02:779.440 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: QeInit [SYS_WARN]: QELog: initialization of database failed. retrying 0 time src/contrail-analytics/contrail-query-engine/query.cc 1065 2026-03-19 Thu 05:36:02:779.495 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642055108352, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-03-19 Thu 05:36:02:779.527 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642055108352, Pid 1]: SANDESH: Send FAILED: 1773898562779408 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Disconnect : EvCollectorUpdate -> Connect ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Number of connections:2, Expected:3 Missing: Redis-Query:Query ], ] ] ] 2026-03-19 Thu 05:36:02:779.535 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642055108352, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-03-19 Thu 05:36:07:793.849 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: Database initialization failed src/contrail-analytics/contrail-query-engine/query.cc 997 2026-03-19 Thu 05:36:07:793.906 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: Older table exists. will query both the tables src/contrail-analytics/contrail-query-engine/query.cc 1043 2026-03-19 Thu 05:36:07:793.950 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: QeInit [SYS_WARN]: QELog: initialization of database failed. retrying 1 time src/contrail-analytics/contrail-query-engine/query.cc 1065 2026-03-19 Thu 05:36:12:796.850 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: Database initialization failed src/contrail-analytics/contrail-query-engine/query.cc 997 2026-03-19 Thu 05:36:12:796.898 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: Older table exists. will query both the tables src/contrail-analytics/contrail-query-engine/query.cc 1043 2026-03-19 Thu 05:36:12:796.922 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: QeInit [SYS_WARN]: QELog: initialization of database failed. retrying 2 time src/contrail-analytics/contrail-query-engine/query.cc 1065 2026-03-19 Thu 05:36:17:809.871 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: Database initialization failed src/contrail-analytics/contrail-query-engine/query.cc 997 2026-03-19 Thu 05:36:17:809.933 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: Older table exists. will query both the tables src/contrail-analytics/contrail-query-engine/query.cc 1043 2026-03-19 Thu 05:36:17:809.965 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: QeInit [SYS_WARN]: QELog: initialization of database failed. retrying 3 time src/contrail-analytics/contrail-query-engine/query.cc 1065 2026-03-19 Thu 05:36:22:839.813 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: : Create/Set KEYSPACE: ContrailAnalyticsCql FAILED src/contrail-analytics/contrail-query-engine/query.cc 1004 2026-03-19 Thu 05:36:22:839.865 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: Older table exists. will query both the tables src/contrail-analytics/contrail-query-engine/query.cc 1043 2026-03-19 Thu 05:36:22:839.887 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: QeInit [SYS_WARN]: QELog: initialization of database failed. retrying 4 time src/contrail-analytics/contrail-query-engine/query.cc 1065 2026-03-19 Thu 05:36:27:918.622 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: : Create/Set KEYSPACE: ContrailAnalyticsCql FAILED src/contrail-analytics/contrail-query-engine/query.cc 1004 2026-03-19 Thu 05:36:27:918.669 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: Older table exists. will query both the tables src/contrail-analytics/contrail-query-engine/query.cc 1043 2026-03-19 Thu 05:36:27:918.702 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: QeInit [SYS_WARN]: QELog: initialization of database failed. retrying 5 time src/contrail-analytics/contrail-query-engine/query.cc 1065 2026-03-19 Thu 05:36:32:965.271 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: : Create/Set KEYSPACE: ContrailAnalyticsCql FAILED src/contrail-analytics/contrail-query-engine/query.cc 1004 2026-03-19 Thu 05:36:32:965.336 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: Older table exists. will query both the tables src/contrail-analytics/contrail-query-engine/query.cc 1043 2026-03-19 Thu 05:36:32:965.360 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: QeInit [SYS_WARN]: QELog: initialization of database failed. retrying 6 time src/contrail-analytics/contrail-query-engine/query.cc 1065 2026-03-19 Thu 05:36:38:037.943 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: Older table exists. will query both the tables src/contrail-analytics/contrail-query-engine/query.cc 1043 2026-03-19 Thu 05:36:38:038.027 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: QeInit [SYS_WARN]: QELog: initialization of database failed. retrying 7 time src/contrail-analytics/contrail-query-engine/query.cc 1065 2026-03-19 Thu 05:36:43:063.086 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: Older table does not exist. will query only the new table src/contrail-analytics/contrail-query-engine/query.cc 1037 2026-03-19 Thu 05:36:48:068.669 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642025719552, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-03-19 Thu 05:36:48:068.737 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: 0 0x1f215f0 2026-03-19 Thu 05:36:48:068.779 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: Connected to REDIS... 2026-03-19 Thu 05:36:48:068.806 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 0 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-03-19 Thu 05:36:48:068.850 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: -(-1) OnSessionEvent TCP Connect Failed 2026-03-19 Thu 05:36:48:068.860 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: -(-1):EvTcpConnectFail 2026-03-19 Thu 05:36:48:068.878 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: 0 0x1f21dd0 2026-03-19 Thu 05:36:48:068.887 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: Connected to REDIS... 2026-03-19 Thu 05:36:48:068.901 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 1 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-03-19 Thu 05:36:48:068.915 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: 0 0x1f1b1d0 2026-03-19 Thu 05:36:48:068.923 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: Connected to REDIS... 2026-03-19 Thu 05:36:48:068.936 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 2 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-03-19 Thu 05:36:48:068.950 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: 0 0x1f22360 2026-03-19 Thu 05:36:48:068.956 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: Connected to REDIS... 2026-03-19 Thu 05:36:48:068.965 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 3 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-03-19 Thu 05:36:48:068.975 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: 0 0x1f22970 2026-03-19 Thu 05:36:48:068.980 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: Connected to REDIS... 2026-03-19 Thu 05:36:48:068.989 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 4 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-03-19 Thu 05:36:48:068.999 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: 0 0x1f23280 2026-03-19 Thu 05:36:48:069.004 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: Connected to REDIS... 2026-03-19 Thu 05:36:48:069.013 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 0 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-03-19 Thu 05:36:48:069.034 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: 0 0x1f23bc0 2026-03-19 Thu 05:36:48:069.040 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: Connected to REDIS... 2026-03-19 Thu 05:36:48:069.049 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 1 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-03-19 Thu 05:36:48:069.058 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: 0 0x1f244c0 2026-03-19 Thu 05:36:48:069.063 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: Connected to REDIS... 2026-03-19 Thu 05:36:48:069.072 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 2 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-03-19 Thu 05:36:48:069.081 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: 0 0x1f24e00 2026-03-19 Thu 05:36:48:069.087 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: Connected to REDIS... 2026-03-19 Thu 05:36:48:069.096 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 3 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-03-19 Thu 05:36:48:069.107 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: 0 0x1f26080 2026-03-19 Thu 05:36:48:069.113 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: Connected to REDIS... 2026-03-19 Thu 05:36:48:069.122 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 0 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-03-19 Thu 05:36:48:069.132 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: 0 0x1f269c0 2026-03-19 Thu 05:36:48:069.138 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: Connected to REDIS... 2026-03-19 Thu 05:36:48:069.146 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 1 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-03-19 Thu 05:36:48:069.155 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: 0 0x1f25740 2026-03-19 Thu 05:36:48:069.161 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: Connected to REDIS... 2026-03-19 Thu 05:36:48:069.169 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 4 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-03-19 Thu 05:36:48:069.178 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: 0 0x1f27c40 2026-03-19 Thu 05:36:48:069.184 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: Connected to REDIS... 2026-03-19 Thu 05:36:48:069.191 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 3 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-03-19 Thu 05:36:48:069.201 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: 0 0x1f28580 2026-03-19 Thu 05:36:48:069.206 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: Connected to REDIS... 2026-03-19 Thu 05:36:48:069.214 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 4 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-03-19 Thu 05:36:48:069.227 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: 0 0x1f27300 2026-03-19 Thu 05:36:48:069.232 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: Connected to REDIS... 2026-03-19 Thu 05:36:48:069.240 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 2 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-03-19 Thu 05:36:48:069.499 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-03-19 Thu 05:36:48:069.518 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-03-19 Thu 05:36:48:069.534 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-03-19 Thu 05:36:48:069.544 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-03-19 Thu 05:36:48:069.595 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB0 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-03-19 Thu 05:36:48:069.624 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB4 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-03-19 Thu 05:36:48:069.640 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-03-19 Thu 05:36:48:069.650 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-03-19 Thu 05:36:48:069.663 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-03-19 Thu 05:36:48:069.673 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-03-19 Thu 05:36:48:069.686 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-03-19 Thu 05:36:48:069.704 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-03-19 Thu 05:36:48:069.724 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB3 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-03-19 Thu 05:36:48:069.767 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB2 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-03-19 Thu 05:36:48:069.796 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB1 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-03-19 Thu 05:36:48:070.062 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-03-19 Thu 05:36:48:070.103 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-03-19 Thu 05:36:48:070.120 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-03-19 Thu 05:36:48:070.130 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-03-19 Thu 05:36:48:070.143 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-03-19 Thu 05:36:48:070.153 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-03-19 Thu 05:36:48:070.167 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-03-19 Thu 05:36:48:070.176 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-03-19 Thu 05:36:48:070.189 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-03-19 Thu 05:36:48:070.198 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-03-19 Thu 05:36:48:070.211 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-03-19 Thu 05:36:48:070.220 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-03-19 Thu 05:36:48:070.233 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-03-19 Thu 05:36:48:070.243 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-03-19 Thu 05:36:48:070.255 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-03-19 Thu 05:36:48:070.264 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-03-19 Thu 05:36:48:070.276 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-03-19 Thu 05:36:48:070.294 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-03-19 Thu 05:36:48:070.308 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-03-19 Thu 05:36:48:070.318 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-03-19 Thu 05:36:48:070.344 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB1 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-03-19 Thu 05:36:48:068.757 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642025719552, Pid 1]: SANDESH: Send FAILED: 1773898608068572 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Disconnect : EvCollectorUpdate -> Connect ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], ] ] flag_infos= [ [ ] ] description = Number of connections:2, Expected:3 Missing: Redis-Query:Query ], ] ] ] 2026-03-19 Thu 05:36:48:070.386 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642025719552, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-03-19 Thu 05:36:48:070.393 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB2 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-03-19 Thu 05:36:48:070.405 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642025719552, Pid 1]: Processing scm::EvTcpConnectFail in state Connect 2026-03-19 Thu 05:36:48:070.415 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642025719552, Pid 1]: Connect : EvTcpConnectFail 2026-03-19 Thu 05:36:48:070.417 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB0 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-03-19 Thu 05:36:48:070.508 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB1 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-03-19 Thu 05:36:48:070.557 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642025719552, Pid 1]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 312 2026-03-19 Thu 05:36:48:070.575 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB0 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-03-19 Thu 05:36:48:070.593 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB4 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-03-19 Thu 05:36:48:070.610 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642025719552, Pid 1]: Idle 2026-03-19 Thu 05:36:48:070.615 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB3 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-03-19 Thu 05:36:48:070.657 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB2 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-03-19 Thu 05:36:48:070.708 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642025719552, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-03-19 Thu 05:36:48:070.712 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB3 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-03-19 Thu 05:36:48:070.742 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642025719552, Pid 1]: SANDESH: Send FAILED: 1773898608069582 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Disconnect : EvCollectorUpdate -> Connect ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-03-19 Thu 05:36:48:070.765 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642025719552, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-03-19 Thu 05:36:48:070.774 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB4 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-03-19 Thu 05:36:48:070.777 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642025719552, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-03-19 Thu 05:36:48:070.830 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642025719552, Pid 1]: SANDESH: Send FAILED: 1773898608069619 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Disconnect : EvCollectorUpdate -> Connect ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = 10.0.0.50:6379::Down.10.0.0.33:6379::Down. ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-03-19 Thu 05:36:48:070.837 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642025719552, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-03-19 Thu 05:36:48:070.846 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642025719552, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-03-19 Thu 05:36:48:070.871 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642025719552, Pid 1]: SANDESH: Send FAILED: 1773898608070381 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Disconnect : EvCollectorUpdate -> Connect ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = 10.0.0.33:6379::Down. ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-03-19 Thu 05:36:48:070.883 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642025719552, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-03-19 Thu 05:36:48:070.892 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642025719552, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-03-19 Thu 05:36:48:070.996 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642025719552, Pid 1]: SANDESH: Send FAILED: 1773898608070504 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1.:Database:contrail-query-engine:0 client_info= [ status = Connect successful_connections = 0 pid = 1 http_port = 8091 start_time = 1773898562749429 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 77 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 78, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 7, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-03-19 Thu 05:36:48:071.013 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642025719552, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-03-19 Thu 05:36:48:071.023 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642025719552, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-03-19 Thu 05:36:48:071.042 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642025719552, Pid 1]: SANDESH: Send FAILED: 1773898608070528 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = 10.0.0.33:6379::Down. ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-03-19 Thu 05:36:48:071.049 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642025719552, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-03-19 Thu 05:36:48:071.057 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642025719552, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-03-19 Thu 05:36:48:071.075 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642025719552, Pid 1]: SANDESH: Send FAILED: 1773898608070559 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-03-19 Thu 05:36:48:071.081 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642025719552, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-03-19 Thu 05:36:48:071.089 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642025719552, Pid 1]: Processing scm::EvTcpDeleteSession in state Idle 2026-03-19 Thu 05:36:48:071.133 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642025719552, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-03-19 Thu 05:36:48:071.235 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642025719552, Pid 1]: SANDESH: Send FAILED: 1773898608070689 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1.:Database:contrail-query-engine:0 client_info= [ status = Idle successful_connections = 0 pid = 1 http_port = 8091 start_time = 1773898562749429 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 82 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 84, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 7, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-03-19 Thu 05:36:48:071.255 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642025719552, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-03-19 Thu 05:36:48:071.345 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642025719552, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-03-19 Thu 05:36:48:071.389 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642025719552, Pid 1]: SANDESH: Send FAILED: 1773898608071281 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1.:Database:contrail-query-engine:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 14 dequeues = 14 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 18 dequeues = 18 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1773898608070608 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1773898608071131 ] max_sm_queue_count = 8 ] 2026-03-19 Thu 05:36:48:071.397 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642025719552, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-03-19 Thu 05:36:48:071.405 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642025719552, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-03-19 Thu 05:36:48:071.500 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642025719552, Pid 1]: SANDESH: Send FAILED: 1773898608071333 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1.:Database:contrail-query-engine:0 client_info= [ status = Idle successful_connections = 0 pid = 1 http_port = 8091 start_time = 1773898562749429 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 84 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 5 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 86, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 14, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-03-19 Thu 05:36:48:071.515 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642025719552, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-03-19 Thu 05:36:49:070.765 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642034116352, Pid 1]: Processing scm::EvIdleHoldTimerExpired in state Idle 2026-03-19 Thu 05:36:49:071.020 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642034116352, Pid 1]: Connect : Start Connect timer 10.0.0.33:8086 2026-03-19 Thu 05:36:49:071.147 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642034116352, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-03-19 Thu 05:36:49:071.179 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642034116352, Pid 1]: SANDESH: Send FAILED: 1773898609070852 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-03-19 Thu 05:36:49:071.198 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642034116352, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-03-19 Thu 05:36:49:071.207 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642034116352, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-03-19 Thu 05:36:49:071.328 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642034116352, Pid 1]: SANDESH: Send FAILED: 1773898609071133 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1.:Database:contrail-query-engine:0 client_info= [ status = Connect successful_connections = 0 pid = 1 http_port = 8091 start_time = 1773898562749429 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 84 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 86, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 16, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-03-19 Thu 05:36:49:071.343 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642034116352, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-03-19 Thu 05:36:49:074.236 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.38:33954::10.0.0.33:8086(-1) < Active session connection complete src/contrail-common/io/tcp_session.cc 280 2026-03-19 Thu 05:36:49:074.281 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: 10.0.0.38:33954::10.0.0.33:8086(-1) OnSessionEvent TCP Connected 2026-03-19 Thu 05:36:49:074.294 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: 10.0.0.38:33954::10.0.0.33:8086(-1):EvTcpConnected 2026-03-19 Thu 05:36:49:074.367 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642025719552, Pid 1]: Processing scm::EvTcpConnected in state Connect 2026-03-19 Thu 05:36:49:074.394 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642025719552, Pid 1]: Connect : EvTcpConnected : Cancelling Connect timer 2026-03-19 Thu 05:36:49:074.434 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642025719552, Pid 1]: ClientInit 2026-03-19 Thu 05:36:49:074.458 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642025719552, Pid 1]: Sending Ctrl Message for cn-jenkins-deploy-platform-ansible-os-5495-1.:contrail-query-engine:0:Database count 1 2026-03-19 Thu 05:36:49:074.544 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642025719552, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-03-19 Thu 05:36:49:074.553 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642025719552, Pid 1]: ClientInit : EvSandeshSend : NodeStatusUVE 2026-03-19 Thu 05:36:49:074.578 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642025719552, Pid 1]: SANDESH: Send FAILED: 1773898609074426 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-03-19 Thu 05:36:49:074.594 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642025719552, Pid 1]: Received UVE message in wrong state : NodeStatusUVE 2026-03-19 Thu 05:36:49:074.603 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642025719552, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-03-19 Thu 05:36:49:074.610 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642025719552, Pid 1]: ClientInit : EvSandeshSend : SandeshCtrlClientToServer 2026-03-19 Thu 05:36:49:074.622 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642025719552, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-03-19 Thu 05:36:49:074.628 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642025719552, Pid 1]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2026-03-19 Thu 05:36:49:074.740 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642025719552, Pid 1]: SANDESH: Send FAILED: 1773898609074534 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1.:Database:contrail-query-engine:0 client_info= [ status = ClientInit successful_connections = 1 pid = 1 http_port = 8091 start_time = 1773898562749429 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 84 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 87, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 18, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-03-19 Thu 05:36:49:074.755 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642025719552, Pid 1]: Received UVE message in wrong state : SandeshModuleClientTrace 2026-03-19 Thu 05:36:49:079.769 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642059306752, Pid 1]: OnMessage control in state: ClientInit 2026-03-19 Thu 05:36:49:079.864 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642059306752, Pid 1]: Processing scm::EvSandeshMessageRecv in state ClientInit 2026-03-19 Thu 05:36:49:079.876 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642059306752, Pid 1]: ClientInit : EvSandeshMessageRecv 2026-03-19 Thu 05:36:49:079.902 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642059306752, Pid 1]: Received Ctrl Message with size 0 2026-03-19 Thu 05:36:49:080.092 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642059306752, Pid 1]: SyncUVE Syncing ObjectGeneratorInfo val name = cn-jenkins-deploy-platform-ansible-os-5495-1.:Database:contrail-query-engine:0 client_info= [ status = ClientInit successful_connections = 1 pid = 1 http_port = 8091 start_time = 1773898562749429 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 14 dequeues = 14 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 18 dequeues = 18 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1773898608070608 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1773898608071131 ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 84 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 87, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 18, _iter59->first = sent _iter59->second = 0, ] ] max_sm_queue_count = 8 proxy seq 10 2026-03-19 Thu 05:36:49:080.136 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642059306752, Pid 1]: SyncAllMaps for ModuleClientState:0 period 0 cycle 0 without seqno , total = 1 2026-03-19 Thu 05:36:49:080.174 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642059306752, Pid 1]: SyncUVE Syncing ObjectDatabaseInfo val name = cn-jenkins-deploy-platform-ansible-os-5495-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] proxy seq 11 2026-03-19 Thu 05:36:49:080.197 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642059306752, Pid 1]: SyncAllMaps for NodeStatus:0 period 0 cycle 0 without seqno , total = 1 2026-03-19 Thu 05:36:49:080.208 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642059306752, Pid 1]: SyncAllMaps for QEDbStats:0 period 0 cycle 0 without seqno , total = 0 2026-03-19 Thu 05:36:49:080.219 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642059306752, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:0 period 60 cycle 0 without seqno , total = 0 2026-03-19 Thu 05:36:49:080.270 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642059306752, Pid 1]: Established 2026-03-19 Thu 05:36:57:424.377 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642038314752, Pid 1]: OnMessage control in state: Established 2026-03-19 Thu 05:36:57:424.463 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642038314752, Pid 1]: Processing scm::EvSandeshMessageRecv in state Established 2026-03-19 Thu 05:36:57:424.476 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642038314752, Pid 1]: Established : EvSandeshMessageRecv 2026-03-19 Thu 05:36:57:424.520 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642038314752, Pid 1]: Received Ctrl Message with size 3 2026-03-19 Thu 05:36:57:424.535 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642038314752, Pid 1]: SyncAllMaps for ModuleClientState:0 period 0 cycle 0 with seqno 11, total = 0 2026-03-19 Thu 05:36:57:424.547 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642038314752, Pid 1]: SyncAllMaps for NodeStatus:0 period 0 cycle 0 with seqno 13, total = 0 2026-03-19 Thu 05:36:57:424.560 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642038314752, Pid 1]: SyncAllMaps for QEDbStats:0 period 0 cycle 0 with seqno 1, total = 0 2026-03-19 Thu 05:36:57:424.568 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642038314752, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:0 period 60 cycle 0 without seqno , total = 0 2026-03-19 Thu 05:37:18:071.642 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642038314752, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 1 without seqno , total = 0 2026-03-19 Thu 05:38:18:072.265 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642038314752, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 2 without seqno , total = 0 2026-03-19 Thu 05:39:18:073.949 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642050909952, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 3 without seqno , total = 0 2026-03-19 Thu 05:40:18:074.535 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642050909952, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 4 without seqno , total = 0 2026-03-19 Thu 05:40:51:648.910 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.38:8091::10.0.0.38:49760 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 254 2026-03-19 Thu 05:41:15:314.405 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.38:8091::10.0.0.38:58200 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 254 2026-03-19 Thu 05:41:18:075.233 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140641999865600, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 5 without seqno , total = 0 2026-03-19 Thu 05:42:18:076.342 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642050909952, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 6 without seqno , total = 0 2026-03-19 Thu 05:43:18:076.970 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642017322752, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 7 without seqno , total = 0 2026-03-19 Thu 05:43:24:985.340 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Disconnect status: -1 0x1f269c0 2026-03-19 Thu 05:43:24:985.423 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..1 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-03-19 Thu 05:43:24:985.485 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Disconnect status: -1 0x1f28580 2026-03-19 Thu 05:43:24:985.503 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..4 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-03-19 Thu 05:43:24:985.535 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Disconnect status: -1 0x1f27300 2026-03-19 Thu 05:43:24:985.555 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..2 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-03-19 Thu 05:43:24:985.585 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Disconnect status: -1 0x1f27c40 2026-03-19 Thu 05:43:24:985.602 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..3 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-03-19 Thu 05:43:24:985.895 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: CallbackProcess: received NULL reply from redis src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1049 2026-03-19 Thu 05:43:24:985.912 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Disconnect status: -1 0x1f26080 2026-03-19 Thu 05:43:24:985.926 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..0 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-03-19 Thu 05:43:24:986.251 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f269c0 2026-03-19 Thu 05:43:24:986.340 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f28580 2026-03-19 Thu 05:43:24:986.370 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f27300 2026-03-19 Thu 05:43:24:986.389 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f27c40 2026-03-19 Thu 05:43:24:986.410 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f26080 2026-03-19 Thu 05:43:25:087.300 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Disconnect status: -1 0x1f25740 2026-03-19 Thu 05:43:25:087.395 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..4 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-03-19 Thu 05:43:25:087.731 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Disconnect status: -1 0x1f24e00 2026-03-19 Thu 05:43:25:087.751 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..3 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-03-19 Thu 05:43:25:087.972 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: CallbackProcess: received NULL reply from redis src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1049 2026-03-19 Thu 05:43:25:087.987 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Disconnect status: -1 0x1f23280 2026-03-19 Thu 05:43:25:088.001 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..0 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-03-19 Thu 05:43:25:088.039 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Disconnect status: -1 0x1f23bc0 2026-03-19 Thu 05:43:25:088.055 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..1 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-03-19 Thu 05:43:25:088.079 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Disconnect status: -1 0x1f244c0 2026-03-19 Thu 05:43:25:088.090 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..2 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-03-19 Thu 05:43:25:088.320 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f25740 2026-03-19 Thu 05:43:25:088.351 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f24e00 2026-03-19 Thu 05:43:25:089.480 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f23bc0 2026-03-19 Thu 05:43:25:089.560 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f23280 2026-03-19 Thu 05:43:25:089.587 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f244c0 2026-03-19 Thu 05:43:29:986.342 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f269c0 2026-03-19 Thu 05:43:29:986.527 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f28580 2026-03-19 Thu 05:43:29:986.576 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f27300 2026-03-19 Thu 05:43:29:986.607 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f27c40 2026-03-19 Thu 05:43:29:986.638 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f26080 2026-03-19 Thu 05:43:29:987.138 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f269c0 2026-03-19 Thu 05:43:29:987.222 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f27300 2026-03-19 Thu 05:43:29:987.238 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f28580 2026-03-19 Thu 05:43:29:987.252 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f26080 2026-03-19 Thu 05:43:29:987.280 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f27c40 2026-03-19 Thu 05:43:30:088.444 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f25740 2026-03-19 Thu 05:43:30:088.687 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f24e00 2026-03-19 Thu 05:43:30:089.531 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f25740 2026-03-19 Thu 05:43:30:089.611 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f24e00 2026-03-19 Thu 05:43:30:089.636 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f23bc0 2026-03-19 Thu 05:43:30:089.702 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f23280 2026-03-19 Thu 05:43:30:089.736 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f244c0 2026-03-19 Thu 05:43:30:090.069 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f23bc0 2026-03-19 Thu 05:43:30:090.199 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f23280 2026-03-19 Thu 05:43:30:090.222 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f244c0 2026-03-19 Thu 05:43:34:957.876 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: TCP [SYS_ERR]: TcpSessionMessageLog: Session 10.0.0.38:33954::10.0.0.33:8086(-1) < Read failed due to error system 104 : Connection reset by peer src/contrail-common/io/tcp_session.cc 507 2026-03-19 Thu 05:43:34:957.923 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: TCP [SYS_ERR]: TcpSessionMessageLog: Session 10.0.0.38:33954::10.0.0.33:8086(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 312 2026-03-19 Thu 05:43:34:957.946 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: 10.0.0.38:33954::10.0.0.33:8086(-1) OnSessionEvent TCP Connection Closed 2026-03-19 Thu 05:43:34:957.955 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: 10.0.0.38:33954::10.0.0.33:8086(-1):EvTcpClose 2026-03-19 Thu 05:43:34:958.016 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642017322752, Pid 1]: Processing scm::EvTcpClose in state Established 2026-03-19 Thu 05:43:34:958.027 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642017322752, Pid 1]: Established : EvTcpClose 2026-03-19 Thu 05:43:34:958.268 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642017322752, Pid 1]: Connect : Start Connect timer 10.0.0.50:8086 2026-03-19 Thu 05:43:34:958.364 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642017322752, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-03-19 Thu 05:43:34:958.530 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642017322752, Pid 1]: SANDESH: Send FAILED: 1773899014958116 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1.:Database:contrail-query-engine:0 client_info= [ status = Established successful_connections = 1 pid = 1 http_port = 8091 start_time = 1773898562749429 collector_name = cn-jenkins-deploy-platform-ansible-os-5495-3. collector_ip = 10.0.0.50:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 2447 calls = 4 average_bytes = 611 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 359669 calls = 40 average_bytes = 8991 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 11 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 7 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 96 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 1 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 28 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 103, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 20, _iter59->first = sent _iter59->second = 41, ] ] ] 2026-03-19 Thu 05:43:34:958.553 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642017322752, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-03-19 Thu 05:43:34:958.564 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642017322752, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-03-19 Thu 05:43:34:958.588 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642017322752, Pid 1]: SANDESH: Send FAILED: 1773899014958146 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Initializing description = Established : EvTcpClose -> Connect ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = 10.0.0.50:6379::Down.10.0.0.33:6379::Down. ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-03-19 Thu 05:43:34:958.595 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642017322752, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-03-19 Thu 05:43:34:958.605 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642017322752, Pid 1]: Processing scm::EvTcpDeleteSession in state Connect 2026-03-19 Thu 05:43:34:958.655 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642017322752, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-03-19 Thu 05:43:34:958.792 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642017322752, Pid 1]: SANDESH: Send FAILED: 1773899014958345 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1.:Database:contrail-query-engine:0 client_info= [ status = Connect successful_connections = 1 pid = 1 http_port = 8091 start_time = 1773898562749429 collector_name = collector_ip = 10.0.0.50:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 2447 calls = 4 average_bytes = 611 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 359669 calls = 40 average_bytes = 8991 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 11 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 7 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 96 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 1 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 28 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 103, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 20, _iter59->first = sent _iter59->second = 41, ] ] ] 2026-03-19 Thu 05:43:34:958.813 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642017322752, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-03-19 Thu 05:43:34:962.755 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.38:39974::10.0.0.50:8086(-1) < Active session connection complete src/contrail-common/io/tcp_session.cc 280 2026-03-19 Thu 05:43:34:962.781 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: 10.0.0.38:39974::10.0.0.50:8086(-1) OnSessionEvent TCP Connected 2026-03-19 Thu 05:43:34:962.789 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: 10.0.0.38:39974::10.0.0.50:8086(-1):EvTcpConnected 2026-03-19 Thu 05:43:34:968.768 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642034116352, Pid 1]: Processing scm::EvTcpConnected in state Connect 2026-03-19 Thu 05:43:34:968.795 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642034116352, Pid 1]: Connect : EvTcpConnected : Cancelling Connect timer 2026-03-19 Thu 05:43:34:968.837 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642034116352, Pid 1]: ClientInit 2026-03-19 Thu 05:43:34:968.856 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642034116352, Pid 1]: Sending Ctrl Message for cn-jenkins-deploy-platform-ansible-os-5495-1.:contrail-query-engine:0:Database count 2 2026-03-19 Thu 05:43:34:968.952 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642034116352, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-03-19 Thu 05:43:34:968.961 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642034116352, Pid 1]: ClientInit : EvSandeshSend : NodeStatusUVE 2026-03-19 Thu 05:43:34:968.986 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642034116352, Pid 1]: SANDESH: Send FAILED: 1773899014968828 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.50:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = 10.0.0.50:6379::Down.10.0.0.33:6379::Down. ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-03-19 Thu 05:43:34:968.999 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642034116352, Pid 1]: Received UVE message in wrong state : NodeStatusUVE 2026-03-19 Thu 05:43:34:969.008 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642034116352, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-03-19 Thu 05:43:34:969.015 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642034116352, Pid 1]: ClientInit : EvSandeshSend : SandeshCtrlClientToServer 2026-03-19 Thu 05:43:34:969.027 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642034116352, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-03-19 Thu 05:43:34:969.034 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642034116352, Pid 1]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2026-03-19 Thu 05:43:34:969.213 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642034116352, Pid 1]: SANDESH: Send FAILED: 1773899014968940 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1.:Database:contrail-query-engine:0 client_info= [ status = ClientInit successful_connections = 2 pid = 1 http_port = 8091 start_time = 1773898562749429 collector_name = collector_ip = 10.0.0.50:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 2447 calls = 4 average_bytes = 611 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 359669 calls = 40 average_bytes = 8991 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 7 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 96 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 1 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 28 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 11 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 7 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 104, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 23, _iter59->first = sent _iter59->second = 41, ] ] ] 2026-03-19 Thu 05:43:34:969.233 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642034116352, Pid 1]: Received UVE message in wrong state : SandeshModuleClientTrace 2026-03-19 Thu 05:43:34:987.253 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f269c0 2026-03-19 Thu 05:43:34:987.376 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f27300 2026-03-19 Thu 05:43:34:987.438 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f28580 2026-03-19 Thu 05:43:34:987.487 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f26080 2026-03-19 Thu 05:43:34:987.531 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f27c40 2026-03-19 Thu 05:43:34:987.590 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f269c0 2026-03-19 Thu 05:43:34:987.628 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f27300 2026-03-19 Thu 05:43:34:987.772 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f28580 2026-03-19 Thu 05:43:34:987.806 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f26080 2026-03-19 Thu 05:43:34:987.828 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f27c40 2026-03-19 Thu 05:43:35:022.662 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: TCP [SYS_ERR]: TcpSessionMessageLog: Session 10.0.0.38:39974::10.0.0.50:8086(-1) < Read failed due to error system 104 : Connection reset by peer src/contrail-common/io/tcp_session.cc 507 2026-03-19 Thu 05:43:35:022.722 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: TCP [SYS_ERR]: TcpSessionMessageLog: Session 10.0.0.38:39974::10.0.0.50:8086(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 312 2026-03-19 Thu 05:43:35:022.750 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: 10.0.0.38:39974::10.0.0.50:8086(-1) OnSessionEvent TCP Connection Closed 2026-03-19 Thu 05:43:35:022.763 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: 10.0.0.38:39974::10.0.0.50:8086(-1):EvTcpClose 2026-03-19 Thu 05:43:35:033.219 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642034116352, Pid 1]: Processing scm::EvTcpClose in state ClientInit 2026-03-19 Thu 05:43:35:033.261 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642034116352, Pid 1]: ClientInit : EvTcpClose 2026-03-19 Thu 05:43:35:033.395 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642034116352, Pid 1]: Return to idle with 1000 2026-03-19 Thu 05:43:35:033.405 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642034116352, Pid 1]: Idle 2026-03-19 Thu 05:43:35:033.459 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642034116352, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-03-19 Thu 05:43:35:033.615 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642034116352, Pid 1]: SANDESH: Send FAILED: 1773899015033348 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1.:Database:contrail-query-engine:0 client_info= [ status = ClientInit successful_connections = 2 pid = 1 http_port = 8091 start_time = 1773898562749429 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 2447 calls = 4 average_bytes = 611 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 361242 calls = 41 average_bytes = 8810 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 13 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 7 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 96 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 28 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 9 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 106, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 25, _iter59->first = sent _iter59->second = 42, ] ] ] 2026-03-19 Thu 05:43:35:033.638 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642034116352, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-03-19 Thu 05:43:35:033.648 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642034116352, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-03-19 Thu 05:43:35:033.671 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642034116352, Pid 1]: SANDESH: Send FAILED: 1773899015033380 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.50:8086, ] ] status = Down description = ClientInit : EvTcpClose ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = 10.0.0.50:6379::Down.10.0.0.33:6379::Down. ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-03-19 Thu 05:43:35:033.679 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642034116352, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-03-19 Thu 05:43:35:033.688 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642034116352, Pid 1]: Processing scm::EvTcpDeleteSession in state Idle 2026-03-19 Thu 05:43:35:033.717 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642034116352, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-03-19 Thu 05:43:35:033.861 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642034116352, Pid 1]: SANDESH: Send FAILED: 1773899015033451 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1.:Database:contrail-query-engine:0 client_info= [ status = Idle successful_connections = 2 pid = 1 http_port = 8091 start_time = 1773898562749429 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 2447 calls = 4 average_bytes = 611 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 361242 calls = 41 average_bytes = 8810 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 13 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 7 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 96 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 28 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 9 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 106, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 25, _iter59->first = sent _iter59->second = 42, ] ] ] 2026-03-19 Thu 05:43:35:033.877 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642034116352, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-03-19 Thu 05:43:35:090.408 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f25740 2026-03-19 Thu 05:43:35:090.535 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f24e00 2026-03-19 Thu 05:43:35:090.595 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f23bc0 2026-03-19 Thu 05:43:35:090.640 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f23280 2026-03-19 Thu 05:43:35:090.681 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f244c0 2026-03-19 Thu 05:43:35:091.919 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f25740 2026-03-19 Thu 05:43:35:091.964 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f24e00 2026-03-19 Thu 05:43:35:091.986 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f23280 2026-03-19 Thu 05:43:35:092.004 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f23bc0 2026-03-19 Thu 05:43:35:092.025 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f244c0 2026-03-19 Thu 05:43:36:033.532 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642034116352, Pid 1]: Processing scm::EvIdleHoldTimerExpired in state Idle 2026-03-19 Thu 05:43:36:033.856 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642034116352, Pid 1]: Connect : Start Connect timer 10.0.0.38:8086 2026-03-19 Thu 05:43:36:034.090 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642034116352, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-03-19 Thu 05:43:36:034.129 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642034116352, Pid 1]: SANDESH: Send FAILED: 1773899016033634 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = 10.0.0.50:6379::Down.10.0.0.33:6379::Down. ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-03-19 Thu 05:43:36:034.139 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642034116352, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-03-19 Thu 05:43:36:034.148 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642034116352, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-03-19 Thu 05:43:36:034.329 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642034116352, Pid 1]: SANDESH: Send FAILED: 1773899016034075 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1.:Database:contrail-query-engine:0 client_info= [ status = Connect successful_connections = 2 pid = 1 http_port = 8091 start_time = 1773898562749429 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 2447 calls = 4 average_bytes = 611 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 361242 calls = 41 average_bytes = 8810 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 7 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 96 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 28 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 9 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 106, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 28, _iter59->first = sent _iter59->second = 42, ] ] ] 2026-03-19 Thu 05:43:36:034.356 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642034116352, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-03-19 Thu 05:43:36:036.792 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.38:50412::10.0.0.38:8086(-1) < Active session connection complete src/contrail-common/io/tcp_session.cc 280 2026-03-19 Thu 05:43:36:036.817 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: 10.0.0.38:50412::10.0.0.38:8086(-1) OnSessionEvent TCP Connected 2026-03-19 Thu 05:43:36:036.825 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: 10.0.0.38:50412::10.0.0.38:8086(-1):EvTcpConnected 2026-03-19 Thu 05:43:36:036.908 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642029917952, Pid 1]: Processing scm::EvTcpConnected in state Connect 2026-03-19 Thu 05:43:36:036.942 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642029917952, Pid 1]: Connect : EvTcpConnected : Cancelling Connect timer 2026-03-19 Thu 05:43:36:036.984 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642029917952, Pid 1]: ClientInit 2026-03-19 Thu 05:43:36:037.015 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642029917952, Pid 1]: Sending Ctrl Message for cn-jenkins-deploy-platform-ansible-os-5495-1.:contrail-query-engine:0:Database count 3 2026-03-19 Thu 05:43:36:037.130 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642029917952, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-03-19 Thu 05:43:36:037.146 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642029917952, Pid 1]: ClientInit : EvSandeshSend : NodeStatusUVE 2026-03-19 Thu 05:43:36:037.211 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642029917952, Pid 1]: SANDESH: Send FAILED: 1773899016036976 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = 10.0.0.50:6379::Down.10.0.0.33:6379::Down. ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-03-19 Thu 05:43:36:037.226 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642029917952, Pid 1]: Received UVE message in wrong state : NodeStatusUVE 2026-03-19 Thu 05:43:36:037.241 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642029917952, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-03-19 Thu 05:43:36:037.252 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642029917952, Pid 1]: ClientInit : EvSandeshSend : SandeshCtrlClientToServer 2026-03-19 Thu 05:43:36:037.270 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642029917952, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-03-19 Thu 05:43:36:037.280 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642029917952, Pid 1]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2026-03-19 Thu 05:43:36:037.450 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642029917952, Pid 1]: SANDESH: Send FAILED: 1773899016037115 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1.:Database:contrail-query-engine:0 client_info= [ status = ClientInit successful_connections = 3 pid = 1 http_port = 8091 start_time = 1773898562749429 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 2447 calls = 4 average_bytes = 611 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 361242 calls = 41 average_bytes = 8810 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 15 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 7 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 96 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 28 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 15 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 107, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 30, _iter59->first = sent _iter59->second = 42, ] ] ] 2026-03-19 Thu 05:43:36:037.481 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642029917952, Pid 1]: Received UVE message in wrong state : SandeshModuleClientTrace 2026-03-19 Thu 05:43:36:038.869 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642038314752, Pid 1]: OnMessage control in state: ClientInit 2026-03-19 Thu 05:43:36:038.917 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642038314752, Pid 1]: Processing scm::EvSandeshMessageRecv in state ClientInit 2026-03-19 Thu 05:43:36:038.926 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642038314752, Pid 1]: ClientInit : EvSandeshMessageRecv 2026-03-19 Thu 05:43:36:038.949 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642038314752, Pid 1]: Received Ctrl Message with size 0 2026-03-19 Thu 05:43:36:039.235 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642038314752, Pid 1]: SyncUVE Syncing ObjectGeneratorInfo val name = cn-jenkins-deploy-platform-ansible-os-5495-1.:Database:contrail-query-engine:0 client_info= [ status = ClientInit successful_connections = 3 pid = 1 http_port = 8091 start_time = 1773898562749429 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 2447 calls = 4 average_bytes = 611 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 361242 calls = 41 average_bytes = 8810 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshMessageRecv enqueues = 2 dequeues = 2 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 57 dequeues = 57 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnected enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 65 dequeues = 65 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Established state_since = 1773898609080268 last_state = Established last_event = scm::EvSandeshSend last_event_at = 1773898973089621 ] session_stats= [ num_recv_msg = 0 num_recv_msg_fail = 0 num_recv_fail = 0 num_send_msg = 37 num_send_msg_fail = 0 num_send_buffer_fail = 0 num_wait_msgq_enqueue = 0 num_wait_msgq_dequeue = 0 num_write_ready_cb_error = 0 ] session_rx_socket_stats= [ bytes = 2447 calls = 4 average_bytes = 611 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] session_tx_socket_stats= [ bytes = 331365 calls = 36 average_bytes = 9204 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 15 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 7 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 96 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 28 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 15 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 107, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 30, _iter59->first = sent _iter59->second = 42, ] ] max_sm_queue_count = 8 proxy seq 44 2026-03-19 Thu 05:43:36:039.282 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642038314752, Pid 1]: SyncAllMaps for ModuleClientState:0 period 0 cycle 0 without seqno , total = 1 2026-03-19 Thu 05:43:36:039.311 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642038314752, Pid 1]: SyncUVE Syncing ObjectDatabaseInfo val name = cn-jenkins-deploy-platform-ansible-os-5495-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = 10.0.0.50:6379::Down.10.0.0.33:6379::Down. ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] proxy seq 20 2026-03-19 Thu 05:43:36:039.328 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642038314752, Pid 1]: SyncAllMaps for NodeStatus:0 period 0 cycle 0 without seqno , total = 1 2026-03-19 Thu 05:43:36:039.359 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642038314752, Pid 1]: SyncUVE Syncing ObjectDatabaseInfo val name = cn-jenkins-deploy-platform-ansible-os-5495-1. table_info= [ [ ] ] errors= [ write_tablespace_fails = 0 read_tablespace_fails = 0 write_table_fails = 0 read_table_fails = 0 write_column_fails = 0 write_batch_column_fails = 0 read_column_fails = 0 ] stats_info= [ [ ] ] cql_stats= [ requests_one_minute_rate = 0.4 stats= [ total_connections = 6 available_connections = 6 exceeded_pending_requests_water_mark = 0 exceeded_write_bytes_water_mark = 0 ] errors= [ connection_timeouts = 0 pending_request_timeouts = 0 request_timeouts = 0 ] ] proxy seq 7 2026-03-19 Thu 05:43:36:039.370 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642038314752, Pid 1]: SyncAllMaps for QEDbStats:0 period 0 cycle 0 without seqno , total = 1 2026-03-19 Thu 05:43:36:039.378 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642038314752, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:0 period 60 cycle 0 without seqno , total = 0 2026-03-19 Thu 05:43:36:039.411 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642038314752, Pid 1]: Established 2026-03-19 Thu 05:43:39:987.660 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f269c0 2026-03-19 Thu 05:43:39:987.776 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f27300 2026-03-19 Thu 05:43:39:987.822 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f28580 2026-03-19 Thu 05:43:39:987.881 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f26080 2026-03-19 Thu 05:43:39:987.917 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f27c40 2026-03-19 Thu 05:43:39:988.168 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f269c0 2026-03-19 Thu 05:43:39:988.225 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f27300 2026-03-19 Thu 05:43:39:988.242 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f28580 2026-03-19 Thu 05:43:39:988.257 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f26080 2026-03-19 Thu 05:43:39:988.273 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f27c40 2026-03-19 Thu 05:43:40:091.998 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f25740 2026-03-19 Thu 05:43:40:092.135 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f24e00 2026-03-19 Thu 05:43:40:092.175 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f23280 2026-03-19 Thu 05:43:40:092.217 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f23bc0 2026-03-19 Thu 05:43:40:092.458 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f244c0 2026-03-19 Thu 05:43:40:092.584 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: 0 0x1f25740 2026-03-19 Thu 05:43:40:092.596 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: Connected to REDIS... 2026-03-19 Thu 05:43:40:092.641 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 4 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-03-19 Thu 05:43:40:092.654 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: 0 0x1f23280 2026-03-19 Thu 05:43:40:092.659 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: Connected to REDIS... 2026-03-19 Thu 05:43:40:092.668 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 0 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-03-19 Thu 05:43:40:092.690 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: 0 0x1f24e00 2026-03-19 Thu 05:43:40:092.695 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: Connected to REDIS... 2026-03-19 Thu 05:43:40:092.703 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 3 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-03-19 Thu 05:43:40:092.807 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: 0 0x1f23bc0 2026-03-19 Thu 05:43:40:092.816 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: Connected to REDIS... 2026-03-19 Thu 05:43:40:092.825 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 1 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-03-19 Thu 05:43:40:092.834 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: 0 0x1f244c0 2026-03-19 Thu 05:43:40:092.839 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: Connected to REDIS... 2026-03-19 Thu 05:43:40:092.847 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 2 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-03-19 Thu 05:43:40:093.043 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-03-19 Thu 05:43:40:093.078 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-03-19 Thu 05:43:40:093.118 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB0 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-03-19 Thu 05:43:40:093.140 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-03-19 Thu 05:43:40:093.156 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-03-19 Thu 05:43:40:093.179 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-03-19 Thu 05:43:40:093.198 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-03-19 Thu 05:43:40:094.562 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB4 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-03-19 Thu 05:43:40:094.593 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB3 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-03-19 Thu 05:43:40:094.613 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-03-19 Thu 05:43:40:094.622 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-03-19 Thu 05:43:40:094.636 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-03-19 Thu 05:43:40:094.645 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-03-19 Thu 05:43:40:094.693 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB2 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-03-19 Thu 05:43:40:094.708 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB1 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-03-19 Thu 05:43:44:988.276 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f269c0 2026-03-19 Thu 05:43:44:988.560 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f27300 2026-03-19 Thu 05:43:44:988.609 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f28580 2026-03-19 Thu 05:43:44:988.640 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f26080 2026-03-19 Thu 05:43:44:988.669 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f27c40 2026-03-19 Thu 05:43:44:989.119 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: 0 0x1f269c0 2026-03-19 Thu 05:43:44:989.140 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: Connected to REDIS... 2026-03-19 Thu 05:43:44:989.172 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 1 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-03-19 Thu 05:43:44:989.185 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: 0 0x1f27300 2026-03-19 Thu 05:43:44:989.190 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: Connected to REDIS... 2026-03-19 Thu 05:43:44:989.199 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 2 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-03-19 Thu 05:43:44:989.207 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: 0 0x1f28580 2026-03-19 Thu 05:43:44:989.213 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: Connected to REDIS... 2026-03-19 Thu 05:43:44:989.233 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 4 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-03-19 Thu 05:43:44:989.252 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: 0 0x1f26080 2026-03-19 Thu 05:43:44:989.257 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: Connected to REDIS... 2026-03-19 Thu 05:43:44:989.265 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 0 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-03-19 Thu 05:43:44:989.274 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: 0 0x1f27c40 2026-03-19 Thu 05:43:44:989.279 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: Connected to REDIS... 2026-03-19 Thu 05:43:44:989.287 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 3 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-03-19 Thu 05:43:44:989.673 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-03-19 Thu 05:43:44:989.701 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-03-19 Thu 05:43:44:989.716 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-03-19 Thu 05:43:44:989.725 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-03-19 Thu 05:43:44:989.759 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-03-19 Thu 05:43:44:989.770 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-03-19 Thu 05:43:44:989.793 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB4 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-03-19 Thu 05:43:44:998.126 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB1 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-03-19 Thu 05:43:44:998.181 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB0 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-03-19 Thu 05:43:44:998.211 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-03-19 Thu 05:43:44:998.232 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-03-19 Thu 05:43:44:998.250 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-03-19 Thu 05:43:44:998.274 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-03-19 Thu 05:43:44:998.337 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB2 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-03-19 Thu 05:43:44:998.352 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB3 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-03-19 Thu 05:44:18:079.153 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642059306752, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 8 without seqno , total = 0 2026-03-19 Thu 05:45:18:082.837 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642038314752, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 9 without seqno , total = 0 2026-03-19 Thu 05:46:18:085.145 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140641501693696, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 10 without seqno , total = 0 2026-03-19 Thu 05:46:21:026.286 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Disconnect status: -1 0x1f27c40 2026-03-19 Thu 05:46:21:026.367 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..3 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-03-19 Thu 05:46:21:026.532 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: CallbackProcess: received NULL reply from redis src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1049 2026-03-19 Thu 05:46:21:026.544 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Disconnect status: -1 0x1f26080 2026-03-19 Thu 05:46:21:026.555 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..0 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-03-19 Thu 05:46:21:026.580 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Disconnect status: -1 0x1f28580 2026-03-19 Thu 05:46:21:026.590 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..4 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-03-19 Thu 05:46:21:026.611 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Disconnect status: -1 0x1f27300 2026-03-19 Thu 05:46:21:026.621 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..2 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-03-19 Thu 05:46:21:026.642 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Disconnect status: -1 0x1f269c0 2026-03-19 Thu 05:46:21:026.652 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..1 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-03-19 Thu 05:46:21:026.871 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f27c40 2026-03-19 Thu 05:46:21:034.942 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f26080 2026-03-19 Thu 05:46:21:035.017 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f28580 2026-03-19 Thu 05:46:21:035.035 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f27300 2026-03-19 Thu 05:46:21:035.058 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f269c0 2026-03-19 Thu 05:46:21:061.202 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Disconnect status: -1 0x1f244c0 2026-03-19 Thu 05:46:21:061.269 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..2 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-03-19 Thu 05:46:21:061.333 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Disconnect status: -1 0x1f23bc0 2026-03-19 Thu 05:46:21:061.349 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..1 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-03-19 Thu 05:46:21:061.377 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Disconnect status: -1 0x1f24e00 2026-03-19 Thu 05:46:21:061.387 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..3 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-03-19 Thu 05:46:21:061.423 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Disconnect status: -1 0x1f25740 2026-03-19 Thu 05:46:21:061.434 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..4 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-03-19 Thu 05:46:21:061.459 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: CallbackProcess: received NULL reply from redis src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1049 2026-03-19 Thu 05:46:21:061.468 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Disconnect status: -1 0x1f23280 2026-03-19 Thu 05:46:21:061.477 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..0 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-03-19 Thu 05:46:21:063.592 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f25740 2026-03-19 Thu 05:46:21:063.665 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f23bc0 2026-03-19 Thu 05:46:21:063.693 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f23280 2026-03-19 Thu 05:46:21:063.720 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f244c0 2026-03-19 Thu 05:46:21:063.735 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f24e00 2026-03-19 Thu 05:46:21:135.328 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Disconnect status: -1 0x1f22970 2026-03-19 Thu 05:46:21:135.410 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..4 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-03-19 Thu 05:46:21:135.477 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Disconnect status: -1 0x1f22360 2026-03-19 Thu 05:46:21:135.492 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..3 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-03-19 Thu 05:46:21:135.529 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Disconnect status: -1 0x1f1b1d0 2026-03-19 Thu 05:46:21:135.552 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..2 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-03-19 Thu 05:46:21:135.585 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Disconnect status: -1 0x1f21dd0 2026-03-19 Thu 05:46:21:135.598 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..1 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-03-19 Thu 05:46:21:135.636 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: CallbackProcess: received NULL reply from redis src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1049 2026-03-19 Thu 05:46:21:135.646 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Disconnect status: -1 0x1f215f0 2026-03-19 Thu 05:46:21:135.659 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnDown.. DOWN.. Reconnect..0 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1016 2026-03-19 Thu 05:46:21:138.899 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f22970 2026-03-19 Thu 05:46:21:138.956 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f22360 2026-03-19 Thu 05:46:21:138.979 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f1b1d0 2026-03-19 Thu 05:46:21:138.996 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f21dd0 2026-03-19 Thu 05:46:21:139.012 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f215f0 2026-03-19 Thu 05:46:21:171.299 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: 10.0.0.38:50412::10.0.0.38:8086(-1) OnSessionEvent TCP Connection Closed 2026-03-19 Thu 05:46:21:171.345 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: 10.0.0.38:50412::10.0.0.38:8086(-1):EvTcpClose 2026-03-19 Thu 05:46:21:172.881 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140641999865600, Pid 1]: Processing scm::EvTcpClose in state Established 2026-03-19 Thu 05:46:21:172.899 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140641999865600, Pid 1]: Established : EvTcpClose 2026-03-19 Thu 05:46:21:173.177 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140641999865600, Pid 1]: Connect : Start Connect timer 10.0.0.33:8086 2026-03-19 Thu 05:46:21:173.269 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140641999865600, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-03-19 Thu 05:46:21:173.455 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140641999865600, Pid 1]: SANDESH: Send FAILED: 1773899181172997 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1.:Database:contrail-query-engine:0 client_info= [ status = Established successful_connections = 3 pid = 1 http_port = 8091 start_time = 1773898562749429 collector_name = cn-jenkins-deploy-platform-ansible-os-5495-1. collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 571897 calls = 65 average_bytes = 8798 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 16 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 11 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 154 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 42 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 16 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 165, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 32, _iter59->first = sent _iter59->second = 69, ] ] ] 2026-03-19 Thu 05:46:21:173.480 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140641999865600, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-03-19 Thu 05:46:21:173.492 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140641999865600, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-03-19 Thu 05:46:21:173.515 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140641999865600, Pid 1]: SANDESH: Send FAILED: 1773899181173029 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Established : EvTcpClose -> Connect ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-03-19 Thu 05:46:21:173.523 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140641999865600, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-03-19 Thu 05:46:21:173.533 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140641999865600, Pid 1]: Processing scm::EvTcpDeleteSession in state Connect 2026-03-19 Thu 05:46:21:173.544 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140641999865600, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-03-19 Thu 05:46:21:173.674 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140641999865600, Pid 1]: SANDESH: Send FAILED: 1773899181173253 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1.:Database:contrail-query-engine:0 client_info= [ status = Connect successful_connections = 3 pid = 1 http_port = 8091 start_time = 1773898562749429 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 571897 calls = 65 average_bytes = 8798 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 16 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 11 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 154 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 42 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 16 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 165, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 32, _iter59->first = sent _iter59->second = 69, ] ] ] 2026-03-19 Thu 05:46:21:173.689 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140641999865600, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-03-19 Thu 05:46:21:175.865 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.38:54080::10.0.0.33:8086(-1) < Active session connection complete src/contrail-common/io/tcp_session.cc 280 2026-03-19 Thu 05:46:21:175.888 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: 10.0.0.38:54080::10.0.0.33:8086(-1) OnSessionEvent TCP Connected 2026-03-19 Thu 05:46:21:175.897 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: 10.0.0.38:54080::10.0.0.33:8086(-1):EvTcpConnected 2026-03-19 Thu 05:46:21:184.615 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642050909952, Pid 1]: Processing scm::EvTcpConnected in state Connect 2026-03-19 Thu 05:46:21:184.657 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642050909952, Pid 1]: Connect : EvTcpConnected : Cancelling Connect timer 2026-03-19 Thu 05:46:21:184.716 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642050909952, Pid 1]: ClientInit 2026-03-19 Thu 05:46:21:184.750 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642050909952, Pid 1]: Sending Ctrl Message for cn-jenkins-deploy-platform-ansible-os-5495-1.:contrail-query-engine:0:Database count 4 2026-03-19 Thu 05:46:21:184.879 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642050909952, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-03-19 Thu 05:46:21:184.892 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642050909952, Pid 1]: ClientInit : EvSandeshSend : NodeStatusUVE 2026-03-19 Thu 05:46:21:184.923 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642050909952, Pid 1]: SANDESH: Send FAILED: 1773899181184704 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-03-19 Thu 05:46:21:184.946 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642050909952, Pid 1]: Received UVE message in wrong state : NodeStatusUVE 2026-03-19 Thu 05:46:21:184.958 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642050909952, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-03-19 Thu 05:46:21:184.967 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642050909952, Pid 1]: ClientInit : EvSandeshSend : SandeshCtrlClientToServer 2026-03-19 Thu 05:46:21:184.985 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642050909952, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-03-19 Thu 05:46:21:184.994 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642050909952, Pid 1]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2026-03-19 Thu 05:46:21:185.190 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642050909952, Pid 1]: SANDESH: Send FAILED: 1773899181184864 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1.:Database:contrail-query-engine:0 client_info= [ status = ClientInit successful_connections = 4 pid = 1 http_port = 8091 start_time = 1773898562749429 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 571897 calls = 65 average_bytes = 8798 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 17 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 11 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 154 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 42 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 18 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 11 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 166, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 35, _iter59->first = sent _iter59->second = 69, ] ] ] 2026-03-19 Thu 05:46:21:185.210 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642050909952, Pid 1]: Received UVE message in wrong state : SandeshModuleClientTrace 2026-03-19 Thu 05:46:21:201.221 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: 10.0.0.38:54080::10.0.0.33:8086(-1) OnSessionEvent TCP Connection Closed 2026-03-19 Thu 05:46:21:201.253 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: 10.0.0.38:54080::10.0.0.33:8086(-1):EvTcpClose 2026-03-19 Thu 05:46:21:203.455 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642038314752, Pid 1]: Processing scm::EvTcpClose in state ClientInit 2026-03-19 Thu 05:46:21:203.477 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642038314752, Pid 1]: ClientInit : EvTcpClose 2026-03-19 Thu 05:46:21:203.622 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642038314752, Pid 1]: Return to idle with 1000 2026-03-19 Thu 05:46:21:203.632 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642038314752, Pid 1]: Idle 2026-03-19 Thu 05:46:21:203.690 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642038314752, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-03-19 Thu 05:46:21:203.833 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642038314752, Pid 1]: SANDESH: Send FAILED: 1773899181203564 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1.:Database:contrail-query-engine:0 client_info= [ status = ClientInit successful_connections = 4 pid = 1 http_port = 8091 start_time = 1773898562749429 collector_name = collector_ip = 10.0.0.50:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 573470 calls = 66 average_bytes = 8688 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 18 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 11 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 154 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 4 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 42 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 19 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 11 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 166, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 37, _iter59->first = sent _iter59->second = 70, ] ] ] 2026-03-19 Thu 05:46:21:203.855 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642038314752, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-03-19 Thu 05:46:21:203.866 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642038314752, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-03-19 Thu 05:46:21:203.888 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642038314752, Pid 1]: SANDESH: Send FAILED: 1773899181203599 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Down description = ClientInit : EvTcpClose ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-03-19 Thu 05:46:21:203.898 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642038314752, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-03-19 Thu 05:46:21:203.906 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642038314752, Pid 1]: Processing scm::EvTcpDeleteSession in state Idle 2026-03-19 Thu 05:46:21:203.915 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642038314752, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-03-19 Thu 05:46:21:204.050 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642038314752, Pid 1]: SANDESH: Send FAILED: 1773899181203681 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1.:Database:contrail-query-engine:0 client_info= [ status = Idle successful_connections = 4 pid = 1 http_port = 8091 start_time = 1773898562749429 collector_name = collector_ip = 10.0.0.50:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 573470 calls = 66 average_bytes = 8688 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 18 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 11 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 154 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 4 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 42 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 19 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 11 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 166, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 37, _iter59->first = sent _iter59->second = 70, ] ] ] 2026-03-19 Thu 05:46:21:204.064 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642038314752, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-03-19 Thu 05:46:22:208.177 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140641999865600, Pid 1]: Processing scm::EvIdleHoldTimerExpired in state Idle 2026-03-19 Thu 05:46:22:208.407 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140641999865600, Pid 1]: Connect : Start Connect timer 10.0.0.50:8086 2026-03-19 Thu 05:46:22:208.516 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140641999865600, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-03-19 Thu 05:46:22:208.546 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140641999865600, Pid 1]: SANDESH: Send FAILED: 1773899182208262 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.50:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-03-19 Thu 05:46:22:208.555 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140641999865600, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-03-19 Thu 05:46:22:208.564 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140641999865600, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-03-19 Thu 05:46:22:208.704 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140641999865600, Pid 1]: SANDESH: Send FAILED: 1773899182208502 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1.:Database:contrail-query-engine:0 client_info= [ status = Connect successful_connections = 4 pid = 1 http_port = 8091 start_time = 1773898562749429 collector_name = collector_ip = 10.0.0.50:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 573470 calls = 66 average_bytes = 8688 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 19 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 11 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 154 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 4 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 42 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 21 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 11 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 166, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 40, _iter59->first = sent _iter59->second = 70, ] ] ] 2026-03-19 Thu 05:46:22:208.725 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140641999865600, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-03-19 Thu 05:46:22:222.151 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.38:54098::10.0.0.50:8086(-1) < Active session connection complete src/contrail-common/io/tcp_session.cc 280 2026-03-19 Thu 05:46:22:222.191 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: 10.0.0.38:54098::10.0.0.50:8086(-1) OnSessionEvent TCP Connected 2026-03-19 Thu 05:46:22:222.200 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: 10.0.0.38:54098::10.0.0.50:8086(-1):EvTcpConnected 2026-03-19 Thu 05:46:22:225.132 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140641505892096, Pid 1]: Processing scm::EvTcpConnected in state Connect 2026-03-19 Thu 05:46:22:225.164 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140641505892096, Pid 1]: Connect : EvTcpConnected : Cancelling Connect timer 2026-03-19 Thu 05:46:22:225.482 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140641505892096, Pid 1]: ClientInit 2026-03-19 Thu 05:46:22:225.512 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140641505892096, Pid 1]: Sending Ctrl Message for cn-jenkins-deploy-platform-ansible-os-5495-1.:contrail-query-engine:0:Database count 5 2026-03-19 Thu 05:46:22:225.611 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140641505892096, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-03-19 Thu 05:46:22:225.623 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140641505892096, Pid 1]: ClientInit : EvSandeshSend : NodeStatusUVE 2026-03-19 Thu 05:46:22:225.648 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140641505892096, Pid 1]: SANDESH: Send FAILED: 1773899182225466 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.50:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-03-19 Thu 05:46:22:225.657 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140641505892096, Pid 1]: Received UVE message in wrong state : NodeStatusUVE 2026-03-19 Thu 05:46:22:225.665 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140641505892096, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-03-19 Thu 05:46:22:225.672 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140641505892096, Pid 1]: ClientInit : EvSandeshSend : SandeshCtrlClientToServer 2026-03-19 Thu 05:46:22:225.683 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140641505892096, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-03-19 Thu 05:46:22:225.689 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140641505892096, Pid 1]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2026-03-19 Thu 05:46:22:226.425 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140641505892096, Pid 1]: SANDESH: Send FAILED: 1773899182225596 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1.:Database:contrail-query-engine:0 client_info= [ status = ClientInit successful_connections = 5 pid = 1 http_port = 8091 start_time = 1773898562749429 collector_name = collector_ip = 10.0.0.50:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 573470 calls = 66 average_bytes = 8688 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 20 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 11 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 154 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 4 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 42 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 22 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 12 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 167, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 42, _iter59->first = sent _iter59->second = 70, ] ] ] 2026-03-19 Thu 05:46:22:226.464 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140641505892096, Pid 1]: Received UVE message in wrong state : SandeshModuleClientTrace 2026-03-19 Thu 05:46:22:229.166 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: 10.0.0.38:54098::10.0.0.50:8086(-1) OnSessionEvent TCP Connection Closed 2026-03-19 Thu 05:46:22:229.198 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: 10.0.0.38:54098::10.0.0.50:8086(-1):EvTcpClose 2026-03-19 Thu 05:46:22:238.342 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140641999865600, Pid 1]: Processing scm::EvTcpClose in state ClientInit 2026-03-19 Thu 05:46:22:238.387 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140641999865600, Pid 1]: ClientInit : EvTcpClose 2026-03-19 Thu 05:46:22:238.543 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140641999865600, Pid 1]: Return to idle with 1000 2026-03-19 Thu 05:46:22:238.553 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140641999865600, Pid 1]: Idle 2026-03-19 Thu 05:46:22:238.611 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140641999865600, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-03-19 Thu 05:46:22:238.757 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140641999865600, Pid 1]: SANDESH: Send FAILED: 1773899182238483 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1.:Database:contrail-query-engine:0 client_info= [ status = ClientInit successful_connections = 5 pid = 1 http_port = 8091 start_time = 1773898562749429 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 575043 calls = 67 average_bytes = 8582 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 21 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 11 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 154 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 42 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 23 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 12 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 167, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 44, _iter59->first = sent _iter59->second = 71, ] ] ] 2026-03-19 Thu 05:46:22:238.784 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140641999865600, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-03-19 Thu 05:46:22:238.794 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140641999865600, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-03-19 Thu 05:46:22:238.816 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140641999865600, Pid 1]: SANDESH: Send FAILED: 1773899182238515 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.50:8086, ] ] status = Down description = ClientInit : EvTcpClose ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-03-19 Thu 05:46:22:238.824 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140641999865600, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-03-19 Thu 05:46:22:238.832 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140641999865600, Pid 1]: Processing scm::EvTcpDeleteSession in state Idle 2026-03-19 Thu 05:46:22:238.841 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140641999865600, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-03-19 Thu 05:46:22:238.977 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140641999865600, Pid 1]: SANDESH: Send FAILED: 1773899182238601 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1.:Database:contrail-query-engine:0 client_info= [ status = Idle successful_connections = 5 pid = 1 http_port = 8091 start_time = 1773898562749429 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 575043 calls = 67 average_bytes = 8582 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 21 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 11 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 154 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 42 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 23 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 12 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 167, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 44, _iter59->first = sent _iter59->second = 71, ] ] ] 2026-03-19 Thu 05:46:22:238.992 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140641999865600, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-03-19 Thu 05:46:23:241.229 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140641505892096, Pid 1]: Processing scm::EvIdleHoldTimerExpired in state Idle 2026-03-19 Thu 05:46:23:241.520 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140641505892096, Pid 1]: Connect : Start Connect timer 10.0.0.38:8086 2026-03-19 Thu 05:46:23:241.635 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140641505892096, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-03-19 Thu 05:46:23:241.665 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140641505892096, Pid 1]: SANDESH: Send FAILED: 1773899183241316 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-03-19 Thu 05:46:23:241.675 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140641505892096, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-03-19 Thu 05:46:23:241.684 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140641505892096, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-03-19 Thu 05:46:23:241.854 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140641505892096, Pid 1]: SANDESH: Send FAILED: 1773899183241616 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1.:Database:contrail-query-engine:0 client_info= [ status = Connect successful_connections = 5 pid = 1 http_port = 8091 start_time = 1773898562749429 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 575043 calls = 67 average_bytes = 8582 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 22 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 11 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 154 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 42 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 25 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 12 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 167, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 47, _iter59->first = sent _iter59->second = 71, ] ] ] 2026-03-19 Thu 05:46:23:241.880 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140641505892096, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-03-19 Thu 05:46:23:248.775 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.38:36292::10.0.0.38:8086(-1) < Active session connection complete src/contrail-common/io/tcp_session.cc 280 2026-03-19 Thu 05:46:23:248.838 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: 10.0.0.38:36292::10.0.0.38:8086(-1) OnSessionEvent TCP Connected 2026-03-19 Thu 05:46:23:248.851 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: 10.0.0.38:36292::10.0.0.38:8086(-1):EvTcpConnected 2026-03-19 Thu 05:46:23:248.961 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642055108352, Pid 1]: Processing scm::EvTcpConnected in state Connect 2026-03-19 Thu 05:46:23:248.999 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642055108352, Pid 1]: Connect : EvTcpConnected : Cancelling Connect timer 2026-03-19 Thu 05:46:23:249.048 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642055108352, Pid 1]: ClientInit 2026-03-19 Thu 05:46:23:249.080 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642055108352, Pid 1]: Sending Ctrl Message for cn-jenkins-deploy-platform-ansible-os-5495-1.:contrail-query-engine:0:Database count 6 2026-03-19 Thu 05:46:23:249.237 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642055108352, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-03-19 Thu 05:46:23:249.268 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642055108352, Pid 1]: ClientInit : EvSandeshSend : NodeStatusUVE 2026-03-19 Thu 05:46:23:249.304 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642055108352, Pid 1]: SANDESH: Send FAILED: 1773899183249038 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-03-19 Thu 05:46:23:249.316 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642055108352, Pid 1]: Received UVE message in wrong state : NodeStatusUVE 2026-03-19 Thu 05:46:23:249.331 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642055108352, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-03-19 Thu 05:46:23:249.342 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642055108352, Pid 1]: ClientInit : EvSandeshSend : SandeshCtrlClientToServer 2026-03-19 Thu 05:46:23:249.358 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642055108352, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-03-19 Thu 05:46:23:249.368 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642055108352, Pid 1]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2026-03-19 Thu 05:46:23:249.707 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642055108352, Pid 1]: SANDESH: Send FAILED: 1773899183249218 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1.:Database:contrail-query-engine:0 client_info= [ status = ClientInit successful_connections = 6 pid = 1 http_port = 8091 start_time = 1773898562749429 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 575043 calls = 67 average_bytes = 8582 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 23 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 11 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 154 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 42 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 26 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 13 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 168, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 49, _iter59->first = sent _iter59->second = 71, ] ] ] 2026-03-19 Thu 05:46:23:249.784 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642055108352, Pid 1]: Received UVE message in wrong state : SandeshModuleClientTrace 2026-03-19 Thu 05:46:26:026.944 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f27c40 2026-03-19 Thu 05:46:26:027.494 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f27c40 2026-03-19 Thu 05:46:26:035.079 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f26080 2026-03-19 Thu 05:46:26:035.258 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f28580 2026-03-19 Thu 05:46:26:035.301 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f27300 2026-03-19 Thu 05:46:26:035.332 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f269c0 2026-03-19 Thu 05:46:26:035.586 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f26080 2026-03-19 Thu 05:46:26:035.741 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f28580 2026-03-19 Thu 05:46:26:035.758 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f27300 2026-03-19 Thu 05:46:26:035.772 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f269c0 2026-03-19 Thu 05:46:26:063.728 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f25740 2026-03-19 Thu 05:46:26:063.939 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f23bc0 2026-03-19 Thu 05:46:26:063.996 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f23280 2026-03-19 Thu 05:46:26:064.034 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f244c0 2026-03-19 Thu 05:46:26:064.063 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f24e00 2026-03-19 Thu 05:46:26:064.780 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f25740 2026-03-19 Thu 05:46:26:064.832 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f23bc0 2026-03-19 Thu 05:46:26:064.848 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f244c0 2026-03-19 Thu 05:46:26:064.863 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f23280 2026-03-19 Thu 05:46:26:064.878 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f24e00 2026-03-19 Thu 05:46:26:139.019 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f22970 2026-03-19 Thu 05:46:26:139.395 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f22970 2026-03-19 Thu 05:46:26:139.493 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f22360 2026-03-19 Thu 05:46:26:139.572 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f1b1d0 2026-03-19 Thu 05:46:26:139.616 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f21dd0 2026-03-19 Thu 05:46:26:139.650 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f215f0 2026-03-19 Thu 05:46:26:139.694 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f22360 2026-03-19 Thu 05:46:26:139.713 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f1b1d0 2026-03-19 Thu 05:46:26:139.726 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f21dd0 2026-03-19 Thu 05:46:26:139.738 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f215f0 2026-03-19 Thu 05:46:27:249.356 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642021521152, Pid 1]: 10.0.0.38:8086 EvConnectTimerExpired in state ClientInit 2026-03-19 Thu 05:46:27:249.442 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642021521152, Pid 1]: Processing scm::EvConnectTimerExpired in state ClientInit 2026-03-19 Thu 05:46:27:249.455 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642021521152, Pid 1]: ClientInit : EvConnectTimerExpired 2026-03-19 Thu 05:46:27:249.667 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642021521152, Pid 1]: Return to idle with 1000 2026-03-19 Thu 05:46:27:249.678 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642021521152, Pid 1]: Idle 2026-03-19 Thu 05:46:27:249.774 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642021521152, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-03-19 Thu 05:46:27:249.933 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642021521152, Pid 1]: SANDESH: Send FAILED: 1773899187249553 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1.:Database:contrail-query-engine:0 client_info= [ status = ClientInit successful_connections = 6 pid = 1 http_port = 8091 start_time = 1773898562749429 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 576616 calls = 68 average_bytes = 8479 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 24 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 11 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 154 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 42 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 27 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 13 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 168, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 51, _iter59->first = sent _iter59->second = 72, ] ] ] 2026-03-19 Thu 05:46:27:249.960 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642021521152, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-03-19 Thu 05:46:27:249.972 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642021521152, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-03-19 Thu 05:46:27:249.995 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642021521152, Pid 1]: SANDESH: Send FAILED: 1773899187249589 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Down description = ClientInit : EvConnectTimerExpired ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-03-19 Thu 05:46:27:250.002 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642021521152, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-03-19 Thu 05:46:27:250.011 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642021521152, Pid 1]: Processing scm::EvTcpDeleteSession in state Idle 2026-03-19 Thu 05:46:27:250.021 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642021521152, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-03-19 Thu 05:46:27:250.151 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642021521152, Pid 1]: SANDESH: Send FAILED: 1773899187249731 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1.:Database:contrail-query-engine:0 client_info= [ status = Idle successful_connections = 6 pid = 1 http_port = 8091 start_time = 1773898562749429 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 576616 calls = 68 average_bytes = 8479 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 24 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 11 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 154 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 42 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 27 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 13 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 168, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 51, _iter59->first = sent _iter59->second = 72, ] ] ] 2026-03-19 Thu 05:46:27:250.168 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642021521152, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-03-19 Thu 05:46:28:250.517 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642025719552, Pid 1]: Processing scm::EvIdleHoldTimerExpired in state Idle 2026-03-19 Thu 05:46:28:250.766 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642025719552, Pid 1]: Connect : Start Connect timer 10.0.0.33:8086 2026-03-19 Thu 05:46:28:250.891 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642025719552, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-03-19 Thu 05:46:28:250.922 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642025719552, Pid 1]: SANDESH: Send FAILED: 1773899188250599 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-03-19 Thu 05:46:28:250.932 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642025719552, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-03-19 Thu 05:46:28:250.942 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642025719552, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-03-19 Thu 05:46:28:251.092 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642025719552, Pid 1]: SANDESH: Send FAILED: 1773899188250876 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1.:Database:contrail-query-engine:0 client_info= [ status = Connect successful_connections = 6 pid = 1 http_port = 8091 start_time = 1773898562749429 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 576616 calls = 68 average_bytes = 8479 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 25 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 11 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 154 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 42 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 29 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 13 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 168, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 54, _iter59->first = sent _iter59->second = 72, ] ] ] 2026-03-19 Thu 05:46:28:251.115 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642025719552, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-03-19 Thu 05:46:28:253.156 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.38:54696::10.0.0.33:8086(-1) < Active session connection complete src/contrail-common/io/tcp_session.cc 280 2026-03-19 Thu 05:46:28:253.186 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: 10.0.0.38:54696::10.0.0.33:8086(-1) OnSessionEvent TCP Connected 2026-03-19 Thu 05:46:28:253.194 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: 10.0.0.38:54696::10.0.0.33:8086(-1):EvTcpConnected 2026-03-19 Thu 05:46:28:253.972 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642021521152, Pid 1]: Processing scm::EvTcpConnected in state Connect 2026-03-19 Thu 05:46:28:253.995 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642021521152, Pid 1]: Connect : EvTcpConnected : Cancelling Connect timer 2026-03-19 Thu 05:46:28:254.035 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642021521152, Pid 1]: ClientInit 2026-03-19 Thu 05:46:28:254.052 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642021521152, Pid 1]: Sending Ctrl Message for cn-jenkins-deploy-platform-ansible-os-5495-1.:contrail-query-engine:0:Database count 7 2026-03-19 Thu 05:46:28:254.138 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642021521152, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-03-19 Thu 05:46:28:254.148 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642021521152, Pid 1]: ClientInit : EvSandeshSend : NodeStatusUVE 2026-03-19 Thu 05:46:28:254.171 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642021521152, Pid 1]: SANDESH: Send FAILED: 1773899188254027 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-03-19 Thu 05:46:28:254.178 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642021521152, Pid 1]: Received UVE message in wrong state : NodeStatusUVE 2026-03-19 Thu 05:46:28:254.196 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642021521152, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-03-19 Thu 05:46:28:254.203 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642021521152, Pid 1]: ClientInit : EvSandeshSend : SandeshCtrlClientToServer 2026-03-19 Thu 05:46:28:254.214 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642021521152, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-03-19 Thu 05:46:28:254.220 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642021521152, Pid 1]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2026-03-19 Thu 05:46:28:254.354 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642021521152, Pid 1]: SANDESH: Send FAILED: 1773899188254128 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1.:Database:contrail-query-engine:0 client_info= [ status = ClientInit successful_connections = 7 pid = 1 http_port = 8091 start_time = 1773898562749429 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 576616 calls = 68 average_bytes = 8479 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 26 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 11 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 154 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 42 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 30 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 169, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 56, _iter59->first = sent _iter59->second = 72, ] ] ] 2026-03-19 Thu 05:46:28:254.368 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642021521152, Pid 1]: Received UVE message in wrong state : SandeshModuleClientTrace 2026-03-19 Thu 05:46:31:027.700 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f27c40 2026-03-19 Thu 05:46:31:028.295 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f27c40 2026-03-19 Thu 05:46:31:035.669 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f26080 2026-03-19 Thu 05:46:31:035.990 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f28580 2026-03-19 Thu 05:46:31:036.064 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f27300 2026-03-19 Thu 05:46:31:036.102 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f269c0 2026-03-19 Thu 05:46:31:036.444 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f26080 2026-03-19 Thu 05:46:31:036.514 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f28580 2026-03-19 Thu 05:46:31:036.535 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f27300 2026-03-19 Thu 05:46:31:036.555 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f269c0 2026-03-19 Thu 05:46:31:064.878 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f25740 2026-03-19 Thu 05:46:31:065.048 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f23bc0 2026-03-19 Thu 05:46:31:065.092 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f244c0 2026-03-19 Thu 05:46:31:065.123 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f23280 2026-03-19 Thu 05:46:31:065.251 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f24e00 2026-03-19 Thu 05:46:31:065.536 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f25740 2026-03-19 Thu 05:46:31:065.576 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f23bc0 2026-03-19 Thu 05:46:31:065.591 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f23280 2026-03-19 Thu 05:46:31:065.608 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f244c0 2026-03-19 Thu 05:46:31:065.621 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f24e00 2026-03-19 Thu 05:46:31:139.522 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f22970 2026-03-19 Thu 05:46:31:139.716 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f22970 2026-03-19 Thu 05:46:31:139.764 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f22360 2026-03-19 Thu 05:46:31:139.915 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f1b1d0 2026-03-19 Thu 05:46:31:139.960 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f21dd0 2026-03-19 Thu 05:46:31:139.993 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f215f0 2026-03-19 Thu 05:46:31:140.042 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f22360 2026-03-19 Thu 05:46:31:140.062 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f1b1d0 2026-03-19 Thu 05:46:31:140.074 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f21dd0 2026-03-19 Thu 05:46:31:140.086 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f215f0 2026-03-19 Thu 05:46:32:796.459 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: TCP [SYS_ERR]: TcpSessionMessageLog: Session 10.0.0.38:54696::10.0.0.33:8086(-1) < Read failed due to error system 104 : Connection reset by peer src/contrail-common/io/tcp_session.cc 507 2026-03-19 Thu 05:46:32:796.522 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: TCP [SYS_ERR]: TcpSessionMessageLog: Session 10.0.0.38:54696::10.0.0.33:8086(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 312 2026-03-19 Thu 05:46:32:796.546 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: 10.0.0.38:54696::10.0.0.33:8086(-1) OnSessionEvent TCP Connection Closed 2026-03-19 Thu 05:46:32:796.555 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: 10.0.0.38:54696::10.0.0.33:8086(-1):EvTcpClose 2026-03-19 Thu 05:46:32:798.973 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642021521152, Pid 1]: Processing scm::EvTcpClose in state ClientInit 2026-03-19 Thu 05:46:32:798.992 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642021521152, Pid 1]: ClientInit : EvTcpClose 2026-03-19 Thu 05:46:32:799.151 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642021521152, Pid 1]: Return to idle with 1000 2026-03-19 Thu 05:46:32:799.162 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642021521152, Pid 1]: Idle 2026-03-19 Thu 05:46:32:799.230 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642021521152, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-03-19 Thu 05:46:32:799.386 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642021521152, Pid 1]: SANDESH: Send FAILED: 1773899192799083 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1.:Database:contrail-query-engine:0 client_info= [ status = ClientInit successful_connections = 7 pid = 1 http_port = 8091 start_time = 1773898562749429 collector_name = collector_ip = 10.0.0.50:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 578189 calls = 69 average_bytes = 8379 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 27 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 11 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 154 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 7 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 42 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 31 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 16 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 171, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 58, _iter59->first = sent _iter59->second = 73, ] ] ] 2026-03-19 Thu 05:46:32:799.420 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642021521152, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-03-19 Thu 05:46:32:799.431 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642021521152, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-03-19 Thu 05:46:32:799.454 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642021521152, Pid 1]: SANDESH: Send FAILED: 1773899192799118 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Down description = ClientInit : EvTcpClose ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-03-19 Thu 05:46:32:799.461 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642021521152, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-03-19 Thu 05:46:32:799.470 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642021521152, Pid 1]: Processing scm::EvTcpDeleteSession in state Idle 2026-03-19 Thu 05:46:32:799.478 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642021521152, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-03-19 Thu 05:46:32:799.605 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642021521152, Pid 1]: SANDESH: Send FAILED: 1773899192799220 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1.:Database:contrail-query-engine:0 client_info= [ status = Idle successful_connections = 7 pid = 1 http_port = 8091 start_time = 1773898562749429 collector_name = collector_ip = 10.0.0.50:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 578189 calls = 69 average_bytes = 8379 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 27 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 11 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 154 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 7 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 42 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 31 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 16 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 171, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 58, _iter59->first = sent _iter59->second = 73, ] ] ] 2026-03-19 Thu 05:46:32:799.621 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642021521152, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-03-19 Thu 05:46:33:800.047 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642059306752, Pid 1]: Processing scm::EvIdleHoldTimerExpired in state Idle 2026-03-19 Thu 05:46:33:800.321 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642059306752, Pid 1]: Connect : Start Connect timer 10.0.0.50:8086 2026-03-19 Thu 05:46:33:800.437 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642059306752, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-03-19 Thu 05:46:33:800.470 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642059306752, Pid 1]: SANDESH: Send FAILED: 1773899193800141 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.50:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-03-19 Thu 05:46:33:800.479 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642059306752, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-03-19 Thu 05:46:33:800.488 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642059306752, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-03-19 Thu 05:46:33:800.640 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642059306752, Pid 1]: SANDESH: Send FAILED: 1773899193800425 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1.:Database:contrail-query-engine:0 client_info= [ status = Connect successful_connections = 7 pid = 1 http_port = 8091 start_time = 1773898562749429 collector_name = collector_ip = 10.0.0.50:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 578189 calls = 69 average_bytes = 8379 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 28 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 11 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 154 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 7 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 42 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 33 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 16 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 171, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 61, _iter59->first = sent _iter59->second = 73, ] ] ] 2026-03-19 Thu 05:46:33:800.665 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642059306752, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-03-19 Thu 05:46:33:803.059 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: -(-1) OnSessionEvent TCP Connect Failed 2026-03-19 Thu 05:46:33:803.079 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: -(-1):EvTcpConnectFail 2026-03-19 Thu 05:46:33:805.490 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642046711552, Pid 1]: Processing scm::EvTcpConnectFail in state Connect 2026-03-19 Thu 05:46:33:805.511 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642046711552, Pid 1]: Connect : EvTcpConnectFail 2026-03-19 Thu 05:46:33:805.651 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642046711552, Pid 1]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 312 2026-03-19 Thu 05:46:33:805.693 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642046711552, Pid 1]: Idle 2026-03-19 Thu 05:46:33:805.781 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642046711552, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-03-19 Thu 05:46:33:805.934 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642046711552, Pid 1]: SANDESH: Send FAILED: 1773899193805595 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1.:Database:contrail-query-engine:0 client_info= [ status = Connect successful_connections = 7 pid = 1 http_port = 8091 start_time = 1773898562749429 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 578189 calls = 69 average_bytes = 8379 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 29 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 11 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 154 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 7 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 42 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 34 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 16 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 171, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 63, _iter59->first = sent _iter59->second = 73, ] ] ] 2026-03-19 Thu 05:46:33:805.958 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642046711552, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-03-19 Thu 05:46:33:805.969 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642046711552, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-03-19 Thu 05:46:33:805.991 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642046711552, Pid 1]: SANDESH: Send FAILED: 1773899193805626 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-03-19 Thu 05:46:33:805.999 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642046711552, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-03-19 Thu 05:46:33:806.007 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642046711552, Pid 1]: Processing scm::EvTcpDeleteSession in state Idle 2026-03-19 Thu 05:46:33:806.016 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642046711552, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-03-19 Thu 05:46:33:806.148 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642046711552, Pid 1]: SANDESH: Send FAILED: 1773899193805766 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1.:Database:contrail-query-engine:0 client_info= [ status = Idle successful_connections = 7 pid = 1 http_port = 8091 start_time = 1773898562749429 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 578189 calls = 69 average_bytes = 8379 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 29 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 11 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 154 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 7 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 42 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 34 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 172, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 63, _iter59->first = sent _iter59->second = 73, ] ] ] 2026-03-19 Thu 05:46:33:806.164 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642046711552, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-03-19 Thu 05:46:36:028.446 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f27c40 2026-03-19 Thu 05:46:36:028.924 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f27c40 2026-03-19 Thu 05:46:36:036.779 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f26080 2026-03-19 Thu 05:46:36:036.917 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f28580 2026-03-19 Thu 05:46:36:036.957 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f27300 2026-03-19 Thu 05:46:36:037.011 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f269c0 2026-03-19 Thu 05:46:36:037.269 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f26080 2026-03-19 Thu 05:46:36:037.302 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f28580 2026-03-19 Thu 05:46:36:037.316 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f27300 2026-03-19 Thu 05:46:36:037.330 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f269c0 2026-03-19 Thu 05:46:36:065.611 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f25740 2026-03-19 Thu 05:46:36:065.776 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f23bc0 2026-03-19 Thu 05:46:36:065.823 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f23280 2026-03-19 Thu 05:46:36:065.857 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f244c0 2026-03-19 Thu 05:46:36:065.887 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f24e00 2026-03-19 Thu 05:46:36:066.254 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f25740 2026-03-19 Thu 05:46:36:066.293 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f23bc0 2026-03-19 Thu 05:46:36:066.322 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f244c0 2026-03-19 Thu 05:46:36:066.387 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f23280 2026-03-19 Thu 05:46:36:066.833 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f24e00 2026-03-19 Thu 05:46:36:139.784 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f22970 2026-03-19 Thu 05:46:36:139.939 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f22970 2026-03-19 Thu 05:46:36:140.075 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f22360 2026-03-19 Thu 05:46:36:140.135 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f22360 2026-03-19 Thu 05:46:36:140.150 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f1b1d0 2026-03-19 Thu 05:46:36:140.179 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f21dd0 2026-03-19 Thu 05:46:36:140.325 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f215f0 2026-03-19 Thu 05:46:36:140.395 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f1b1d0 2026-03-19 Thu 05:46:36:140.428 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f21dd0 2026-03-19 Thu 05:46:36:140.442 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f215f0 2026-03-19 Thu 05:46:41:028.999 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f27c40 2026-03-19 Thu 05:46:41:029.610 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f27c40 2026-03-19 Thu 05:46:41:037.344 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f26080 2026-03-19 Thu 05:46:41:037.499 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f28580 2026-03-19 Thu 05:46:41:037.550 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f27300 2026-03-19 Thu 05:46:41:037.612 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f269c0 2026-03-19 Thu 05:46:41:037.910 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f26080 2026-03-19 Thu 05:46:41:037.944 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f28580 2026-03-19 Thu 05:46:41:037.959 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f27300 2026-03-19 Thu 05:46:41:037.976 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f269c0 2026-03-19 Thu 05:46:41:066.339 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f25740 2026-03-19 Thu 05:46:41:066.508 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f23bc0 2026-03-19 Thu 05:46:41:066.549 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f244c0 2026-03-19 Thu 05:46:41:066.779 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f23280 2026-03-19 Thu 05:46:41:066.880 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f24e00 2026-03-19 Thu 05:46:41:067.253 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f25740 2026-03-19 Thu 05:46:41:067.321 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f23bc0 2026-03-19 Thu 05:46:41:067.340 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f244c0 2026-03-19 Thu 05:46:41:067.355 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f23280 2026-03-19 Thu 05:46:41:067.369 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f24e00 2026-03-19 Thu 05:46:41:140.009 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f22970 2026-03-19 Thu 05:46:41:140.183 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f22970 2026-03-19 Thu 05:46:41:140.214 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f22360 2026-03-19 Thu 05:46:41:140.253 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f22360 2026-03-19 Thu 05:46:41:140.464 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f1b1d0 2026-03-19 Thu 05:46:41:140.509 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f21dd0 2026-03-19 Thu 05:46:41:140.539 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f215f0 2026-03-19 Thu 05:46:41:140.574 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f1b1d0 2026-03-19 Thu 05:46:41:140.592 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f21dd0 2026-03-19 Thu 05:46:41:140.604 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f215f0 2026-03-19 Thu 05:46:46:029.699 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f27c40 2026-03-19 Thu 05:46:46:030.256 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f27c40 2026-03-19 Thu 05:46:46:037.984 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f26080 2026-03-19 Thu 05:46:46:038.122 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f28580 2026-03-19 Thu 05:46:46:038.173 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f27300 2026-03-19 Thu 05:46:46:038.212 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f269c0 2026-03-19 Thu 05:46:46:038.573 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f26080 2026-03-19 Thu 05:46:46:038.620 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f28580 2026-03-19 Thu 05:46:46:038.642 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f27300 2026-03-19 Thu 05:46:46:038.664 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f269c0 2026-03-19 Thu 05:46:46:067.378 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f25740 2026-03-19 Thu 05:46:46:067.596 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f23bc0 2026-03-19 Thu 05:46:46:067.668 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f244c0 2026-03-19 Thu 05:46:46:067.720 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f23280 2026-03-19 Thu 05:46:46:067.766 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f24e00 2026-03-19 Thu 05:46:46:068.032 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f23bc0 2026-03-19 Thu 05:46:46:068.121 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f25740 2026-03-19 Thu 05:46:46:068.142 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f244c0 2026-03-19 Thu 05:46:46:068.158 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f23280 2026-03-19 Thu 05:46:46:068.174 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f24e00 2026-03-19 Thu 05:46:46:140.277 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f22970 2026-03-19 Thu 05:46:46:140.436 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f22970 2026-03-19 Thu 05:46:46:140.471 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f22360 2026-03-19 Thu 05:46:46:140.525 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f22360 2026-03-19 Thu 05:46:46:140.597 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f1b1d0 2026-03-19 Thu 05:46:46:140.644 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f1b1d0 2026-03-19 Thu 05:46:46:140.657 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f21dd0 2026-03-19 Thu 05:46:46:140.689 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f215f0 2026-03-19 Thu 05:46:46:140.739 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f21dd0 2026-03-19 Thu 05:46:46:140.757 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f215f0 2026-03-19 Thu 05:46:48:085.461 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642059306752, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-03-19 Thu 05:46:48:085.574 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642059306752, Pid 1]: SANDESH: Send FAILED: 1773899208085316 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1.:Database:contrail-query-engine:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvConnectTimerExpired enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 6 dequeues = 6 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshMessageRecv enqueues = 3 dequeues = 3 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 139 dequeues = 139 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpClose enqueues = 6 dequeues = 6 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 2 dequeues = 2 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnected enqueues = 7 dequeues = 7 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 9 dequeues = 9 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 175 dequeues = 175 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1773899193805691 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1773899193806015 ] max_sm_queue_count = 8 ] 2026-03-19 Thu 05:46:48:085.603 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642059306752, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-03-19 Thu 05:46:48:085.612 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642059306752, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-03-19 Thu 05:46:48:085.783 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642059306752, Pid 1]: SANDESH: Send FAILED: 1773899208085436 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1.:Database:contrail-query-engine:0 client_info= [ status = Idle successful_connections = 7 pid = 1 http_port = 8091 start_time = 1773898562749429 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 578189 calls = 69 average_bytes = 8379 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 30 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 11 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 154 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 7 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 42 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 36 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 172, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 66, _iter59->first = sent _iter59->second = 73, ] ] ] 2026-03-19 Thu 05:46:48:085.804 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642059306752, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-03-19 Thu 05:46:49:805.880 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642055108352, Pid 1]: Processing scm::EvIdleHoldTimerExpired in state Idle 2026-03-19 Thu 05:46:49:806.178 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642055108352, Pid 1]: Connect : Start Connect timer 10.0.0.38:8086 2026-03-19 Thu 05:46:49:806.281 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642055108352, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-03-19 Thu 05:46:49:806.288 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: -(-1) OnSessionEvent TCP Connect Failed 2026-03-19 Thu 05:46:49:806.371 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: -(-1):EvTcpConnectFail 2026-03-19 Thu 05:46:49:806.334 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642055108352, Pid 1]: SANDESH: Send FAILED: 1773899209805975 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-03-19 Thu 05:46:49:806.423 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642055108352, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-03-19 Thu 05:46:49:806.434 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642055108352, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-03-19 Thu 05:46:49:806.578 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642055108352, Pid 1]: SANDESH: Send FAILED: 1773899209806263 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1.:Database:contrail-query-engine:0 client_info= [ status = Connect successful_connections = 7 pid = 1 http_port = 8091 start_time = 1773898562749429 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 578189 calls = 69 average_bytes = 8379 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 30 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 11 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 154 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 7 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 42 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 38 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 172, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 68, _iter59->first = sent _iter59->second = 73, ] ] ] 2026-03-19 Thu 05:46:49:806.596 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642055108352, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-03-19 Thu 05:46:49:806.606 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642055108352, Pid 1]: Processing scm::EvTcpConnectFail in state Connect 2026-03-19 Thu 05:46:49:806.614 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642055108352, Pid 1]: Connect : EvTcpConnectFail 2026-03-19 Thu 05:46:49:806.716 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642055108352, Pid 1]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 312 2026-03-19 Thu 05:46:49:806.752 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642055108352, Pid 1]: Idle 2026-03-19 Thu 05:46:49:806.808 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642055108352, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-03-19 Thu 05:46:49:806.951 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642055108352, Pid 1]: SANDESH: Send FAILED: 1773899209806667 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1.:Database:contrail-query-engine:0 client_info= [ status = Connect successful_connections = 7 pid = 1 http_port = 8091 start_time = 1773898562749429 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 578189 calls = 69 average_bytes = 8379 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 31 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 11 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 154 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 7 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 42 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 39 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 172, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 70, _iter59->first = sent _iter59->second = 73, ] ] ] 2026-03-19 Thu 05:46:49:806.966 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642055108352, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-03-19 Thu 05:46:49:806.976 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642055108352, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-03-19 Thu 05:46:49:806.997 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642055108352, Pid 1]: SANDESH: Send FAILED: 1773899209806692 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-03-19 Thu 05:46:49:807.007 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642055108352, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-03-19 Thu 05:46:49:807.017 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642055108352, Pid 1]: Processing scm::EvTcpDeleteSession in state Idle 2026-03-19 Thu 05:46:49:807.036 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642055108352, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-03-19 Thu 05:46:49:807.169 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642055108352, Pid 1]: SANDESH: Send FAILED: 1773899209806800 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1.:Database:contrail-query-engine:0 client_info= [ status = Idle successful_connections = 7 pid = 1 http_port = 8091 start_time = 1773898562749429 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 578189 calls = 69 average_bytes = 8379 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 31 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 11 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 154 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 7 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 42 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 39 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 18 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 173, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 70, _iter59->first = sent _iter59->second = 73, ] ] ] 2026-03-19 Thu 05:46:49:807.182 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642055108352, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-03-19 Thu 05:46:51:030.349 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f27c40 2026-03-19 Thu 05:46:51:030.958 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f27c40 2026-03-19 Thu 05:46:51:038.654 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f26080 2026-03-19 Thu 05:46:51:038.788 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f28580 2026-03-19 Thu 05:46:51:038.825 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f27300 2026-03-19 Thu 05:46:51:038.854 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f269c0 2026-03-19 Thu 05:46:51:039.192 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f26080 2026-03-19 Thu 05:46:51:039.259 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f28580 2026-03-19 Thu 05:46:51:039.275 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f27300 2026-03-19 Thu 05:46:51:039.288 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f269c0 2026-03-19 Thu 05:46:51:068.120 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f23bc0 2026-03-19 Thu 05:46:51:068.298 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f25740 2026-03-19 Thu 05:46:51:068.350 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f244c0 2026-03-19 Thu 05:46:51:068.404 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f23280 2026-03-19 Thu 05:46:51:068.434 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f24e00 2026-03-19 Thu 05:46:51:068.753 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f23bc0 2026-03-19 Thu 05:46:51:068.789 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f25740 2026-03-19 Thu 05:46:51:068.803 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f244c0 2026-03-19 Thu 05:46:51:068.837 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f23280 2026-03-19 Thu 05:46:51:068.873 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f24e00 2026-03-19 Thu 05:46:51:140.505 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f22970 2026-03-19 Thu 05:46:51:140.727 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f22970 2026-03-19 Thu 05:46:51:140.767 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f22360 2026-03-19 Thu 05:46:51:140.802 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f1b1d0 2026-03-19 Thu 05:46:51:140.841 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f22360 2026-03-19 Thu 05:46:51:140.857 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f1b1d0 2026-03-19 Thu 05:46:51:140.880 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f21dd0 2026-03-19 Thu 05:46:51:140.909 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f215f0 2026-03-19 Thu 05:46:51:140.940 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f21dd0 2026-03-19 Thu 05:46:51:140.954 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f215f0 2026-03-19 Thu 05:46:53:097.079 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642038314752, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-03-19 Thu 05:46:53:097.178 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642038314752, Pid 1]: SANDESH: Send FAILED: 1773899213097031 [SYS_NOTICE]: QEDbStatsUve: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1. table_info= [ [ ] ] errors= [ write_tablespace_fails = 0 read_tablespace_fails = 0 write_table_fails = 0 read_table_fails = 0 write_column_fails = 0 write_batch_column_fails = 0 read_column_fails = 0 ] stats_info= [ [ ] ] cql_stats= [ requests_one_minute_rate = 0.4 stats= [ total_connections = 6 available_connections = 6 exceeded_pending_requests_water_mark = 0 exceeded_write_bytes_water_mark = 0 ] errors= [ connection_timeouts = 0 pending_request_timeouts = 0 request_timeouts = 0 ] ] ] 2026-03-19 Thu 05:46:53:097.190 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642038314752, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: QEDbStatsUve 2026-03-19 Thu 05:46:56:031.042 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f27c40 2026-03-19 Thu 05:46:56:031.587 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f27c40 2026-03-19 Thu 05:46:56:039.322 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f26080 2026-03-19 Thu 05:46:56:039.522 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f28580 2026-03-19 Thu 05:46:56:039.585 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f27300 2026-03-19 Thu 05:46:56:039.629 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f269c0 2026-03-19 Thu 05:46:56:039.864 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f26080 2026-03-19 Thu 05:46:56:039.904 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f28580 2026-03-19 Thu 05:46:56:039.926 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f27300 2026-03-19 Thu 05:46:56:039.945 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f269c0 2026-03-19 Thu 05:46:56:068.826 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f23bc0 2026-03-19 Thu 05:46:56:068.995 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f25740 2026-03-19 Thu 05:46:56:069.031 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f244c0 2026-03-19 Thu 05:46:56:069.059 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f23280 2026-03-19 Thu 05:46:56:069.088 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f24e00 2026-03-19 Thu 05:46:56:069.693 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f244c0 2026-03-19 Thu 05:46:56:069.781 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f25740 2026-03-19 Thu 05:46:56:069.800 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f23280 2026-03-19 Thu 05:46:56:069.815 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f23bc0 2026-03-19 Thu 05:46:56:069.830 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f24e00 2026-03-19 Thu 05:46:56:140.793 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f22970 2026-03-19 Thu 05:46:56:140.943 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f22970 2026-03-19 Thu 05:46:56:140.972 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f22360 2026-03-19 Thu 05:46:56:141.005 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f1b1d0 2026-03-19 Thu 05:46:56:141.046 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f22360 2026-03-19 Thu 05:46:56:141.062 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f1b1d0 2026-03-19 Thu 05:46:56:141.072 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f21dd0 2026-03-19 Thu 05:46:56:141.098 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f215f0 2026-03-19 Thu 05:46:56:141.143 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f21dd0 2026-03-19 Thu 05:46:56:149.860 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f215f0 2026-03-19 Thu 05:47:01:031.679 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f27c40 2026-03-19 Thu 05:47:01:032.220 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f27c40 2026-03-19 Thu 05:47:01:039.961 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f26080 2026-03-19 Thu 05:47:01:040.171 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f28580 2026-03-19 Thu 05:47:01:040.242 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f27300 2026-03-19 Thu 05:47:01:040.286 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f269c0 2026-03-19 Thu 05:47:01:040.536 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f26080 2026-03-19 Thu 05:47:01:040.590 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f28580 2026-03-19 Thu 05:47:01:040.615 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f27300 2026-03-19 Thu 05:47:01:040.635 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f269c0 2026-03-19 Thu 05:47:01:069.814 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f244c0 2026-03-19 Thu 05:47:01:070.005 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f25740 2026-03-19 Thu 05:47:01:070.061 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f23280 2026-03-19 Thu 05:47:01:070.107 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f23bc0 2026-03-19 Thu 05:47:01:070.140 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f24e00 2026-03-19 Thu 05:47:01:070.485 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f25740 2026-03-19 Thu 05:47:01:070.530 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f23280 2026-03-19 Thu 05:47:01:070.548 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f24e00 2026-03-19 Thu 05:47:01:070.562 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f23bc0 2026-03-19 Thu 05:47:01:070.695 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f244c0 2026-03-19 Thu 05:47:01:141.034 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f22970 2026-03-19 Thu 05:47:01:141.293 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f22970 2026-03-19 Thu 05:47:01:141.348 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f22360 2026-03-19 Thu 05:47:01:141.423 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f1b1d0 2026-03-19 Thu 05:47:01:141.455 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f21dd0 2026-03-19 Thu 05:47:01:141.491 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f22360 2026-03-19 Thu 05:47:01:141.508 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f1b1d0 2026-03-19 Thu 05:47:01:141.520 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f21dd0 2026-03-19 Thu 05:47:01:150.010 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f215f0 2026-03-19 Thu 05:47:01:150.182 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f215f0 2026-03-19 Thu 05:47:06:032.302 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f27c40 2026-03-19 Thu 05:47:06:032.767 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f27c40 2026-03-19 Thu 05:47:06:040.619 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f26080 2026-03-19 Thu 05:47:06:040.752 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f28580 2026-03-19 Thu 05:47:06:040.789 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f27300 2026-03-19 Thu 05:47:06:040.818 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f269c0 2026-03-19 Thu 05:47:06:041.151 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f26080 2026-03-19 Thu 05:47:06:041.183 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f269c0 2026-03-19 Thu 05:47:06:041.198 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f28580 2026-03-19 Thu 05:47:06:041.212 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f27300 2026-03-19 Thu 05:47:06:070.585 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f25740 2026-03-19 Thu 05:47:06:070.711 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f23280 2026-03-19 Thu 05:47:06:070.750 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f24e00 2026-03-19 Thu 05:47:06:070.779 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f23bc0 2026-03-19 Thu 05:47:06:070.807 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f244c0 2026-03-19 Thu 05:47:06:071.186 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f25740 2026-03-19 Thu 05:47:06:071.215 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f24e00 2026-03-19 Thu 05:47:06:071.262 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f244c0 2026-03-19 Thu 05:47:06:071.278 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f23280 2026-03-19 Thu 05:47:06:071.407 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f23bc0 2026-03-19 Thu 05:47:06:145.859 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f22970 2026-03-19 Thu 05:47:06:146.013 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f22360 2026-03-19 Thu 05:47:06:146.054 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f1b1d0 2026-03-19 Thu 05:47:06:146.082 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f21dd0 2026-03-19 Thu 05:47:06:146.123 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f22970 2026-03-19 Thu 05:47:06:146.148 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f22360 2026-03-19 Thu 05:47:06:146.161 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f1b1d0 2026-03-19 Thu 05:47:06:146.173 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f21dd0 2026-03-19 Thu 05:47:06:152.371 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f215f0 2026-03-19 Thu 05:47:06:152.492 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f215f0 2026-03-19 Thu 05:47:11:032.842 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f27c40 2026-03-19 Thu 05:47:11:033.336 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f27c40 2026-03-19 Thu 05:47:11:041.236 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f26080 2026-03-19 Thu 05:47:11:041.368 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f269c0 2026-03-19 Thu 05:47:11:041.492 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f28580 2026-03-19 Thu 05:47:11:041.525 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f27300 2026-03-19 Thu 05:47:11:041.662 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f26080 2026-03-19 Thu 05:47:11:041.689 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f269c0 2026-03-19 Thu 05:47:11:041.725 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f28580 2026-03-19 Thu 05:47:11:041.759 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f27300 2026-03-19 Thu 05:47:11:071.291 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f25740 2026-03-19 Thu 05:47:11:071.427 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f24e00 2026-03-19 Thu 05:47:11:071.471 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f244c0 2026-03-19 Thu 05:47:11:071.505 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f23280 2026-03-19 Thu 05:47:11:071.539 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f23bc0 2026-03-19 Thu 05:47:11:071.927 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f25740 2026-03-19 Thu 05:47:11:072.082 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f24e00 2026-03-19 Thu 05:47:11:072.225 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f23280 2026-03-19 Thu 05:47:11:072.266 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f244c0 2026-03-19 Thu 05:47:11:072.281 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f23bc0 2026-03-19 Thu 05:47:11:146.226 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f22970 2026-03-19 Thu 05:47:11:146.439 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f22360 2026-03-19 Thu 05:47:11:146.511 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f22970 2026-03-19 Thu 05:47:11:146.554 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f22360 2026-03-19 Thu 05:47:11:146.570 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f1b1d0 2026-03-19 Thu 05:47:11:146.614 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f21dd0 2026-03-19 Thu 05:47:11:146.657 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f1b1d0 2026-03-19 Thu 05:47:11:146.677 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f21dd0 2026-03-19 Thu 05:47:11:152.605 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f215f0 2026-03-19 Thu 05:47:11:152.828 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f215f0 2026-03-19 Thu 05:47:16:033.442 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f27c40 2026-03-19 Thu 05:47:16:034.012 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f27c40 2026-03-19 Thu 05:47:16:041.768 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f26080 2026-03-19 Thu 05:47:16:041.939 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f269c0 2026-03-19 Thu 05:47:16:041.978 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f28580 2026-03-19 Thu 05:47:16:042.008 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f27300 2026-03-19 Thu 05:47:16:042.310 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f26080 2026-03-19 Thu 05:47:16:042.360 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f269c0 2026-03-19 Thu 05:47:16:042.376 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f28580 2026-03-19 Thu 05:47:16:042.434 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f27300 2026-03-19 Thu 05:47:16:071.997 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f25740 2026-03-19 Thu 05:47:16:072.169 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f24e00 2026-03-19 Thu 05:47:16:072.278 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f23280 2026-03-19 Thu 05:47:16:072.321 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f244c0 2026-03-19 Thu 05:47:16:072.359 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f23bc0 2026-03-19 Thu 05:47:16:072.702 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f24e00 2026-03-19 Thu 05:47:16:072.731 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f23280 2026-03-19 Thu 05:47:16:072.745 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f25740 2026-03-19 Thu 05:47:16:072.767 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f244c0 2026-03-19 Thu 05:47:16:072.927 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f23bc0 2026-03-19 Thu 05:47:16:146.589 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f22970 2026-03-19 Thu 05:47:16:146.745 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f22970 2026-03-19 Thu 05:47:16:146.773 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f22360 2026-03-19 Thu 05:47:16:146.819 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f1b1d0 2026-03-19 Thu 05:47:16:146.848 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f21dd0 2026-03-19 Thu 05:47:16:146.882 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f22360 2026-03-19 Thu 05:47:16:146.899 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f1b1d0 2026-03-19 Thu 05:47:16:146.912 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f21dd0 2026-03-19 Thu 05:47:16:152.901 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f215f0 2026-03-19 Thu 05:47:16:153.045 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f215f0 2026-03-19 Thu 05:47:18:086.064 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642055108352, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 11 without seqno , total = 0 2026-03-19 Thu 05:47:18:086.147 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642055108352, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-03-19 Thu 05:47:18:086.220 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642055108352, Pid 1]: SANDESH: Send FAILED: 1773899238085939 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1.:Database:contrail-query-engine:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvConnectTimerExpired enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 7 dequeues = 7 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshMessageRecv enqueues = 3 dequeues = 3 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 147 dequeues = 147 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpClose enqueues = 6 dequeues = 6 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 3 dequeues = 3 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnected enqueues = 7 dequeues = 7 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 10 dequeues = 10 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 186 dequeues = 186 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1773899209806751 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1773899213097062 ] max_sm_queue_count = 8 ] 2026-03-19 Thu 05:47:18:086.234 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642055108352, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-03-19 Thu 05:47:18:086.244 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642055108352, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-03-19 Thu 05:47:18:086.401 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642055108352, Pid 1]: SANDESH: Send FAILED: 1773899238086044 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1.:Database:contrail-query-engine:0 client_info= [ status = Idle successful_connections = 7 pid = 1 http_port = 8091 start_time = 1773898562749429 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 578189 calls = 69 average_bytes = 8379 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 32 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 11 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 154 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 7 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 42 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 41 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 18 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 173, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 74, _iter59->first = sent _iter59->second = 73, ] ] ] 2026-03-19 Thu 05:47:18:086.425 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642055108352, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-03-19 Thu 05:47:19:811.225 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140641505892096, Pid 1]: Processing scm::EvIdleHoldTimerExpired in state Idle 2026-03-19 Thu 05:47:19:811.491 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140641505892096, Pid 1]: Connect : Start Connect timer 10.0.0.33:8086 2026-03-19 Thu 05:47:19:811.663 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140641505892096, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-03-19 Thu 05:47:19:811.719 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140641505892096, Pid 1]: SANDESH: Send FAILED: 1773899239811304 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-03-19 Thu 05:47:19:811.734 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140641505892096, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-03-19 Thu 05:47:19:811.748 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140641505892096, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-03-19 Thu 05:47:19:811.960 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140641505892096, Pid 1]: SANDESH: Send FAILED: 1773899239811642 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1.:Database:contrail-query-engine:0 client_info= [ status = Connect successful_connections = 7 pid = 1 http_port = 8091 start_time = 1773898562749429 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 578189 calls = 69 average_bytes = 8379 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 32 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 11 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 154 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 7 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 42 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 43 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 18 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 173, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 76, _iter59->first = sent _iter59->second = 73, ] ] ] 2026-03-19 Thu 05:47:19:811.992 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140641505892096, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-03-19 Thu 05:47:19:822.193 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: -(-1) OnSessionEvent TCP Connect Failed 2026-03-19 Thu 05:47:19:823.750 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: -(-1):EvTcpConnectFail 2026-03-19 Thu 05:47:19:826.900 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642055108352, Pid 1]: Processing scm::EvTcpConnectFail in state Connect 2026-03-19 Thu 05:47:19:828.239 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642055108352, Pid 1]: Connect : EvTcpConnectFail 2026-03-19 Thu 05:47:19:828.410 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642055108352, Pid 1]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 312 2026-03-19 Thu 05:47:19:828.450 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642055108352, Pid 1]: Idle 2026-03-19 Thu 05:47:19:828.512 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642055108352, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-03-19 Thu 05:47:19:828.660 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642055108352, Pid 1]: SANDESH: Send FAILED: 1773899239828350 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1.:Database:contrail-query-engine:0 client_info= [ status = Connect successful_connections = 7 pid = 1 http_port = 8091 start_time = 1773898562749429 collector_name = collector_ip = 10.0.0.50:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 578189 calls = 69 average_bytes = 8379 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 33 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 11 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 154 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 7 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 42 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 44 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 18 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 173, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 78, _iter59->first = sent _iter59->second = 73, ] ] ] 2026-03-19 Thu 05:47:19:828.683 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642055108352, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-03-19 Thu 05:47:19:828.694 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642055108352, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-03-19 Thu 05:47:19:828.724 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642055108352, Pid 1]: SANDESH: Send FAILED: 1773899239828385 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-03-19 Thu 05:47:19:828.735 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642055108352, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-03-19 Thu 05:47:19:828.749 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642055108352, Pid 1]: Processing scm::EvTcpDeleteSession in state Idle 2026-03-19 Thu 05:47:19:828.785 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642055108352, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-03-19 Thu 05:47:19:828.922 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642055108352, Pid 1]: SANDESH: Send FAILED: 1773899239828502 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1.:Database:contrail-query-engine:0 client_info= [ status = Idle successful_connections = 7 pid = 1 http_port = 8091 start_time = 1773898562749429 collector_name = collector_ip = 10.0.0.50:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 578189 calls = 69 average_bytes = 8379 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 33 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 11 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 154 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 7 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 42 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 44 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 19 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 174, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 78, _iter59->first = sent _iter59->second = 73, ] ] ] 2026-03-19 Thu 05:47:19:828.938 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642055108352, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-03-19 Thu 05:47:21:037.922 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f27c40 2026-03-19 Thu 05:47:21:038.386 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f27c40 2026-03-19 Thu 05:47:21:042.384 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f26080 2026-03-19 Thu 05:47:21:042.518 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f269c0 2026-03-19 Thu 05:47:21:042.568 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f28580 2026-03-19 Thu 05:47:21:042.607 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f27300 2026-03-19 Thu 05:47:21:042.791 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f26080 2026-03-19 Thu 05:47:21:042.849 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f28580 2026-03-19 Thu 05:47:21:042.866 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f269c0 2026-03-19 Thu 05:47:21:042.900 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f27300 2026-03-19 Thu 05:47:21:072.781 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f24e00 2026-03-19 Thu 05:47:21:072.937 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f23280 2026-03-19 Thu 05:47:21:072.978 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f25740 2026-03-19 Thu 05:47:21:073.014 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f244c0 2026-03-19 Thu 05:47:21:073.046 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f23bc0 2026-03-19 Thu 05:47:21:073.414 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f24e00 2026-03-19 Thu 05:47:21:073.616 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f25740 2026-03-19 Thu 05:47:21:073.639 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f23280 2026-03-19 Thu 05:47:21:073.653 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f244c0 2026-03-19 Thu 05:47:21:073.666 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f23bc0 2026-03-19 Thu 05:47:21:146.811 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f22970 2026-03-19 Thu 05:47:21:146.973 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f22970 2026-03-19 Thu 05:47:21:146.998 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f22360 2026-03-19 Thu 05:47:21:147.033 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f1b1d0 2026-03-19 Thu 05:47:21:147.062 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f21dd0 2026-03-19 Thu 05:47:21:147.095 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f22360 2026-03-19 Thu 05:47:21:147.110 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f1b1d0 2026-03-19 Thu 05:47:21:147.122 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f21dd0 2026-03-19 Thu 05:47:21:153.096 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f215f0 2026-03-19 Thu 05:47:21:153.228 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f215f0 2026-03-19 Thu 05:47:26:038.627 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f27c40 2026-03-19 Thu 05:47:26:039.101 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f27c40 2026-03-19 Thu 05:47:26:051.202 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f26080 2026-03-19 Thu 05:47:26:051.355 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f28580 2026-03-19 Thu 05:47:26:051.394 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f269c0 2026-03-19 Thu 05:47:26:051.424 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f27300 2026-03-19 Thu 05:47:26:051.722 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f26080 2026-03-19 Thu 05:47:26:051.756 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f269c0 2026-03-19 Thu 05:47:26:051.770 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f28580 2026-03-19 Thu 05:47:26:051.783 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f27300 2026-03-19 Thu 05:47:26:081.644 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f24e00 2026-03-19 Thu 05:47:26:081.822 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f25740 2026-03-19 Thu 05:47:26:081.861 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f23280 2026-03-19 Thu 05:47:26:081.891 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f244c0 2026-03-19 Thu 05:47:26:081.919 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f23bc0 2026-03-19 Thu 05:47:26:085.812 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f24e00 2026-03-19 Thu 05:47:26:085.886 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f23280 2026-03-19 Thu 05:47:26:085.904 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f25740 2026-03-19 Thu 05:47:26:085.919 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f244c0 2026-03-19 Thu 05:47:26:085.934 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f23bc0 2026-03-19 Thu 05:47:26:156.104 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f22970 2026-03-19 Thu 05:47:26:156.255 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f22360 2026-03-19 Thu 05:47:26:156.291 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f1b1d0 2026-03-19 Thu 05:47:26:156.320 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f21dd0 2026-03-19 Thu 05:47:26:156.360 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f215f0 2026-03-19 Thu 05:47:26:156.402 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f22970 2026-03-19 Thu 05:47:26:156.426 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f22360 2026-03-19 Thu 05:47:26:156.440 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f1b1d0 2026-03-19 Thu 05:47:26:156.452 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f21dd0 2026-03-19 Thu 05:47:26:156.463 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f215f0 2026-03-19 Thu 05:47:31:039.193 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f27c40 2026-03-19 Thu 05:47:31:039.819 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f27c40 2026-03-19 Thu 05:47:31:051.798 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f26080 2026-03-19 Thu 05:47:31:051.911 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f269c0 2026-03-19 Thu 05:47:31:051.952 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f28580 2026-03-19 Thu 05:47:31:051.981 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f27300 2026-03-19 Thu 05:47:31:052.229 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f26080 2026-03-19 Thu 05:47:31:052.261 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f269c0 2026-03-19 Thu 05:47:31:052.276 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f28580 2026-03-19 Thu 05:47:31:052.290 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f27300 2026-03-19 Thu 05:47:31:086.363 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f24e00 2026-03-19 Thu 05:47:31:086.529 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f23280 2026-03-19 Thu 05:47:31:086.571 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f25740 2026-03-19 Thu 05:47:31:086.606 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f244c0 2026-03-19 Thu 05:47:31:086.644 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f23bc0 2026-03-19 Thu 05:47:31:087.181 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f24e00 2026-03-19 Thu 05:47:31:087.229 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f23280 2026-03-19 Thu 05:47:31:087.253 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f244c0 2026-03-19 Thu 05:47:31:087.272 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f25740 2026-03-19 Thu 05:47:31:087.293 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f23bc0 2026-03-19 Thu 05:47:31:162.276 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f22970 2026-03-19 Thu 05:47:31:162.446 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f22360 2026-03-19 Thu 05:47:31:162.491 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f1b1d0 2026-03-19 Thu 05:47:31:162.530 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f21dd0 2026-03-19 Thu 05:47:31:162.568 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f215f0 2026-03-19 Thu 05:47:31:162.625 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f22970 2026-03-19 Thu 05:47:31:162.653 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f22360 2026-03-19 Thu 05:47:31:162.671 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f1b1d0 2026-03-19 Thu 05:47:31:162.698 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f21dd0 2026-03-19 Thu 05:47:31:162.715 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f215f0 2026-03-19 Thu 05:47:36:039.926 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f27c40 2026-03-19 Thu 05:47:36:040.390 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f27c40 2026-03-19 Thu 05:47:36:052.302 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f26080 2026-03-19 Thu 05:47:36:052.550 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f269c0 2026-03-19 Thu 05:47:36:052.589 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f28580 2026-03-19 Thu 05:47:36:052.619 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f27300 2026-03-19 Thu 05:47:36:052.834 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f26080 2026-03-19 Thu 05:47:36:052.865 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f269c0 2026-03-19 Thu 05:47:36:052.879 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f28580 2026-03-19 Thu 05:47:36:052.892 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f27300 2026-03-19 Thu 05:47:36:087.297 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f24e00 2026-03-19 Thu 05:47:36:087.534 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f23280 2026-03-19 Thu 05:47:36:087.603 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f244c0 2026-03-19 Thu 05:47:36:087.645 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f25740 2026-03-19 Thu 05:47:36:087.685 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f23bc0 2026-03-19 Thu 05:47:36:088.107 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f24e00 2026-03-19 Thu 05:47:36:088.602 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f23280 2026-03-19 Thu 05:47:36:088.659 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f244c0 2026-03-19 Thu 05:47:36:088.702 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f25740 2026-03-19 Thu 05:47:36:088.746 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f23bc0 2026-03-19 Thu 05:47:36:162.712 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f22970 2026-03-19 Thu 05:47:36:162.921 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f22360 2026-03-19 Thu 05:47:36:163.010 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f22970 2026-03-19 Thu 05:47:36:163.044 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f22360 2026-03-19 Thu 05:47:36:163.071 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f1b1d0 2026-03-19 Thu 05:47:36:163.114 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f21dd0 2026-03-19 Thu 05:47:36:163.152 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f215f0 2026-03-19 Thu 05:47:36:163.197 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f1b1d0 2026-03-19 Thu 05:47:36:163.217 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f21dd0 2026-03-19 Thu 05:47:36:163.235 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f215f0 2026-03-19 Thu 05:47:41:040.478 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f27c40 2026-03-19 Thu 05:47:41:040.905 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f27c40 2026-03-19 Thu 05:47:41:052.901 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f26080 2026-03-19 Thu 05:47:41:053.058 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f269c0 2026-03-19 Thu 05:47:41:053.093 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f28580 2026-03-19 Thu 05:47:41:053.124 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f27300 2026-03-19 Thu 05:47:41:053.454 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f269c0 2026-03-19 Thu 05:47:41:053.499 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f26080 2026-03-19 Thu 05:47:41:053.575 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f27300 2026-03-19 Thu 05:47:41:053.603 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f28580 2026-03-19 Thu 05:47:41:088.603 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f24e00 2026-03-19 Thu 05:47:41:088.745 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f23280 2026-03-19 Thu 05:47:41:088.789 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f244c0 2026-03-19 Thu 05:47:41:088.820 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f25740 2026-03-19 Thu 05:47:41:088.852 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f23bc0 2026-03-19 Thu 05:47:41:089.341 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f25740 2026-03-19 Thu 05:47:41:089.384 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f24e00 2026-03-19 Thu 05:47:41:089.402 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f23bc0 2026-03-19 Thu 05:47:41:089.416 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f244c0 2026-03-19 Thu 05:47:41:089.439 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f23280 2026-03-19 Thu 05:47:41:163.083 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f22970 2026-03-19 Thu 05:47:41:163.243 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f22970 2026-03-19 Thu 05:47:41:163.268 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f22360 2026-03-19 Thu 05:47:41:163.303 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f1b1d0 2026-03-19 Thu 05:47:41:163.349 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f22360 2026-03-19 Thu 05:47:41:163.365 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f1b1d0 2026-03-19 Thu 05:47:41:163.375 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f21dd0 2026-03-19 Thu 05:47:41:163.404 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f215f0 2026-03-19 Thu 05:47:41:163.433 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f21dd0 2026-03-19 Thu 05:47:41:163.447 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: -1 0x1f215f0 2026-03-19 Thu 05:47:46:041.146 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f27c40 2026-03-19 Thu 05:47:46:041.542 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: 0 0x1f27c40 2026-03-19 Thu 05:47:46:041.553 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: Connected to REDIS... 2026-03-19 Thu 05:47:46:041.577 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 3 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-03-19 Thu 05:47:46:042.347 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-03-19 Thu 05:47:46:042.367 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-03-19 Thu 05:47:46:042.393 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB3 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-03-19 Thu 05:47:46:053.554 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f269c0 2026-03-19 Thu 05:47:46:053.695 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f26080 2026-03-19 Thu 05:47:46:053.790 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f27300 2026-03-19 Thu 05:47:46:053.826 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f28580 2026-03-19 Thu 05:47:46:054.074 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: 0 0x1f269c0 2026-03-19 Thu 05:47:46:054.087 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: Connected to REDIS... 2026-03-19 Thu 05:47:46:054.124 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 1 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-03-19 Thu 05:47:46:054.151 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: 0 0x1f26080 2026-03-19 Thu 05:47:46:054.157 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: Connected to REDIS... 2026-03-19 Thu 05:47:46:054.165 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 0 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-03-19 Thu 05:47:46:054.173 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: 0 0x1f27300 2026-03-19 Thu 05:47:46:054.179 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: Connected to REDIS... 2026-03-19 Thu 05:47:46:054.186 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 2 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-03-19 Thu 05:47:46:054.229 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: 0 0x1f28580 2026-03-19 Thu 05:47:46:054.235 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: Connected to REDIS... 2026-03-19 Thu 05:47:46:054.243 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 4 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-03-19 Thu 05:47:46:054.711 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-03-19 Thu 05:47:46:054.742 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-03-19 Thu 05:47:46:054.757 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-03-19 Thu 05:47:46:054.766 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-03-19 Thu 05:47:46:054.779 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-03-19 Thu 05:47:46:054.788 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-03-19 Thu 05:47:46:054.801 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-03-19 Thu 05:47:46:054.810 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-03-19 Thu 05:47:46:054.892 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB2 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-03-19 Thu 05:47:46:054.913 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB0 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-03-19 Thu 05:47:46:054.944 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB4 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-03-19 Thu 05:47:46:054.959 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB1 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-03-19 Thu 05:47:46:054.920 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140641505892096, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-03-19 Thu 05:47:46:057.341 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140641505892096, Pid 1]: SANDESH: Send FAILED: 1773899266054849 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = 10.0.0.38:6379::Down.10.0.0.50:6379::Down. ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-03-19 Thu 05:47:46:057.352 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140641505892096, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-03-19 Thu 05:47:46:089.433 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f25740 2026-03-19 Thu 05:47:46:089.566 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f24e00 2026-03-19 Thu 05:47:46:089.615 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f23bc0 2026-03-19 Thu 05:47:46:089.644 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f244c0 2026-03-19 Thu 05:47:46:089.673 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f23280 2026-03-19 Thu 05:47:46:090.022 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: 0 0x1f25740 2026-03-19 Thu 05:47:46:090.046 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: Connected to REDIS... 2026-03-19 Thu 05:47:46:090.075 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 4 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-03-19 Thu 05:47:46:090.089 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: 0 0x1f24e00 2026-03-19 Thu 05:47:46:090.095 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: Connected to REDIS... 2026-03-19 Thu 05:47:46:090.104 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 3 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-03-19 Thu 05:47:46:090.114 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: 0 0x1f23bc0 2026-03-19 Thu 05:47:46:090.119 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: Connected to REDIS... 2026-03-19 Thu 05:47:46:090.127 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 1 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-03-19 Thu 05:47:46:090.187 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: 0 0x1f23280 2026-03-19 Thu 05:47:46:090.194 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: Connected to REDIS... 2026-03-19 Thu 05:47:46:090.202 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 0 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-03-19 Thu 05:47:46:090.219 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: 0 0x1f244c0 2026-03-19 Thu 05:47:46:090.225 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: Connected to REDIS... 2026-03-19 Thu 05:47:46:090.233 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 2 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-03-19 Thu 05:47:46:090.844 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-03-19 Thu 05:47:46:090.889 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-03-19 Thu 05:47:46:090.906 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-03-19 Thu 05:47:46:090.917 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-03-19 Thu 05:47:46:090.946 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB4 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-03-19 Thu 05:47:46:091.852 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642050909952, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-03-19 Thu 05:47:46:091.929 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642050909952, Pid 1]: SANDESH: Send FAILED: 1773899266090984 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = 10.0.0.38:6379::Down. ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-03-19 Thu 05:47:46:091.943 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642050909952, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-03-19 Thu 05:47:46:091.982 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB3 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-03-19 Thu 05:47:46:092.014 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-03-19 Thu 05:47:46:092.034 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-03-19 Thu 05:47:46:092.049 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-03-19 Thu 05:47:46:092.058 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-03-19 Thu 05:47:46:092.072 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-03-19 Thu 05:47:46:092.081 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-03-19 Thu 05:47:46:092.122 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB2 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-03-19 Thu 05:47:46:092.138 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB0 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-03-19 Thu 05:47:46:092.152 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB1 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-03-19 Thu 05:47:46:163.525 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f22970 2026-03-19 Thu 05:47:46:163.763 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: 0 0x1f22970 2026-03-19 Thu 05:47:46:163.776 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: Connected to REDIS... 2026-03-19 Thu 05:47:46:163.808 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 4 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-03-19 Thu 05:47:46:163.819 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f22360 2026-03-19 Thu 05:47:46:163.873 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f1b1d0 2026-03-19 Thu 05:47:46:163.920 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f21dd0 2026-03-19 Thu 05:47:46:163.965 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RedisAsyncConnection::RAC_Reconnect initiated 0x1f215f0 2026-03-19 Thu 05:47:46:164.183 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: 0 0x1f22360 2026-03-19 Thu 05:47:46:164.198 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: Connected to REDIS... 2026-03-19 Thu 05:47:46:164.213 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 3 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-03-19 Thu 05:47:46:164.226 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: 0 0x1f1b1d0 2026-03-19 Thu 05:47:46:164.234 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: Connected to REDIS... 2026-03-19 Thu 05:47:46:164.267 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 2 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-03-19 Thu 05:47:46:164.278 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: 0 0x1f21dd0 2026-03-19 Thu 05:47:46:164.286 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: Connected to REDIS... 2026-03-19 Thu 05:47:46:164.298 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 1 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-03-19 Thu 05:47:46:164.320 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: RAC_Connect status: 0 0x1f215f0 2026-03-19 Thu 05:47:46:164.328 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: Connected to REDIS... 2026-03-19 Thu 05:47:46:164.338 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 0 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 973 2026-03-19 Thu 05:47:46:164.380 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-03-19 Thu 05:47:46:164.425 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-03-19 Thu 05:47:46:164.523 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB4 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-03-19 Thu 05:47:46:164.544 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-03-19 Thu 05:47:46:164.553 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-03-19 Thu 05:47:46:164.599 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-03-19 Thu 05:47:46:164.610 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-03-19 Thu 05:47:46:164.763 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB3 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-03-19 Thu 05:47:46:164.786 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB0 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-03-19 Thu 05:47:46:164.802 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-03-19 Thu 05:47:46:164.807 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140641497495296, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-03-19 Thu 05:47:46:164.899 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140641497495296, Pid 1]: SANDESH: Send FAILED: 1773899266164666 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-03-19 Thu 05:47:46:164.925 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140641497495296, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-03-19 Thu 05:47:46:164.813 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-03-19 Thu 05:47:46:166.023 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1025 2026-03-19 Thu 05:47:46:166.178 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1032 2026-03-19 Thu 05:47:46:166.201 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB1 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-03-19 Thu 05:47:46:166.215 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB2 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 682 2026-03-19 Thu 05:47:48:087.197 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140641497495296, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-03-19 Thu 05:47:48:087.417 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140641497495296, Pid 1]: SANDESH: Send FAILED: 1773899268087008 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1.:Database:contrail-query-engine:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvConnectTimerExpired enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 8 dequeues = 8 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshMessageRecv enqueues = 3 dequeues = 3 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 157 dequeues = 157 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpClose enqueues = 6 dequeues = 6 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 4 dequeues = 4 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnected enqueues = 7 dequeues = 7 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 11 dequeues = 11 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 199 dequeues = 199 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1773899239828449 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1773899266164773 ] max_sm_queue_count = 8 ] 2026-03-19 Thu 05:47:48:087.436 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140641497495296, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-03-19 Thu 05:47:48:087.447 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140641497495296, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-03-19 Thu 05:47:48:087.606 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140641497495296, Pid 1]: SANDESH: Send FAILED: 1773899268087160 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1.:Database:contrail-query-engine:0 client_info= [ status = Idle successful_connections = 7 pid = 1 http_port = 8091 start_time = 1773898562749429 collector_name = collector_ip = 10.0.0.50:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 578189 calls = 69 average_bytes = 8379 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 37 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 11 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 214 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 7 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 42 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 46 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 19 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 234, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 84, _iter59->first = sent _iter59->second = 73, ] ] ] 2026-03-19 Thu 05:47:48:087.621 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140641497495296, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-03-19 Thu 05:47:49:829.694 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642055108352, Pid 1]: Processing scm::EvIdleHoldTimerExpired in state Idle 2026-03-19 Thu 05:47:49:829.926 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642055108352, Pid 1]: Connect : Start Connect timer 10.0.0.50:8086 2026-03-19 Thu 05:47:49:830.024 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642055108352, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-03-19 Thu 05:47:49:830.053 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642055108352, Pid 1]: SANDESH: Send FAILED: 1773899269829793 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.50:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-03-19 Thu 05:47:49:830.070 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642055108352, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-03-19 Thu 05:47:49:830.080 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642055108352, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-03-19 Thu 05:47:49:830.220 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642055108352, Pid 1]: SANDESH: Send FAILED: 1773899269830012 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1.:Database:contrail-query-engine:0 client_info= [ status = Connect successful_connections = 7 pid = 1 http_port = 8091 start_time = 1773898562749429 collector_name = collector_ip = 10.0.0.50:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 578189 calls = 69 average_bytes = 8379 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 37 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 11 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 214 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 7 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 42 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 48 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 19 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 234, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 86, _iter59->first = sent _iter59->second = 73, ] ] ] 2026-03-19 Thu 05:47:49:830.234 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642055108352, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-03-19 Thu 05:47:49:832.176 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: -(-1) OnSessionEvent TCP Connect Failed 2026-03-19 Thu 05:47:49:832.187 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: -(-1):EvTcpConnectFail 2026-03-19 Thu 05:47:49:834.640 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642021521152, Pid 1]: Processing scm::EvTcpConnectFail in state Connect 2026-03-19 Thu 05:47:49:834.661 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642021521152, Pid 1]: Connect : EvTcpConnectFail 2026-03-19 Thu 05:47:49:834.783 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642021521152, Pid 1]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 312 2026-03-19 Thu 05:47:49:834.817 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642021521152, Pid 1]: Idle 2026-03-19 Thu 05:47:49:834.883 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642021521152, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-03-19 Thu 05:47:49:835.024 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642021521152, Pid 1]: SANDESH: Send FAILED: 1773899269834732 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1.:Database:contrail-query-engine:0 client_info= [ status = Connect successful_connections = 7 pid = 1 http_port = 8091 start_time = 1773898562749429 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 578189 calls = 69 average_bytes = 8379 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 38 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 11 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 214 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 7 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 42 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 49 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 19 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 234, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 88, _iter59->first = sent _iter59->second = 73, ] ] ] 2026-03-19 Thu 05:47:49:835.040 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642021521152, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-03-19 Thu 05:47:49:835.050 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642021521152, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-03-19 Thu 05:47:49:835.072 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642021521152, Pid 1]: SANDESH: Send FAILED: 1773899269834760 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-03-19 Thu 05:47:49:835.080 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642021521152, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-03-19 Thu 05:47:49:835.091 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642021521152, Pid 1]: Processing scm::EvTcpDeleteSession in state Idle 2026-03-19 Thu 05:47:49:835.101 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642021521152, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-03-19 Thu 05:47:49:835.227 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642021521152, Pid 1]: SANDESH: Send FAILED: 1773899269834874 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1.:Database:contrail-query-engine:0 client_info= [ status = Idle successful_connections = 7 pid = 1 http_port = 8091 start_time = 1773898562749429 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 578189 calls = 69 average_bytes = 8379 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 38 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 11 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 214 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 7 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 42 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 49 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 20 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 235, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 88, _iter59->first = sent _iter59->second = 73, ] ] ] 2026-03-19 Thu 05:47:49:835.239 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642021521152, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-03-19 Thu 05:47:53:099.411 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642042513152, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-03-19 Thu 05:47:53:099.507 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642042513152, Pid 1]: SANDESH: Send FAILED: 1773899273099364 [SYS_NOTICE]: QEDbStatsUve: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1. table_info= [ [ ] ] errors= [ write_tablespace_fails = 0 read_tablespace_fails = 0 write_table_fails = 0 read_table_fails = 0 write_column_fails = 0 write_batch_column_fails = 0 read_column_fails = 0 ] stats_info= [ [ ] ] cql_stats= [ requests_one_minute_rate = 0.4 stats= [ total_connections = 6 available_connections = 6 exceeded_pending_requests_water_mark = 0 exceeded_write_bytes_water_mark = 0 ] errors= [ connection_timeouts = 0 pending_request_timeouts = 0 request_timeouts = 0 ] ] ] 2026-03-19 Thu 05:47:53:099.519 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642042513152, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: QEDbStatsUve 2026-03-19 Thu 05:48:18:088.772 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642042513152, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 12 without seqno , total = 0 2026-03-19 Thu 05:48:18:088.834 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642042513152, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-03-19 Thu 05:48:18:088.901 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642042513152, Pid 1]: SANDESH: Send FAILED: 1773899298088642 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1.:Database:contrail-query-engine:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvConnectTimerExpired enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 9 dequeues = 9 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshMessageRecv enqueues = 3 dequeues = 3 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 165 dequeues = 165 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpClose enqueues = 6 dequeues = 6 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 5 dequeues = 5 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnected enqueues = 7 dequeues = 7 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 12 dequeues = 12 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 210 dequeues = 210 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1773899269834816 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1773899273099385 ] max_sm_queue_count = 8 ] 2026-03-19 Thu 05:48:18:088.914 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642042513152, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-03-19 Thu 05:48:18:088.923 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642042513152, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-03-19 Thu 05:48:18:090.469 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642042513152, Pid 1]: SANDESH: Send FAILED: 1773899298088753 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1.:Database:contrail-query-engine:0 client_info= [ status = Idle successful_connections = 7 pid = 1 http_port = 8091 start_time = 1773898562749429 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 578189 calls = 69 average_bytes = 8379 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 39 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 11 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 214 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 7 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 42 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 51 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 20 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 235, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 92, _iter59->first = sent _iter59->second = 73, ] ] ] 2026-03-19 Thu 05:48:18:090.505 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642042513152, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-03-19 Thu 05:48:19:836.425 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140641497495296, Pid 1]: Processing scm::EvIdleHoldTimerExpired in state Idle 2026-03-19 Thu 05:48:19:836.704 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140641497495296, Pid 1]: Connect : Start Connect timer 10.0.0.38:8086 2026-03-19 Thu 05:48:19:836.810 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140641497495296, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-03-19 Thu 05:48:19:836.840 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140641497495296, Pid 1]: SANDESH: Send FAILED: 1773899299836518 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-03-19 Thu 05:48:19:836.852 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140641497495296, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-03-19 Thu 05:48:19:836.865 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140641497495296, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-03-19 Thu 05:48:19:837.021 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140641497495296, Pid 1]: SANDESH: Send FAILED: 1773899299836793 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1.:Database:contrail-query-engine:0 client_info= [ status = Connect successful_connections = 7 pid = 1 http_port = 8091 start_time = 1773898562749429 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 578189 calls = 69 average_bytes = 8379 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 39 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 11 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 214 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 7 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 42 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 53 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 20 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 235, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 94, _iter59->first = sent _iter59->second = 73, ] ] ] 2026-03-19 Thu 05:48:19:837.043 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140641497495296, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-03-19 Thu 05:48:19:846.301 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.38:49194::10.0.0.38:8086(-1) < Active session connection complete src/contrail-common/io/tcp_session.cc 280 2026-03-19 Thu 05:48:19:846.342 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: 10.0.0.38:49194::10.0.0.38:8086(-1) OnSessionEvent TCP Connected 2026-03-19 Thu 05:48:19:846.352 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642202978432, Pid 1]: 10.0.0.38:49194::10.0.0.38:8086(-1):EvTcpConnected 2026-03-19 Thu 05:48:19:849.298 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642055108352, Pid 1]: Processing scm::EvTcpConnected in state Connect 2026-03-19 Thu 05:48:19:849.321 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642055108352, Pid 1]: Connect : EvTcpConnected : Cancelling Connect timer 2026-03-19 Thu 05:48:19:849.370 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642055108352, Pid 1]: ClientInit 2026-03-19 Thu 05:48:19:849.388 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642055108352, Pid 1]: Sending Ctrl Message for cn-jenkins-deploy-platform-ansible-os-5495-1.:contrail-query-engine:0:Database count 8 2026-03-19 Thu 05:48:19:849.486 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642055108352, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-03-19 Thu 05:48:19:849.500 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642055108352, Pid 1]: ClientInit : EvSandeshSend : NodeStatusUVE 2026-03-19 Thu 05:48:19:849.530 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642055108352, Pid 1]: SANDESH: Send FAILED: 1773899299849358 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-03-19 Thu 05:48:19:849.543 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642055108352, Pid 1]: Received UVE message in wrong state : NodeStatusUVE 2026-03-19 Thu 05:48:19:849.565 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642055108352, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-03-19 Thu 05:48:19:849.573 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642055108352, Pid 1]: ClientInit : EvSandeshSend : SandeshCtrlClientToServer 2026-03-19 Thu 05:48:19:849.585 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642055108352, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-03-19 Thu 05:48:19:849.592 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642055108352, Pid 1]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2026-03-19 Thu 05:48:19:849.795 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642055108352, Pid 1]: SANDESH: Send FAILED: 1773899299849471 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1.:Database:contrail-query-engine:0 client_info= [ status = ClientInit successful_connections = 8 pid = 1 http_port = 8091 start_time = 1773898562749429 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 578189 calls = 69 average_bytes = 8379 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 40 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 11 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 214 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 7 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 42 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 54 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 21 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 236, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 96, _iter59->first = sent _iter59->second = 73, ] ] ] 2026-03-19 Thu 05:48:19:849.818 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642055108352, Pid 1]: Received UVE message in wrong state : SandeshModuleClientTrace 2026-03-19 Thu 05:48:19:860.196 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642025719552, Pid 1]: OnMessage control in state: ClientInit 2026-03-19 Thu 05:48:19:860.280 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642025719552, Pid 1]: Processing scm::EvSandeshMessageRecv in state ClientInit 2026-03-19 Thu 05:48:19:860.291 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642025719552, Pid 1]: ClientInit : EvSandeshMessageRecv 2026-03-19 Thu 05:48:19:860.319 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642025719552, Pid 1]: Received Ctrl Message with size 0 2026-03-19 Thu 05:48:19:860.574 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642025719552, Pid 1]: SyncUVE Syncing ObjectGeneratorInfo val name = cn-jenkins-deploy-platform-ansible-os-5495-1.:Database:contrail-query-engine:0 client_info= [ status = ClientInit successful_connections = 8 pid = 1 http_port = 8091 start_time = 1773898562749429 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 3464 calls = 6 average_bytes = 577 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 578189 calls = 69 average_bytes = 8379 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvConnectTimerExpired enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 9 dequeues = 9 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshMessageRecv enqueues = 3 dequeues = 3 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 165 dequeues = 165 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpClose enqueues = 6 dequeues = 6 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 5 dequeues = 5 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnected enqueues = 7 dequeues = 7 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 12 dequeues = 12 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 210 dequeues = 210 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1773899269834816 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1773899273099385 ] session_stats= [ num_recv_msg = 0 num_recv_msg_fail = 0 num_recv_fail = 0 num_send_msg = 22 num_send_msg_fail = 0 num_send_buffer_fail = 0 num_wait_msgq_enqueue = 0 num_wait_msgq_dequeue = 0 num_write_ready_cb_error = 0 ] session_rx_socket_stats= [ bytes = 1017 calls = 2 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] session_tx_socket_stats= [ bytes = 179165 calls = 19 average_bytes = 9429 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 13 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 40 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 11 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 214 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 7 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 42 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 54 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 21 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 236, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 96, _iter59->first = sent _iter59->second = 73, ] ] max_sm_queue_count = 8 proxy seq 96 2026-03-19 Thu 05:48:19:860.609 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642025719552, Pid 1]: SyncAllMaps for ModuleClientState:0 period 0 cycle 0 without seqno , total = 1 2026-03-19 Thu 05:48:19:860.641 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642025719552, Pid 1]: SyncUVE Syncing ObjectDatabaseInfo val name = cn-jenkins-deploy-platform-ansible-os-5495-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] proxy seq 52 2026-03-19 Thu 05:48:19:860.654 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642025719552, Pid 1]: SyncAllMaps for NodeStatus:0 period 0 cycle 0 without seqno , total = 1 2026-03-19 Thu 05:48:19:860.693 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642025719552, Pid 1]: SyncUVE Syncing ObjectDatabaseInfo val name = cn-jenkins-deploy-platform-ansible-os-5495-1. table_info= [ [ ] ] errors= [ write_tablespace_fails = 0 read_tablespace_fails = 0 write_table_fails = 0 read_table_fails = 0 write_column_fails = 0 write_batch_column_fails = 0 read_column_fails = 0 ] stats_info= [ [ ] ] cql_stats= [ requests_one_minute_rate = 0.4 stats= [ total_connections = 6 available_connections = 6 exceeded_pending_requests_water_mark = 0 exceeded_write_bytes_water_mark = 0 ] errors= [ connection_timeouts = 0 pending_request_timeouts = 0 request_timeouts = 0 ] ] proxy seq 12 2026-03-19 Thu 05:48:19:860.704 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642025719552, Pid 1]: SyncAllMaps for QEDbStats:0 period 0 cycle 0 without seqno , total = 1 2026-03-19 Thu 05:48:19:860.711 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642025719552, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:0 period 60 cycle 0 without seqno , total = 0 2026-03-19 Thu 05:48:19:860.746 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642025719552, Pid 1]: Established 2026-03-19 Thu 05:49:18:094.897 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642025719552, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 13 without seqno , total = 0 2026-03-19 Thu 05:50:18:095.795 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642025719552, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 14 without seqno , total = 0 2026-03-19 Thu 05:51:12:103.768 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642025719552, Pid 1]: Processing scm::EvStop in state Established 2026-03-19 Thu 05:51:12:103.867 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642025719552, Pid 1]: Idle 2026-03-19 Thu 05:51:12:103.968 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642025719552, Pid 1]: Processing scm::EvTcpDeleteSession in state Idle 2026-03-19 Thu 05:51:12:103.979 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642025719552, Pid 1]: Processing scm::EvSandeshSend in state Idle 2026-03-19 Thu 05:51:12:104.136 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642025719552, Pid 1]: SANDESH: Send FAILED: 1773899472103957 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1.:Database:contrail-query-engine:0 client_info= [ status = Idle successful_connections = 8 pid = 1 http_port = 8091 start_time = 1773898562749429 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 4481 calls = 8 average_bytes = 560 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 753998 calls = 86 average_bytes = 8767 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 16 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 41 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QEDbStatsUve [ messages_sent = 15 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 214 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 8 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 54 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 55 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 21 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 236, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 98, _iter59->first = sent _iter59->second = 93, ] ] ] 2026-03-19 Thu 05:51:12:104.162 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140642025719552, Pid 1]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-03-19 Thu 05:51:38:527.545 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: http-server-port 8091 2026-03-19 Thu 05:51:38:527.622 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: Max-tasks 16 2026-03-19 Thu 05:51:38:527.629 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: Max-slice 100 2026-03-19 Thu 05:51:38:527.635 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: Collectors 10.0.0.38:8086 2026-03-19 Thu 05:51:38:527.639 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: Collectors 10.0.0.50:8086 2026-03-19 Thu 05:51:38:527.644 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: Collectors 10.0.0.33:8086 2026-03-19 Thu 05:51:38:527.666 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: SANDESH: ROLE : Generator 2026-03-19 Thu 05:51:38:527.671 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: SANDESH: MODULE : contrail-query-engine 2026-03-19 Thu 05:51:38:527.676 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: SANDESH: SOURCE : cn-jenkins-deploy-platform-ansible-os-5495-1. 2026-03-19 Thu 05:51:38:527.681 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: SANDESH: NODE TYPE : Database 2026-03-19 Thu 05:51:38:527.686 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: SANDESH: INSTANCE ID : 0 2026-03-19 Thu 05:51:38:527.691 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: SANDESH: HTTP SERVER PORT : 8091 2026-03-19 Thu 05:51:38:527.697 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: SANDESH: System Log Send Rate Limit: 0 -> 0 2026-03-19 Thu 05:51:38:530.811 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: HTTP Introspect Init 2026-03-19 Thu 05:51:38:530.936 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: TCP [SYS_DEBUG]: TcpServerMessageLog: Server 0.0.0.0:8091 Initialization complete src/contrail-common/io/tcp_server.cc 138 2026-03-19 Thu 05:51:38:531.056 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: Sandesh Http Server Port: 8091 2026-03-19 Thu 05:51:38:531.126 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: SANDESH: NOT Writing http_port 8091TO : /tmp/contrail-query-engine.0.http_port 2026-03-19 Thu 05:51:38:531.135 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: SANDESH: CONNECT TO COLLECTOR: 1 2026-03-19 Thu 05:51:38:531.496 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: Idle 2026-03-19 Thu 05:51:38:531.648 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: SANDESH: No Client: 1773899498531592 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1.:Database:contrail-query-engine:0 client_info= [ status = Idle successful_connections = 0 pid = 1 http_port = 8091 start_time = 1773899498531513 collector_name = collector_ip = 0.0.0.0:0 collector_list= [ [ ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 0, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 0, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-03-19 Thu 05:51:38:531.795 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: SANDESH: Logging: DISABLED -> ENABLED 2026-03-19 Thu 05:51:38:531.810 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: SANDESH: Logging: LEVEL: [ INVALID ] -> [ SYS_DEBUG ] log4level: [ NOTSET ] -> [ DEBUG ] 2026-03-19 Thu 05:51:38:531.821 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: Cassandra Servers: 10.0.0.38:9042 10.0.0.50:9042 10.0.0.33:9042 2026-03-19 Thu 05:51:38:531.925 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603308529216, Pid 1]: Processing scm::EvStart in state Idle 2026-03-19 Thu 05:51:38:532.005 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603308529216, Pid 1]: Disconnect 2026-03-19 Thu 05:51:38:532.082 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603308529216, Pid 1]: Processing scm::EvCollectorUpdate in state Disconnect 2026-03-19 Thu 05:51:38:532.275 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603308529216, Pid 1]: Connect : Start Connect timer 10.0.0.33:8086 2026-03-19 Thu 05:51:38:532.331 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603308529216, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-03-19 Thu 05:51:38:532.349 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603308529216, Pid 1]: SANDESH: Send FAILED: 1773899498531998 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Idle : EvStart -> Disconnect ], ] ] flag_infos= [ [ ] ] description = Number of connections:1, Expected:3 Missing: Database,Redis-Query:Query ], ] ] ] 2026-03-19 Thu 05:51:38:532.358 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603308529216, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-03-19 Thu 05:51:38:532.366 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603308529216, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-03-19 Thu 05:51:38:532.429 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603308529216, Pid 1]: SANDESH: Send FAILED: 1773899498532069 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1.:Database:contrail-query-engine:0 client_info= [ status = Disconnect successful_connections = 0 pid = 1 http_port = 8091 start_time = 1773899498531513 collector_name = collector_ip = 0.0.0.0:0 collector_list= [ [ ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 0, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-03-19 Thu 05:51:38:532.444 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603308529216, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-03-19 Thu 05:51:38:532.453 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603308529216, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-03-19 Thu 05:51:38:532.497 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603308529216, Pid 1]: SANDESH: Send FAILED: 1773899498532112 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1.:Database:contrail-query-engine:0 client_info= [ status = Disconnect successful_connections = 0 pid = 1 http_port = 8091 start_time = 1773899498531513 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 0, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-03-19 Thu 05:51:38:532.506 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603308529216, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-03-19 Thu 05:51:38:532.513 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603308529216, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-03-19 Thu 05:51:38:532.525 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603308529216, Pid 1]: SANDESH: Send FAILED: 1773899498532129 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Initializing description = Disconnect : EvCollectorUpdate -> Connect ], ] ] flag_infos= [ [ ] ] description = Number of connections:1, Expected:3 Missing: Database,Redis-Query:Query ], ] ] ] 2026-03-19 Thu 05:51:38:532.533 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603308529216, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-03-19 Thu 05:51:38:532.540 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603308529216, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-03-19 Thu 05:51:38:532.578 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603308529216, Pid 1]: SANDESH: Send FAILED: 1773899498532320 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1.:Database:contrail-query-engine:0 client_info= [ status = Connect successful_connections = 0 pid = 1 http_port = 8091 start_time = 1773899498531513 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 0, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-03-19 Thu 05:51:38:532.587 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603308529216, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-03-19 Thu 05:51:38:610.427 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: [SYS_INFO]: QELog: Older table does not exist. will query only the new table src/contrail-analytics/contrail-query-engine/query.cc 1037 2026-03-19 Thu 05:51:38:617.136 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.38:59458::10.0.0.33:8086(-1) < Active session connection complete src/contrail-common/io/tcp_session.cc 282 2026-03-19 Thu 05:51:38:617.172 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: 10.0.0.38:59458::10.0.0.33:8086(-1) OnSessionEvent TCP Connected 2026-03-19 Thu 05:51:38:617.182 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: 10.0.0.38:59458::10.0.0.33:8086(-1):EvTcpConnected 2026-03-19 Thu 05:51:38:617.213 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: RAC_Connect status: 0 0x1bf3540 2026-03-19 Thu 05:51:38:617.218 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: Connected to REDIS... 2026-03-19 Thu 05:51:38:617.231 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 0 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1028 2026-03-19 Thu 05:51:38:617.252 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: RAC_Connect status: 0 0x1bf8540 2026-03-19 Thu 05:51:38:617.259 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: Connected to REDIS... 2026-03-19 Thu 05:51:38:617.269 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 1 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1028 2026-03-19 Thu 05:51:38:617.284 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: RAC_Connect status: 0 0x1bf8df0 2026-03-19 Thu 05:51:38:617.292 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: Connected to REDIS... 2026-03-19 Thu 05:51:38:617.303 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 2 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1028 2026-03-19 Thu 05:51:38:617.317 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: RAC_Connect status: 0 0x1bf9ab0 2026-03-19 Thu 05:51:38:617.324 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: Connected to REDIS... 2026-03-19 Thu 05:51:38:617.336 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 3 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1028 2026-03-19 Thu 05:51:38:617.351 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: RAC_Connect status: 0 0x1bfa770 2026-03-19 Thu 05:51:38:617.358 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: Connected to REDIS... 2026-03-19 Thu 05:51:38:617.369 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 4 With Redis:10.0.0.38 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1028 2026-03-19 Thu 05:51:38:617.382 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: RAC_Connect status: 0 0x1bff0f0 2026-03-19 Thu 05:51:38:617.390 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: Connected to REDIS... 2026-03-19 Thu 05:51:38:617.399 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 0 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1028 2026-03-19 Thu 05:51:38:617.414 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: RAC_Connect status: 0 0x1bffd30 2026-03-19 Thu 05:51:38:617.431 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: Connected to REDIS... 2026-03-19 Thu 05:51:38:617.442 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 1 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1028 2026-03-19 Thu 05:51:38:617.457 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: RAC_Connect status: 0 0x1c00970 2026-03-19 Thu 05:51:38:617.464 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: Connected to REDIS... 2026-03-19 Thu 05:51:38:617.475 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 2 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1028 2026-03-19 Thu 05:51:38:617.489 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: RAC_Connect status: 0 0x1bfb430 2026-03-19 Thu 05:51:38:617.496 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: Connected to REDIS... 2026-03-19 Thu 05:51:38:617.507 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 0 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1028 2026-03-19 Thu 05:51:38:617.520 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: RAC_Connect status: 0 0x1bfcc30 2026-03-19 Thu 05:51:38:617.528 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: Connected to REDIS... 2026-03-19 Thu 05:51:38:617.539 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 2 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1028 2026-03-19 Thu 05:51:38:617.552 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: RAC_Connect status: 0 0x1bfe4b0 2026-03-19 Thu 05:51:38:617.559 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: Connected to REDIS... 2026-03-19 Thu 05:51:38:617.570 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 4 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1028 2026-03-19 Thu 05:51:38:629.865 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: RAC_Connect status: 0 0x1c015b0 2026-03-19 Thu 05:51:38:629.879 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: Connected to REDIS... 2026-03-19 Thu 05:51:38:629.900 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 3 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1028 2026-03-19 Thu 05:51:38:629.912 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: RAC_Connect status: 0 0x1bfbff0 2026-03-19 Thu 05:51:38:629.917 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: Connected to REDIS... 2026-03-19 Thu 05:51:38:629.924 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 1 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1028 2026-03-19 Thu 05:51:38:629.935 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: RAC_Connect status: 0 0x1c021f0 2026-03-19 Thu 05:51:38:629.940 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: Connected to REDIS... 2026-03-19 Thu 05:51:38:629.947 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 4 With Redis:10.0.0.33 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1028 2026-03-19 Thu 05:51:38:629.966 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: RAC_Connect status: 0 0x1bfd870 2026-03-19 Thu 05:51:38:629.971 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: Connected to REDIS... 2026-03-19 Thu 05:51:38:629.978 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: [SYS_INFO]: QELog: ConnUp.. UP 3 With Redis:10.0.0.50 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1028 2026-03-19 Thu 05:51:38:630.242 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1080 2026-03-19 Thu 05:51:38:630.254 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1087 2026-03-19 Thu 05:51:38:630.317 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB0 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 705 2026-03-19 Thu 05:51:38:630.332 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1080 2026-03-19 Thu 05:51:38:630.340 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1087 2026-03-19 Thu 05:51:38:630.351 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1080 2026-03-19 Thu 05:51:38:630.358 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1087 2026-03-19 Thu 05:51:38:630.369 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1080 2026-03-19 Thu 05:51:38:630.376 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1087 2026-03-19 Thu 05:51:38:630.387 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1080 2026-03-19 Thu 05:51:38:630.394 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.38:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1087 2026-03-19 Thu 05:51:38:630.418 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB4 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 705 2026-03-19 Thu 05:51:38:630.430 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB3 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 705 2026-03-19 Thu 05:51:38:630.441 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB2 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 705 2026-03-19 Thu 05:51:38:630.452 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB1 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 705 2026-03-19 Thu 05:51:38:617.670 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603304330816, Pid 1]: Processing scm::EvSandeshSend in state Connect 2026-03-19 Thu 05:51:38:632.630 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603304330816, Pid 1]: SANDESH: Send FAILED: 1773899498615315 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Initializing description = Disconnect : EvCollectorUpdate -> Connect ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], ] ] flag_infos= [ [ ] ] description = Number of connections:2, Expected:3 Missing: Redis-Query:Query ], ] ] ] 2026-03-19 Thu 05:51:38:632.644 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603304330816, Pid 1]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-03-19 Thu 05:51:38:632.662 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603304330816, Pid 1]: Processing scm::EvTcpConnected in state Connect 2026-03-19 Thu 05:51:38:632.673 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603304330816, Pid 1]: Connect : EvTcpConnected : Cancelling Connect timer 2026-03-19 Thu 05:51:38:632.721 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603304330816, Pid 1]: ClientInit 2026-03-19 Thu 05:51:38:632.746 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603304330816, Pid 1]: Sending Ctrl Message for cn-jenkins-deploy-platform-ansible-os-5495-1.:contrail-query-engine:0:Database count 1 2026-03-19 Thu 05:51:38:632.857 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603304330816, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-03-19 Thu 05:51:38:632.869 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603304330816, Pid 1]: ClientInit : EvSandeshSend : NodeStatusUVE 2026-03-19 Thu 05:51:38:632.893 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603304330816, Pid 1]: SANDESH: Send FAILED: 1773899498630294 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Initializing description = Disconnect : EvCollectorUpdate -> Connect ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Down description = ], ] ] flag_infos= [ [ ] ] description = Collector, Redis-Query:Query connection down ], ] ] ] 2026-03-19 Thu 05:51:38:632.903 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603304330816, Pid 1]: Received UVE message in wrong state : NodeStatusUVE 2026-03-19 Thu 05:51:38:632.913 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603304330816, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-03-19 Thu 05:51:38:632.921 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603304330816, Pid 1]: ClientInit : EvSandeshSend : NodeStatusUVE 2026-03-19 Thu 05:51:38:632.946 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603304330816, Pid 1]: SANDESH: Send FAILED: 1773899498630413 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Initializing description = Disconnect : EvCollectorUpdate -> Connect ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = 10.0.0.50:6379::Down.10.0.0.33:6379::Down. ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-03-19 Thu 05:51:38:632.959 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603304330816, Pid 1]: Received UVE message in wrong state : NodeStatusUVE 2026-03-19 Thu 05:51:38:632.968 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603304330816, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-03-19 Thu 05:51:38:632.977 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603304330816, Pid 1]: ClientInit : EvSandeshSend : NodeStatusUVE 2026-03-19 Thu 05:51:38:632.996 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603304330816, Pid 1]: SANDESH: Send FAILED: 1773899498632712 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = 10.0.0.50:6379::Down.10.0.0.33:6379::Down. ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-03-19 Thu 05:51:38:633.004 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603304330816, Pid 1]: Received UVE message in wrong state : NodeStatusUVE 2026-03-19 Thu 05:51:38:633.013 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603304330816, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-03-19 Thu 05:51:38:633.022 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603304330816, Pid 1]: ClientInit : EvSandeshSend : SandeshCtrlClientToServer 2026-03-19 Thu 05:51:38:633.060 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603304330816, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-03-19 Thu 05:51:38:633.069 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603304330816, Pid 1]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2026-03-19 Thu 05:51:38:633.193 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603304330816, Pid 1]: SANDESH: Send FAILED: 1773899498632841 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1.:Database:contrail-query-engine:0 client_info= [ status = ClientInit successful_connections = 1 pid = 1 http_port = 8091 start_time = 1773899498531513 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 31 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 33, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 6, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-03-19 Thu 05:51:38:633.218 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603304330816, Pid 1]: Received UVE message in wrong state : SandeshModuleClientTrace 2026-03-19 Thu 05:51:38:636.768 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1080 2026-03-19 Thu 05:51:38:636.808 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1087 2026-03-19 Thu 05:51:38:636.839 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1080 2026-03-19 Thu 05:51:38:636.852 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1087 2026-03-19 Thu 05:51:38:636.870 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1080 2026-03-19 Thu 05:51:38:636.882 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1087 2026-03-19 Thu 05:51:38:636.901 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1080 2026-03-19 Thu 05:51:38:636.913 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1087 2026-03-19 Thu 05:51:38:636.930 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1080 2026-03-19 Thu 05:51:38:636.962 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.33:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1087 2026-03-19 Thu 05:51:38:637.011 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1080 2026-03-19 Thu 05:51:38:637.024 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1087 2026-03-19 Thu 05:51:38:637.055 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1080 2026-03-19 Thu 05:51:38:637.068 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1087 2026-03-19 Thu 05:51:38:637.087 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1080 2026-03-19 Thu 05:51:38:637.100 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1087 2026-03-19 Thu 05:51:38:637.118 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1080 2026-03-19 Thu 05:51:38:637.137 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1087 2026-03-19 Thu 05:51:38:637.154 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: [SYS_INFO]: QELog: UP ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1080 2026-03-19 Thu 05:51:38:637.170 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: [SYS_INFO]: QELog: In ConnectCallbackProcess..10.0.0.50:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 1087 2026-03-19 Thu 05:51:38:637.208 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB0 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 705 2026-03-19 Thu 05:51:38:637.855 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603308529216, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-03-19 Thu 05:51:38:637.874 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603308529216, Pid 1]: ClientInit : EvSandeshSend : NodeStatusUVE 2026-03-19 Thu 05:51:38:637.904 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603308529216, Pid 1]: SANDESH: Send FAILED: 1773899498637250 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = 10.0.0.50:6379::Down. ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-03-19 Thu 05:51:38:637.915 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603308529216, Pid 1]: Received UVE message in wrong state : NodeStatusUVE 2026-03-19 Thu 05:51:38:637.974 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB2 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 705 2026-03-19 Thu 05:51:38:638.001 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB1 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 705 2026-03-19 Thu 05:51:38:638.016 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB3 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 705 2026-03-19 Thu 05:51:38:638.027 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB4 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 705 2026-03-19 Thu 05:51:38:638.058 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB0 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 705 2026-03-19 Thu 05:51:38:638.492 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603300132416, Pid 1]: Processing scm::EvSandeshSend in state ClientInit 2026-03-19 Thu 05:51:38:638.505 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603300132416, Pid 1]: ClientInit : EvSandeshSend : NodeStatusUVE 2026-03-19 Thu 05:51:38:638.522 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603300132416, Pid 1]: SANDESH: Send FAILED: 1773899498638079 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5495-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-03-19 Thu 05:51:38:638.536 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603300132416, Pid 1]: Received UVE message in wrong state : NodeStatusUVE 2026-03-19 Thu 05:51:38:638.549 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB3 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 705 2026-03-19 Thu 05:51:38:638.561 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB1 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 705 2026-03-19 Thu 05:51:38:638.571 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB2 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 705 2026-03-19 Thu 05:51:38:638.581 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: [SYS_INFO]: QELog: ConnUp SetCB4 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 705 2026-03-19 Thu 05:51:38:639.234 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603308529216, Pid 1]: OnMessage control in state: ClientInit 2026-03-19 Thu 05:51:38:639.265 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603308529216, Pid 1]: Processing scm::EvSandeshMessageRecv in state ClientInit 2026-03-19 Thu 05:51:38:639.273 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603308529216, Pid 1]: ClientInit : EvSandeshMessageRecv 2026-03-19 Thu 05:51:38:639.290 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603308529216, Pid 1]: Received Ctrl Message with size 0 2026-03-19 Thu 05:51:38:639.383 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603308529216, Pid 1]: SyncUVE Syncing ObjectGeneratorInfo val name = cn-jenkins-deploy-platform-ansible-os-5495-1.:Database:contrail-query-engine:0 client_info= [ status = ClientInit successful_connections = 1 pid = 1 http_port = 8091 start_time = 1773899498531513 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = QELog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 31 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 33, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 6, _iter59->first = sent _iter59->second = 0, ] ] proxy seq 5 2026-03-19 Thu 05:51:38:639.410 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603308529216, Pid 1]: SyncAllMaps for ModuleClientState:0 period 0 cycle 0 without seqno , total = 1 2026-03-19 Thu 05:51:38:639.430 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603308529216, Pid 1]: SyncUVE Syncing ObjectDatabaseInfo val name = cn-jenkins-deploy-platform-ansible-os-5495-1. process_status= [ [ [ module_id = contrail-query-engine instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = server_addrs= [ [ (*_iter6) = 10.0.0.38:9042, (*_iter6) = 10.0.0.50:9042, (*_iter6) = 10.0.0.33:9042, ] ] status = Up description = ], [ type = Redis-Query name = Query server_addrs= [ [ (*_iter6) = 10.0.0.38:6379, (*_iter6) = 10.0.0.50:6379, (*_iter6) = 10.0.0.33:6379, ] ] status = Up description = ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] proxy seq 8 2026-03-19 Thu 05:51:38:639.441 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603308529216, Pid 1]: SyncAllMaps for NodeStatus:0 period 0 cycle 0 without seqno , total = 1 2026-03-19 Thu 05:51:38:639.448 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603308529216, Pid 1]: SyncAllMaps for QEDbStats:0 period 0 cycle 0 without seqno , total = 0 2026-03-19 Thu 05:51:38:639.455 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603308529216, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:0 period 60 cycle 0 without seqno , total = 0 2026-03-19 Thu 05:51:38:639.484 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603308529216, Pid 1]: Established 2026-03-19 Thu 05:52:38:532.186 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603291735616, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 1 without seqno , total = 0 2026-03-19 Thu 05:53:38:532.677 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602763085376, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 2 without seqno , total = 0 2026-03-19 Thu 05:54:38:533.458 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602771482176, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 3 without seqno , total = 0 2026-03-19 Thu 05:55:38:534.018 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602754688576, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 4 without seqno , total = 0 2026-03-19 Thu 05:56:38:534.594 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602767283776, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 5 without seqno , total = 0 2026-03-19 Thu 05:57:38:535.152 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602750490176, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 6 without seqno , total = 0 2026-03-19 Thu 05:58:38:538.341 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602784077376, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 7 without seqno , total = 0 2026-03-19 Thu 05:59:38:540.927 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602754688576, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 8 without seqno , total = 0 2026-03-19 Thu 06:00:38:541.604 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602763085376, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 9 without seqno , total = 0 2026-03-19 Thu 06:01:38:542.167 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602754688576, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 10 without seqno , total = 0 2026-03-19 Thu 06:02:38:542.734 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602750490176, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 11 without seqno , total = 0 2026-03-19 Thu 06:03:38:543.459 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602758886976, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 12 without seqno , total = 0 2026-03-19 Thu 06:04:38:544.138 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603276179008, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 13 without seqno , total = 0 2026-03-19 Thu 06:05:38:545.047 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603276179008, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 14 without seqno , total = 0 2026-03-19 Thu 06:06:38:546.019 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602779878976, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 15 without seqno , total = 0 2026-03-19 Thu 06:07:38:546.930 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602750490176, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 16 without seqno , total = 0 2026-03-19 Thu 06:08:38:547.523 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602750490176, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 17 without seqno , total = 0 2026-03-19 Thu 06:09:38:548.293 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602758886976, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 18 without seqno , total = 0 2026-03-19 Thu 06:10:16:317.503 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.38:8091::10.0.0.38:52094 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2026-03-19 Thu 06:10:25:223.214 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.38:8091::10.0.0.38:53590 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2026-03-19 Thu 06:10:38:549.379 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602771482176, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 19 without seqno , total = 0 2026-03-19 Thu 06:11:38:550.108 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602767283776, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 20 without seqno , total = 0 2026-03-19 Thu 06:12:00:862.611 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.38:8091::10.0.0.38:42490 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2026-03-19 Thu 06:12:23:578.101 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: [SYS_INFO]: QELog: StartPipeline on : Redis:10.0.0.38 Port:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 812 2026-03-19 Thu 06:12:23:578.732 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: [SYS_INFO]: QELog: Got Query to prepare for QID 98617f74-235a-11f1-9dd1-00000a000026 src/contrail-analytics/contrail-query-engine/query.cc 1165 2026-03-19 Thu 06:12:23:578.815 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: [SYS_INFO]: QEQueryLog: 98617f74-235a-11f1-9dd1-00000a000026 :0:json query is: { end_time: 1773900743508604 , enqueue_time: 1773900743577616 , query_metadata: {"enqueue_time": 1773900743577512} , select_fields: ["ObjectId", "Source", "ObjectLog", "SystemLog", "Messagetype", "ModuleId", "MessageTS"] , start_time: 1773900725601133 , table: "ObjectVNTable" , table_schema: [{"name": "MessageTS", "datatype": "int", "index": false, "suffixes": null}, {"name": "ObjectId", "datatype": "string", "index": true, "suffixes": null}, {"name": "Source", "datatype": "string", "index": true, "suffixes": null}, {"name": "ModuleId", "datatype": "string", "index": true, "suffixes": null}, {"name": "Messagetype", "datatype": "string", "index": true, "suffixes": null}, {"name": "ObjectLog", "datatype": "string", "index": false, "suffixes": null}, {"name": "SystemLog", "datatype": "string", "index": false, "suffixes": null}] , where: [[{"name": "ObjectId", "value": "default-domain:ctest-AnalyticsBasicTestSanity-47331943:ctest-vn22-00062871", "op": 1, "value2": null}]] } src/contrail-analytics/contrail-query-engine/query.cc 467 2026-03-19 Thu 06:12:23:579.048 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: [SYS_INFO]: QELog: Chunks: 3 Need Merge: 0 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 936 2026-03-19 Thu 06:12:23:579.073 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: [SYS_INFO]: QELog: Getting Least Loaded Conn as :0 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 986 2026-03-19 Thu 06:12:23:579.139 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: [SYS_INFO]: QELog: Starting Pipeline for 98617f74-235a-11f1-9dd1-00000a000026 , 1 conn, 16 tasks src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 994 2026-03-19 Thu 06:12:23:579.229 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602763085376, Pid 1]: [SYS_INFO]: QEQueryLog: 98617f74-235a-11f1-9dd1-00000a000026 :0:json query is: { end_time: 1773900743508604 , enqueue_time: 1773900743577616 , query_metadata: {"enqueue_time": 1773900743577512} , select_fields: ["ObjectId", "Source", "ObjectLog", "SystemLog", "Messagetype", "ModuleId", "MessageTS"] , start_time: 1773900725601133 , table: "ObjectVNTable" , table_schema: [{"name": "MessageTS", "datatype": "int", "index": false, "suffixes": null}, {"name": "ObjectId", "datatype": "string", "index": true, "suffixes": null}, {"name": "Source", "datatype": "string", "index": true, "suffixes": null}, {"name": "ModuleId", "datatype": "string", "index": true, "suffixes": null}, {"name": "Messagetype", "datatype": "string", "index": true, "suffixes": null}, {"name": "ObjectLog", "datatype": "string", "index": false, "suffixes": null}, {"name": "SystemLog", "datatype": "string", "index": false, "suffixes": null}] , where: [[{"name": "ObjectId", "value": "default-domain:ctest-AnalyticsBasicTestSanity-47331943:ctest-vn22-00062871", "op": 1, "value2": null}]] } src/contrail-analytics/contrail-query-engine/query.cc 467 2026-03-19 Thu 06:12:23:692.643 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602784077376, Pid 1]: [SYS_INFO]: QELog: 98617f74-235a-11f1-9dd1-00000a000026 UNION between tables of sizes 0 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-03-19 Thu 06:12:23:692.704 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602784077376, Pid 1]: [SYS_INFO]: QELog: 98617f74-235a-11f1-9dd1-00000a000026 Resulting size of set 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-03-19 Thu 06:12:23:692.717 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602784077376, Pid 1]: [SYS_INFO]: QELog: 98617f74-235a-11f1-9dd1-00000a000026 UNION between tables of sizes 0 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-03-19 Thu 06:12:23:692.727 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602784077376, Pid 1]: [SYS_INFO]: QELog: 98617f74-235a-11f1-9dd1-00000a000026 Resulting size of set 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-03-19 Thu 06:12:23:692.736 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602784077376, Pid 1]: [SYS_INFO]: QELog: 98617f74-235a-11f1-9dd1-00000a000026 UNION between tables of sizes 0 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-03-19 Thu 06:12:23:692.746 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602784077376, Pid 1]: [SYS_INFO]: QELog: 98617f74-235a-11f1-9dd1-00000a000026 Resulting size of set 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-03-19 Thu 06:12:23:692.755 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602784077376, Pid 1]: [SYS_INFO]: QELog: 98617f74-235a-11f1-9dd1-00000a000026 UNION between tables of sizes 0 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-03-19 Thu 06:12:23:692.763 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602784077376, Pid 1]: [SYS_INFO]: QELog: 98617f74-235a-11f1-9dd1-00000a000026 Resulting size of set 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-03-19 Thu 06:12:23:692.772 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602784077376, Pid 1]: [SYS_INFO]: QELog: 98617f74-235a-11f1-9dd1-00000a000026 UNION between tables of sizes 0 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-03-19 Thu 06:12:23:692.781 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602784077376, Pid 1]: [SYS_INFO]: QELog: 98617f74-235a-11f1-9dd1-00000a000026 Resulting size of set 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-03-19 Thu 06:12:23:693.109 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602784077376, Pid 1]: [SYS_INFO]: QELog: 98617f74-235a-11f1-9dd1-00000a000026 UNION between tables of sizes 0 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-03-19 Thu 06:12:23:693.132 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602784077376, Pid 1]: [SYS_INFO]: QELog: 98617f74-235a-11f1-9dd1-00000a000026 Resulting size of set 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-03-19 Thu 06:12:23:693.144 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602784077376, Pid 1]: [SYS_INFO]: QELog: 98617f74-235a-11f1-9dd1-00000a000026 UNION between tables of sizes 0 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-03-19 Thu 06:12:23:693.170 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602784077376, Pid 1]: [SYS_INFO]: QELog: 98617f74-235a-11f1-9dd1-00000a000026 Resulting size of set 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-03-19 Thu 06:12:23:693.181 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602784077376, Pid 1]: [SYS_INFO]: QELog: 98617f74-235a-11f1-9dd1-00000a000026 UNION between tables of sizes 0 and 33 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-03-19 Thu 06:12:23:693.295 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602784077376, Pid 1]: [SYS_INFO]: QELog: 98617f74-235a-11f1-9dd1-00000a000026 Resulting size of set 33 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-03-19 Thu 06:12:23:693.310 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602784077376, Pid 1]: [SYS_INFO]: QELog: 98617f74-235a-11f1-9dd1-00000a000026 UNION between tables of sizes 33 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-03-19 Thu 06:12:23:693.338 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602784077376, Pid 1]: [SYS_INFO]: QELog: 98617f74-235a-11f1-9dd1-00000a000026 Resulting size of set 33 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-03-19 Thu 06:12:23:693.349 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602784077376, Pid 1]: [SYS_INFO]: QELog: 98617f74-235a-11f1-9dd1-00000a000026 UNION between tables of sizes 33 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-03-19 Thu 06:12:23:693.376 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602784077376, Pid 1]: [SYS_INFO]: QELog: 98617f74-235a-11f1-9dd1-00000a000026 Resulting size of set 33 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-03-19 Thu 06:12:23:693.462 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602784077376, Pid 1]: [SYS_INFO]: QEQueryLog: 98617f74-235a-11f1-9dd1-00000a000026 :0:json query is: { end_time: 1773900743508604 , enqueue_time: 1773900743577616 , query_metadata: {"enqueue_time": 1773900743577512} , select_fields: ["ObjectId", "Source", "ObjectLog", "SystemLog", "Messagetype", "ModuleId", "MessageTS"] , start_time: 1773900725601133 , table: "ObjectVNTable" , table_schema: [{"name": "MessageTS", "datatype": "int", "index": false, "suffixes": null}, {"name": "ObjectId", "datatype": "string", "index": true, "suffixes": null}, {"name": "Source", "datatype": "string", "index": true, "suffixes": null}, {"name": "ModuleId", "datatype": "string", "index": true, "suffixes": null}, {"name": "Messagetype", "datatype": "string", "index": true, "suffixes": null}, {"name": "ObjectLog", "datatype": "string", "index": false, "suffixes": null}, {"name": "SystemLog", "datatype": "string", "index": false, "suffixes": null}] , where: [[{"name": "ObjectId", "value": "default-domain:ctest-AnalyticsBasicTestSanity-47331943:ctest-vn22-00062871", "op": 1, "value2": null}]] } src/contrail-analytics/contrail-query-engine/query.cc 467 2026-03-19 Thu 06:12:23:693.624 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602784077376, Pid 1]: [SYS_INFO]: QEQueryLog: 98617f74-235a-11f1-9dd1-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-47331943:ctest-vn22-00062871 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-03-19 Thu 06:12:23:693.732 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602784077376, Pid 1]: [SYS_INFO]: QEQueryLog: 98617f74-235a-11f1-9dd1-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-47331943:ctest-vn22-00062871 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-03-19 Thu 06:12:23:693.858 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602784077376, Pid 1]: [SYS_INFO]: QEQueryLog: 98617f74-235a-11f1-9dd1-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-47331943:ctest-vn22-00062871 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-03-19 Thu 06:12:23:693.960 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602784077376, Pid 1]: [SYS_INFO]: QEQueryLog: 98617f74-235a-11f1-9dd1-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-47331943:ctest-vn22-00062871 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-03-19 Thu 06:12:23:694.050 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602784077376, Pid 1]: [SYS_INFO]: QEQueryLog: 98617f74-235a-11f1-9dd1-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-47331943:ctest-vn22-00062871 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-03-19 Thu 06:12:23:694.139 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602784077376, Pid 1]: [SYS_INFO]: QEQueryLog: 98617f74-235a-11f1-9dd1-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-47331943:ctest-vn22-00062871 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-03-19 Thu 06:12:23:694.229 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602784077376, Pid 1]: [SYS_INFO]: QEQueryLog: 98617f74-235a-11f1-9dd1-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-47331943:ctest-vn22-00062871 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-03-19 Thu 06:12:23:694.320 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602784077376, Pid 1]: [SYS_INFO]: QEQueryLog: 98617f74-235a-11f1-9dd1-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-47331943:ctest-vn22-00062871 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-03-19 Thu 06:12:23:694.410 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602784077376, Pid 1]: [SYS_INFO]: QEQueryLog: 98617f74-235a-11f1-9dd1-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-47331943:ctest-vn22-00062871 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-03-19 Thu 06:12:23:694.539 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602784077376, Pid 1]: [SYS_INFO]: QEQueryLog: 98617f74-235a-11f1-9dd1-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-47331943:ctest-vn22-00062871 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-03-19 Thu 06:12:23:694.681 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602784077376, Pid 1]: [SYS_INFO]: QEQueryLog: 98617f74-235a-11f1-9dd1-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-47331943:ctest-vn22-00062871 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-03-19 Thu 06:12:23:694.823 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602784077376, Pid 1]: [SYS_INFO]: QEQueryLog: 98617f74-235a-11f1-9dd1-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-47331943:ctest-vn22-00062871 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-03-19 Thu 06:12:23:694.936 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602784077376, Pid 1]: [SYS_INFO]: QEQueryLog: 98617f74-235a-11f1-9dd1-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-47331943:ctest-vn22-00062871 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-03-19 Thu 06:12:23:695.028 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602784077376, Pid 1]: [SYS_INFO]: QEQueryLog: 98617f74-235a-11f1-9dd1-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-47331943:ctest-vn22-00062871 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-03-19 Thu 06:12:23:695.128 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602784077376, Pid 1]: [SYS_INFO]: QEQueryLog: 98617f74-235a-11f1-9dd1-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-47331943:ctest-vn22-00062871 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-03-19 Thu 06:12:23:695.220 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602784077376, Pid 1]: [SYS_INFO]: QEQueryLog: 98617f74-235a-11f1-9dd1-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-47331943:ctest-vn22-00062871 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-03-19 Thu 06:12:23:695.311 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602784077376, Pid 1]: [SYS_INFO]: QEQueryLog: 98617f74-235a-11f1-9dd1-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-47331943:ctest-vn22-00062871 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-03-19 Thu 06:12:23:695.430 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602784077376, Pid 1]: [SYS_INFO]: QEQueryLog: 98617f74-235a-11f1-9dd1-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-47331943:ctest-vn22-00062871 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-03-19 Thu 06:12:23:695.575 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602784077376, Pid 1]: [SYS_INFO]: QEQueryLog: 98617f74-235a-11f1-9dd1-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-47331943:ctest-vn22-00062871 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-03-19 Thu 06:12:23:695.721 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602784077376, Pid 1]: [SYS_INFO]: QEQueryLog: 98617f74-235a-11f1-9dd1-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-47331943:ctest-vn22-00062871 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-03-19 Thu 06:12:23:695.845 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602784077376, Pid 1]: [SYS_INFO]: QEQueryLog: 98617f74-235a-11f1-9dd1-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-47331943:ctest-vn22-00062871 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-03-19 Thu 06:12:23:695.945 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602784077376, Pid 1]: [SYS_INFO]: QEQueryLog: 98617f74-235a-11f1-9dd1-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-47331943:ctest-vn22-00062871 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-03-19 Thu 06:12:23:696.033 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602784077376, Pid 1]: [SYS_INFO]: QEQueryLog: 98617f74-235a-11f1-9dd1-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-47331943:ctest-vn22-00062871 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-03-19 Thu 06:12:23:696.120 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602784077376, Pid 1]: [SYS_INFO]: QEQueryLog: 98617f74-235a-11f1-9dd1-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-47331943:ctest-vn22-00062871 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-03-19 Thu 06:12:23:696.225 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602784077376, Pid 1]: [SYS_INFO]: QEQueryLog: 98617f74-235a-11f1-9dd1-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-47331943:ctest-vn22-00062871 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-03-19 Thu 06:12:23:696.355 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602784077376, Pid 1]: [SYS_INFO]: QEQueryLog: 98617f74-235a-11f1-9dd1-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-47331943:ctest-vn22-00062871 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-03-19 Thu 06:12:23:696.512 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602784077376, Pid 1]: [SYS_INFO]: QEQueryLog: 98617f74-235a-11f1-9dd1-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-47331943:ctest-vn22-00062871 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-03-19 Thu 06:12:23:696.658 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602784077376, Pid 1]: [SYS_INFO]: QEQueryLog: 98617f74-235a-11f1-9dd1-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-47331943:ctest-vn22-00062871 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-03-19 Thu 06:12:23:696.822 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602784077376, Pid 1]: [SYS_INFO]: QEQueryLog: 98617f74-235a-11f1-9dd1-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-47331943:ctest-vn22-00062871 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-03-19 Thu 06:12:23:696.963 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602784077376, Pid 1]: [SYS_INFO]: QEQueryLog: 98617f74-235a-11f1-9dd1-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-47331943:ctest-vn22-00062871 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-03-19 Thu 06:12:23:697.087 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602784077376, Pid 1]: [SYS_INFO]: QEQueryLog: 98617f74-235a-11f1-9dd1-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-47331943:ctest-vn22-00062871 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-03-19 Thu 06:12:23:697.177 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602784077376, Pid 1]: [SYS_INFO]: QEQueryLog: 98617f74-235a-11f1-9dd1-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-47331943:ctest-vn22-00062871 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-03-19 Thu 06:12:23:697.260 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602784077376, Pid 1]: [SYS_INFO]: QEQueryLog: 98617f74-235a-11f1-9dd1-00000a000026 ObjectVNTable :0:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-47331943:ctest-vn22-00062871 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-03-19 Thu 06:12:23:712.269 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603300132416, Pid 1]: [SYS_INFO]: QELog: 98617f74-235a-11f1-9dd1-00000a000026 UNION between tables of sizes 0 and 10 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-03-19 Thu 06:12:23:712.365 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603300132416, Pid 1]: [SYS_INFO]: QELog: 98617f74-235a-11f1-9dd1-00000a000026 Resulting size of set 10 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-03-19 Thu 06:12:23:712.377 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603300132416, Pid 1]: [SYS_INFO]: QELog: 98617f74-235a-11f1-9dd1-00000a000026 UNION between tables of sizes 10 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-03-19 Thu 06:12:23:712.407 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603300132416, Pid 1]: [SYS_INFO]: QELog: 98617f74-235a-11f1-9dd1-00000a000026 Resulting size of set 10 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-03-19 Thu 06:12:23:712.429 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603300132416, Pid 1]: [SYS_INFO]: QELog: 98617f74-235a-11f1-9dd1-00000a000026 UNION between tables of sizes 10 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-03-19 Thu 06:12:23:712.446 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603300132416, Pid 1]: [SYS_INFO]: QELog: 98617f74-235a-11f1-9dd1-00000a000026 Resulting size of set 10 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-03-19 Thu 06:12:23:712.455 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603300132416, Pid 1]: [SYS_INFO]: QELog: 98617f74-235a-11f1-9dd1-00000a000026 UNION between tables of sizes 10 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-03-19 Thu 06:12:23:712.469 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603300132416, Pid 1]: [SYS_INFO]: QELog: 98617f74-235a-11f1-9dd1-00000a000026 Resulting size of set 10 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-03-19 Thu 06:12:23:712.477 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603300132416, Pid 1]: [SYS_INFO]: QELog: 98617f74-235a-11f1-9dd1-00000a000026 UNION between tables of sizes 10 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-03-19 Thu 06:12:23:712.490 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603300132416, Pid 1]: [SYS_INFO]: QELog: 98617f74-235a-11f1-9dd1-00000a000026 Resulting size of set 10 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-03-19 Thu 06:12:23:712.738 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603300132416, Pid 1]: [SYS_INFO]: QEQueryLog: 98617f74-235a-11f1-9dd1-00000a000026 ObjectVNTable :1:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-47331943:ctest-vn22-00062871 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-03-19 Thu 06:12:23:712.862 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603300132416, Pid 1]: [SYS_INFO]: QEQueryLog: 98617f74-235a-11f1-9dd1-00000a000026 ObjectVNTable :1:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-47331943:ctest-vn22-00062871 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-03-19 Thu 06:12:23:712.955 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603300132416, Pid 1]: [SYS_INFO]: QEQueryLog: 98617f74-235a-11f1-9dd1-00000a000026 ObjectVNTable :1:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-47331943:ctest-vn22-00062871 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-03-19 Thu 06:12:23:713.047 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603300132416, Pid 1]: [SYS_INFO]: QEQueryLog: 98617f74-235a-11f1-9dd1-00000a000026 ObjectVNTable :1:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-47331943:ctest-vn22-00062871 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-03-19 Thu 06:12:23:713.136 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603300132416, Pid 1]: [SYS_INFO]: QEQueryLog: 98617f74-235a-11f1-9dd1-00000a000026 ObjectVNTable :1:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-47331943:ctest-vn22-00062871 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-03-19 Thu 06:12:23:713.226 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603300132416, Pid 1]: [SYS_INFO]: QEQueryLog: 98617f74-235a-11f1-9dd1-00000a000026 ObjectVNTable :1:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-47331943:ctest-vn22-00062871 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-03-19 Thu 06:12:23:713.322 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603300132416, Pid 1]: [SYS_INFO]: QEQueryLog: 98617f74-235a-11f1-9dd1-00000a000026 ObjectVNTable :1:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-47331943:ctest-vn22-00062871 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-03-19 Thu 06:12:23:713.425 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603300132416, Pid 1]: [SYS_INFO]: QEQueryLog: 98617f74-235a-11f1-9dd1-00000a000026 ObjectVNTable :1:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-47331943:ctest-vn22-00062871 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-03-19 Thu 06:12:23:713.514 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603300132416, Pid 1]: [SYS_INFO]: QEQueryLog: 98617f74-235a-11f1-9dd1-00000a000026 ObjectVNTable :1:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-47331943:ctest-vn22-00062871 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-03-19 Thu 06:12:23:713.605 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603300132416, Pid 1]: [SYS_INFO]: QEQueryLog: 98617f74-235a-11f1-9dd1-00000a000026 ObjectVNTable :1:Column ObjectTypeName1 Table ObjectVNTable object_id default-domain:ctest-AnalyticsBasicTestSanity-47331943:ctest-vn22-00062871 src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-03-19 Thu 06:12:23:713.844 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603300132416, Pid 1]: [SYS_INFO]: QELog: Will Jsonify #rows 43 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 542 2026-03-19 Thu 06:12:23:714.234 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603300132416, Pid 1]: [SYS_INFO]: QELog: Did Jsonify #rows 43 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 551 2026-03-19 Thu 06:12:23:714.381 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603300132416, Pid 1]: [SYS_INFO]: QELog: QE Query Result is {"progress":100, "lines":6, "count":43} src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 586 2026-03-19 Thu 06:12:23:715.022 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602771482176, Pid 1]: [SYS_INFO]: QELog: Finished: QID 98617f74-235a-11f1-9dd1-00000a000026 Table ObjectVNTable Time(ms) 136 RedisTime(ms) 0 MergeTime(ms) 0 Rows 43 EnQ-delay1 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 661 2026-03-19 Thu 06:12:23:715.140 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602771482176, Pid 1]: [SYS_INFO]: QELog: Result 1 , 1 conn src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 699 2026-03-19 Thu 06:12:23:715.190 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602771482176, Pid 1]: [SYS_INFO]: QueryPerfInfo: name = cn-jenkins-deploy-platform-ansible-os-5495-1. table = ObjectVNTable query_stats= [ qid = 98617f74-235a-11f1-9dd1-00000a000026 where = [[{"name": "ObjectId", "value": "default-domain:ctest-AnalyticsBasicTestSanity-47331943:ctest-vn22-00062871", "op": 1, "value2": null}]] select = { "select" : ["ObjectId", "Source", "ObjectLog", "SystemLog", "Messagetype", "ModuleId", "MessageTS"] } post = time_span = 8 error = None chunks = 3 chunk_where_time = 133,0, 114,0, 113,0, chunk_select_time = 0,0, 0,3, 0,0, chunk_postproc_time = 0,0, 0,0, 0,0, chunk_merge_time = final_merge_time = 0 time = 136 rows = 43 enq_delay = 1 ] file = src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc line = 657 2026-03-19 Thu 06:12:26:738.616 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: [SYS_INFO]: QELog: StartPipeline on : Redis:10.0.0.38 Port:6379 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 812 2026-03-19 Thu 06:12:26:738.949 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: [SYS_INFO]: QELog: Got Query to prepare for QID 9a43cfdb-235a-11f1-91ba-00000a000026 src/contrail-analytics/contrail-query-engine/query.cc 1165 2026-03-19 Thu 06:12:26:739.000 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: [SYS_INFO]: QEQueryLog: 9a43cfdb-235a-11f1-91ba-00000a000026 :0:json query is: { end_time: 1773900746723764 , enqueue_time: 1773900746738250 , query_metadata: {"enqueue_time": 1773900746738151} , select_fields: ["ObjectId", "Source", "ObjectLog", "SystemLog", "ModuleId", "Messagetype", "MessageTS"] , start_time: 1773900725601133 , table: "ObjectVMTable" , table_schema: [{"name": "MessageTS", "datatype": "int", "index": false, "suffixes": null}, {"name": "ObjectId", "datatype": "string", "index": true, "suffixes": null}, {"name": "Source", "datatype": "string", "index": true, "suffixes": null}, {"name": "ModuleId", "datatype": "string", "index": true, "suffixes": null}, {"name": "Messagetype", "datatype": "string", "index": true, "suffixes": null}, {"name": "ObjectLog", "datatype": "string", "index": false, "suffixes": null}, {"name": "SystemLog", "datatype": "string", "index": false, "suffixes": null}] , where: [[{"name": "ObjectId", "value": "00d7eb24-0beb-4c53-8aa7-7b03c789c5fa", "op": 1, "value2": null}]] } src/contrail-analytics/contrail-query-engine/query.cc 467 2026-03-19 Thu 06:12:26:739.146 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: [SYS_INFO]: QELog: Chunks: 3 Need Merge: 0 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 936 2026-03-19 Thu 06:12:26:739.163 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: [SYS_INFO]: QELog: Getting Least Loaded Conn as :0 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 986 2026-03-19 Thu 06:12:26:739.212 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: [SYS_INFO]: QELog: Starting Pipeline for 9a43cfdb-235a-11f1-91ba-00000a000026 , 1 conn, 16 tasks src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 994 2026-03-19 Thu 06:12:26:739.329 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603291735616, Pid 1]: [SYS_INFO]: QEQueryLog: 9a43cfdb-235a-11f1-91ba-00000a000026 :0:json query is: { end_time: 1773900746723764 , enqueue_time: 1773900746738250 , query_metadata: {"enqueue_time": 1773900746738151} , select_fields: ["ObjectId", "Source", "ObjectLog", "SystemLog", "ModuleId", "Messagetype", "MessageTS"] , start_time: 1773900725601133 , table: "ObjectVMTable" , table_schema: [{"name": "MessageTS", "datatype": "int", "index": false, "suffixes": null}, {"name": "ObjectId", "datatype": "string", "index": true, "suffixes": null}, {"name": "Source", "datatype": "string", "index": true, "suffixes": null}, {"name": "ModuleId", "datatype": "string", "index": true, "suffixes": null}, {"name": "Messagetype", "datatype": "string", "index": true, "suffixes": null}, {"name": "ObjectLog", "datatype": "string", "index": false, "suffixes": null}, {"name": "SystemLog", "datatype": "string", "index": false, "suffixes": null}] , where: [[{"name": "ObjectId", "value": "00d7eb24-0beb-4c53-8aa7-7b03c789c5fa", "op": 1, "value2": null}]] } src/contrail-analytics/contrail-query-engine/query.cc 467 2026-03-19 Thu 06:12:26:784.625 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602784077376, Pid 1]: [SYS_INFO]: QELog: 9a43cfdb-235a-11f1-91ba-00000a000026 UNION between tables of sizes 0 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-03-19 Thu 06:12:26:784.675 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602784077376, Pid 1]: [SYS_INFO]: QELog: 9a43cfdb-235a-11f1-91ba-00000a000026 Resulting size of set 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-03-19 Thu 06:12:26:784.683 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602784077376, Pid 1]: [SYS_INFO]: QELog: 9a43cfdb-235a-11f1-91ba-00000a000026 UNION between tables of sizes 0 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-03-19 Thu 06:12:26:784.702 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602784077376, Pid 1]: [SYS_INFO]: QELog: 9a43cfdb-235a-11f1-91ba-00000a000026 Resulting size of set 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-03-19 Thu 06:12:26:784.708 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602784077376, Pid 1]: [SYS_INFO]: QELog: 9a43cfdb-235a-11f1-91ba-00000a000026 UNION between tables of sizes 0 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-03-19 Thu 06:12:26:784.715 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602784077376, Pid 1]: [SYS_INFO]: QELog: 9a43cfdb-235a-11f1-91ba-00000a000026 Resulting size of set 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-03-19 Thu 06:12:26:784.721 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602784077376, Pid 1]: [SYS_INFO]: QELog: 9a43cfdb-235a-11f1-91ba-00000a000026 UNION between tables of sizes 0 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-03-19 Thu 06:12:26:784.728 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602784077376, Pid 1]: [SYS_INFO]: QELog: 9a43cfdb-235a-11f1-91ba-00000a000026 Resulting size of set 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-03-19 Thu 06:12:26:784.735 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602784077376, Pid 1]: [SYS_INFO]: QELog: 9a43cfdb-235a-11f1-91ba-00000a000026 UNION between tables of sizes 0 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-03-19 Thu 06:12:26:784.741 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602784077376, Pid 1]: [SYS_INFO]: QELog: 9a43cfdb-235a-11f1-91ba-00000a000026 Resulting size of set 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-03-19 Thu 06:12:26:784.860 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602784077376, Pid 1]: [SYS_INFO]: QELog: 9a43cfdb-235a-11f1-91ba-00000a000026 UNION between tables of sizes 0 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-03-19 Thu 06:12:26:784.868 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602784077376, Pid 1]: [SYS_INFO]: QELog: 9a43cfdb-235a-11f1-91ba-00000a000026 Resulting size of set 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-03-19 Thu 06:12:26:784.874 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602784077376, Pid 1]: [SYS_INFO]: QELog: 9a43cfdb-235a-11f1-91ba-00000a000026 UNION between tables of sizes 0 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-03-19 Thu 06:12:26:784.881 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602784077376, Pid 1]: [SYS_INFO]: QELog: 9a43cfdb-235a-11f1-91ba-00000a000026 Resulting size of set 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-03-19 Thu 06:12:26:784.887 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602784077376, Pid 1]: [SYS_INFO]: QELog: 9a43cfdb-235a-11f1-91ba-00000a000026 UNION between tables of sizes 0 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-03-19 Thu 06:12:26:784.894 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602784077376, Pid 1]: [SYS_INFO]: QELog: 9a43cfdb-235a-11f1-91ba-00000a000026 Resulting size of set 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-03-19 Thu 06:12:26:784.900 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602784077376, Pid 1]: [SYS_INFO]: QELog: 9a43cfdb-235a-11f1-91ba-00000a000026 UNION between tables of sizes 0 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-03-19 Thu 06:12:26:784.906 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602784077376, Pid 1]: [SYS_INFO]: QELog: 9a43cfdb-235a-11f1-91ba-00000a000026 Resulting size of set 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-03-19 Thu 06:12:26:784.916 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602784077376, Pid 1]: [SYS_INFO]: QELog: 9a43cfdb-235a-11f1-91ba-00000a000026 UNION between tables of sizes 0 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-03-19 Thu 06:12:26:784.922 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602784077376, Pid 1]: [SYS_INFO]: QELog: 9a43cfdb-235a-11f1-91ba-00000a000026 Resulting size of set 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-03-19 Thu 06:12:26:784.940 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602784077376, Pid 1]: [SYS_INFO]: QELog: 9a43cfdb-235a-11f1-91ba-00000a000026 UNION between tables of sizes 0 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-03-19 Thu 06:12:26:784.947 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602784077376, Pid 1]: [SYS_INFO]: QELog: 9a43cfdb-235a-11f1-91ba-00000a000026 Resulting size of set 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-03-19 Thu 06:12:26:784.954 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602784077376, Pid 1]: [SYS_INFO]: QELog: 9a43cfdb-235a-11f1-91ba-00000a000026 UNION between tables of sizes 0 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-03-19 Thu 06:12:26:784.961 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602784077376, Pid 1]: [SYS_INFO]: QELog: 9a43cfdb-235a-11f1-91ba-00000a000026 Resulting size of set 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-03-19 Thu 06:12:26:784.968 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602784077376, Pid 1]: [SYS_INFO]: QELog: 9a43cfdb-235a-11f1-91ba-00000a000026 UNION between tables of sizes 0 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-03-19 Thu 06:12:26:784.974 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602784077376, Pid 1]: [SYS_INFO]: QELog: 9a43cfdb-235a-11f1-91ba-00000a000026 Resulting size of set 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-03-19 Thu 06:12:26:784.981 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602784077376, Pid 1]: [SYS_INFO]: QELog: 9a43cfdb-235a-11f1-91ba-00000a000026 UNION between tables of sizes 0 and 5 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-03-19 Thu 06:12:26:784.996 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602784077376, Pid 1]: [SYS_INFO]: QELog: 9a43cfdb-235a-11f1-91ba-00000a000026 Resulting size of set 5 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-03-19 Thu 06:12:26:785.004 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602784077376, Pid 1]: [SYS_INFO]: QELog: 9a43cfdb-235a-11f1-91ba-00000a000026 UNION between tables of sizes 5 and 0 src/contrail-analytics/contrail-query-engine/set_operation.cc 59 2026-03-19 Thu 06:12:26:785.018 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602784077376, Pid 1]: [SYS_INFO]: QELog: 9a43cfdb-235a-11f1-91ba-00000a000026 Resulting size of set 5 src/contrail-analytics/contrail-query-engine/set_operation.cc 66 2026-03-19 Thu 06:12:26:785.246 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602784077376, Pid 1]: [SYS_INFO]: QEQueryLog: 9a43cfdb-235a-11f1-91ba-00000a000026 :0:json query is: { end_time: 1773900746723764 , enqueue_time: 1773900746738250 , query_metadata: {"enqueue_time": 1773900746738151} , select_fields: ["ObjectId", "Source", "ObjectLog", "SystemLog", "ModuleId", "Messagetype", "MessageTS"] , start_time: 1773900725601133 , table: "ObjectVMTable" , table_schema: [{"name": "MessageTS", "datatype": "int", "index": false, "suffixes": null}, {"name": "ObjectId", "datatype": "string", "index": true, "suffixes": null}, {"name": "Source", "datatype": "string", "index": true, "suffixes": null}, {"name": "ModuleId", "datatype": "string", "index": true, "suffixes": null}, {"name": "Messagetype", "datatype": "string", "index": true, "suffixes": null}, {"name": "ObjectLog", "datatype": "string", "index": false, "suffixes": null}, {"name": "SystemLog", "datatype": "string", "index": false, "suffixes": null}] , where: [[{"name": "ObjectId", "value": "00d7eb24-0beb-4c53-8aa7-7b03c789c5fa", "op": 1, "value2": null}]] } src/contrail-analytics/contrail-query-engine/query.cc 467 2026-03-19 Thu 06:12:26:785.496 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602784077376, Pid 1]: [SYS_INFO]: QEQueryLog: 9a43cfdb-235a-11f1-91ba-00000a000026 ObjectVMTable :1:Column ObjectTypeName1 Table ObjectVMTable object_id 00d7eb24-0beb-4c53-8aa7-7b03c789c5fa src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-03-19 Thu 06:12:26:785.593 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602784077376, Pid 1]: [SYS_INFO]: QEQueryLog: 9a43cfdb-235a-11f1-91ba-00000a000026 ObjectVMTable :1:Column ObjectTypeName1 Table ObjectVMTable object_id 00d7eb24-0beb-4c53-8aa7-7b03c789c5fa src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-03-19 Thu 06:12:26:785.679 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602784077376, Pid 1]: [SYS_INFO]: QEQueryLog: 9a43cfdb-235a-11f1-91ba-00000a000026 ObjectVMTable :1:Column ObjectTypeName1 Table ObjectVMTable object_id 00d7eb24-0beb-4c53-8aa7-7b03c789c5fa src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-03-19 Thu 06:12:26:785.790 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602784077376, Pid 1]: [SYS_INFO]: QEQueryLog: 9a43cfdb-235a-11f1-91ba-00000a000026 ObjectVMTable :1:Column ObjectTypeName1 Table ObjectVMTable object_id 00d7eb24-0beb-4c53-8aa7-7b03c789c5fa src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-03-19 Thu 06:12:26:785.884 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602784077376, Pid 1]: [SYS_INFO]: QEQueryLog: 9a43cfdb-235a-11f1-91ba-00000a000026 ObjectVMTable :1:Column ObjectTypeName1 Table ObjectVMTable object_id 00d7eb24-0beb-4c53-8aa7-7b03c789c5fa src/contrail-analytics/contrail-query-engine/select.cc 1022 2026-03-19 Thu 06:12:26:786.098 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602784077376, Pid 1]: [SYS_INFO]: QELog: Will Jsonify #rows 5 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 542 2026-03-19 Thu 06:12:26:786.610 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602784077376, Pid 1]: [SYS_INFO]: QELog: Did Jsonify #rows 5 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 551 2026-03-19 Thu 06:12:26:786.756 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602784077376, Pid 1]: [SYS_INFO]: QELog: QE Query Result is {"progress":100, "lines":1, "count":5} src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 586 2026-03-19 Thu 06:12:26:787.061 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602754688576, Pid 1]: [SYS_INFO]: QELog: Finished: QID 9a43cfdb-235a-11f1-91ba-00000a000026 Table ObjectVMTable Time(ms) 48 RedisTime(ms) 0 MergeTime(ms) 0 Rows 5 EnQ-delay0 src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 661 2026-03-19 Thu 06:12:26:787.148 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602754688576, Pid 1]: [SYS_INFO]: QELog: Result 1 , 1 conn src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc 699 2026-03-19 Thu 06:12:26:787.164 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603304330816, Pid 1]: [SYS_INFO]: QueryPerfInfo: name = cn-jenkins-deploy-platform-ansible-os-5495-1. table = ObjectVMTable query_stats= [ qid = 9a43cfdb-235a-11f1-91ba-00000a000026 where = [[{"name": "ObjectId", "value": "00d7eb24-0beb-4c53-8aa7-7b03c789c5fa", "op": 1, "value2": null}]] select = { "select" : ["ObjectId", "Source", "ObjectLog", "SystemLog", "ModuleId", "Messagetype", "MessageTS"] } post = time_span = 8 error = None chunks = 3 chunk_where_time = 44,0, 43,0, 44,0, chunk_select_time = 0,0, 0,0, 0,0, chunk_postproc_time = 0,0, 0,0, 0,0, chunk_merge_time = final_merge_time = 0 time = 48 rows = 5 enq_delay = 0 ] file = src/contrail-analytics/contrail-query-engine/QEOpServerProxy.cc line = 657 2026-03-19 Thu 06:12:38:551.251 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602775680576, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 21 without seqno , total = 0 2026-03-19 Thu 06:13:38:551.781 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602775680576, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 22 without seqno , total = 0 2026-03-19 Thu 06:14:38:552.392 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602771482176, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 23 without seqno , total = 0 2026-03-19 Thu 06:15:38:552.972 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603300132416, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 24 without seqno , total = 0 2026-03-19 Thu 06:16:38:554.164 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602775680576, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 25 without seqno , total = 0 2026-03-19 Thu 06:17:38:555.200 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602775680576, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 26 without seqno , total = 0 2026-03-19 Thu 06:18:38:555.718 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602775680576, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 27 without seqno , total = 0 2026-03-19 Thu 06:19:38:556.660 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602750490176, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 28 without seqno , total = 0 2026-03-19 Thu 06:20:38:557.238 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602775680576, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 29 without seqno , total = 0 2026-03-19 Thu 06:21:38:561.240 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602775680576, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 30 without seqno , total = 0 2026-03-19 Thu 06:22:38:562.075 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602750490176, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 31 without seqno , total = 0 2026-03-19 Thu 06:23:38:562.855 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140602750490176, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 32 without seqno , total = 0 2026-03-19 Thu 06:24:32:099.092 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.38:8091::10.0.0.38:34412 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2026-03-19 Thu 06:24:38:563.656 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603312727616, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 33 without seqno , total = 0 2026-03-19 Thu 06:25:38:564.353 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603308529216, Pid 1]: SyncAllMaps for UserDefinedLogStatistic:2 period 60 cycle 34 without seqno , total = 0 2026-03-19 Thu 06:26:28:935.214 UTC cn-jenkins-deploy-platform-ansible-os-5495-1 [Thread 140603319867520, Pid 1]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.38:8091::10.0.0.38:36710 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256