2026-06-08 Mon 05:44:40:312.574 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710049530944, Pid 25272]: DnsAgent [SYS_INFO]: VersionInfoLog: {"build-info": [{"build-version": "v0+nightly.r25.1", "build-time": "2026-05-28 00:08:01.734303", "build-user": "root", "build-hostname": "jenkins-build-prepare-sandbox-15063-1.novalocal", "build-id": "v0+nightly.r25.1", "build-number": "202605280007"}]} src/contrail-common/base/misc_utils.cc 39 2026-06-08 Mon 05:44:40:312.648 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710049530944, Pid 25272]: SANDESH: ROLE : Generator 2026-06-08 Mon 05:44:40:312.657 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710049530944, Pid 25272]: SANDESH: MODULE : contrail-dns 2026-06-08 Mon 05:44:40:312.662 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710049530944, Pid 25272]: SANDESH: SOURCE : cn-jenkins-deploy-platform-ansible-os-5902-3 2026-06-08 Mon 05:44:40:312.667 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710049530944, Pid 25272]: SANDESH: NODE TYPE : Control 2026-06-08 Mon 05:44:40:312.672 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710049530944, Pid 25272]: SANDESH: INSTANCE ID : 0 2026-06-08 Mon 05:44:40:312.677 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710049530944, Pid 25272]: SANDESH: HTTP SERVER PORT : 8092 2026-06-08 Mon 05:44:40:312.683 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710049530944, Pid 25272]: SANDESH: System Log Send Rate Limit: 0 -> 0 2026-06-08 Mon 05:44:40:329.908 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710049530944, Pid 25272]: HTTP Introspect Init 2026-06-08 Mon 05:44:40:330.046 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710049530944, Pid 25272]: TCP [SYS_DEBUG]: TcpServerMessageLog: Server 0.0.0.0:8092 Initialization complete src/contrail-common/io/tcp_server.cc 137 2026-06-08 Mon 05:44:40:330.130 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710049530944, Pid 25272]: Sandesh Http Server Port: 8092 2026-06-08 Mon 05:44:40:330.213 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710049530944, Pid 25272]: SANDESH: NOT Writing http_port 8092TO : /tmp/contrail-dns.25125.http_port 2026-06-08 Mon 05:44:40:330.222 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710049530944, Pid 25272]: SANDESH: CONNECT TO COLLECTOR: 1 2026-06-08 Mon 05:44:40:330.562 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710049530944, Pid 25272]: Idle 2026-06-08 Mon 05:44:40:330.699 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710049530944, Pid 25272]: SANDESH: No Client: 1780897480330643 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3:Control:contrail-dns:0 client_info= [ status = Idle successful_connections = 0 pid = 25272 http_port = 8092 start_time = 1780897480330573 collector_name = collector_ip = 0.0.0.0:0 collector_list= [ [ ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 0, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 0, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-06-08 Mon 05:44:40:332.073 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710049530944, Pid 25272]: SANDESH: Logging: DISABLED -> ENABLED 2026-06-08 Mon 05:44:40:332.100 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710049530944, Pid 25272]: SANDESH: Logging: LEVEL: [ INVALID ] -> [ SYS_DEBUG ] log4level: [ NOTSET ] -> [ DEBUG ] 2026-06-08 Mon 05:44:40:333.328 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710036584000, Pid 25272]: Processing scm::EvStart in state Idle 2026-06-08 Mon 05:44:40:333.361 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710036584000, Pid 25272]: Disconnect 2026-06-08 Mon 05:44:40:333.423 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710036584000, Pid 25272]: Processing scm::EvCollectorUpdate in state Disconnect 2026-06-08 Mon 05:44:40:333.603 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710036584000, Pid 25272]: Connect : Start Connect timer 10.0.0.38:8086 2026-06-08 Mon 05:44:40:333.655 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710036584000, Pid 25272]: Processing scm::EvSandeshSend in state Connect 2026-06-08 Mon 05:44:40:333.720 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710036584000, Pid 25272]: SANDESH: Send FAILED: 1780897480333412 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3:Control:contrail-dns:0 client_info= [ status = Disconnect successful_connections = 0 pid = 25272 http_port = 8092 start_time = 1780897480330573 collector_name = collector_ip = 0.0.0.0:0 collector_list= [ [ ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 0, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-06-08 Mon 05:44:40:333.736 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710036584000, Pid 25272]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:44:40:333.745 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710036584000, Pid 25272]: Processing scm::EvSandeshSend in state Connect 2026-06-08 Mon 05:44:40:333.791 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710036584000, Pid 25272]: SANDESH: Send FAILED: 1780897480333454 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3:Control:contrail-dns:0 client_info= [ status = Disconnect successful_connections = 0 pid = 25272 http_port = 8092 start_time = 1780897480330573 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 0, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-06-08 Mon 05:44:40:333.800 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710036584000, Pid 25272]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:44:40:333.810 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710036584000, Pid 25272]: Processing scm::EvSandeshSend in state Connect 2026-06-08 Mon 05:44:40:333.848 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710036584000, Pid 25272]: SANDESH: Send FAILED: 1780897480333647 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 0 pid = 25272 http_port = 8092 start_time = 1780897480330573 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 0, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-06-08 Mon 05:44:40:333.856 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710036584000, Pid 25272]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:44:40:834.060 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710049530944, Pid 25272]: TCP [SYS_DEBUG]: TcpServerMessageLog: Server 10.20.0.14:8093 Initialization complete src/contrail-common/io/tcp_server.cc 137 2026-06-08 Mon 05:44:40:834.160 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710049530944, Pid 25272]: DnsAgent [SYS_NOTICE]: DnsOperational: Bulk sync has been started controller/src/dns/main.cc 401 2026-06-08 Mon 05:44:40:834.220 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Config Client Mgr SM: Start RabbitMqReader and init Database src/contrail-common/config-client-mgr/config_client_manager.cc 281 2026-06-08 Mon 05:44:40:834.304 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: RabbitMQ SM: RabbitMQ connection down src/contrail-common/config-client-mgr/config_amqp_client.cc 157 2026-06-08 Mon 05:44:40:834.313 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: RabbitMQ SM: Connect to Rabbit MQ with queue_delete TRUE src/contrail-common/config-client-mgr/config_amqp_client.cc 167 2026-06-08 Mon 05:44:40:834.427 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710015592000, Pid 25272]: Processing scm::EvSandeshSend in state Connect 2026-06-08 Mon 05:44:40:834.457 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710015592000, Pid 25272]: SANDESH: Send FAILED: 1780897480834289 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Disconnect : EvCollectorUpdate -> Connect ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.38:5673, (*_iter6) = 10.0.0.50:5673, (*_iter6) = 10.0.0.33:5673, ] ] status = Down description = RabbitMQ connection down ], ] ] flag_infos= [ [ ] ] description = Number of connections:2, Expected:3 Missing: Database:Cassandra ], ] ] ] 2026-06-08 Mon 05:44:40:834.464 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710015592000, Pid 25272]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-06-08 Mon 05:44:40:885.063 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: RabbitMQ SM: RabbitMQ connection established src/contrail-common/config-client-mgr/config_amqp_client.cc 150 2026-06-08 Mon 05:44:40:885.103 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Config Client Mgr SM: Init Database src/contrail-common/config-client-mgr/config_client_manager.cc 285 2026-06-08 Mon 05:44:40:885.105 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139709893969472, Pid 25272]: Processing scm::EvSandeshSend in state Connect 2026-06-08 Mon 05:44:40:885.153 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Lost Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 579 2026-06-08 Mon 05:44:40:885.185 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139709893969472, Pid 25272]: SANDESH: Send FAILED: 1780897480885017 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Disconnect : EvCollectorUpdate -> Connect ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.38:5673, (*_iter6) = 10.0.0.50:5673, (*_iter6) = 10.0.0.33:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Number of connections:2, Expected:3 Missing: Database:Cassandra ], ] ] ] 2026-06-08 Mon 05:44:40:885.197 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139709893969472, Pid 25272]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-06-08 Mon 05:44:40:885.206 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139709893969472, Pid 25272]: Processing scm::EvSandeshSend in state Connect 2026-06-08 Mon 05:44:40:885.236 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139709893969472, Pid 25272]: SANDESH: Send FAILED: 1780897480885133 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Disconnect : EvCollectorUpdate -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.38:9041, (*_iter6) = 10.0.0.50:9041, (*_iter6) = 10.0.0.33:9041, ] ] status = Down description = Lost Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.38:5673, (*_iter6) = 10.0.0.50:5673, (*_iter6) = 10.0.0.33:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector, Database:Cassandra connection down ], ] ] ] 2026-06-08 Mon 05:44:40:885.242 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139709893969472, Pid 25272]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-06-08 Mon 05:44:40:885.214 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db Init src/contrail-common/config-client-mgr/config_cassandra_client.cc 87 2026-06-08 Mon 05:44:40:909.518 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db SetTableSpace src/contrail-common/config-client-mgr/config_cassandra_client.cc 94 2026-06-08 Mon 05:44:40:910.104 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db UseColumnFamily uuidTableName src/contrail-common/config-client-mgr/config_cassandra_client.cc 103 2026-06-08 Mon 05:44:40:910.135 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db UseColumnFamily fqnTableName src/contrail-common/config-client-mgr/config_cassandra_client.cc 109 2026-06-08 Mon 05:44:40:910.242 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710023988800, Pid 25272]: Processing scm::EvSandeshSend in state Connect 2026-06-08 Mon 05:44:40:910.279 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710023988800, Pid 25272]: SANDESH: Send FAILED: 1780897480910176 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Disconnect : EvCollectorUpdate -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.38:9041, (*_iter6) = 10.0.0.50:9041, (*_iter6) = 10.0.0.33:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.38:5673, (*_iter6) = 10.0.0.50:5673, (*_iter6) = 10.0.0.33:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-08 Mon 05:44:40:910.289 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710023988800, Pid 25272]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-06-08 Mon 05:44:40:910.431 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:40:910.439 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: BulkDataSync Started src/contrail-common/config-client-mgr/config_cassandra_client.cc 392 2026-06-08 Mon 05:44:40:913.758 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:40:928.345 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:40:933.142 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:40:934.944 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:40:936.865 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:40:938.041 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:40:940.228 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:40:941.523 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:40:943.441 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:40:944.712 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:40:946.763 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:40:947.780 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:40:948.896 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:40:950.652 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:40:951.914 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:40:952.939 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:40:954.305 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:40:961.044 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:40:963.324 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:40:964.938 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:40:965.856 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:40:966.649 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:40:968.369 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:40:969.606 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:40:971.275 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:40:973.249 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:40:976.588 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:40:979.253 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:40:982.210 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:40:983.878 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:40:985.296 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:40:986.836 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:40:989.851 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:40:991.094 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:40:992.446 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:40:993.762 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:40:994.850 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:41:002.311 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:41:012.497 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:41:025.480 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:41:029.252 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:41:033.247 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:41:035.032 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:41:043.791 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:41:046.723 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:41:048.156 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:41:059.225 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:41:061.042 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:41:064.146 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:41:075.334 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:41:090.715 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:41:091.959 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:41:124.616 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:41:126.584 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:41:127.796 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:41:129.823 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:41:131.298 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:41:132.523 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:41:175.258 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:41:185.905 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:41:187.368 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:41:196.509 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:41:200.196 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:41:209.086 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:41:230.550 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:41:232.465 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:41:236.658 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:41:240.286 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:41:241.726 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:41:246.761 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:41:252.103 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:41:253.514 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:41:254.593 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:41:263.373 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:41:266.362 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:41:269.605 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:41:273.048 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:41:274.475 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:41:275.590 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:41:278.242 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:41:280.514 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:41:285.611 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:41:291.004 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:41:297.013 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:41:298.072 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:41:304.187 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:41:306.145 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:41:315.685 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:41:325.358 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:41:329.351 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:41:330.608 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:41:332.594 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:41:333.643 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:41:334.889 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:41:345.424 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:41:347.228 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:41:348.711 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:41:351.667 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:41:353.039 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:41:355.292 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:41:369.086 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:41:376.610 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:41:384.183 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:41:409.119 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:41:411.816 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:41:413.398 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:41:448.007 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:41:451.087 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:41:452.934 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:41:455.450 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:41:456.729 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:41:457.787 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:41:464.113 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:41:466.712 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:41:469.509 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:41:473.338 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:41:475.093 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:41:476.314 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:41:476.415 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710023988800, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: One reader finished BulkSync src/contrail-common/config-client-mgr/config_cassandra_client.cc 557 2026-06-08 Mon 05:44:41:476.523 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710036584000, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: One reader finished BulkSync src/contrail-common/config-client-mgr/config_cassandra_client.cc 557 2026-06-08 Mon 05:44:41:476.656 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139709898167872, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: One reader finished BulkSync src/contrail-common/config-client-mgr/config_cassandra_client.cc 557 2026-06-08 Mon 05:44:41:476.710 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710019790400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: One reader finished BulkSync src/contrail-common/config-client-mgr/config_cassandra_client.cc 557 2026-06-08 Mon 05:44:41:476.769 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710028187200, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: One reader finished BulkSync src/contrail-common/config-client-mgr/config_cassandra_client.cc 557 2026-06-08 Mon 05:44:41:476.879 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710007060032, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: One reader finished BulkSync src/contrail-common/config-client-mgr/config_cassandra_client.cc 557 2026-06-08 Mon 05:44:41:476.930 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710011393600, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: One reader finished BulkSync src/contrail-common/config-client-mgr/config_cassandra_client.cc 557 2026-06-08 Mon 05:44:41:477.776 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: BulkSyncDone by all readers src/contrail-common/config-client-mgr/config_cassandra_client.cc 553 2026-06-08 Mon 05:44:41:477.806 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Config Client Mgr SM: End of RIB computed and notification sent src/contrail-common/config-client-mgr/config_client_manager.cc 187 2026-06-08 Mon 05:44:41:477.938 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710015592000, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: TRUE src/contrail-common/config-client-mgr/config_client_manager.cc 216 2026-06-08 Mon 05:44:41:478.044 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139709998663232, Pid 25272]: Processing scm::EvSandeshSend in state Connect 2026-06-08 Mon 05:44:41:478.074 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139709998663232, Pid 25272]: SANDESH: Send FAILED: 1780897481477857 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Disconnect : EvCollectorUpdate -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.38:9041, (*_iter6) = 10.0.0.50:9041, (*_iter6) = 10.0.0.33:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.38:5673, (*_iter6) = 10.0.0.50:5673, (*_iter6) = 10.0.0.33:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-08 Mon 05:44:41:478.083 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139709998663232, Pid 25272]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-06-08 Mon 05:44:41:485.815 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139709898167872, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:41:486.161 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139709898167872, Pid 25272]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating network-ipam:default-domain:default-project:default-network-ipam controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:44:41:486.197 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139709898167872, Pid 25272]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating project:default-domain:default-project controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:44:41:486.210 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139709898167872, Pid 25272]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-network-ipam controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:44:41:486.222 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139709898167872, Pid 25272]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating virtual-network:default-domain:default-project:ip-fabric controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:44:41:486.244 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139709898167872, Pid 25272]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-virtual-network controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:44:41:486.254 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139709898167872, Pid 25272]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating tag-type:label controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:44:41:486.264 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139709898167872, Pid 25272]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating config-node:default-global-system-config:cn-jenkins-deploy-platform-ansible-os-5902-1. controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:44:41:486.276 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139709898167872, Pid 25272]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating global-system-config:default-global-system-config controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:44:41:486.293 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139709898167872, Pid 25272]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-config-node controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:44:41:486.304 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139709898167872, Pid 25272]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating tag-type:application controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:44:41:487.946 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710036584000, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:41:488.473 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710036584000, Pid 25272]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating application-policy-set:default-policy-management:default-application-policy-set controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:44:41:488.522 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710036584000, Pid 25272]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating policy-management:default-policy-management controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:44:41:488.537 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710036584000, Pid 25272]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating policy-management-application-policy-set controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:44:41:488.552 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710036584000, Pid 25272]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating global-vrouter-config:default-global-system-config:default-global-vrouter-config controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:44:41:488.561 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710036584000, Pid 25272]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating application-policy-set-global-vrouter-config controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:44:41:488.571 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710036584000, Pid 25272]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating global-qos-config:default-global-system-config:default-global-qos-config controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:44:41:488.584 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710036584000, Pid 25272]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-global-qos-config controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:44:41:488.602 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710036584000, Pid 25272]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-xmpp-close-reason controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:44:41:488.621 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710036584000, Pid 25272]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:44:41:488.636 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710036584000, Pid 25272]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating config-database-node:default-global-system-config:cn-jenkins-deploy-platform-ansible-os-5902-3. controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:44:41:488.653 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710036584000, Pid 25272]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-config-database-node controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:44:41:488.694 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710036584000, Pid 25272]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating namespace:default-domain:default-namespace controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:44:41:488.713 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710036584000, Pid 25272]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating domain:default-domain controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:44:41:488.734 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710036584000, Pid 25272]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating domain-namespace controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:44:41:488.749 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710036584000, Pid 25272]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating service-template:default-domain:analyzer-template controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:44:41:488.768 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710036584000, Pid 25272]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating domain-service-template controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:44:41:488.793 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710036584000, Pid 25272]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-core-files controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:44:41:488.814 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710036584000, Pid 25272]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:44:41:488.830 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710036584000, Pid 25272]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-address-mismatch-control controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:44:41:488.852 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710036584000, Pid 25272]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:44:41:488.866 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710036584000, Pid 25272]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-conf-incorrect controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:44:41:488.884 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710036584000, Pid 25272]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:44:41:489.317 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710007060032, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:41:489.428 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710002861632, Pid 25272]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-xmpp-connectivity controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:44:41:489.476 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710002861632, Pid 25272]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:44:41:489.498 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710002861632, Pid 25272]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating tag-type:site controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:44:41:489.511 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710002861632, Pid 25272]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating service-template:default-domain:nat-template controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:44:41:489.527 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710002861632, Pid 25272]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating domain-service-template controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:44:41:489.538 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710002861632, Pid 25272]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-vrouter-l3mh-physical-interface-link-down controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:44:41:489.557 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710007060032, Pid 25272]: IFMap [SYS_WARN]: ConfigurationMalformedReferenceWarningLog: instance_target No entry in metadata map routing_instance c63bdcee-3142-4bf1-9f1c-c919d7e89cfd controller/src/ifmap/client/config_json_parser.cc 222 2026-06-08 Mon 05:44:41:489.560 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710002861632, Pid 25272]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:44:41:489.612 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710002861632, Pid 25272]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-storage-cluster-state controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:44:41:489.630 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710002861632, Pid 25272]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:44:41:497.833 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710023988800, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:41:498.383 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710023988800, Pid 25272]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-vrouter-limit-exceeded controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:44:41:498.412 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710023988800, Pid 25272]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:44:41:498.427 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710023988800, Pid 25272]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-address-mismatch-compute controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:44:41:498.439 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710023988800, Pid 25272]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:44:41:498.450 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710023988800, Pid 25272]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating service-appliance-set:default-global-system-config:native controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:44:41:498.464 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710023988800, Pid 25272]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-service-appliance-set controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:44:41:498.474 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710023988800, Pid 25272]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating security-group:default-domain:default-project:__no_rule__ controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:44:41:498.489 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710023988800, Pid 25272]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-security-group controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:44:41:498.499 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710023988800, Pid 25272]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating service-appliance-set:default-global-system-config:opencontrail controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:44:41:498.519 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710023988800, Pid 25272]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-service-appliance-set controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:44:41:498.548 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710023988800, Pid 25272]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating access-control-list:default-domain:default-project:__no_rule__:ingress-access-control-list controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:44:41:498.561 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710023988800, Pid 25272]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating security-group-access-control-list controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:44:41:498.574 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710023988800, Pid 25272]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating service-template:default-domain:docker-template controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:44:41:498.587 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710023988800, Pid 25272]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating domain-service-template controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:44:41:498.597 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710023988800, Pid 25272]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-prouter-tsn-connectivity controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:44:41:498.609 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710023988800, Pid 25272]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:44:41:498.619 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710023988800, Pid 25272]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating config-database-node:default-global-system-config:cn-jenkins-deploy-platform-ansible-os-5902-2. controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:44:41:498.632 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710023988800, Pid 25272]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-config-database-node controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:44:41:498.646 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710023988800, Pid 25272]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating domain-project controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:44:41:498.656 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710023988800, Pid 25272]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating tag-type:neutron_tag controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:44:41:498.665 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710023988800, Pid 25272]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-pending-cassandra-compaction-tasks controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:44:41:498.676 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710023988800, Pid 25272]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:44:41:498.688 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710023988800, Pid 25272]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating virtual-network:default-domain:default-project:__link_local__ controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:44:41:498.707 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710023988800, Pid 25272]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-virtual-network controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:44:41:498.717 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710023988800, Pid 25272]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-vrouter-interface controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:44:41:498.728 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710023988800, Pid 25272]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:44:41:501.547 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710019790400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:41:501.699 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710036584000, Pid 25272]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-prouter-connectivity controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:44:41:501.756 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710036584000, Pid 25272]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:44:41:501.781 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710036584000, Pid 25272]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-conf-elements-inconsistency controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:44:41:501.803 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710036584000, Pid 25272]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:44:41:501.820 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710036584000, Pid 25272]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-partial-sysinfo controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:44:41:501.838 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710036584000, Pid 25272]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:44:41:501.856 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710036584000, Pid 25272]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating config-node:default-global-system-config:cn-jenkins-deploy-platform-ansible-os-5902-3. controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:44:41:501.878 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710036584000, Pid 25272]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-config-node controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:44:41:501.895 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710036584000, Pid 25272]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating fabric:default-global-system-config:default-fabric controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:44:41:501.913 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710036584000, Pid 25272]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-fabric controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:44:41:505.385 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710011393600, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:41:505.520 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710032385600, Pid 25272]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-package-version-mismatch controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:44:41:505.558 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710011393600, Pid 25272]: IFMap [SYS_WARN]: ConfigurationMalformedReferenceWarningLog: instance_target No entry in metadata map routing_instance 278e34a1-a811-4e5f-aa92-c92c68d34343 controller/src/ifmap/client/config_json_parser.cc 222 2026-06-08 Mon 05:44:41:505.579 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710032385600, Pid 25272]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:44:41:505.614 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710032385600, Pid 25272]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-process-status controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:44:41:505.637 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710032385600, Pid 25272]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:44:41:505.654 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710032385600, Pid 25272]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating api-access-list:default-global-system-config:default-api-access-list controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:44:41:505.676 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710032385600, Pid 25272]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-api-access-list controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:44:41:505.694 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710032385600, Pid 25272]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating virtual-network:default-domain:default-project:default-virtual-network controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:44:41:505.718 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710032385600, Pid 25272]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-virtual-network controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:44:41:505.737 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710032385600, Pid 25272]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating access-control-list:default-domain:default-project:ip-fabric:ip-fabric controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:44:41:505.753 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710011393600, Pid 25272]: IFMap [SYS_WARN]: ConfigurationMalformedReferenceWarningLog: instance_target No entry in metadata map routing_instance c05dd037-d24e-4d96-9fd1-639cc28310b2 controller/src/ifmap/client/config_json_parser.cc 222 2026-06-08 Mon 05:44:41:505.762 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710032385600, Pid 25272]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating virtual-network-access-control-list controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:44:41:505.780 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710032385600, Pid 25272]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating virtual-network:default-domain:default-project:dci-network controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:44:41:505.808 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710032385600, Pid 25272]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-virtual-network controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:44:41:505.821 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710032385600, Pid 25272]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating tag-type:neutron_fwaas controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:44:41:541.532 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710028187200, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:41:541.696 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710036584000, Pid 25272]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating service-template:default-domain:netns-snat-template controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:44:41:541.749 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710028187200, Pid 25272]: IFMap [SYS_WARN]: ConfigurationMalformedReferenceWarningLog: instance_target No entry in metadata map routing_instance 62ebfee9-aabe-45de-ab84-af810d1590da controller/src/ifmap/client/config_json_parser.cc 222 2026-06-08 Mon 05:44:41:541.759 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710036584000, Pid 25272]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating domain-service-template controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:44:41:542.367 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710036584000, Pid 25272]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating config-node:default-global-system-config:cn-jenkins-deploy-platform-ansible-os-5902-2. controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:44:41:542.391 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710036584000, Pid 25272]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-config-node controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:44:41:542.405 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710036584000, Pid 25272]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-bgp-connectivity controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:44:41:542.419 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710036584000, Pid 25272]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:44:41:542.431 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710036584000, Pid 25272]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-process-connectivity controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:44:41:542.445 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710036584000, Pid 25272]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:44:41:542.456 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710036584000, Pid 25272]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-disk-usage-high controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:44:41:542.467 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710036584000, Pid 25272]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:44:41:542.477 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710036584000, Pid 25272]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating application-policy-set:default-domain:default-project:default-application-policy-set controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:44:41:542.491 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710036584000, Pid 25272]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-application-policy-set controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:44:41:542.502 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710036584000, Pid 25272]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-disk-usage-critical controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:44:41:542.530 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710036584000, Pid 25272]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:44:41:543.490 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:41:543.624 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139709893969472, Pid 25272]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating access-control-list:default-domain:default-project:__no_rule__:egress-access-control-list controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:44:41:543.684 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139709893969472, Pid 25272]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating security-group-access-control-list controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:44:41:543.706 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139709893969472, Pid 25272]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating discovery-service-assignment:default-discovery-service-assignment controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:44:41:543.721 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139709893969472, Pid 25272]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating tag-type:tier controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:44:41:543.736 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139709893969472, Pid 25272]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating network-ipam:default-domain:default-project:service-chain-flat-ipam controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:44:41:543.761 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139709893969472, Pid 25272]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-network-ipam controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:44:41:543.800 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139709893969472, Pid 25272]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-global-vrouter-config controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:44:41:543.822 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139709893969472, Pid 25272]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-node-status controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:44:41:543.845 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139709893969472, Pid 25272]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:44:41:543.865 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139709893969472, Pid 25272]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-vrouter-table-limit-exceeded controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:44:41:543.894 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139709893969472, Pid 25272]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:44:41:543.910 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139709893969472, Pid 25272]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating tag-type:deployment controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:44:41:543.925 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139709893969472, Pid 25272]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating config-database-node:default-global-system-config:cn-jenkins-deploy-platform-ansible-os-5902-1. controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:44:41:543.947 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139709893969472, Pid 25272]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-config-database-node controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:44:41:543.974 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139709893969472, Pid 25272]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating service-template:default-domain:haproxy-loadbalancer-template controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:44:41:543.995 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139709893969472, Pid 25272]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating domain-service-template controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:44:41:544.011 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139709893969472, Pid 25272]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-bottle-request-size-limit controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:44:41:544.037 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139709893969472, Pid 25272]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:44:41:544.978 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710049530944, Pid 25272]: DnsAgent [SYS_NOTICE]: DnsOperational: EndOfRib point was reached, IF-MAP objects have been created controller/src/dns/main.cc 83 2026-06-08 Mon 05:44:41:545.001 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710049530944, Pid 25272]: DnsAgent [SYS_NOTICE]: DnsOperational: The IF-MAP database table has been updated: __ifmap__.virtual_DNS.0, counting 0 elements controller/src/dns/main.cc 96 2026-06-08 Mon 05:44:41:545.010 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710049530944, Pid 25272]: DnsAgent [SYS_NOTICE]: DnsOperational: The IF-MAP database table has been updated: __ifmap__.virtual_DNS_record.0, counting 0 elements controller/src/dns/main.cc 96 2026-06-08 Mon 05:44:41:545.017 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710049530944, Pid 25272]: DnsAgent [SYS_NOTICE]: DnsOperational: The IF-MAP database table has been updated: __ifmap__.network_ipam.0, counting 2 elements controller/src/dns/main.cc 96 2026-06-08 Mon 05:44:41:545.024 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710049530944, Pid 25272]: DnsAgent [SYS_NOTICE]: DnsOperational: The IF-MAP database table has been updated: __ifmap__.virtual_network_network_ipam.0, counting 0 elements controller/src/dns/main.cc 96 2026-06-08 Mon 05:44:41:545.031 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710049530944, Pid 25272]: DnsAgent [SYS_NOTICE]: DnsOperational: The IF-MAP database table has been updated: __ifmap__.global_qos_config.0, counting 1 elements controller/src/dns/main.cc 96 2026-06-08 Mon 05:44:41:545.038 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710049530944, Pid 25272]: DnsAgent [SYS_NOTICE]: DnsOperational: VnniConfig has been updated after the bulk sync, the number of elements: 0 controller/src/dns/main.cc 106 2026-06-08 Mon 05:44:41:545.044 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710049530944, Pid 25272]: DnsAgent [SYS_NOTICE]: DnsOperational: IpamConfig has been updated after the bulk sync, the number of elements: 2 controller/src/dns/main.cc 117 2026-06-08 Mon 05:44:41:545.051 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710049530944, Pid 25272]: DnsAgent [SYS_NOTICE]: DnsOperational: VirtualDnsConfig has been updated after the bulk sync, the number of elements: 0 controller/src/dns/main.cc 128 2026-06-08 Mon 05:44:41:545.057 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710049530944, Pid 25272]: DnsAgent [SYS_NOTICE]: DnsOperational: VirtualDnsRecordConfig has been updated after the bulk sync, the number of elements: 0 controller/src/dns/main.cc 139 2026-06-08 Mon 05:44:41:545.064 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710049530944, Pid 25272]: DnsAgent [SYS_NOTICE]: DnsOperational: GlobalQosConfig has been updated after the bulk sync controller/src/dns/main.cc 149 2026-06-08 Mon 05:44:41:545.114 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710049530944, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: RabbitMQ SM: RabbitMQ connection down src/contrail-common/config-client-mgr/config_amqp_client.cc 157 2026-06-08 Mon 05:44:41:545.122 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710049530944, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: RabbitMQ SM: Connect to Rabbit MQ with queue_delete TRUE src/contrail-common/config-client-mgr/config_amqp_client.cc 167 2026-06-08 Mon 05:44:41:545.132 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710036584000, Pid 25272]: Processing scm::EvSandeshSend in state Connect 2026-06-08 Mon 05:44:41:545.196 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710036584000, Pid 25272]: SANDESH: Send FAILED: 1780897481545091 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Disconnect : EvCollectorUpdate -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.38:9041, (*_iter6) = 10.0.0.50:9041, (*_iter6) = 10.0.0.33:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.38:5673, (*_iter6) = 10.0.0.50:5673, (*_iter6) = 10.0.0.33:5673, ] ] status = Down description = RabbitMQ connection down ], ] ] flag_infos= [ [ ] ] description = Collector, Database:RabbitMQ connection down ], ] ] ] 2026-06-08 Mon 05:44:41:545.220 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710036584000, Pid 25272]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-06-08 Mon 05:44:42:188.483 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710049530944, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: RabbitMQ SM: RabbitMQ connection established src/contrail-common/config-client-mgr/config_amqp_client.cc 150 2026-06-08 Mon 05:44:42:188.512 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710019790400, Pid 25272]: Processing scm::EvSandeshSend in state Connect 2026-06-08 Mon 05:44:42:188.644 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710019790400, Pid 25272]: SANDESH: Send FAILED: 1780897482188395 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Disconnect : EvCollectorUpdate -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.38:9041, (*_iter6) = 10.0.0.50:9041, (*_iter6) = 10.0.0.33:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.38:5673, (*_iter6) = 10.0.0.50:5673, (*_iter6) = 10.0.0.33:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-08 Mon 05:44:42:188.656 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710019790400, Pid 25272]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-06-08 Mon 05:44:42:188.581 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710036584000, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: TRUE src/contrail-common/config-client-mgr/config_client_manager.cc 216 2026-06-08 Mon 05:44:42:189.049 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710049530944, Pid 25272]: -(-1) OnSessionEvent TCP Connect Failed 2026-06-08 Mon 05:44:42:189.065 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710049530944, Pid 25272]: -(-1):EvTcpConnectFail 2026-06-08 Mon 05:44:42:189.121 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710002861632, Pid 25272]: Processing scm::EvTcpConnectFail in state Connect 2026-06-08 Mon 05:44:42:189.145 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710002861632, Pid 25272]: Connect : EvTcpConnectFail 2026-06-08 Mon 05:44:42:189.250 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139709998663232, Pid 25272]: DnsAgent [SYS_NOTICE]: DnsOperational: BIND named up; DNS is operational controller/src/dns/mgr/dns_mgr.cc 735 2026-06-08 Mon 05:44:42:189.290 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710002861632, Pid 25272]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2026-06-08 Mon 05:44:42:189.347 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710002861632, Pid 25272]: Idle 2026-06-08 Mon 05:44:42:189.417 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710002861632, Pid 25272]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:44:42:189.518 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710002861632, Pid 25272]: SANDESH: Send FAILED: 1780897482189240 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 0 pid = 25272 http_port = 8092 start_time = 1780897480330573 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 152 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 4 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 12 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 56 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 66 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 292, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 10, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-06-08 Mon 05:44:42:189.535 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710002861632, Pid 25272]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:44:42:189.544 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710002861632, Pid 25272]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:44:42:189.561 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710002861632, Pid 25272]: SANDESH: Send FAILED: 1780897482189266 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.38:9041, (*_iter6) = 10.0.0.50:9041, (*_iter6) = 10.0.0.33:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.38:5673, (*_iter6) = 10.0.0.50:5673, (*_iter6) = 10.0.0.33:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-08 Mon 05:44:42:189.569 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710002861632, Pid 25272]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-06-08 Mon 05:44:42:189.576 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710002861632, Pid 25272]: Processing scm::EvTcpDeleteSession in state Idle 2026-06-08 Mon 05:44:42:189.646 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710002861632, Pid 25272]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:44:42:189.734 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710002861632, Pid 25272]: SANDESH: Send FAILED: 1780897482189395 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3:Control:contrail-dns:0 client_info= [ status = Idle successful_connections = 0 pid = 25272 http_port = 8092 start_time = 1780897480330573 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 152 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 4 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 13 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 56 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 66 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 294, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 10, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-06-08 Mon 05:44:42:189.746 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710002861632, Pid 25272]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:44:43:189.434 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710028187200, Pid 25272]: Processing scm::EvIdleHoldTimerExpired in state Idle 2026-06-08 Mon 05:44:43:189.755 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710028187200, Pid 25272]: Connect : Start Connect timer 10.0.0.33:8086 2026-06-08 Mon 05:44:43:189.791 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710049530944, Pid 25272]: -(-1) OnSessionEvent TCP Connect Failed 2026-06-08 Mon 05:44:43:189.820 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710049530944, Pid 25272]: -(-1):EvTcpConnectFail 2026-06-08 Mon 05:44:43:189.858 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710028187200, Pid 25272]: Processing scm::EvSandeshSend in state Connect 2026-06-08 Mon 05:44:43:189.882 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710028187200, Pid 25272]: SANDESH: Send FAILED: 1780897483189514 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.38:9041, (*_iter6) = 10.0.0.50:9041, (*_iter6) = 10.0.0.33:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.38:5673, (*_iter6) = 10.0.0.50:5673, (*_iter6) = 10.0.0.33:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-08 Mon 05:44:43:189.890 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710028187200, Pid 25272]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-06-08 Mon 05:44:43:189.898 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710028187200, Pid 25272]: Processing scm::EvTcpConnectFail in state Connect 2026-06-08 Mon 05:44:43:189.905 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710028187200, Pid 25272]: Connect : EvTcpConnectFail 2026-06-08 Mon 05:44:43:189.993 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710028187200, Pid 25272]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2026-06-08 Mon 05:44:43:190.022 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710028187200, Pid 25272]: Idle 2026-06-08 Mon 05:44:43:190.070 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710028187200, Pid 25272]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:44:43:190.184 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710028187200, Pid 25272]: SANDESH: Send FAILED: 1780897483189843 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 0 pid = 25272 http_port = 8092 start_time = 1780897480330573 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 152 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 4 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 13 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 56 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 66 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 5 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 294, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 13, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-06-08 Mon 05:44:43:190.208 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710028187200, Pid 25272]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:44:43:190.217 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710028187200, Pid 25272]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:44:43:190.298 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710028187200, Pid 25272]: SANDESH: Send FAILED: 1780897483189941 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 0 pid = 25272 http_port = 8092 start_time = 1780897480330573 collector_name = collector_ip = 10.0.0.50:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 152 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 4 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 13 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 56 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 66 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 5 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 294, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 14, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-06-08 Mon 05:44:43:190.311 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710028187200, Pid 25272]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:44:43:190.319 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710028187200, Pid 25272]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:44:43:190.333 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710028187200, Pid 25272]: SANDESH: Send FAILED: 1780897483189976 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.38:9041, (*_iter6) = 10.0.0.50:9041, (*_iter6) = 10.0.0.33:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.38:5673, (*_iter6) = 10.0.0.50:5673, (*_iter6) = 10.0.0.33:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-08 Mon 05:44:43:190.339 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710028187200, Pid 25272]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-06-08 Mon 05:44:43:190.345 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710028187200, Pid 25272]: Processing scm::EvTcpDeleteSession in state Idle 2026-06-08 Mon 05:44:43:190.368 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710028187200, Pid 25272]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:44:43:190.447 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710028187200, Pid 25272]: SANDESH: Send FAILED: 1780897483190061 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3:Control:contrail-dns:0 client_info= [ status = Idle successful_connections = 0 pid = 25272 http_port = 8092 start_time = 1780897480330573 collector_name = collector_ip = 10.0.0.50:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 152 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 4 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 13 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 56 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 66 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 5 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 295, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 14, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-06-08 Mon 05:44:43:190.461 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710028187200, Pid 25272]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:44:43:222.142 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:43:235.233 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:43:340.993 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:43:341.101 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: IFMap [SYS_WARN]: ConfigurationMalformedReferenceWarningLog: bgp_peering No entry in metadata map bgp_router 52be0fb9-3a40-4991-9fb0-17a2b7266240 controller/src/ifmap/client/config_json_parser.cc 222 2026-06-08 Mon 05:44:43:341.930 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139709893969472, Pid 25272]: DnsAgent [SYS_NOTICE]: DnsOperational: Bulk sync has been done (the DB has been synced with named) controller/src/dns/mgr/dns_mgr.cc 452 2026-06-08 Mon 05:44:43:349.582 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139709893969472, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:43:349.664 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139709893969472, Pid 25272]: IFMap [SYS_WARN]: ConfigurationMalformedReferenceWarningLog: bgp_peering No entry in metadata map bgp_router 7ea058e7-13e3-42e3-ad5c-3a51db612125 controller/src/ifmap/client/config_json_parser.cc 222 2026-06-08 Mon 05:44:43:496.957 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139709893969472, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:43:547.624 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710007060032, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:44:131.867 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710011393600, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:44:201.056 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710032385600, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:44:201.217 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710032385600, Pid 25272]: IFMap [SYS_WARN]: ConfigurationMalformedReferenceWarningLog: bgp_peering No entry in metadata map bgp_router 1ed9429b-8995-4250-afbd-ba8d5ddff253 controller/src/ifmap/client/config_json_parser.cc 222 2026-06-08 Mon 05:44:44:213.455 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710019790400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:44:213.555 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710019790400, Pid 25272]: IFMap [SYS_WARN]: ConfigurationMalformedReferenceWarningLog: bgp_peering No entry in metadata map bgp_router 52be0fb9-3a40-4991-9fb0-17a2b7266240 controller/src/ifmap/client/config_json_parser.cc 222 2026-06-08 Mon 05:44:44:261.861 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139709893969472, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:44:261.945 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139709893969472, Pid 25272]: IFMap [SYS_WARN]: ConfigurationMalformedReferenceWarningLog: bgp_peering No entry in metadata map bgp_router 1ed9429b-8995-4250-afbd-ba8d5ddff253 controller/src/ifmap/client/config_json_parser.cc 222 2026-06-08 Mon 05:44:44:265.290 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139709893969472, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:44:265.377 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139709893969472, Pid 25272]: IFMap [SYS_WARN]: ConfigurationMalformedReferenceWarningLog: bgp_peering No entry in metadata map bgp_router 7ea058e7-13e3-42e3-ad5c-3a51db612125 controller/src/ifmap/client/config_json_parser.cc 222 2026-06-08 Mon 05:44:44:416.086 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139709893969472, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:44:45:190.111 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: Processing scm::EvIdleHoldTimerExpired in state Idle 2026-06-08 Mon 05:44:45:190.381 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: Connect : Start Connect timer 10.0.0.50:8086 2026-06-08 Mon 05:44:45:190.491 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: Processing scm::EvSandeshSend in state Connect 2026-06-08 Mon 05:44:45:190.517 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: SANDESH: Send FAILED: 1780897485190176 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.50:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.38:9041, (*_iter6) = 10.0.0.50:9041, (*_iter6) = 10.0.0.33:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.38:5673, (*_iter6) = 10.0.0.50:5673, (*_iter6) = 10.0.0.33:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-08 Mon 05:44:45:190.525 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-06-08 Mon 05:44:45:190.533 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: Processing scm::EvSandeshSend in state Connect 2026-06-08 Mon 05:44:45:190.604 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710049530944, Pid 25272]: -(-1) OnSessionEvent TCP Connect Failed 2026-06-08 Mon 05:44:45:190.613 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710049530944, Pid 25272]: -(-1):EvTcpConnectFail 2026-06-08 Mon 05:44:45:190.711 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: SANDESH: Send FAILED: 1780897485190475 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 0 pid = 25272 http_port = 8092 start_time = 1780897480330573 collector_name = collector_ip = 10.0.0.50:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 164 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 56 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 66 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 314, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 18, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-06-08 Mon 05:44:45:190.733 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:44:45:190.743 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: Processing scm::EvTcpConnectFail in state Connect 2026-06-08 Mon 05:44:45:190.749 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: Connect : EvTcpConnectFail 2026-06-08 Mon 05:44:45:190.825 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2026-06-08 Mon 05:44:45:190.855 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: Idle 2026-06-08 Mon 05:44:45:190.897 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:44:45:191.055 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: SANDESH: Send FAILED: 1780897485190789 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 0 pid = 25272 http_port = 8092 start_time = 1780897480330573 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 164 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 56 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 66 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 11 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 314, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 20, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-06-08 Mon 05:44:45:191.077 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:44:45:191.088 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:44:45:191.103 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: SANDESH: Send FAILED: 1780897485190809 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.38:9041, (*_iter6) = 10.0.0.50:9041, (*_iter6) = 10.0.0.33:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.38:5673, (*_iter6) = 10.0.0.50:5673, (*_iter6) = 10.0.0.33:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-08 Mon 05:44:45:191.109 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-06-08 Mon 05:44:45:191.116 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: Processing scm::EvTcpDeleteSession in state Idle 2026-06-08 Mon 05:44:45:191.141 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:44:45:191.225 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: SANDESH: Send FAILED: 1780897485190888 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3:Control:contrail-dns:0 client_info= [ status = Idle successful_connections = 0 pid = 25272 http_port = 8092 start_time = 1780897480330573 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 164 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 56 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 66 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 11 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 315, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 20, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-06-08 Mon 05:44:45:191.242 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:44:49:190.998 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710002861632, Pid 25272]: Processing scm::EvIdleHoldTimerExpired in state Idle 2026-06-08 Mon 05:44:49:191.351 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710002861632, Pid 25272]: Connect : Start Connect timer 10.0.0.38:8086 2026-06-08 Mon 05:44:49:191.476 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710002861632, Pid 25272]: Processing scm::EvSandeshSend in state Connect 2026-06-08 Mon 05:44:49:191.506 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710002861632, Pid 25272]: SANDESH: Send FAILED: 1780897489191101 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.38:9041, (*_iter6) = 10.0.0.50:9041, (*_iter6) = 10.0.0.33:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.38:5673, (*_iter6) = 10.0.0.50:5673, (*_iter6) = 10.0.0.33:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-08 Mon 05:44:49:191.517 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710002861632, Pid 25272]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-06-08 Mon 05:44:49:191.527 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710002861632, Pid 25272]: Processing scm::EvSandeshSend in state Connect 2026-06-08 Mon 05:44:49:191.687 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710049530944, Pid 25272]: -(-1) OnSessionEvent TCP Connect Failed 2026-06-08 Mon 05:44:49:191.778 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710049530944, Pid 25272]: -(-1):EvTcpConnectFail 2026-06-08 Mon 05:44:49:191.695 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710002861632, Pid 25272]: SANDESH: Send FAILED: 1780897489191455 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 0 pid = 25272 http_port = 8092 start_time = 1780897480330573 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 164 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 56 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 66 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 11 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 315, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 23, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-06-08 Mon 05:44:49:191.822 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710002861632, Pid 25272]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:44:49:191.836 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710002861632, Pid 25272]: Processing scm::EvTcpConnectFail in state Connect 2026-06-08 Mon 05:44:49:191.845 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710002861632, Pid 25272]: Connect : EvTcpConnectFail 2026-06-08 Mon 05:44:49:191.962 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710002861632, Pid 25272]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2026-06-08 Mon 05:44:49:192.002 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710002861632, Pid 25272]: Idle 2026-06-08 Mon 05:44:49:192.048 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710002861632, Pid 25272]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:44:49:192.164 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710002861632, Pid 25272]: SANDESH: Send FAILED: 1780897489191904 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 0 pid = 25272 http_port = 8092 start_time = 1780897480330573 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 164 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 56 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 66 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 13 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 315, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 25, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-06-08 Mon 05:44:49:192.183 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710002861632, Pid 25272]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:44:49:192.196 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710002861632, Pid 25272]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:44:49:192.215 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710002861632, Pid 25272]: SANDESH: Send FAILED: 1780897489191928 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.38:9041, (*_iter6) = 10.0.0.50:9041, (*_iter6) = 10.0.0.33:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.38:5673, (*_iter6) = 10.0.0.50:5673, (*_iter6) = 10.0.0.33:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-08 Mon 05:44:49:192.224 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710002861632, Pid 25272]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-06-08 Mon 05:44:49:192.230 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710002861632, Pid 25272]: Processing scm::EvTcpDeleteSession in state Idle 2026-06-08 Mon 05:44:49:192.256 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710002861632, Pid 25272]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:44:49:192.339 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710002861632, Pid 25272]: SANDESH: Send FAILED: 1780897489192039 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3:Control:contrail-dns:0 client_info= [ status = Idle successful_connections = 0 pid = 25272 http_port = 8092 start_time = 1780897480330573 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 164 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 56 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 66 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 13 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 4 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 316, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 25, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-06-08 Mon 05:44:49:192.354 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710002861632, Pid 25272]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:44:57:192.158 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710015592000, Pid 25272]: Processing scm::EvIdleHoldTimerExpired in state Idle 2026-06-08 Mon 05:44:57:192.606 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710015592000, Pid 25272]: Connect : Start Connect timer 10.0.0.33:8086 2026-06-08 Mon 05:44:57:192.757 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710015592000, Pid 25272]: Processing scm::EvSandeshSend in state Connect 2026-06-08 Mon 05:44:57:192.788 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710015592000, Pid 25272]: SANDESH: Send FAILED: 1780897497192338 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.38:9041, (*_iter6) = 10.0.0.50:9041, (*_iter6) = 10.0.0.33:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.38:5673, (*_iter6) = 10.0.0.50:5673, (*_iter6) = 10.0.0.33:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-08 Mon 05:44:57:192.805 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710015592000, Pid 25272]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-06-08 Mon 05:44:57:192.813 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710015592000, Pid 25272]: Processing scm::EvSandeshSend in state Connect 2026-06-08 Mon 05:44:57:192.929 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710015592000, Pid 25272]: SANDESH: Send FAILED: 1780897497192736 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 0 pid = 25272 http_port = 8092 start_time = 1780897480330573 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 164 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 56 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 66 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 4 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 316, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 28, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-06-08 Mon 05:44:57:192.975 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710015592000, Pid 25272]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:44:57:193.024 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710049530944, Pid 25272]: -(-1) OnSessionEvent TCP Connect Failed 2026-06-08 Mon 05:44:57:193.037 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710049530944, Pid 25272]: -(-1):EvTcpConnectFail 2026-06-08 Mon 05:44:57:193.073 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710015592000, Pid 25272]: Processing scm::EvTcpConnectFail in state Connect 2026-06-08 Mon 05:44:57:193.084 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710015592000, Pid 25272]: Connect : EvTcpConnectFail 2026-06-08 Mon 05:44:57:193.229 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710015592000, Pid 25272]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2026-06-08 Mon 05:44:57:193.267 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710015592000, Pid 25272]: Idle 2026-06-08 Mon 05:44:57:193.330 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710015592000, Pid 25272]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:44:57:193.504 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710015592000, Pid 25272]: SANDESH: Send FAILED: 1780897497193147 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 0 pid = 25272 http_port = 8092 start_time = 1780897480330573 collector_name = collector_ip = 10.0.0.50:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 164 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 56 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 66 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 15 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 15 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 4 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 316, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 30, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-06-08 Mon 05:44:57:193.530 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710015592000, Pid 25272]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:44:57:193.543 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710015592000, Pid 25272]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:44:57:193.566 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710015592000, Pid 25272]: SANDESH: Send FAILED: 1780897497193206 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.38:9041, (*_iter6) = 10.0.0.50:9041, (*_iter6) = 10.0.0.33:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.38:5673, (*_iter6) = 10.0.0.50:5673, (*_iter6) = 10.0.0.33:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-08 Mon 05:44:57:193.574 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710015592000, Pid 25272]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-06-08 Mon 05:44:57:193.583 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710015592000, Pid 25272]: Processing scm::EvTcpDeleteSession in state Idle 2026-06-08 Mon 05:44:57:193.617 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710015592000, Pid 25272]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:44:57:193.705 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710015592000, Pid 25272]: SANDESH: Send FAILED: 1780897497193316 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3:Control:contrail-dns:0 client_info= [ status = Idle successful_connections = 0 pid = 25272 http_port = 8092 start_time = 1780897480330573 collector_name = collector_ip = 10.0.0.50:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 164 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 56 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 66 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 15 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 15 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 5 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 317, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 30, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-06-08 Mon 05:44:57:193.720 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710015592000, Pid 25272]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:45:10:331.301 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710011393600, Pid 25272]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:45:10:331.397 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710011393600, Pid 25272]: SANDESH: Send FAILED: 1780897510331121 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3:Control:contrail-dns:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 4 dequeues = 4 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 33 dequeues = 33 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 5 dequeues = 5 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 5 dequeues = 5 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 49 dequeues = 49 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1780897497193267 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1780897497193616 ] max_sm_queue_count = 5 ] 2026-06-08 Mon 05:45:10:331.422 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710011393600, Pid 25272]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:45:10:331.433 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710011393600, Pid 25272]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:45:10:331.637 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710011393600, Pid 25272]: SANDESH: Send FAILED: 1780897510331256 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3:Control:contrail-dns:0 client_info= [ status = Idle successful_connections = 0 pid = 25272 http_port = 8092 start_time = 1780897480330573 collector_name = collector_ip = 10.0.0.50:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 164 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 56 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 66 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 16 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 17 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 5 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 317, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 33, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-06-08 Mon 05:45:10:331.657 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710011393600, Pid 25272]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:45:13:193.368 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710015592000, Pid 25272]: Processing scm::EvIdleHoldTimerExpired in state Idle 2026-06-08 Mon 05:45:13:193.565 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710015592000, Pid 25272]: Connect : Start Connect timer 10.0.0.50:8086 2026-06-08 Mon 05:45:13:193.659 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710015592000, Pid 25272]: Processing scm::EvSandeshSend in state Connect 2026-06-08 Mon 05:45:13:193.681 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710015592000, Pid 25272]: SANDESH: Send FAILED: 1780897513193440 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.50:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.38:9041, (*_iter6) = 10.0.0.50:9041, (*_iter6) = 10.0.0.33:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.38:5673, (*_iter6) = 10.0.0.50:5673, (*_iter6) = 10.0.0.33:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-08 Mon 05:45:13:193.688 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710015592000, Pid 25272]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-06-08 Mon 05:45:13:193.696 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710015592000, Pid 25272]: Processing scm::EvSandeshSend in state Connect 2026-06-08 Mon 05:45:13:193.793 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710015592000, Pid 25272]: SANDESH: Send FAILED: 1780897513193646 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 0 pid = 25272 http_port = 8092 start_time = 1780897480330573 collector_name = collector_ip = 10.0.0.50:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 164 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 56 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 66 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 16 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 19 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 5 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 317, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 35, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-06-08 Mon 05:45:13:193.810 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710015592000, Pid 25272]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:45:13:193.825 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710049530944, Pid 25272]: -(-1) OnSessionEvent TCP Connect Failed 2026-06-08 Mon 05:45:13:193.860 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710049530944, Pid 25272]: -(-1):EvTcpConnectFail 2026-06-08 Mon 05:45:13:193.902 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710015592000, Pid 25272]: Processing scm::EvTcpConnectFail in state Connect 2026-06-08 Mon 05:45:13:193.911 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710015592000, Pid 25272]: Connect : EvTcpConnectFail 2026-06-08 Mon 05:45:13:193.997 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710015592000, Pid 25272]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2026-06-08 Mon 05:45:13:194.024 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710015592000, Pid 25272]: Idle 2026-06-08 Mon 05:45:13:194.069 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710015592000, Pid 25272]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:45:13:194.169 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710015592000, Pid 25272]: SANDESH: Send FAILED: 1780897513193961 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 0 pid = 25272 http_port = 8092 start_time = 1780897480330573 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 164 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 56 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 66 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 17 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 20 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 5 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 317, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 37, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-06-08 Mon 05:45:13:194.187 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710015592000, Pid 25272]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:45:13:194.197 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710015592000, Pid 25272]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:45:13:194.212 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710015592000, Pid 25272]: SANDESH: Send FAILED: 1780897513193981 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.38:9041, (*_iter6) = 10.0.0.50:9041, (*_iter6) = 10.0.0.33:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.38:5673, (*_iter6) = 10.0.0.50:5673, (*_iter6) = 10.0.0.33:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-08 Mon 05:45:13:194.218 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710015592000, Pid 25272]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-06-08 Mon 05:45:13:194.225 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710015592000, Pid 25272]: Processing scm::EvTcpDeleteSession in state Idle 2026-06-08 Mon 05:45:13:194.248 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710015592000, Pid 25272]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:45:13:194.331 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710015592000, Pid 25272]: SANDESH: Send FAILED: 1780897513194058 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3:Control:contrail-dns:0 client_info= [ status = Idle successful_connections = 0 pid = 25272 http_port = 8092 start_time = 1780897480330573 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 164 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 56 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 66 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 17 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 20 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 318, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 37, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-06-08 Mon 05:45:13:194.346 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710015592000, Pid 25272]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:45:13:205.922 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710011393600, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:45:13:206.058 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710011393600, Pid 25272]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating database-node:default-global-system-config:cn-jenkins-deploy-platform-ansible-os-5902-3. controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:45:13:206.086 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710011393600, Pid 25272]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-database-node controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:45:13:216.724 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710028187200, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:45:13:216.869 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710028187200, Pid 25272]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating database-node:default-global-system-config:cn-jenkins-deploy-platform-ansible-os-5902-2. controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:45:13:216.899 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710028187200, Pid 25272]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-database-node controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:45:13:860.078 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710028187200, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:45:13:860.197 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710028187200, Pid 25272]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating database-node:default-global-system-config:cn-jenkins-deploy-platform-ansible-os-5902-1. controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:45:13:860.235 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710028187200, Pid 25272]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-database-node controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:45:40:331.518 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139709998663232, Pid 25272]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:45:40:331.586 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139709998663232, Pid 25272]: SANDESH: Send FAILED: 1780897540331390 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3:Control:contrail-dns:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 5 dequeues = 5 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 40 dequeues = 40 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 6 dequeues = 6 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 6 dequeues = 6 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 59 dequeues = 59 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1780897513194024 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1780897513194247 ] max_sm_queue_count = 5 ] 2026-06-08 Mon 05:45:40:331.599 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139709998663232, Pid 25272]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:45:40:331.608 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139709998663232, Pid 25272]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:45:40:331.738 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139709998663232, Pid 25272]: SANDESH: Send FAILED: 1780897540331481 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3:Control:contrail-dns:0 client_info= [ status = Idle successful_connections = 0 pid = 25272 http_port = 8092 start_time = 1780897480330573 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 167 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 59 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 69 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 18 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 22 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 327, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 40, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-06-08 Mon 05:45:40:331.760 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139709998663232, Pid 25272]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:45:40:834.380 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139709893969472, Pid 25272]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:45:40:834.455 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139709893969472, Pid 25272]: SANDESH: Send FAILED: 1780897540834346 [SYS_NOTICE]: UveDnsInfo: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3 start_time = 1780897480834146 build_info = {"build-info": [{"build-version": "v0+nightly.r25.1", "build-time": "2026-05-28 00:08:01.734303", "build-user": "root", "build-hostname": "jenkins-build-prepare-sandbox-15063-1.novalocal", "build-id": "v0+nightly.r25.1", "build-number": "202605280007"}]} collector = 10.0.0.38:8086 self_ip_list= [ [ (*_iter12) = 10.20.0.14, ] ] ] 2026-06-08 Mon 05:45:40:834.469 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139709893969472, Pid 25272]: Wrong state: Idle for event: EvSandeshSend message: UveDnsInfo 2026-06-08 Mon 05:45:43:194.126 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139709898167872, Pid 25272]: Processing scm::EvIdleHoldTimerExpired in state Idle 2026-06-08 Mon 05:45:43:194.380 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139709898167872, Pid 25272]: Connect : Start Connect timer 10.0.0.38:8086 2026-06-08 Mon 05:45:43:194.484 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139709898167872, Pid 25272]: Processing scm::EvSandeshSend in state Connect 2026-06-08 Mon 05:45:43:194.509 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139709898167872, Pid 25272]: SANDESH: Send FAILED: 1780897543194195 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.38:9041, (*_iter6) = 10.0.0.50:9041, (*_iter6) = 10.0.0.33:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.38:5673, (*_iter6) = 10.0.0.50:5673, (*_iter6) = 10.0.0.33:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-08 Mon 05:45:43:194.517 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139709898167872, Pid 25272]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-06-08 Mon 05:45:43:194.524 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139709898167872, Pid 25272]: Processing scm::EvSandeshSend in state Connect 2026-06-08 Mon 05:45:43:194.651 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139709898167872, Pid 25272]: SANDESH: Send FAILED: 1780897543194466 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 0 pid = 25272 http_port = 8092 start_time = 1780897480330573 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 167 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 59 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 69 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 18 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 24 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 327, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 43, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-06-08 Mon 05:45:43:194.673 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139709898167872, Pid 25272]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:45:43:194.910 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710049530944, Pid 25272]: -(-1) OnSessionEvent TCP Connect Failed 2026-06-08 Mon 05:45:43:194.962 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710049530944, Pid 25272]: -(-1):EvTcpConnectFail 2026-06-08 Mon 05:45:43:195.022 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: Processing scm::EvTcpConnectFail in state Connect 2026-06-08 Mon 05:45:43:195.044 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: Connect : EvTcpConnectFail 2026-06-08 Mon 05:45:43:195.172 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2026-06-08 Mon 05:45:43:195.202 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: Idle 2026-06-08 Mon 05:45:43:195.253 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:45:43:195.360 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: SANDESH: Send FAILED: 1780897543195121 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 0 pid = 25272 http_port = 8092 start_time = 1780897480330573 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 167 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 59 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 69 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 19 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 25 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 327, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 45, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-06-08 Mon 05:45:43:195.381 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:45:43:195.393 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:45:43:195.415 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: SANDESH: Send FAILED: 1780897543195151 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.38:9041, (*_iter6) = 10.0.0.50:9041, (*_iter6) = 10.0.0.33:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.38:5673, (*_iter6) = 10.0.0.50:5673, (*_iter6) = 10.0.0.33:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-08 Mon 05:45:43:195.424 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-06-08 Mon 05:45:43:195.435 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: Processing scm::EvTcpDeleteSession in state Idle 2026-06-08 Mon 05:45:43:195.468 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:45:43:195.575 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: SANDESH: Send FAILED: 1780897543195241 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3:Control:contrail-dns:0 client_info= [ status = Idle successful_connections = 0 pid = 25272 http_port = 8092 start_time = 1780897480330573 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 167 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 59 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 69 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 19 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 25 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 7 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 328, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 45, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-06-08 Mon 05:45:43:195.593 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710040782400, Pid 25272]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:45:47:310.880 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139709998663232, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:45:47:311.031 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139709998663232, Pid 25272]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating analytics-node:default-global-system-config:cn-jenkins-deploy-platform-ansible-os-5902-3. controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:45:47:311.062 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139709998663232, Pid 25272]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-analytics-node controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:45:47:352.429 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710023988800, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:45:47:352.553 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710023988800, Pid 25272]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating analytics-node:default-global-system-config:cn-jenkins-deploy-platform-ansible-os-5902-2. controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:45:47:352.583 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710023988800, Pid 25272]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-analytics-node controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:45:47:420.332 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710023988800, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:45:47:420.505 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710028187200, Pid 25272]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating analytics-node:default-global-system-config:cn-jenkins-deploy-platform-ansible-os-5902-1. controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:45:47:420.700 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710028187200, Pid 25272]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-analytics-node controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:46:05:781.226 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710023988800, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:46:05:781.394 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710023988800, Pid 25272]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating analytics-alarm-node:default-global-system-config:cn-jenkins-deploy-platform-ansible-os-5902-3. controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:46:05:781.835 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710023988800, Pid 25272]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-analytics-alarm-node controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:46:05:846.602 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710011393600, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:46:05:851.080 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710011393600, Pid 25272]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating analytics-alarm-node:default-global-system-config:cn-jenkins-deploy-platform-ansible-os-5902-2. controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:46:05:851.117 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710011393600, Pid 25272]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-analytics-alarm-node controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:46:05:899.797 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710019790400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:46:05:899.960 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710019790400, Pid 25272]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating analytics-alarm-node:default-global-system-config:cn-jenkins-deploy-platform-ansible-os-5902-1. controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:46:05:899.992 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710019790400, Pid 25272]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-analytics-alarm-node controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:46:10:331.714 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710032385600, Pid 25272]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:46:10:331.779 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710032385600, Pid 25272]: SANDESH: Send FAILED: 1780897570331582 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3:Control:contrail-dns:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 6 dequeues = 6 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 48 dequeues = 48 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 7 dequeues = 7 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 7 dequeues = 7 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 70 dequeues = 70 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1780897543195202 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1780897543195467 ] max_sm_queue_count = 5 ] 2026-06-08 Mon 05:46:10:331.799 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710032385600, Pid 25272]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:46:10:331.807 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710032385600, Pid 25272]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:46:10:331.916 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710032385600, Pid 25272]: SANDESH: Send FAILED: 1780897570331687 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3:Control:contrail-dns:0 client_info= [ status = Idle successful_connections = 0 pid = 25272 http_port = 8092 start_time = 1780897480330573 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 173 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 65 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 75 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 20 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 27 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 7 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 346, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 48, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-06-08 Mon 05:46:10:331.931 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710032385600, Pid 25272]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:46:13:195.350 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710002861632, Pid 25272]: Processing scm::EvIdleHoldTimerExpired in state Idle 2026-06-08 Mon 05:46:13:195.612 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710002861632, Pid 25272]: Connect : Start Connect timer 10.0.0.33:8086 2026-06-08 Mon 05:46:13:195.726 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710002861632, Pid 25272]: Processing scm::EvSandeshSend in state Connect 2026-06-08 Mon 05:46:13:195.763 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710002861632, Pid 25272]: SANDESH: Send FAILED: 1780897573195443 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.38:9041, (*_iter6) = 10.0.0.50:9041, (*_iter6) = 10.0.0.33:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.38:5673, (*_iter6) = 10.0.0.50:5673, (*_iter6) = 10.0.0.33:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-08 Mon 05:46:13:195.772 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710002861632, Pid 25272]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-06-08 Mon 05:46:13:195.780 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710002861632, Pid 25272]: Processing scm::EvSandeshSend in state Connect 2026-06-08 Mon 05:46:13:199.791 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710049530944, Pid 25272]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.33:38896::10.0.0.33:8086(-1) < Active session connection complete src/contrail-common/io/tcp_session.cc 281 2026-06-08 Mon 05:46:13:199.826 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710049530944, Pid 25272]: 10.0.0.33:38896::10.0.0.33:8086(-1) OnSessionEvent TCP Connected 2026-06-08 Mon 05:46:13:199.837 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710049530944, Pid 25272]: 10.0.0.33:38896::10.0.0.33:8086(-1):EvTcpConnected 2026-06-08 Mon 05:46:13:199.916 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710002861632, Pid 25272]: SANDESH: Send FAILED: 1780897573195711 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 0 pid = 25272 http_port = 8092 start_time = 1780897480330573 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 173 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 65 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 75 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 20 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 29 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 7 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 346, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 50, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-06-08 Mon 05:46:13:199.958 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710002861632, Pid 25272]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:46:13:199.975 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710002861632, Pid 25272]: Processing scm::EvTcpConnected in state Connect 2026-06-08 Mon 05:46:13:199.985 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710002861632, Pid 25272]: Connect : EvTcpConnected : Cancelling Connect timer 2026-06-08 Mon 05:46:13:200.025 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710002861632, Pid 25272]: ClientInit 2026-06-08 Mon 05:46:13:200.044 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710002861632, Pid 25272]: Sending Ctrl Message for cn-jenkins-deploy-platform-ansible-os-5902-3:contrail-dns:0:Control count 1 2026-06-08 Mon 05:46:13:200.141 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710002861632, Pid 25272]: Processing scm::EvSandeshSend in state ClientInit 2026-06-08 Mon 05:46:13:200.153 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710002861632, Pid 25272]: ClientInit : EvSandeshSend : NodeStatusUVE 2026-06-08 Mon 05:46:13:204.615 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710002861632, Pid 25272]: SANDESH: Send FAILED: 1780897573200017 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.38:9041, (*_iter6) = 10.0.0.50:9041, (*_iter6) = 10.0.0.33:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.38:5673, (*_iter6) = 10.0.0.50:5673, (*_iter6) = 10.0.0.33:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-08 Mon 05:46:13:204.649 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710002861632, Pid 25272]: Received UVE message in wrong state : NodeStatusUVE 2026-06-08 Mon 05:46:13:204.667 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710002861632, Pid 25272]: Processing scm::EvSandeshSend in state ClientInit 2026-06-08 Mon 05:46:13:204.679 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710002861632, Pid 25272]: ClientInit : EvSandeshSend : SandeshCtrlClientToServer 2026-06-08 Mon 05:46:13:204.702 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710002861632, Pid 25272]: Processing scm::EvSandeshSend in state ClientInit 2026-06-08 Mon 05:46:13:204.712 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710002861632, Pid 25272]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2026-06-08 Mon 05:46:13:204.823 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710002861632, Pid 25272]: SANDESH: Send FAILED: 1780897573200127 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3:Control:contrail-dns:0 client_info= [ status = ClientInit successful_connections = 1 pid = 25272 http_port = 8092 start_time = 1780897480330573 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 173 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 65 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 75 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 21 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 30 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 8 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 347, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 52, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-06-08 Mon 05:46:13:204.854 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710002861632, Pid 25272]: Received UVE message in wrong state : SandeshModuleClientTrace 2026-06-08 Mon 05:46:13:207.005 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139709998663232, Pid 25272]: OnMessage control in state: ClientInit 2026-06-08 Mon 05:46:13:209.091 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139709998663232, Pid 25272]: Processing scm::EvSandeshMessageRecv in state ClientInit 2026-06-08 Mon 05:46:13:209.932 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139709998663232, Pid 25272]: ClientInit : EvSandeshMessageRecv 2026-06-08 Mon 05:46:13:210.000 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139709998663232, Pid 25272]: Received Ctrl Message with size 0 2026-06-08 Mon 05:46:13:210.027 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139709998663232, Pid 25272]: SyncUVE Syncing ObjectDns val name = cn-jenkins-deploy-platform-ansible-os-5902-3 start_time = 1780897480834146 build_info = {"build-info": [{"build-version": "v0+nightly.r25.1", "build-time": "2026-05-28 00:08:01.734303", "build-user": "root", "build-hostname": "jenkins-build-prepare-sandbox-15063-1.novalocal", "build-id": "v0+nightly.r25.1", "build-number": "202605280007"}]} collector = 10.0.0.38:8086 self_ip_list= [ [ (*_iter12) = 10.20.0.14, ] ] proxy seq 1 2026-06-08 Mon 05:46:13:210.047 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139709998663232, Pid 25272]: SyncAllMaps for DnsState:0 period 0 cycle 0 without seqno , total = 1 2026-06-08 Mon 05:46:13:210.373 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139709998663232, Pid 25272]: SyncUVE Syncing ObjectGeneratorInfo val name = cn-jenkins-deploy-platform-ansible-os-5902-3:Control:contrail-dns:0 client_info= [ status = ClientInit successful_connections = 1 pid = 25272 http_port = 8092 start_time = 1780897480330573 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 6 dequeues = 6 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 48 dequeues = 48 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 7 dequeues = 7 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 7 dequeues = 7 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 70 dequeues = 70 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1780897543195202 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1780897543195467 ] msg_type_agg= [ [ _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 173 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 65 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 75 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 21 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 30 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 8 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 347, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 52, _iter59->first = sent _iter59->second = 0, ] ] max_sm_queue_count = 5 proxy seq 32 2026-06-08 Mon 05:46:13:210.418 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139709998663232, Pid 25272]: SyncAllMaps for ModuleClientState:0 period 0 cycle 0 without seqno , total = 1 2026-06-08 Mon 05:46:13:210.452 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139709998663232, Pid 25272]: SyncUVE Syncing ObjectDns val name = cn-jenkins-deploy-platform-ansible-os-5902-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.38:9041, (*_iter6) = 10.0.0.50:9041, (*_iter6) = 10.0.0.33:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.38:5673, (*_iter6) = 10.0.0.50:5673, (*_iter6) = 10.0.0.33:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] proxy seq 22 2026-06-08 Mon 05:46:13:210.473 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139709998663232, Pid 25272]: SyncAllMaps for NodeStatus:0 period 0 cycle 0 without seqno , total = 1 2026-06-08 Mon 05:46:13:210.484 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139709998663232, Pid 25272]: SyncAllMaps for PeerFlapData:0 period 0 cycle 0 without seqno , total = 0 2026-06-08 Mon 05:46:13:210.494 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139709998663232, Pid 25272]: SyncAllMaps for PeerStatsData:0 period 0 cycle 0 without seqno , total = 0 2026-06-08 Mon 05:46:13:210.505 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139709998663232, Pid 25272]: SyncAllMaps for XmppPeerInfoData:0 period 0 cycle 0 without seqno , total = 0 2026-06-08 Mon 05:46:13:210.554 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139709998663232, Pid 25272]: Established 2026-06-08 Mon 05:46:21:750.610 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710023988800, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:46:21:750.760 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710023988800, Pid 25272]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating analytics-snmp-node:default-global-system-config:cn-jenkins-deploy-platform-ansible-os-5902-2. controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:46:21:750.791 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710023988800, Pid 25272]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-analytics-snmp-node controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:46:21:818.881 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710023988800, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:46:21:819.040 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710023988800, Pid 25272]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating analytics-snmp-node:default-global-system-config:cn-jenkins-deploy-platform-ansible-os-5902-3. controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:46:21:819.073 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710023988800, Pid 25272]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-analytics-snmp-node controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:46:22:693.713 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139709893969472, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:46:22:693.837 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139709893969472, Pid 25272]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating analytics-snmp-node:default-global-system-config:cn-jenkins-deploy-platform-ansible-os-5902-1. controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:46:22:693.866 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139709893969472, Pid 25272]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-analytics-snmp-node controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:48:10:643.082 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139709893969472, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:48:10:781.979 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710028187200, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:48:10:782.141 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710028187200, Pid 25272]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating virtual-router:default-global-system-config:an-jenkins-deploy-platform-ansible-os-5902-2 controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:48:10:782.176 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710028187200, Pid 25272]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-virtual-router controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:48:11:160.811 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139709893969472, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:48:11:161.021 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139709893969472, Pid 25272]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating virtual-machine-interface:default-global-system-config:an-jenkins-deploy-platform-ansible-os-5902-2:vhost0 controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:48:11:161.072 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139709893969472, Pid 25272]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating virtual-router-virtual-machine-interface controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:48:11:161.097 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139709893969472, Pid 25272]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating virtual-machine-interface-virtual-network controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:48:11:239.378 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139709893969472, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:48:11:740.620 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710023988800, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:49:01:530.900 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710049530944, Pid 25272]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.14:8092::10.20.0.14:53336 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2026-06-08 Mon 05:49:02:033.656 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710049530944, Pid 25272]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.14:8092::10.20.0.14:53338 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2026-06-08 Mon 05:49:02:212.471 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710049530944, Pid 25272]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.14:8092::10.20.0.254:42146 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2026-06-08 Mon 05:49:02:545.866 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710049530944, Pid 25272]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.14:8092::10.20.0.254:42156 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2026-06-08 Mon 05:49:03:124.432 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710023988800, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:49:03:124.609 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710023988800, Pid 25272]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating virtual-router:default-global-system-config:an-jenkins-deploy-platform-ansible-os-5902-1 controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:49:03:124.658 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710023988800, Pid 25272]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-virtual-router controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:49:03:493.511 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710011393600, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:49:03:493.668 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710011393600, Pid 25272]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating virtual-machine-interface:default-global-system-config:an-jenkins-deploy-platform-ansible-os-5902-1:vhost0 controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:49:03:493.701 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710011393600, Pid 25272]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating virtual-router-virtual-machine-interface controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:49:03:493.719 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710011393600, Pid 25272]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating virtual-machine-interface-virtual-network controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:49:03:557.000 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710019790400, Pid 25272]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:49:03:959.771 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710049530944, Pid 25272]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.14:8092::10.20.0.17:39576 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2026-06-08 Mon 05:49:04:218.634 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139710049530944, Pid 25272]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.14:8092::10.20.0.17:39588 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2026-06-08 Mon 05:51:39:157.951 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625491276864, Pid 43016]: DnsAgent [SYS_INFO]: VersionInfoLog: {"build-info": [{"build-version": "v0+nightly.r25.1", "build-time": "2026-05-28 00:08:01.734303", "build-user": "root", "build-hostname": "jenkins-build-prepare-sandbox-15063-1.novalocal", "build-id": "v0+nightly.r25.1", "build-number": "202605280007"}]} src/contrail-common/base/misc_utils.cc 39 2026-06-08 Mon 05:51:39:158.042 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625491276864, Pid 43016]: SANDESH: ROLE : Generator 2026-06-08 Mon 05:51:39:158.053 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625491276864, Pid 43016]: SANDESH: MODULE : contrail-dns 2026-06-08 Mon 05:51:39:158.059 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625491276864, Pid 43016]: SANDESH: SOURCE : cn-jenkins-deploy-platform-ansible-os-5902-3 2026-06-08 Mon 05:51:39:158.064 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625491276864, Pid 43016]: SANDESH: NODE TYPE : Control 2026-06-08 Mon 05:51:39:158.068 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625491276864, Pid 43016]: SANDESH: INSTANCE ID : 0 2026-06-08 Mon 05:51:39:158.073 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625491276864, Pid 43016]: SANDESH: HTTP SERVER PORT : 8092 2026-06-08 Mon 05:51:39:158.086 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625491276864, Pid 43016]: SANDESH: System Log Send Rate Limit: 0 -> 0 2026-06-08 Mon 05:51:39:170.818 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625491276864, Pid 43016]: HTTP Introspect Init 2026-06-08 Mon 05:51:39:170.957 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625491276864, Pid 43016]: TCP [SYS_DEBUG]: TcpServerMessageLog: Server 0.0.0.0:8092 Initialization complete src/contrail-common/io/tcp_server.cc 137 2026-06-08 Mon 05:51:39:171.036 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625491276864, Pid 43016]: Sandesh Http Server Port: 8092 2026-06-08 Mon 05:51:39:171.103 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625491276864, Pid 43016]: SANDESH: NOT Writing http_port 8092TO : /tmp/contrail-dns.42985.http_port 2026-06-08 Mon 05:51:39:171.111 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625491276864, Pid 43016]: SANDESH: CONNECT TO COLLECTOR: 1 2026-06-08 Mon 05:51:39:171.481 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625491276864, Pid 43016]: Idle 2026-06-08 Mon 05:51:39:171.620 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625491276864, Pid 43016]: SANDESH: No Client: 1780897899171564 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3:Control:contrail-dns:0 client_info= [ status = Idle successful_connections = 0 pid = 43016 http_port = 8092 start_time = 1780897899171493 collector_name = collector_ip = 0.0.0.0:0 collector_list= [ [ ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 0, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 0, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-06-08 Mon 05:51:39:171.743 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625491276864, Pid 43016]: SANDESH: Logging: DISABLED -> ENABLED 2026-06-08 Mon 05:51:39:171.757 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625491276864, Pid 43016]: SANDESH: Logging: LEVEL: [ INVALID ] -> [ SYS_DEBUG ] log4level: [ NOTSET ] -> [ DEBUG ] 2026-06-08 Mon 05:51:39:173.903 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: Processing scm::EvStart in state Idle 2026-06-08 Mon 05:51:39:173.939 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: Disconnect 2026-06-08 Mon 05:51:39:174.009 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: Processing scm::EvCollectorUpdate in state Disconnect 2026-06-08 Mon 05:51:39:177.869 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: Connect : Start Connect timer 10.0.0.38:8086 2026-06-08 Mon 05:51:39:177.962 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: Processing scm::EvSandeshSend in state Connect 2026-06-08 Mon 05:51:39:178.034 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: SANDESH: Send FAILED: 1780897899173998 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3:Control:contrail-dns:0 client_info= [ status = Disconnect successful_connections = 0 pid = 43016 http_port = 8092 start_time = 1780897899171493 collector_name = collector_ip = 0.0.0.0:0 collector_list= [ [ ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 0, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-06-08 Mon 05:51:39:178.046 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:51:39:178.054 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: Processing scm::EvSandeshSend in state Connect 2026-06-08 Mon 05:51:39:178.100 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: SANDESH: Send FAILED: 1780897899174041 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3:Control:contrail-dns:0 client_info= [ status = Disconnect successful_connections = 0 pid = 43016 http_port = 8092 start_time = 1780897899171493 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 0, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-06-08 Mon 05:51:39:178.114 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:51:39:178.121 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: Processing scm::EvSandeshSend in state Connect 2026-06-08 Mon 05:51:39:178.159 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: SANDESH: Send FAILED: 1780897899177946 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 0 pid = 43016 http_port = 8092 start_time = 1780897899171493 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 0, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-06-08 Mon 05:51:39:178.170 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:51:39:628.716 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625491276864, Pid 43016]: TCP [SYS_DEBUG]: TcpServerMessageLog: Server 10.20.0.14:8093 Initialization complete src/contrail-common/io/tcp_server.cc 137 2026-06-08 Mon 05:51:39:628.807 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625491276864, Pid 43016]: DnsAgent [SYS_NOTICE]: DnsOperational: Bulk sync has been started controller/src/dns/main.cc 401 2026-06-08 Mon 05:51:39:628.872 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625474131520, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Config Client Mgr SM: Start RabbitMqReader and init Database src/contrail-common/config-client-mgr/config_client_manager.cc 281 2026-06-08 Mon 05:51:39:628.948 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625474131520, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: RabbitMQ SM: RabbitMQ connection down src/contrail-common/config-client-mgr/config_amqp_client.cc 157 2026-06-08 Mon 05:51:39:628.957 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625474131520, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: RabbitMQ SM: Connect to Rabbit MQ with queue_delete TRUE src/contrail-common/config-client-mgr/config_amqp_client.cc 167 2026-06-08 Mon 05:51:39:629.079 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625457337920, Pid 43016]: Processing scm::EvSandeshSend in state Connect 2026-06-08 Mon 05:51:39:629.108 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625457337920, Pid 43016]: SANDESH: Send FAILED: 1780897899628931 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Disconnect : EvCollectorUpdate -> Connect ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.38:5673, (*_iter6) = 10.0.0.50:5673, (*_iter6) = 10.0.0.33:5673, ] ] status = Down description = RabbitMQ connection down ], ] ] flag_infos= [ [ ] ] description = Number of connections:2, Expected:3 Missing: Database:Cassandra ], ] ] ] 2026-06-08 Mon 05:51:39:629.124 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625457337920, Pid 43016]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-06-08 Mon 05:51:39:651.855 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625474131520, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: RabbitMQ SM: RabbitMQ connection established src/contrail-common/config-client-mgr/config_amqp_client.cc 150 2026-06-08 Mon 05:51:39:651.890 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625474131520, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Config Client Mgr SM: Init Database src/contrail-common/config-client-mgr/config_client_manager.cc 285 2026-06-08 Mon 05:51:39:651.922 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625474131520, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Lost Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 579 2026-06-08 Mon 05:51:39:651.931 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625474131520, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db Init src/contrail-common/config-client-mgr/config_cassandra_client.cc 87 2026-06-08 Mon 05:51:39:652.057 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: Processing scm::EvSandeshSend in state Connect 2026-06-08 Mon 05:51:39:652.122 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: SANDESH: Send FAILED: 1780897899651808 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Disconnect : EvCollectorUpdate -> Connect ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.38:5673, (*_iter6) = 10.0.0.50:5673, (*_iter6) = 10.0.0.33:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Number of connections:2, Expected:3 Missing: Database:Cassandra ], ] ] ] 2026-06-08 Mon 05:51:39:652.132 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-06-08 Mon 05:51:39:652.141 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: Processing scm::EvSandeshSend in state Connect 2026-06-08 Mon 05:51:39:652.160 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: SANDESH: Send FAILED: 1780897899651912 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Disconnect : EvCollectorUpdate -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.38:9041, (*_iter6) = 10.0.0.50:9041, (*_iter6) = 10.0.0.33:9041, ] ] status = Down description = Lost Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.38:5673, (*_iter6) = 10.0.0.50:5673, (*_iter6) = 10.0.0.33:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector, Database:Cassandra connection down ], ] ] ] 2026-06-08 Mon 05:51:39:652.166 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-06-08 Mon 05:51:39:701.562 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625474131520, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db SetTableSpace src/contrail-common/config-client-mgr/config_cassandra_client.cc 94 2026-06-08 Mon 05:51:39:703.330 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625474131520, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db UseColumnFamily uuidTableName src/contrail-common/config-client-mgr/config_cassandra_client.cc 103 2026-06-08 Mon 05:51:39:703.368 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625474131520, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db UseColumnFamily fqnTableName src/contrail-common/config-client-mgr/config_cassandra_client.cc 109 2026-06-08 Mon 05:51:39:703.427 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625474131520, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:51:39:703.436 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625474131520, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: BulkDataSync Started src/contrail-common/config-client-mgr/config_cassandra_client.cc 392 2026-06-08 Mon 05:51:39:703.480 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: Processing scm::EvSandeshSend in state Connect 2026-06-08 Mon 05:51:39:703.509 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: SANDESH: Send FAILED: 1780897899703403 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Disconnect : EvCollectorUpdate -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.38:9041, (*_iter6) = 10.0.0.50:9041, (*_iter6) = 10.0.0.33:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.38:5673, (*_iter6) = 10.0.0.50:5673, (*_iter6) = 10.0.0.33:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-08 Mon 05:51:39:703.517 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-06-08 Mon 05:51:39:711.083 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Lost Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 579 2026-06-08 Mon 05:51:39:711.117 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625469933120, Pid 43016]: Processing scm::EvSandeshSend in state Connect 2026-06-08 Mon 05:51:39:711.165 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625469933120, Pid 43016]: SANDESH: Send FAILED: 1780897899711036 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Disconnect : EvCollectorUpdate -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.38:9041, (*_iter6) = 10.0.0.50:9041, (*_iter6) = 10.0.0.33:9041, ] ] status = Down description = Lost Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.38:5673, (*_iter6) = 10.0.0.50:5673, (*_iter6) = 10.0.0.33:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector, Database:Cassandra connection down ], ] ] ] 2026-06-08 Mon 05:51:39:711.182 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625469933120, Pid 43016]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-06-08 Mon 05:51:39:711.125 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_WARN]: ConfigClientGetRowError: GetRow failed for table obj_fq_name_table access_control_list src/contrail-common/config-client-mgr/config_cassandra_client.cc 479 2026-06-08 Mon 05:51:44:712.673 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Lost Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 579 2026-06-08 Mon 05:51:44:712.713 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_WARN]: ConfigClientGetRowError: GetRow failed for table obj_fq_name_table access_control_list src/contrail-common/config-client-mgr/config_cassandra_client.cc 479 2026-06-08 Mon 05:51:49:715.858 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Lost Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 579 2026-06-08 Mon 05:51:49:715.916 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_WARN]: ConfigClientGetRowError: GetRow failed for table obj_fq_name_table access_control_list src/contrail-common/config-client-mgr/config_cassandra_client.cc 479 2026-06-08 Mon 05:51:54:717.140 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Lost Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 579 2026-06-08 Mon 05:51:54:717.191 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_WARN]: ConfigClientGetRowError: GetRow failed for table obj_fq_name_table access_control_list src/contrail-common/config-client-mgr/config_cassandra_client.cc 479 2026-06-08 Mon 05:51:59:718.447 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Lost Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 579 2026-06-08 Mon 05:51:59:718.486 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_WARN]: ConfigClientGetRowError: GetRow failed for table obj_fq_name_table access_control_list src/contrail-common/config-client-mgr/config_cassandra_client.cc 479 2026-06-08 Mon 05:52:04:719.642 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Lost Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 579 2026-06-08 Mon 05:52:04:719.681 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_WARN]: ConfigClientGetRowError: GetRow failed for table obj_fq_name_table access_control_list src/contrail-common/config-client-mgr/config_cassandra_client.cc 479 2026-06-08 Mon 05:52:09:724.600 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Lost Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 579 2026-06-08 Mon 05:52:09:724.636 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_WARN]: ConfigClientGetRowError: GetRow failed for table obj_fq_name_table access_control_list src/contrail-common/config-client-mgr/config_cassandra_client.cc 479 2026-06-08 Mon 05:52:14:733.536 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:14:733.590 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625440409152, Pid 43016]: Processing scm::EvSandeshSend in state Connect 2026-06-08 Mon 05:52:14:733.655 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625440409152, Pid 43016]: SANDESH: Send FAILED: 1780897934733496 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Disconnect : EvCollectorUpdate -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.38:9041, (*_iter6) = 10.0.0.50:9041, (*_iter6) = 10.0.0.33:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.38:5673, (*_iter6) = 10.0.0.50:5673, (*_iter6) = 10.0.0.33:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-08 Mon 05:52:14:733.671 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625440409152, Pid 43016]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-06-08 Mon 05:52:14:742.189 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:14:744.951 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:14:747.826 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:14:749.271 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:14:751.921 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:14:756.521 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:14:759.385 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:14:761.524 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:14:766.330 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:14:775.811 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:14:779.264 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:14:784.303 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:14:792.305 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:14:794.706 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:14:797.134 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:14:799.278 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:14:803.143 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:14:806.584 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:14:811.132 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:14:814.561 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:14:816.141 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:14:819.191 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:14:821.048 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:14:825.374 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:14:827.943 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:14:829.779 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:14:832.000 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:14:833.913 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:14:837.241 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:14:840.881 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:14:842.153 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:14:843.697 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:14:849.313 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:14:854.119 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:14:865.497 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:14:869.267 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:14:886.588 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:14:890.029 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:14:891.925 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:14:903.184 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:14:910.659 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:14:913.213 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:14:920.295 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:14:924.810 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:14:929.901 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:14:931.694 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:14:955.184 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:14:960.212 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:14:966.394 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:14:977.701 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:14:987.564 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:14:998.488 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:15:003.999 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:15:006.760 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:15:008.787 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:15:014.371 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:15:015.949 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:15:017.265 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:15:018.399 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:15:021.410 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:15:027.687 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:15:060.967 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:15:068.295 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:15:077.984 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:15:086.089 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:15:096.030 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:15:108.416 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:15:114.000 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:15:118.792 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:15:122.724 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:15:126.701 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:15:129.602 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:15:134.338 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:15:148.134 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:15:152.284 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:15:157.293 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:15:167.607 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:15:182.701 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:15:190.885 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:15:198.927 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:15:201.667 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:15:214.173 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:15:217.548 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:15:231.860 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:15:240.666 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:15:243.631 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:15:254.399 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:15:261.119 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:15:275.373 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:15:285.241 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:15:289.719 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:15:295.519 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:15:298.627 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:15:316.924 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:15:329.304 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:15:338.182 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:15:348.309 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:15:357.327 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:15:363.149 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:15:367.593 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:15:371.025 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:15:372.354 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:15:373.541 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:15:374.847 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:15:376.077 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:15:377.991 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:15:379.969 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:15:381.537 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:15:382.565 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:15:383.556 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:15:384.754 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:15:386.029 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:15:389.450 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:15:391.074 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:15:392.445 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:15:395.374 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:15:396.452 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:15:396.516 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: One reader finished BulkSync src/contrail-common/config-client-mgr/config_cassandra_client.cc 557 2026-06-08 Mon 05:52:15:396.545 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625465734720, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: One reader finished BulkSync src/contrail-common/config-client-mgr/config_cassandra_client.cc 557 2026-06-08 Mon 05:52:15:396.761 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625440409152, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: One reader finished BulkSync src/contrail-common/config-client-mgr/config_cassandra_client.cc 557 2026-06-08 Mon 05:52:15:396.865 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: One reader finished BulkSync src/contrail-common/config-client-mgr/config_cassandra_client.cc 557 2026-06-08 Mon 05:52:15:397.072 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625453139520, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: One reader finished BulkSync src/contrail-common/config-client-mgr/config_cassandra_client.cc 557 2026-06-08 Mon 05:52:15:397.285 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625457337920, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: One reader finished BulkSync src/contrail-common/config-client-mgr/config_cassandra_client.cc 557 2026-06-08 Mon 05:52:15:397.435 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625448805952, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: One reader finished BulkSync src/contrail-common/config-client-mgr/config_cassandra_client.cc 557 2026-06-08 Mon 05:52:15:397.622 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625444607552, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: BulkSyncDone by all readers src/contrail-common/config-client-mgr/config_cassandra_client.cc 553 2026-06-08 Mon 05:52:15:397.661 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625444607552, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Config Client Mgr SM: End of RIB computed and notification sent src/contrail-common/config-client-mgr/config_client_manager.cc 187 2026-06-08 Mon 05:52:15:397.685 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625474131520, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: TRUE src/contrail-common/config-client-mgr/config_client_manager.cc 216 2026-06-08 Mon 05:52:15:397.826 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625469933120, Pid 43016]: Processing scm::EvSandeshSend in state Connect 2026-06-08 Mon 05:52:15:397.874 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625469933120, Pid 43016]: SANDESH: Send FAILED: 1780897935397694 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Disconnect : EvCollectorUpdate -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.38:9041, (*_iter6) = 10.0.0.50:9041, (*_iter6) = 10.0.0.33:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.38:5673, (*_iter6) = 10.0.0.50:5673, (*_iter6) = 10.0.0.33:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-08 Mon 05:52:15:397.885 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625469933120, Pid 43016]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-06-08 Mon 05:52:15:404.185 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625465734720, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:15:404.416 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625469933120, Pid 43016]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating network-ipam:default-domain:default-project:default-network-ipam controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:52:15:404.480 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625469933120, Pid 43016]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating project:default-domain:default-project controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:52:15:404.510 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625469933120, Pid 43016]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-network-ipam controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:52:15:404.539 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625469933120, Pid 43016]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating virtual-network:default-domain:default-project:ip-fabric controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:52:15:404.577 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625469933120, Pid 43016]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-virtual-network controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:52:15:404.594 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625469933120, Pid 43016]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating analytics-node:default-global-system-config:cn-jenkins-deploy-platform-ansible-os-5902-2. controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:52:15:404.613 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625469933120, Pid 43016]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating global-system-config:default-global-system-config controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:52:15:404.628 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625469933120, Pid 43016]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-analytics-node controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:52:15:404.485 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625465734720, Pid 43016]: IFMap [SYS_WARN]: ConfigurationMalformedReferenceWarningLog: bgp_peering No entry in metadata map bgp_router 52be0fb9-3a40-4991-9fb0-17a2b7266240 controller/src/ifmap/client/config_json_parser.cc 222 2026-06-08 Mon 05:52:15:405.767 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625465734720, Pid 43016]: IFMap [SYS_WARN]: ConfigurationMalformedReferenceWarningLog: bgp_peering No entry in metadata map bgp_router 52be0fb9-3a40-4991-9fb0-17a2b7266240 controller/src/ifmap/client/config_json_parser.cc 222 2026-06-08 Mon 05:52:15:405.975 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625465734720, Pid 43016]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating tag-type:label controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:52:15:405.999 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625465734720, Pid 43016]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating config-node:default-global-system-config:cn-jenkins-deploy-platform-ansible-os-5902-1. controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:52:15:406.023 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625465734720, Pid 43016]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-config-node controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:52:15:406.041 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625465734720, Pid 43016]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating tag-type:application controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:52:15:421.693 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625444607552, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:15:422.281 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625444607552, Pid 43016]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating analytics-snmp-node:default-global-system-config:cn-jenkins-deploy-platform-ansible-os-5902-2. controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:52:15:422.335 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625444607552, Pid 43016]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-analytics-snmp-node controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:52:15:422.357 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625444607552, Pid 43016]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating application-policy-set:default-policy-management:default-application-policy-set controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:52:15:422.389 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625444607552, Pid 43016]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating policy-management:default-policy-management controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:52:15:422.403 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625444607552, Pid 43016]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating policy-management-application-policy-set controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:52:15:422.420 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625444607552, Pid 43016]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating global-vrouter-config:default-global-system-config:default-global-vrouter-config controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:52:15:422.433 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625444607552, Pid 43016]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating application-policy-set-global-vrouter-config controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:52:15:422.449 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625444607552, Pid 43016]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating global-qos-config:default-global-system-config:default-global-qos-config controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:52:15:422.479 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625444607552, Pid 43016]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-global-qos-config controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:52:15:422.498 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625444607552, Pid 43016]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-xmpp-close-reason controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:52:15:422.519 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625444607552, Pid 43016]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:52:15:422.536 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625444607552, Pid 43016]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating config-database-node:default-global-system-config:cn-jenkins-deploy-platform-ansible-os-5902-3. controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:52:15:422.556 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625444607552, Pid 43016]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-config-database-node controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:52:15:422.572 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625444607552, Pid 43016]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating namespace:default-domain:default-namespace controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:52:15:422.592 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625444607552, Pid 43016]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating domain:default-domain controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:52:15:422.606 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625444607552, Pid 43016]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating domain-namespace controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:52:15:422.622 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625444607552, Pid 43016]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating service-template:default-domain:analyzer-template controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:52:15:422.643 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625444607552, Pid 43016]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating domain-service-template controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:52:15:422.659 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625444607552, Pid 43016]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-core-files controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:52:15:422.685 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625444607552, Pid 43016]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:52:15:422.706 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625444607552, Pid 43016]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-address-mismatch-control controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:52:15:422.727 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625444607552, Pid 43016]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:52:15:422.743 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625444607552, Pid 43016]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-conf-incorrect controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:52:15:422.762 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625444607552, Pid 43016]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:52:15:423.268 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625457337920, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:15:423.455 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625457337920, Pid 43016]: IFMap [SYS_WARN]: ConfigurationMalformedReferenceWarningLog: bgp_peering No entry in metadata map bgp_router 1ed9429b-8995-4250-afbd-ba8d5ddff253 controller/src/ifmap/client/config_json_parser.cc 222 2026-06-08 Mon 05:52:15:423.475 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625457337920, Pid 43016]: IFMap [SYS_WARN]: ConfigurationMalformedReferenceWarningLog: bgp_peering No entry in metadata map bgp_router 1ed9429b-8995-4250-afbd-ba8d5ddff253 controller/src/ifmap/client/config_json_parser.cc 222 2026-06-08 Mon 05:52:15:423.641 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625457337920, Pid 43016]: IFMap [SYS_WARN]: ConfigurationMalformedReferenceWarningLog: bgp_peering No entry in metadata map bgp_router 7ea058e7-13e3-42e3-ad5c-3a51db612125 controller/src/ifmap/client/config_json_parser.cc 222 2026-06-08 Mon 05:52:15:423.657 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625457337920, Pid 43016]: IFMap [SYS_WARN]: ConfigurationMalformedReferenceWarningLog: bgp_peering No entry in metadata map bgp_router 7ea058e7-13e3-42e3-ad5c-3a51db612125 controller/src/ifmap/client/config_json_parser.cc 222 2026-06-08 Mon 05:52:15:423.934 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625457337920, Pid 43016]: IFMap [SYS_WARN]: ConfigurationMalformedReferenceWarningLog: instance_target No entry in metadata map routing_instance c63bdcee-3142-4bf1-9f1c-c919d7e89cfd controller/src/ifmap/client/config_json_parser.cc 222 2026-06-08 Mon 05:52:15:424.051 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625457337920, Pid 43016]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-xmpp-connectivity controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:52:15:424.082 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625457337920, Pid 43016]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:52:15:424.100 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625457337920, Pid 43016]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating tag-type:site controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:52:15:424.123 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625457337920, Pid 43016]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating virtual-machine-interface:default-global-system-config:an-jenkins-deploy-platform-ansible-os-5902-1:vhost0 controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:52:15:424.165 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625457337920, Pid 43016]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating virtual-router:default-global-system-config:an-jenkins-deploy-platform-ansible-os-5902-1 controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:52:15:424.178 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625457337920, Pid 43016]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating virtual-router-virtual-machine-interface controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:52:15:424.200 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625457337920, Pid 43016]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating virtual-machine-interface-virtual-network controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:52:15:424.215 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625457337920, Pid 43016]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating database-node:default-global-system-config:cn-jenkins-deploy-platform-ansible-os-5902-3. controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:52:15:424.234 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625457337920, Pid 43016]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-database-node controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:52:15:424.257 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625457337920, Pid 43016]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-virtual-router controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:52:15:424.273 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625457337920, Pid 43016]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating analytics-snmp-node:default-global-system-config:cn-jenkins-deploy-platform-ansible-os-5902-1. controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:52:15:424.292 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625457337920, Pid 43016]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-analytics-snmp-node controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:52:15:424.307 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625457337920, Pid 43016]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating service-template:default-domain:nat-template controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:52:15:424.330 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625457337920, Pid 43016]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating domain-service-template controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:52:15:424.346 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625457337920, Pid 43016]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-vrouter-l3mh-physical-interface-link-down controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:52:15:424.364 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625457337920, Pid 43016]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:52:15:424.380 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625457337920, Pid 43016]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-storage-cluster-state controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:52:15:424.398 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625457337920, Pid 43016]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:52:15:424.656 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625440409152, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:15:424.872 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625440409152, Pid 43016]: IFMap [SYS_WARN]: ConfigurationMalformedReferenceWarningLog: instance_target No entry in metadata map routing_instance 278e34a1-a811-4e5f-aa92-c92c68d34343 controller/src/ifmap/client/config_json_parser.cc 222 2026-06-08 Mon 05:52:15:425.220 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625440409152, Pid 43016]: IFMap [SYS_WARN]: ConfigurationMalformedReferenceWarningLog: instance_target No entry in metadata map routing_instance c05dd037-d24e-4d96-9fd1-639cc28310b2 controller/src/ifmap/client/config_json_parser.cc 222 2026-06-08 Mon 05:52:15:425.342 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625440409152, Pid 43016]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-package-version-mismatch controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:52:15:425.372 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625440409152, Pid 43016]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:52:15:425.391 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625440409152, Pid 43016]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-process-status controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:52:15:425.409 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625440409152, Pid 43016]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:52:15:425.424 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625440409152, Pid 43016]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating analytics-alarm-node:default-global-system-config:cn-jenkins-deploy-platform-ansible-os-5902-2. controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:52:15:425.446 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625440409152, Pid 43016]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-analytics-alarm-node controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:52:15:425.461 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625440409152, Pid 43016]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating api-access-list:default-global-system-config:default-api-access-list controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:52:15:425.478 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625440409152, Pid 43016]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-api-access-list controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:52:15:425.494 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625440409152, Pid 43016]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating virtual-network:default-domain:default-project:default-virtual-network controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:52:15:425.515 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625440409152, Pid 43016]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-virtual-network controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:52:15:425.529 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625440409152, Pid 43016]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating access-control-list:default-domain:default-project:ip-fabric:ip-fabric controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:52:15:425.570 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625440409152, Pid 43016]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating virtual-network-access-control-list controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:52:15:425.593 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625440409152, Pid 43016]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating virtual-network:default-domain:default-project:dci-network controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:52:15:425.624 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625440409152, Pid 43016]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-virtual-network controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:52:15:425.639 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625440409152, Pid 43016]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating tag-type:neutron_fwaas controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:52:15:425.947 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625453139520, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:15:426.713 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625453139520, Pid 43016]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-vrouter-limit-exceeded controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:52:15:426.751 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625453139520, Pid 43016]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:52:15:426.771 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625453139520, Pid 43016]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-address-mismatch-compute controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:52:15:426.791 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625453139520, Pid 43016]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:52:15:426.807 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625453139520, Pid 43016]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating service-appliance-set:default-global-system-config:native controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:52:15:426.829 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625453139520, Pid 43016]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-service-appliance-set controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:52:15:426.858 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625453139520, Pid 43016]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating security-group:default-domain:default-project:__no_rule__ controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:52:15:426.877 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625453139520, Pid 43016]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-security-group controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:52:15:426.892 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625453139520, Pid 43016]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating service-appliance-set:default-global-system-config:opencontrail controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:52:15:426.910 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625453139520, Pid 43016]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-service-appliance-set controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:52:15:426.957 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625453139520, Pid 43016]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating access-control-list:default-domain:default-project:__no_rule__:ingress-access-control-list controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:52:15:426.978 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625453139520, Pid 43016]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating security-group-access-control-list controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:52:15:427.001 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625453139520, Pid 43016]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating service-template:default-domain:docker-template controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:52:15:427.023 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625453139520, Pid 43016]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating domain-service-template controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:52:15:427.039 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625453139520, Pid 43016]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-prouter-tsn-connectivity controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:52:15:427.059 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625453139520, Pid 43016]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:52:15:427.075 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625453139520, Pid 43016]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating config-database-node:default-global-system-config:cn-jenkins-deploy-platform-ansible-os-5902-2. controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:52:15:427.096 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625453139520, Pid 43016]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-config-database-node controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:52:15:427.117 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625453139520, Pid 43016]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating domain-project controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:52:15:427.132 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625453139520, Pid 43016]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating tag-type:neutron_tag controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:52:15:427.147 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625453139520, Pid 43016]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-pending-cassandra-compaction-tasks controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:52:15:427.170 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625453139520, Pid 43016]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:52:15:427.186 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625453139520, Pid 43016]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating virtual-network:default-domain:default-project:__link_local__ controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:52:15:427.227 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625453139520, Pid 43016]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-virtual-network controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:52:15:427.242 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625453139520, Pid 43016]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-vrouter-interface controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:52:15:427.264 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625453139520, Pid 43016]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:52:15:435.330 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:15:435.556 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625444607552, Pid 43016]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-prouter-connectivity controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:52:15:435.625 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625444607552, Pid 43016]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:52:15:435.649 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625444607552, Pid 43016]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-conf-elements-inconsistency controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:52:15:435.670 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625444607552, Pid 43016]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:52:15:435.694 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625444607552, Pid 43016]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating analytics-snmp-node:default-global-system-config:cn-jenkins-deploy-platform-ansible-os-5902-3. controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:52:15:435.715 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625444607552, Pid 43016]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-analytics-snmp-node controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:52:15:435.733 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625444607552, Pid 43016]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-partial-sysinfo controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:52:15:435.752 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625444607552, Pid 43016]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:52:15:435.770 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625444607552, Pid 43016]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating config-node:default-global-system-config:cn-jenkins-deploy-platform-ansible-os-5902-3. controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:52:15:435.789 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625444607552, Pid 43016]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-config-node controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:52:15:435.804 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625444607552, Pid 43016]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating fabric:default-global-system-config:default-fabric controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:52:15:435.822 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625444607552, Pid 43016]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-fabric controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:52:15:440.965 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625448805952, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:15:441.293 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625448805952, Pid 43016]: IFMap [SYS_WARN]: ConfigurationMalformedReferenceWarningLog: instance_target No entry in metadata map routing_instance 62ebfee9-aabe-45de-ab84-af810d1590da controller/src/ifmap/client/config_json_parser.cc 222 2026-06-08 Mon 05:52:15:441.660 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625448805952, Pid 43016]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating service-template:default-domain:netns-snat-template controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:52:15:441.712 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625448805952, Pid 43016]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating domain-service-template controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:52:15:441.733 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625448805952, Pid 43016]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating config-node:default-global-system-config:cn-jenkins-deploy-platform-ansible-os-5902-2. controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:52:15:441.756 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625448805952, Pid 43016]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-config-node controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:52:15:441.773 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625448805952, Pid 43016]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating virtual-machine-interface:default-global-system-config:an-jenkins-deploy-platform-ansible-os-5902-2:vhost0 controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:52:15:441.811 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625448805952, Pid 43016]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating virtual-router:default-global-system-config:an-jenkins-deploy-platform-ansible-os-5902-2 controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:52:15:441.824 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625448805952, Pid 43016]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating virtual-router-virtual-machine-interface controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:52:15:441.858 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625448805952, Pid 43016]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating virtual-machine-interface-virtual-network controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:52:15:441.876 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625448805952, Pid 43016]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-bgp-connectivity controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:52:15:441.898 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625448805952, Pid 43016]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:52:15:441.916 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625448805952, Pid 43016]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating analytics-alarm-node:default-global-system-config:cn-jenkins-deploy-platform-ansible-os-5902-3. controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:52:15:441.935 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625448805952, Pid 43016]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-analytics-alarm-node controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:52:15:441.952 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625448805952, Pid 43016]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-process-connectivity controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:52:15:441.970 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625448805952, Pid 43016]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:52:15:441.986 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625448805952, Pid 43016]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-disk-usage-high controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:52:15:442.009 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625448805952, Pid 43016]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:52:15:442.024 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625448805952, Pid 43016]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating database-node:default-global-system-config:cn-jenkins-deploy-platform-ansible-os-5902-1. controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:52:15:442.050 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625448805952, Pid 43016]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-database-node controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:52:15:442.066 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625448805952, Pid 43016]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating application-policy-set:default-domain:default-project:default-application-policy-set controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:52:15:442.084 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625448805952, Pid 43016]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-application-policy-set controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:52:15:442.101 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625448805952, Pid 43016]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-disk-usage-critical controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:52:15:442.119 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625448805952, Pid 43016]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:52:15:482.699 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:15:483.477 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating access-control-list:default-domain:default-project:__no_rule__:egress-access-control-list controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:52:15:483.525 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating security-group-access-control-list controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:52:15:483.544 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating analytics-node:default-global-system-config:cn-jenkins-deploy-platform-ansible-os-5902-3. controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:52:15:483.565 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-analytics-node controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:52:15:483.581 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating discovery-service-assignment:default-discovery-service-assignment controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:52:15:483.595 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating tag-type:tier controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:52:15:483.609 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating network-ipam:default-domain:default-project:service-chain-flat-ipam controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:52:15:483.629 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-network-ipam controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:52:15:483.660 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-global-vrouter-config controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:52:15:483.689 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-node-status controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:52:15:483.709 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:52:15:483.730 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-vrouter-table-limit-exceeded controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:52:15:483.748 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:52:15:483.765 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating tag-type:deployment controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:52:15:483.780 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating config-database-node:default-global-system-config:cn-jenkins-deploy-platform-ansible-os-5902-1. controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:52:15:483.797 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-config-database-node controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:52:15:483.814 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating analytics-alarm-node:default-global-system-config:cn-jenkins-deploy-platform-ansible-os-5902-1. controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:52:15:483.848 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-analytics-alarm-node controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:52:15:483.866 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating service-template:default-domain:haproxy-loadbalancer-template controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:52:15:483.889 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating domain-service-template controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:52:15:483.916 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-virtual-router controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:52:15:483.932 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-bottle-request-size-limit controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:52:15:483.954 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:52:15:483.969 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating analytics-node:default-global-system-config:cn-jenkins-deploy-platform-ansible-os-5902-1. controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:52:15:483.998 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-analytics-node controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:52:15:484.014 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating database-node:default-global-system-config:cn-jenkins-deploy-platform-ansible-os-5902-2. controller/src/ifmap/ifmap_server_table.cc 95 2026-06-08 Mon 05:52:15:484.035 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-database-node controller/src/ifmap/ifmap_server_table.cc 130 2026-06-08 Mon 05:52:15:484.299 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625491276864, Pid 43016]: DnsAgent [SYS_NOTICE]: DnsOperational: EndOfRib point was reached, IF-MAP objects have been created controller/src/dns/main.cc 83 2026-06-08 Mon 05:52:15:484.323 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625491276864, Pid 43016]: DnsAgent [SYS_NOTICE]: DnsOperational: The IF-MAP database table has been updated: __ifmap__.virtual_DNS.0, counting 0 elements controller/src/dns/main.cc 96 2026-06-08 Mon 05:52:15:484.336 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625491276864, Pid 43016]: DnsAgent [SYS_NOTICE]: DnsOperational: The IF-MAP database table has been updated: __ifmap__.virtual_DNS_record.0, counting 0 elements controller/src/dns/main.cc 96 2026-06-08 Mon 05:52:15:484.347 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625491276864, Pid 43016]: DnsAgent [SYS_NOTICE]: DnsOperational: The IF-MAP database table has been updated: __ifmap__.network_ipam.0, counting 2 elements controller/src/dns/main.cc 96 2026-06-08 Mon 05:52:15:484.360 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625491276864, Pid 43016]: DnsAgent [SYS_NOTICE]: DnsOperational: The IF-MAP database table has been updated: __ifmap__.virtual_network_network_ipam.0, counting 0 elements controller/src/dns/main.cc 96 2026-06-08 Mon 05:52:15:484.372 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625491276864, Pid 43016]: DnsAgent [SYS_NOTICE]: DnsOperational: The IF-MAP database table has been updated: __ifmap__.global_qos_config.0, counting 1 elements controller/src/dns/main.cc 96 2026-06-08 Mon 05:52:15:484.384 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625491276864, Pid 43016]: DnsAgent [SYS_NOTICE]: DnsOperational: VnniConfig has been updated after the bulk sync, the number of elements: 0 controller/src/dns/main.cc 106 2026-06-08 Mon 05:52:15:484.394 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625491276864, Pid 43016]: DnsAgent [SYS_NOTICE]: DnsOperational: IpamConfig has been updated after the bulk sync, the number of elements: 2 controller/src/dns/main.cc 117 2026-06-08 Mon 05:52:15:484.405 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625491276864, Pid 43016]: DnsAgent [SYS_NOTICE]: DnsOperational: VirtualDnsConfig has been updated after the bulk sync, the number of elements: 0 controller/src/dns/main.cc 128 2026-06-08 Mon 05:52:15:484.414 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625491276864, Pid 43016]: DnsAgent [SYS_NOTICE]: DnsOperational: VirtualDnsRecordConfig has been updated after the bulk sync, the number of elements: 0 controller/src/dns/main.cc 139 2026-06-08 Mon 05:52:15:484.424 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625491276864, Pid 43016]: DnsAgent [SYS_NOTICE]: DnsOperational: GlobalQosConfig has been updated after the bulk sync controller/src/dns/main.cc 149 2026-06-08 Mon 05:52:15:484.994 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625448805952, Pid 43016]: Processing scm::EvSandeshSend in state Connect 2026-06-08 Mon 05:52:15:485.038 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625448805952, Pid 43016]: SANDESH: Send FAILED: 1780897935484466 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Disconnect : EvCollectorUpdate -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.38:9041, (*_iter6) = 10.0.0.50:9041, (*_iter6) = 10.0.0.33:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.38:5673, (*_iter6) = 10.0.0.50:5673, (*_iter6) = 10.0.0.33:5673, ] ] status = Down description = RabbitMQ connection down ], ] ] flag_infos= [ [ ] ] description = Collector, Database:RabbitMQ connection down ], ] ] ] 2026-06-08 Mon 05:52:15:485.058 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625448805952, Pid 43016]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-06-08 Mon 05:52:15:485.083 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625491276864, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: RabbitMQ SM: RabbitMQ connection down src/contrail-common/config-client-mgr/config_amqp_client.cc 157 2026-06-08 Mon 05:52:15:485.096 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625491276864, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: RabbitMQ SM: Connect to Rabbit MQ with queue_delete TRUE src/contrail-common/config-client-mgr/config_amqp_client.cc 167 2026-06-08 Mon 05:52:15:548.377 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625491276864, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: RabbitMQ SM: RabbitMQ connection established src/contrail-common/config-client-mgr/config_amqp_client.cc 150 2026-06-08 Mon 05:52:15:548.484 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625453139520, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: TRUE src/contrail-common/config-client-mgr/config_client_manager.cc 216 2026-06-08 Mon 05:52:15:548.488 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: Processing scm::EvSandeshSend in state Connect 2026-06-08 Mon 05:52:15:548.949 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: SANDESH: Send FAILED: 1780897935548341 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Disconnect : EvCollectorUpdate -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.38:9041, (*_iter6) = 10.0.0.50:9041, (*_iter6) = 10.0.0.33:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.38:5673, (*_iter6) = 10.0.0.50:5673, (*_iter6) = 10.0.0.33:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-08 Mon 05:52:15:548.958 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-06-08 Mon 05:52:15:554.801 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625491276864, Pid 43016]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.14:8092::10.20.0.254:38744 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2026-06-08 Mon 05:52:15:556.501 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625491276864, Pid 43016]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.33:53306::10.0.0.38:8086(-1) < Active session connection complete src/contrail-common/io/tcp_session.cc 281 2026-06-08 Mon 05:52:15:556.536 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625491276864, Pid 43016]: 10.0.0.33:53306::10.0.0.38:8086(-1) OnSessionEvent TCP Connected 2026-06-08 Mon 05:52:15:556.545 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625491276864, Pid 43016]: 10.0.0.33:53306::10.0.0.38:8086(-1):EvTcpConnected 2026-06-08 Mon 05:52:15:556.606 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: Processing scm::EvTcpConnected in state Connect 2026-06-08 Mon 05:52:15:556.636 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: Connect : EvTcpConnected : Cancelling Connect timer 2026-06-08 Mon 05:52:15:556.689 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: ClientInit 2026-06-08 Mon 05:52:15:556.702 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: Sending Ctrl Message for cn-jenkins-deploy-platform-ansible-os-5902-3:contrail-dns:0:Control count 1 2026-06-08 Mon 05:52:15:556.723 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625474131520, Pid 43016]: DnsAgent [SYS_NOTICE]: DnsOperational: BIND named up; DNS is operational controller/src/dns/mgr/dns_mgr.cc 735 2026-06-08 Mon 05:52:15:556.795 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: Processing scm::EvSandeshSend in state ClientInit 2026-06-08 Mon 05:52:15:556.804 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: ClientInit : EvSandeshSend : NodeStatusUVE 2026-06-08 Mon 05:52:15:556.821 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: SANDESH: Send FAILED: 1780897935556682 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.38:9041, (*_iter6) = 10.0.0.50:9041, (*_iter6) = 10.0.0.33:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.38:5673, (*_iter6) = 10.0.0.50:5673, (*_iter6) = 10.0.0.33:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-08 Mon 05:52:15:556.828 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: Received UVE message in wrong state : NodeStatusUVE 2026-06-08 Mon 05:52:15:556.847 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: Processing scm::EvSandeshSend in state ClientInit 2026-06-08 Mon 05:52:15:556.853 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: ClientInit : EvSandeshSend : SandeshCtrlClientToServer 2026-06-08 Mon 05:52:15:556.872 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: Processing scm::EvSandeshSend in state ClientInit 2026-06-08 Mon 05:52:15:556.878 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2026-06-08 Mon 05:52:15:556.995 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: SANDESH: Send FAILED: 1780897935556780 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3:Control:contrail-dns:0 client_info= [ status = ClientInit successful_connections = 1 pid = 43016 http_port = 8092 start_time = 1780897899171493 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 7 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 159 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 13 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 74 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 82 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 349, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 12, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-06-08 Mon 05:52:15:557.016 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: Received UVE message in wrong state : SandeshModuleClientTrace 2026-06-08 Mon 05:52:15:557.193 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: Processing scm::EvSandeshSend in state ClientInit 2026-06-08 Mon 05:52:15:557.205 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2026-06-08 Mon 05:52:15:557.244 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: SANDESH: Send FAILED: 1780897935557059 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3:Control:contrail-dns:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 15 dequeues = 15 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnected enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 18 dequeues = 18 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = ClientInit state_since = 1780897935556688 last_state = ClientInit last_event = scm::EvSandeshSend last_event_at = 1780897935556871 ] session_stats= [ num_recv_msg = 0 num_recv_msg_fail = 0 num_recv_fail = 0 num_send_msg = 0 num_send_msg_fail = 0 num_send_buffer_fail = 0 num_wait_msgq_enqueue = 0 num_wait_msgq_dequeue = 0 num_write_ready_cb_error = 0 ] session_rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] session_tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] max_sm_queue_count = 3 ] 2026-06-08 Mon 05:52:15:557.258 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: Received UVE message in wrong state : SandeshModuleClientTrace 2026-06-08 Mon 05:52:15:557.265 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: Processing scm::EvSandeshSend in state ClientInit 2026-06-08 Mon 05:52:15:557.271 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2026-06-08 Mon 05:52:15:557.366 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: SANDESH: Send FAILED: 1780897935557091 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3:Control:contrail-dns:0 client_info= [ status = ClientInit successful_connections = 1 pid = 43016 http_port = 8092 start_time = 1780897899171493 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 7 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 159 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 13 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 74 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 82 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 349, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 14, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-06-08 Mon 05:52:15:557.380 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: Received UVE message in wrong state : SandeshModuleClientTrace 2026-06-08 Mon 05:52:15:562.382 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625491276864, Pid 43016]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.14:8092::10.20.0.14:51506 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2026-06-08 Mon 05:52:15:567.577 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625491276864, Pid 43016]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.14:8092::10.20.0.254:34868 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2026-06-08 Mon 05:52:15:572.497 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625491276864, Pid 43016]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.14:8092::10.20.0.14:51516 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2026-06-08 Mon 05:52:15:572.731 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625457337920, Pid 43016]: OnMessage control in state: ClientInit 2026-06-08 Mon 05:52:15:572.900 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625457337920, Pid 43016]: Processing scm::EvSandeshMessageRecv in state ClientInit 2026-06-08 Mon 05:52:15:572.953 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625457337920, Pid 43016]: ClientInit : EvSandeshMessageRecv 2026-06-08 Mon 05:52:15:572.970 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625457337920, Pid 43016]: Received Ctrl Message with size 0 2026-06-08 Mon 05:52:15:572.979 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625457337920, Pid 43016]: SyncAllMaps for DnsState:0 period 0 cycle 0 without seqno , total = 0 2026-06-08 Mon 05:52:15:573.142 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625457337920, Pid 43016]: SyncUVE Syncing ObjectGeneratorInfo val name = cn-jenkins-deploy-platform-ansible-os-5902-3:Control:contrail-dns:0 client_info= [ status = ClientInit successful_connections = 1 pid = 43016 http_port = 8092 start_time = 1780897899171493 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 15 dequeues = 15 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnected enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 18 dequeues = 18 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = ClientInit state_since = 1780897935556688 last_state = ClientInit last_event = scm::EvSandeshSend last_event_at = 1780897935556871 ] session_stats= [ num_recv_msg = 0 num_recv_msg_fail = 0 num_recv_fail = 0 num_send_msg = 0 num_send_msg_fail = 0 num_send_buffer_fail = 0 num_wait_msgq_enqueue = 0 num_wait_msgq_dequeue = 0 num_write_ready_cb_error = 0 ] session_rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] session_tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 7 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 159 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 13 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 74 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 82 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 349, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 14, _iter59->first = sent _iter59->second = 0, ] ] max_sm_queue_count = 3 proxy seq 7 2026-06-08 Mon 05:52:15:573.192 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625457337920, Pid 43016]: SyncAllMaps for ModuleClientState:0 period 0 cycle 0 without seqno , total = 1 2026-06-08 Mon 05:52:15:573.217 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625457337920, Pid 43016]: SyncUVE Syncing ObjectDns val name = cn-jenkins-deploy-platform-ansible-os-5902-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.38:9041, (*_iter6) = 10.0.0.50:9041, (*_iter6) = 10.0.0.33:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.38:5673, (*_iter6) = 10.0.0.50:5673, (*_iter6) = 10.0.0.33:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] proxy seq 10 2026-06-08 Mon 05:52:15:573.228 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625457337920, Pid 43016]: SyncAllMaps for NodeStatus:0 period 0 cycle 0 without seqno , total = 1 2026-06-08 Mon 05:52:15:573.235 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625457337920, Pid 43016]: SyncAllMaps for PeerFlapData:0 period 0 cycle 0 without seqno , total = 0 2026-06-08 Mon 05:52:15:573.240 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625457337920, Pid 43016]: SyncAllMaps for PeerStatsData:0 period 0 cycle 0 without seqno , total = 0 2026-06-08 Mon 05:52:15:573.247 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625457337920, Pid 43016]: SyncAllMaps for XmppPeerInfoData:0 period 0 cycle 0 without seqno , total = 0 2026-06-08 Mon 05:52:15:573.276 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625457337920, Pid 43016]: Established 2026-06-08 Mon 05:52:17:341.461 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: DnsAgent [SYS_NOTICE]: DnsOperational: Bulk sync has been done (the DB has been synced with named) controller/src/dns/mgr/dns_mgr.cc 452 2026-06-08 Mon 05:52:17:352.039 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625474131520, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:17:950.317 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625457337920, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:18:137.039 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625440409152, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:18:443.409 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625457337920, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:18:694.362 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625474131520, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:23:054.660 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625448805952, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:23:137.922 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625448805952, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:23:248.686 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625444607552, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:23:596.850 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625474131520, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:23:666.366 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625448805952, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:24:751.276 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625444607552, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:25:336.445 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625474131520, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:25:478.121 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:25:484.220 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:25:596.774 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625444607552, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:25:656.567 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625474131520, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:25:703.647 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625448805952, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:25:869.147 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625469933120, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:25:898.402 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625440409152, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:26:101.651 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625448805952, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:26:140.843 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625444607552, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:26:336.637 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625440409152, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:26:342.706 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625440409152, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:26:547.083 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625440409152, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:26:633.052 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:26:746.822 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:26:814.016 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:26:944.597 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625436210752, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:27:007.899 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625465734720, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:27:120.039 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625436210752, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:27:194.056 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625440409152, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:27:349.316 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625474131520, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:27:434.066 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625474131520, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:27:559.901 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625457337920, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:27:678.172 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625440409152, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:27:753.127 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:27:882.028 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:27:953.011 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625440409152, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:28:100.503 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625444607552, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:28:177.686 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625444607552, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:28:342.621 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625432012352, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:28:420.519 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:28:527.212 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625444607552, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:28:643.418 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625444607552, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:28:736.582 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:28:864.917 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625469933120, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:28:961.355 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625465734720, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:29:071.154 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625440409152, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:29:197.650 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:29:312.606 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:29:453.103 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:29:507.985 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625469933120, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:29:626.197 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625432012352, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:29:719.328 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625465734720, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:29:825.524 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:29:909.420 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:30:018.290 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:30:103.880 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:30:232.318 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:31:665.014 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625465734720, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:31:745.505 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625440409152, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:32:721.135 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:32:770.490 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625465734720, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:34:035.600 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:52:34:096.234 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625432012352, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:53:39:568.693 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625491276864, Pid 43016]: 10.0.0.33:53306::10.0.0.38:8086(-1) OnSessionEvent TCP Connection Closed 2026-06-08 Mon 05:53:39:568.732 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625491276864, Pid 43016]: 10.0.0.33:53306::10.0.0.38:8086(-1):EvTcpClose 2026-06-08 Mon 05:53:39:568.934 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: Processing scm::EvTcpClose in state Established 2026-06-08 Mon 05:53:39:568.956 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: Established : EvTcpClose 2026-06-08 Mon 05:53:39:569.696 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: Connect : Start Connect timer 10.0.0.33:8086 2026-06-08 Mon 05:53:39:569.717 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625491276864, Pid 43016]: -(-1) OnSessionEvent TCP Connect Failed 2026-06-08 Mon 05:53:39:569.766 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625491276864, Pid 43016]: -(-1):EvTcpConnectFail 2026-06-08 Mon 05:53:39:569.855 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: Processing scm::EvSandeshSend in state Connect 2026-06-08 Mon 05:53:39:570.104 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: SANDESH: Send FAILED: 1780898019569060 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3:Control:contrail-dns:0 client_info= [ status = Established successful_connections = 1 pid = 43016 http_port = 8092 start_time = 1780897899171493 collector_name = cn-jenkins-deploy-platform-ansible-os-5902-1. collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 1017 calls = 2 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 115646 calls = 9 average_bytes = 12849 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 7 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 224 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 74 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 82 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 1 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 5 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 1 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 418, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 16, _iter59->first = sent _iter59->second = 11, ] ] ] 2026-06-08 Mon 05:53:39:570.144 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:53:39:570.160 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: Processing scm::EvSandeshSend in state Connect 2026-06-08 Mon 05:53:39:570.185 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: SANDESH: Send FAILED: 1780898019569148 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Established : EvTcpClose -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.38:9041, (*_iter6) = 10.0.0.50:9041, (*_iter6) = 10.0.0.33:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.38:5673, (*_iter6) = 10.0.0.50:5673, (*_iter6) = 10.0.0.33:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-08 Mon 05:53:39:570.200 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-06-08 Mon 05:53:39:570.212 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: Processing scm::EvTcpDeleteSession in state Connect 2026-06-08 Mon 05:53:39:570.248 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: Processing scm::EvTcpConnectFail in state Connect 2026-06-08 Mon 05:53:39:570.258 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: Connect : EvTcpConnectFail 2026-06-08 Mon 05:53:39:570.379 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2026-06-08 Mon 05:53:39:570.413 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: Idle 2026-06-08 Mon 05:53:39:570.485 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:53:39:570.758 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: SANDESH: Send FAILED: 1780898019569831 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 1 pid = 43016 http_port = 8092 start_time = 1780897899171493 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 1017 calls = 2 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 115646 calls = 9 average_bytes = 12849 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 7 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 224 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 74 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 82 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 1 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 5 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 1 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 418, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 16, _iter59->first = sent _iter59->second = 11, ] ] ] 2026-06-08 Mon 05:53:39:570.802 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:53:39:570.818 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:53:39:571.032 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: SANDESH: Send FAILED: 1780898019570320 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 1 pid = 43016 http_port = 8092 start_time = 1780897899171493 collector_name = collector_ip = 10.0.0.50:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 1017 calls = 2 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 115646 calls = 9 average_bytes = 12849 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 7 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 224 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 74 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 82 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 11 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 1 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 5 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 1 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 418, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 18, _iter59->first = sent _iter59->second = 11, ] ] ] 2026-06-08 Mon 05:53:39:571.059 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:53:39:571.072 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:53:39:571.097 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: SANDESH: Send FAILED: 1780898019570351 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.38:9041, (*_iter6) = 10.0.0.50:9041, (*_iter6) = 10.0.0.33:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.38:5673, (*_iter6) = 10.0.0.50:5673, (*_iter6) = 10.0.0.33:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-08 Mon 05:53:39:571.106 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-06-08 Mon 05:53:39:571.117 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: Processing scm::EvTcpDeleteSession in state Idle 2026-06-08 Mon 05:53:39:571.144 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:53:39:571.342 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: SANDESH: Send FAILED: 1780898019570468 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3:Control:contrail-dns:0 client_info= [ status = Idle successful_connections = 1 pid = 43016 http_port = 8092 start_time = 1780897899171493 collector_name = collector_ip = 10.0.0.50:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 1017 calls = 2 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 115646 calls = 9 average_bytes = 12849 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 7 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 224 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 74 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 82 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 11 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 1 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 1 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 419, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 18, _iter59->first = sent _iter59->second = 11, ] ] ] 2026-06-08 Mon 05:53:39:571.369 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:53:40:570.519 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: Processing scm::EvIdleHoldTimerExpired in state Idle 2026-06-08 Mon 05:53:40:570.740 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: Connect : Start Connect timer 10.0.0.50:8086 2026-06-08 Mon 05:53:40:570.852 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: Processing scm::EvSandeshSend in state Connect 2026-06-08 Mon 05:53:40:570.874 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: SANDESH: Send FAILED: 1780898020570589 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.50:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.38:9041, (*_iter6) = 10.0.0.50:9041, (*_iter6) = 10.0.0.33:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.38:5673, (*_iter6) = 10.0.0.50:5673, (*_iter6) = 10.0.0.33:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-08 Mon 05:53:40:570.882 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-06-08 Mon 05:53:40:570.890 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: Processing scm::EvSandeshSend in state Connect 2026-06-08 Mon 05:53:40:571.028 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: SANDESH: Send FAILED: 1780898020570837 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 1 pid = 43016 http_port = 8092 start_time = 1780897899171493 collector_name = collector_ip = 10.0.0.50:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 1017 calls = 2 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 115646 calls = 9 average_bytes = 12849 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 7 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 224 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 74 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 82 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 1 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 1 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 419, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 22, _iter59->first = sent _iter59->second = 11, ] ] ] 2026-06-08 Mon 05:53:40:571.049 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:53:40:572.452 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625491276864, Pid 43016]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.33:47978::10.0.0.50:8086(-1) < Active session connection complete src/contrail-common/io/tcp_session.cc 281 2026-06-08 Mon 05:53:40:572.482 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625491276864, Pid 43016]: 10.0.0.33:47978::10.0.0.50:8086(-1) OnSessionEvent TCP Connected 2026-06-08 Mon 05:53:40:572.490 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625491276864, Pid 43016]: 10.0.0.33:47978::10.0.0.50:8086(-1):EvTcpConnected 2026-06-08 Mon 05:53:40:572.553 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: Processing scm::EvTcpConnected in state Connect 2026-06-08 Mon 05:53:40:572.586 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: Connect : EvTcpConnected : Cancelling Connect timer 2026-06-08 Mon 05:53:40:572.634 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: ClientInit 2026-06-08 Mon 05:53:40:572.655 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: Sending Ctrl Message for cn-jenkins-deploy-platform-ansible-os-5902-3:contrail-dns:0:Control count 2 2026-06-08 Mon 05:53:40:572.778 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: Processing scm::EvSandeshSend in state ClientInit 2026-06-08 Mon 05:53:40:572.791 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: ClientInit : EvSandeshSend : NodeStatusUVE 2026-06-08 Mon 05:53:40:572.814 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: SANDESH: Send FAILED: 1780898020572625 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.50:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.38:9041, (*_iter6) = 10.0.0.50:9041, (*_iter6) = 10.0.0.33:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.38:5673, (*_iter6) = 10.0.0.50:5673, (*_iter6) = 10.0.0.33:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-08 Mon 05:53:40:572.833 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: Received UVE message in wrong state : NodeStatusUVE 2026-06-08 Mon 05:53:40:572.842 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: Processing scm::EvSandeshSend in state ClientInit 2026-06-08 Mon 05:53:40:572.848 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: ClientInit : EvSandeshSend : SandeshCtrlClientToServer 2026-06-08 Mon 05:53:40:572.861 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: Processing scm::EvSandeshSend in state ClientInit 2026-06-08 Mon 05:53:40:572.867 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2026-06-08 Mon 05:53:40:573.035 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: SANDESH: Send FAILED: 1780898020572760 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3:Control:contrail-dns:0 client_info= [ status = ClientInit successful_connections = 2 pid = 43016 http_port = 8092 start_time = 1780897899171493 collector_name = collector_ip = 10.0.0.50:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 1017 calls = 2 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 115646 calls = 9 average_bytes = 12849 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 7 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 224 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 74 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 82 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 13 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 1 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 11 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 7 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 1 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 420, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 24, _iter59->first = sent _iter59->second = 11, ] ] ] 2026-06-08 Mon 05:53:40:573.076 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: Received UVE message in wrong state : SandeshModuleClientTrace 2026-06-08 Mon 05:53:40:581.159 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625491276864, Pid 43016]: 10.0.0.33:47978::10.0.0.50:8086(-1) OnSessionEvent TCP Connection Closed 2026-06-08 Mon 05:53:40:581.214 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625491276864, Pid 43016]: 10.0.0.33:47978::10.0.0.50:8086(-1):EvTcpClose 2026-06-08 Mon 05:53:40:581.291 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: Processing scm::EvTcpClose in state ClientInit 2026-06-08 Mon 05:53:40:581.324 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: ClientInit : EvTcpClose 2026-06-08 Mon 05:53:40:581.454 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: Return to idle with 1000 2026-06-08 Mon 05:53:40:581.463 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: Idle 2026-06-08 Mon 05:53:40:581.510 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:53:40:581.636 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: SANDESH: Send FAILED: 1780898020581409 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3:Control:contrail-dns:0 client_info= [ status = ClientInit successful_connections = 2 pid = 43016 http_port = 8092 start_time = 1780897899171493 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 1017 calls = 2 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 117256 calls = 10 average_bytes = 11725 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 7 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 224 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 74 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 82 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 7 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 1 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 420, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 26, _iter59->first = sent _iter59->second = 12, ] ] ] 2026-06-08 Mon 05:53:40:581.660 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:53:40:581.669 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:53:40:581.685 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: SANDESH: Send FAILED: 1780898020581442 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.50:8086, ] ] status = Down description = ClientInit : EvTcpClose ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.38:9041, (*_iter6) = 10.0.0.50:9041, (*_iter6) = 10.0.0.33:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.38:5673, (*_iter6) = 10.0.0.50:5673, (*_iter6) = 10.0.0.33:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-08 Mon 05:53:40:581.699 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-06-08 Mon 05:53:40:581.706 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: Processing scm::EvTcpDeleteSession in state Idle 2026-06-08 Mon 05:53:40:581.739 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:53:40:581.860 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: SANDESH: Send FAILED: 1780898020581499 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3:Control:contrail-dns:0 client_info= [ status = Idle successful_connections = 2 pid = 43016 http_port = 8092 start_time = 1780897899171493 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 1017 calls = 2 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 117256 calls = 10 average_bytes = 11725 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 7 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 224 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 74 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 82 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 7 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 1 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 420, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 26, _iter59->first = sent _iter59->second = 12, ] ] ] 2026-06-08 Mon 05:53:40:581.875 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:53:41:581.557 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625469933120, Pid 43016]: Processing scm::EvIdleHoldTimerExpired in state Idle 2026-06-08 Mon 05:53:41:581.797 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625469933120, Pid 43016]: Connect : Start Connect timer 10.0.0.38:8086 2026-06-08 Mon 05:53:41:581.919 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625469933120, Pid 43016]: Processing scm::EvSandeshSend in state Connect 2026-06-08 Mon 05:53:41:581.941 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625469933120, Pid 43016]: SANDESH: Send FAILED: 1780898021581627 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.38:9041, (*_iter6) = 10.0.0.50:9041, (*_iter6) = 10.0.0.33:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.38:5673, (*_iter6) = 10.0.0.50:5673, (*_iter6) = 10.0.0.33:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-08 Mon 05:53:41:581.956 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625469933120, Pid 43016]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-06-08 Mon 05:53:41:581.964 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625469933120, Pid 43016]: Processing scm::EvSandeshSend in state Connect 2026-06-08 Mon 05:53:41:582.110 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625469933120, Pid 43016]: SANDESH: Send FAILED: 1780898021581900 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 2 pid = 43016 http_port = 8092 start_time = 1780897899171493 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 1017 calls = 2 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 117256 calls = 10 average_bytes = 11725 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 7 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 224 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 74 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 82 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 15 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 7 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 1 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 420, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 29, _iter59->first = sent _iter59->second = 12, ] ] ] 2026-06-08 Mon 05:53:41:582.131 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625469933120, Pid 43016]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:53:41:583.615 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625491276864, Pid 43016]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.33:48140::10.0.0.38:8086(-1) < Active session connection complete src/contrail-common/io/tcp_session.cc 281 2026-06-08 Mon 05:53:41:583.640 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625491276864, Pid 43016]: 10.0.0.33:48140::10.0.0.38:8086(-1) OnSessionEvent TCP Connected 2026-06-08 Mon 05:53:41:583.647 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625491276864, Pid 43016]: 10.0.0.33:48140::10.0.0.38:8086(-1):EvTcpConnected 2026-06-08 Mon 05:53:41:583.690 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625474131520, Pid 43016]: Processing scm::EvTcpConnected in state Connect 2026-06-08 Mon 05:53:41:583.700 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625474131520, Pid 43016]: Connect : EvTcpConnected : Cancelling Connect timer 2026-06-08 Mon 05:53:41:583.729 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625474131520, Pid 43016]: ClientInit 2026-06-08 Mon 05:53:41:583.742 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625474131520, Pid 43016]: Sending Ctrl Message for cn-jenkins-deploy-platform-ansible-os-5902-3:contrail-dns:0:Control count 3 2026-06-08 Mon 05:53:41:583.818 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625474131520, Pid 43016]: Processing scm::EvSandeshSend in state ClientInit 2026-06-08 Mon 05:53:41:583.838 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625474131520, Pid 43016]: ClientInit : EvSandeshSend : NodeStatusUVE 2026-06-08 Mon 05:53:41:583.856 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625474131520, Pid 43016]: SANDESH: Send FAILED: 1780898021583724 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.38:9041, (*_iter6) = 10.0.0.50:9041, (*_iter6) = 10.0.0.33:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.38:5673, (*_iter6) = 10.0.0.50:5673, (*_iter6) = 10.0.0.33:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-08 Mon 05:53:41:583.868 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625474131520, Pid 43016]: Received UVE message in wrong state : NodeStatusUVE 2026-06-08 Mon 05:53:41:583.875 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625474131520, Pid 43016]: Processing scm::EvSandeshSend in state ClientInit 2026-06-08 Mon 05:53:41:583.881 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625474131520, Pid 43016]: ClientInit : EvSandeshSend : SandeshCtrlClientToServer 2026-06-08 Mon 05:53:41:583.902 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625474131520, Pid 43016]: Processing scm::EvSandeshSend in state ClientInit 2026-06-08 Mon 05:53:41:583.907 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625474131520, Pid 43016]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2026-06-08 Mon 05:53:41:584.034 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625474131520, Pid 43016]: SANDESH: Send FAILED: 1780898021583807 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3:Control:contrail-dns:0 client_info= [ status = ClientInit successful_connections = 3 pid = 43016 http_port = 8092 start_time = 1780897899171493 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 1017 calls = 2 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 117256 calls = 10 average_bytes = 11725 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 7 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 224 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 74 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 82 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 16 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 15 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 8 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 1 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 421, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 31, _iter59->first = sent _iter59->second = 12, ] ] ] 2026-06-08 Mon 05:53:41:584.053 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625474131520, Pid 43016]: Received UVE message in wrong state : SandeshModuleClientTrace 2026-06-08 Mon 05:53:42:872.010 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625491276864, Pid 43016]: 10.0.0.33:48140::10.0.0.38:8086(-1) OnSessionEvent TCP Connection Closed 2026-06-08 Mon 05:53:42:872.099 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625491276864, Pid 43016]: 10.0.0.33:48140::10.0.0.38:8086(-1):EvTcpClose 2026-06-08 Mon 05:53:42:872.201 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625432012352, Pid 43016]: Processing scm::EvTcpClose in state ClientInit 2026-06-08 Mon 05:53:42:872.244 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625432012352, Pid 43016]: ClientInit : EvTcpClose 2026-06-08 Mon 05:53:42:872.410 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625432012352, Pid 43016]: Return to idle with 1000 2026-06-08 Mon 05:53:42:872.420 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625432012352, Pid 43016]: Idle 2026-06-08 Mon 05:53:42:872.487 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625432012352, Pid 43016]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:53:42:872.668 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625432012352, Pid 43016]: SANDESH: Send FAILED: 1780898022872359 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3:Control:contrail-dns:0 client_info= [ status = ClientInit successful_connections = 3 pid = 43016 http_port = 8092 start_time = 1780897899171493 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 1017 calls = 2 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 118866 calls = 11 average_bytes = 10806 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 7 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 224 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 74 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 82 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 17 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 16 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 8 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 1 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 421, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 33, _iter59->first = sent _iter59->second = 13, ] ] ] 2026-06-08 Mon 05:53:42:872.693 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625432012352, Pid 43016]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:53:42:872.703 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625432012352, Pid 43016]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:53:42:872.722 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625432012352, Pid 43016]: SANDESH: Send FAILED: 1780898022872397 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Down description = ClientInit : EvTcpClose ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.38:9041, (*_iter6) = 10.0.0.50:9041, (*_iter6) = 10.0.0.33:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.38:5673, (*_iter6) = 10.0.0.50:5673, (*_iter6) = 10.0.0.33:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-08 Mon 05:53:42:872.728 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625432012352, Pid 43016]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-06-08 Mon 05:53:42:872.736 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625432012352, Pid 43016]: Processing scm::EvTcpDeleteSession in state Idle 2026-06-08 Mon 05:53:42:872.770 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625432012352, Pid 43016]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:53:42:872.933 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625432012352, Pid 43016]: SANDESH: Send FAILED: 1780898022872472 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3:Control:contrail-dns:0 client_info= [ status = Idle successful_connections = 3 pid = 43016 http_port = 8092 start_time = 1780897899171493 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 1017 calls = 2 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 118866 calls = 11 average_bytes = 10806 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 7 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 224 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 74 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 82 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 17 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 16 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 8 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 1 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 421, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 33, _iter59->first = sent _iter59->second = 13, ] ] ] 2026-06-08 Mon 05:53:42:872.952 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625432012352, Pid 43016]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:53:43:872.531 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625474131520, Pid 43016]: Processing scm::EvIdleHoldTimerExpired in state Idle 2026-06-08 Mon 05:53:43:872.797 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625474131520, Pid 43016]: Connect : Start Connect timer 10.0.0.33:8086 2026-06-08 Mon 05:53:43:872.943 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625474131520, Pid 43016]: Processing scm::EvSandeshSend in state Connect 2026-06-08 Mon 05:53:43:872.979 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625474131520, Pid 43016]: SANDESH: Send FAILED: 1780898023872602 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.38:9041, (*_iter6) = 10.0.0.50:9041, (*_iter6) = 10.0.0.33:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.38:5673, (*_iter6) = 10.0.0.50:5673, (*_iter6) = 10.0.0.33:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-08 Mon 05:53:43:872.990 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625474131520, Pid 43016]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-06-08 Mon 05:53:43:873.001 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625474131520, Pid 43016]: Processing scm::EvSandeshSend in state Connect 2026-06-08 Mon 05:53:43:873.257 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625474131520, Pid 43016]: SANDESH: Send FAILED: 1780898023872921 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 3 pid = 43016 http_port = 8092 start_time = 1780897899171493 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 1017 calls = 2 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 118866 calls = 11 average_bytes = 10806 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 7 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 224 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 74 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 82 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 18 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 18 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 8 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 1 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 421, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 36, _iter59->first = sent _iter59->second = 13, ] ] ] 2026-06-08 Mon 05:53:43:873.297 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625474131520, Pid 43016]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:53:43:874.555 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625491276864, Pid 43016]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.33:44272::10.0.0.33:8086(-1) < Active session connection complete src/contrail-common/io/tcp_session.cc 281 2026-06-08 Mon 05:53:43:874.578 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625491276864, Pid 43016]: 10.0.0.33:44272::10.0.0.33:8086(-1) OnSessionEvent TCP Connected 2026-06-08 Mon 05:53:43:874.585 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625491276864, Pid 43016]: 10.0.0.33:44272::10.0.0.33:8086(-1):EvTcpConnected 2026-06-08 Mon 05:53:43:874.633 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625465734720, Pid 43016]: Processing scm::EvTcpConnected in state Connect 2026-06-08 Mon 05:53:43:874.653 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625465734720, Pid 43016]: Connect : EvTcpConnected : Cancelling Connect timer 2026-06-08 Mon 05:53:43:874.693 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625465734720, Pid 43016]: ClientInit 2026-06-08 Mon 05:53:43:874.707 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625465734720, Pid 43016]: Sending Ctrl Message for cn-jenkins-deploy-platform-ansible-os-5902-3:contrail-dns:0:Control count 4 2026-06-08 Mon 05:53:43:874.840 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625465734720, Pid 43016]: Processing scm::EvSandeshSend in state ClientInit 2026-06-08 Mon 05:53:43:874.855 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625465734720, Pid 43016]: ClientInit : EvSandeshSend : NodeStatusUVE 2026-06-08 Mon 05:53:43:874.881 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625465734720, Pid 43016]: SANDESH: Send FAILED: 1780898023874686 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.38:9041, (*_iter6) = 10.0.0.50:9041, (*_iter6) = 10.0.0.33:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.38:5673, (*_iter6) = 10.0.0.50:5673, (*_iter6) = 10.0.0.33:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-08 Mon 05:53:43:874.892 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625465734720, Pid 43016]: Received UVE message in wrong state : NodeStatusUVE 2026-06-08 Mon 05:53:43:874.903 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625465734720, Pid 43016]: Processing scm::EvSandeshSend in state ClientInit 2026-06-08 Mon 05:53:43:874.912 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625465734720, Pid 43016]: ClientInit : EvSandeshSend : SandeshCtrlClientToServer 2026-06-08 Mon 05:53:43:874.926 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625465734720, Pid 43016]: Processing scm::EvSandeshSend in state ClientInit 2026-06-08 Mon 05:53:43:874.944 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625465734720, Pid 43016]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2026-06-08 Mon 05:53:43:875.148 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625465734720, Pid 43016]: SANDESH: Send FAILED: 1780898023874804 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3:Control:contrail-dns:0 client_info= [ status = ClientInit successful_connections = 4 pid = 43016 http_port = 8092 start_time = 1780897899171493 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 1017 calls = 2 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 118866 calls = 11 average_bytes = 10806 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 7 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 224 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 74 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 82 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 19 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 19 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 9 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 1 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 422, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 38, _iter59->first = sent _iter59->second = 13, ] ] ] 2026-06-08 Mon 05:53:43:875.183 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625465734720, Pid 43016]: Received UVE message in wrong state : SandeshModuleClientTrace 2026-06-08 Mon 05:53:43:883.130 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625491276864, Pid 43016]: 10.0.0.33:44272::10.0.0.33:8086(-1) OnSessionEvent TCP Connection Closed 2026-06-08 Mon 05:53:43:883.156 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625491276864, Pid 43016]: 10.0.0.33:44272::10.0.0.33:8086(-1):EvTcpClose 2026-06-08 Mon 05:53:43:883.213 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: Processing scm::EvTcpClose in state ClientInit 2026-06-08 Mon 05:53:43:883.238 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: ClientInit : EvTcpClose 2026-06-08 Mon 05:53:43:883.404 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: Return to idle with 1000 2026-06-08 Mon 05:53:43:883.417 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: Idle 2026-06-08 Mon 05:53:43:883.491 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:53:43:883.709 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: SANDESH: Send FAILED: 1780898023883343 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3:Control:contrail-dns:0 client_info= [ status = ClientInit successful_connections = 4 pid = 43016 http_port = 8092 start_time = 1780897899171493 collector_name = collector_ip = 10.0.0.50:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 1017 calls = 2 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 120476 calls = 12 average_bytes = 10039 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 7 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 224 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 74 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 82 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 20 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 4 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 20 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 9 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 1 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 422, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 40, _iter59->first = sent _iter59->second = 14, ] ] ] 2026-06-08 Mon 05:53:43:883.747 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:53:43:883.763 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:53:43:883.788 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: SANDESH: Send FAILED: 1780898023883389 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Down description = ClientInit : EvTcpClose ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.38:9041, (*_iter6) = 10.0.0.50:9041, (*_iter6) = 10.0.0.33:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.38:5673, (*_iter6) = 10.0.0.50:5673, (*_iter6) = 10.0.0.33:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-08 Mon 05:53:43:883.797 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-06-08 Mon 05:53:43:883.808 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: Processing scm::EvTcpDeleteSession in state Idle 2026-06-08 Mon 05:53:43:883.873 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:53:43:884.076 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: SANDESH: Send FAILED: 1780898023883475 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3:Control:contrail-dns:0 client_info= [ status = Idle successful_connections = 4 pid = 43016 http_port = 8092 start_time = 1780897899171493 collector_name = collector_ip = 10.0.0.50:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 1017 calls = 2 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 120476 calls = 12 average_bytes = 10039 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 7 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 224 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 74 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 82 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 20 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 4 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 20 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 9 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 1 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 422, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 40, _iter59->first = sent _iter59->second = 14, ] ] ] 2026-06-08 Mon 05:53:43:884.106 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:53:44:883.496 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625465734720, Pid 43016]: Processing scm::EvIdleHoldTimerExpired in state Idle 2026-06-08 Mon 05:53:44:883.738 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625465734720, Pid 43016]: Connect : Start Connect timer 10.0.0.50:8086 2026-06-08 Mon 05:53:44:883.849 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625465734720, Pid 43016]: Processing scm::EvSandeshSend in state Connect 2026-06-08 Mon 05:53:44:883.873 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625465734720, Pid 43016]: SANDESH: Send FAILED: 1780898024883570 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.50:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.38:9041, (*_iter6) = 10.0.0.50:9041, (*_iter6) = 10.0.0.33:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.38:5673, (*_iter6) = 10.0.0.50:5673, (*_iter6) = 10.0.0.33:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-08 Mon 05:53:44:883.881 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625465734720, Pid 43016]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-06-08 Mon 05:53:44:883.888 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625465734720, Pid 43016]: Processing scm::EvSandeshSend in state Connect 2026-06-08 Mon 05:53:44:884.032 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625465734720, Pid 43016]: SANDESH: Send FAILED: 1780898024883828 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 4 pid = 43016 http_port = 8092 start_time = 1780897899171493 collector_name = collector_ip = 10.0.0.50:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 1017 calls = 2 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 120476 calls = 12 average_bytes = 10039 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 7 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 224 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 74 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 82 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 21 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 4 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 22 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 9 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 1 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 422, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 43, _iter59->first = sent _iter59->second = 14, ] ] ] 2026-06-08 Mon 05:53:44:884.053 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625465734720, Pid 43016]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:53:44:885.920 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625491276864, Pid 43016]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.33:38498::10.0.0.50:8086(-1) < Active session connection complete src/contrail-common/io/tcp_session.cc 281 2026-06-08 Mon 05:53:44:885.941 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625491276864, Pid 43016]: 10.0.0.33:38498::10.0.0.50:8086(-1) OnSessionEvent TCP Connected 2026-06-08 Mon 05:53:44:885.949 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625491276864, Pid 43016]: 10.0.0.33:38498::10.0.0.50:8086(-1):EvTcpConnected 2026-06-08 Mon 05:53:44:886.012 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: Processing scm::EvTcpConnected in state Connect 2026-06-08 Mon 05:53:44:886.057 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: Connect : EvTcpConnected : Cancelling Connect timer 2026-06-08 Mon 05:53:44:886.093 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: ClientInit 2026-06-08 Mon 05:53:44:886.113 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: Sending Ctrl Message for cn-jenkins-deploy-platform-ansible-os-5902-3:contrail-dns:0:Control count 5 2026-06-08 Mon 05:53:44:886.227 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: Processing scm::EvSandeshSend in state ClientInit 2026-06-08 Mon 05:53:44:886.241 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: ClientInit : EvSandeshSend : NodeStatusUVE 2026-06-08 Mon 05:53:44:886.265 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: SANDESH: Send FAILED: 1780898024886087 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.50:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.38:9041, (*_iter6) = 10.0.0.50:9041, (*_iter6) = 10.0.0.33:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.38:5673, (*_iter6) = 10.0.0.50:5673, (*_iter6) = 10.0.0.33:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-08 Mon 05:53:44:886.274 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: Received UVE message in wrong state : NodeStatusUVE 2026-06-08 Mon 05:53:44:886.284 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: Processing scm::EvSandeshSend in state ClientInit 2026-06-08 Mon 05:53:44:886.292 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: ClientInit : EvSandeshSend : SandeshCtrlClientToServer 2026-06-08 Mon 05:53:44:886.306 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: Processing scm::EvSandeshSend in state ClientInit 2026-06-08 Mon 05:53:44:886.314 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2026-06-08 Mon 05:53:44:886.447 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: SANDESH: Send FAILED: 1780898024886205 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3:Control:contrail-dns:0 client_info= [ status = ClientInit successful_connections = 5 pid = 43016 http_port = 8092 start_time = 1780897899171493 collector_name = collector_ip = 10.0.0.50:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 1017 calls = 2 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 120476 calls = 12 average_bytes = 10039 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 7 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 224 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 74 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 82 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 22 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 4 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 23 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 1 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 423, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 45, _iter59->first = sent _iter59->second = 14, ] ] ] 2026-06-08 Mon 05:53:44:886.473 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: Received UVE message in wrong state : SandeshModuleClientTrace 2026-06-08 Mon 05:53:45:557.924 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: Processing scm::EvSandeshSend in state ClientInit 2026-06-08 Mon 05:53:45:557.985 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2026-06-08 Mon 05:53:45:558.092 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: SANDESH: Send FAILED: 1780898025557801 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3:Control:contrail-dns:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 4 dequeues = 4 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshMessageRecv enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 62 dequeues = 62 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpClose enqueues = 4 dequeues = 4 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnected enqueues = 5 dequeues = 5 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 5 dequeues = 5 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 84 dequeues = 84 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = ClientInit state_since = 1780898024886092 last_state = ClientInit last_event = scm::EvSandeshSend last_event_at = 1780898024886305 ] session_stats= [ num_recv_msg = 0 num_recv_msg_fail = 0 num_recv_fail = 0 num_send_msg = 1 num_send_msg_fail = 0 num_send_buffer_fail = 0 num_wait_msgq_enqueue = 0 num_wait_msgq_dequeue = 0 num_write_ready_cb_error = 0 ] session_rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] session_tx_socket_stats= [ bytes = 1610 calls = 1 average_bytes = 1610 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] max_sm_queue_count = 5 ] 2026-06-08 Mon 05:53:45:558.108 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: Received UVE message in wrong state : SandeshModuleClientTrace 2026-06-08 Mon 05:53:45:558.118 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: Processing scm::EvSandeshSend in state ClientInit 2026-06-08 Mon 05:53:45:558.124 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2026-06-08 Mon 05:53:45:558.265 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: SANDESH: Send FAILED: 1780898025557877 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3:Control:contrail-dns:0 client_info= [ status = ClientInit successful_connections = 5 pid = 43016 http_port = 8092 start_time = 1780897899171493 collector_name = collector_ip = 10.0.0.50:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 1017 calls = 2 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 122086 calls = 13 average_bytes = 9391 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 7 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 224 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 74 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 82 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 23 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 24 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 1 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 423, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 47, _iter59->first = sent _iter59->second = 15, ] ] ] 2026-06-08 Mon 05:53:45:558.294 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: Received UVE message in wrong state : SandeshModuleClientTrace 2026-06-08 Mon 05:53:51:300.730 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625491276864, Pid 43016]: TCP [SYS_ERR]: TcpSessionMessageLog: Session 10.0.0.33:38498::10.0.0.50:8086(-1) < Read failed due to error system 104 : Connection reset by peer src/contrail-common/io/tcp_session.cc 506 2026-06-08 Mon 05:53:51:300.787 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625491276864, Pid 43016]: TCP [SYS_ERR]: TcpSessionMessageLog: Session 10.0.0.33:38498::10.0.0.50:8086(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2026-06-08 Mon 05:53:51:300.815 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625491276864, Pid 43016]: 10.0.0.33:38498::10.0.0.50:8086(-1) OnSessionEvent TCP Connection Closed 2026-06-08 Mon 05:53:51:300.836 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625491276864, Pid 43016]: 10.0.0.33:38498::10.0.0.50:8086(-1):EvTcpClose 2026-06-08 Mon 05:53:51:300.912 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625440409152, Pid 43016]: Processing scm::EvTcpClose in state ClientInit 2026-06-08 Mon 05:53:51:300.946 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625440409152, Pid 43016]: ClientInit : EvTcpClose 2026-06-08 Mon 05:53:51:301.235 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625440409152, Pid 43016]: Return to idle with 1000 2026-06-08 Mon 05:53:51:301.254 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625440409152, Pid 43016]: Idle 2026-06-08 Mon 05:53:51:301.323 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625440409152, Pid 43016]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:53:51:301.494 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625440409152, Pid 43016]: SANDESH: Send FAILED: 1780898031301125 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3:Control:contrail-dns:0 client_info= [ status = ClientInit successful_connections = 5 pid = 43016 http_port = 8092 start_time = 1780897899171493 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 1017 calls = 2 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 122086 calls = 13 average_bytes = 9391 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 7 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 224 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 74 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 82 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 23 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 26 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 12 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 1 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 425, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 49, _iter59->first = sent _iter59->second = 15, ] ] ] 2026-06-08 Mon 05:53:51:301.537 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625440409152, Pid 43016]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:53:51:301.555 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625440409152, Pid 43016]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:53:51:301.573 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625440409152, Pid 43016]: SANDESH: Send FAILED: 1780898031301212 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.50:8086, ] ] status = Down description = ClientInit : EvTcpClose ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.38:9041, (*_iter6) = 10.0.0.50:9041, (*_iter6) = 10.0.0.33:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.38:5673, (*_iter6) = 10.0.0.50:5673, (*_iter6) = 10.0.0.33:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-08 Mon 05:53:51:301.592 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625440409152, Pid 43016]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-06-08 Mon 05:53:51:301.600 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625440409152, Pid 43016]: Processing scm::EvTcpDeleteSession in state Idle 2026-06-08 Mon 05:53:51:301.648 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625440409152, Pid 43016]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:53:51:301.766 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625440409152, Pid 43016]: SANDESH: Send FAILED: 1780898031301306 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3:Control:contrail-dns:0 client_info= [ status = Idle successful_connections = 5 pid = 43016 http_port = 8092 start_time = 1780897899171493 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 1017 calls = 2 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 122086 calls = 13 average_bytes = 9391 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 7 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 224 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 74 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 82 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 23 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 26 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 12 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 1 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 425, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 49, _iter59->first = sent _iter59->second = 15, ] ] ] 2026-06-08 Mon 05:53:51:301.783 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625440409152, Pid 43016]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:53:52:301.349 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625469933120, Pid 43016]: Processing scm::EvIdleHoldTimerExpired in state Idle 2026-06-08 Mon 05:53:52:301.576 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625469933120, Pid 43016]: Connect : Start Connect timer 10.0.0.38:8086 2026-06-08 Mon 05:53:52:301.674 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625469933120, Pid 43016]: Processing scm::EvSandeshSend in state Connect 2026-06-08 Mon 05:53:52:301.698 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625469933120, Pid 43016]: SANDESH: Send FAILED: 1780898032301431 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.38:9041, (*_iter6) = 10.0.0.50:9041, (*_iter6) = 10.0.0.33:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.38:5673, (*_iter6) = 10.0.0.50:5673, (*_iter6) = 10.0.0.33:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-08 Mon 05:53:52:301.714 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625469933120, Pid 43016]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-06-08 Mon 05:53:52:301.722 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625469933120, Pid 43016]: Processing scm::EvSandeshSend in state Connect 2026-06-08 Mon 05:53:52:301.813 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625491276864, Pid 43016]: -(-1) OnSessionEvent TCP Connect Failed 2026-06-08 Mon 05:53:52:301.851 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625491276864, Pid 43016]: -(-1):EvTcpConnectFail 2026-06-08 Mon 05:53:52:301.884 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625469933120, Pid 43016]: SANDESH: Send FAILED: 1780898032301660 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 5 pid = 43016 http_port = 8092 start_time = 1780897899171493 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 1017 calls = 2 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 122086 calls = 13 average_bytes = 9391 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 7 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 224 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 74 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 82 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 24 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 28 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 12 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 1 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 425, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 52, _iter59->first = sent _iter59->second = 15, ] ] ] 2026-06-08 Mon 05:53:52:301.904 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625469933120, Pid 43016]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:53:52:301.913 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625469933120, Pid 43016]: Processing scm::EvTcpConnectFail in state Connect 2026-06-08 Mon 05:53:52:301.920 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625469933120, Pid 43016]: Connect : EvTcpConnectFail 2026-06-08 Mon 05:53:52:302.000 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625469933120, Pid 43016]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2026-06-08 Mon 05:53:52:302.037 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625469933120, Pid 43016]: Idle 2026-06-08 Mon 05:53:52:302.105 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625469933120, Pid 43016]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:53:52:302.259 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625469933120, Pid 43016]: SANDESH: Send FAILED: 1780898032301960 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 5 pid = 43016 http_port = 8092 start_time = 1780897899171493 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 1017 calls = 2 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 122086 calls = 13 average_bytes = 9391 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 7 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 224 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 74 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 82 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 25 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 29 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 12 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 1 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 425, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 54, _iter59->first = sent _iter59->second = 15, ] ] ] 2026-06-08 Mon 05:53:52:302.284 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625469933120, Pid 43016]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:53:52:302.293 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625469933120, Pid 43016]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:53:52:302.309 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625469933120, Pid 43016]: SANDESH: Send FAILED: 1780898032301980 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.38:9041, (*_iter6) = 10.0.0.50:9041, (*_iter6) = 10.0.0.33:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.38:5673, (*_iter6) = 10.0.0.50:5673, (*_iter6) = 10.0.0.33:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-08 Mon 05:53:52:302.315 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625469933120, Pid 43016]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-06-08 Mon 05:53:52:302.322 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625469933120, Pid 43016]: Processing scm::EvTcpDeleteSession in state Idle 2026-06-08 Mon 05:53:52:302.345 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625469933120, Pid 43016]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:53:52:302.462 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625469933120, Pid 43016]: SANDESH: Send FAILED: 1780898032302090 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3:Control:contrail-dns:0 client_info= [ status = Idle successful_connections = 5 pid = 43016 http_port = 8092 start_time = 1780897899171493 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 1017 calls = 2 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 122086 calls = 13 average_bytes = 9391 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 7 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 224 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 74 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 82 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 25 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 29 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 13 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 1 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 426, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 54, _iter59->first = sent _iter59->second = 15, ] ] ] 2026-06-08 Mon 05:53:52:302.480 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625469933120, Pid 43016]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:54:15:558.227 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:54:15:558.334 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: SANDESH: Send FAILED: 1780898055558040 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3:Control:contrail-dns:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 5 dequeues = 5 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshMessageRecv enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 72 dequeues = 72 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpClose enqueues = 5 dequeues = 5 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 2 dequeues = 2 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnected enqueues = 5 dequeues = 5 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 7 dequeues = 7 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 99 dequeues = 99 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1780898032302036 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1780898032302344 ] max_sm_queue_count = 5 ] 2026-06-08 Mon 05:54:15:558.351 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:54:15:558.363 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:54:15:558.601 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: SANDESH: Send FAILED: 1780898055558192 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3:Control:contrail-dns:0 client_info= [ status = Idle successful_connections = 5 pid = 43016 http_port = 8092 start_time = 1780897899171493 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 1017 calls = 2 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 122086 calls = 13 average_bytes = 9391 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 7 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 224 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 74 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 82 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 26 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 31 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 13 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 1 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 426, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 57, _iter59->first = sent _iter59->second = 15, ] ] ] 2026-06-08 Mon 05:54:15:558.641 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:54:22:302.131 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: Processing scm::EvIdleHoldTimerExpired in state Idle 2026-06-08 Mon 05:54:22:302.432 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: Connect : Start Connect timer 10.0.0.33:8086 2026-06-08 Mon 05:54:22:302.554 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: Processing scm::EvSandeshSend in state Connect 2026-06-08 Mon 05:54:22:302.582 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: SANDESH: Send FAILED: 1780898062302213 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.33:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.38:9041, (*_iter6) = 10.0.0.50:9041, (*_iter6) = 10.0.0.33:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.38:5673, (*_iter6) = 10.0.0.50:5673, (*_iter6) = 10.0.0.33:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-08 Mon 05:54:22:302.586 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625491276864, Pid 43016]: -(-1) OnSessionEvent TCP Connect Failed 2026-06-08 Mon 05:54:22:302.646 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625491276864, Pid 43016]: -(-1):EvTcpConnectFail 2026-06-08 Mon 05:54:22:302.591 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-06-08 Mon 05:54:22:302.743 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: Processing scm::EvSandeshSend in state Connect 2026-06-08 Mon 05:54:22:302.955 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: SANDESH: Send FAILED: 1780898062302531 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 5 pid = 43016 http_port = 8092 start_time = 1780897899171493 collector_name = collector_ip = 10.0.0.33:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 1017 calls = 2 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 122086 calls = 13 average_bytes = 9391 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 7 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 224 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 74 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 82 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 26 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 33 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 13 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 1 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 426, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 59, _iter59->first = sent _iter59->second = 15, ] ] ] 2026-06-08 Mon 05:54:22:303.052 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:54:22:303.069 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: Processing scm::EvTcpConnectFail in state Connect 2026-06-08 Mon 05:54:22:303.077 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: Connect : EvTcpConnectFail 2026-06-08 Mon 05:54:22:303.214 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2026-06-08 Mon 05:54:22:303.265 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: Idle 2026-06-08 Mon 05:54:22:303.326 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:54:22:303.511 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: SANDESH: Send FAILED: 1780898062303152 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 5 pid = 43016 http_port = 8092 start_time = 1780897899171493 collector_name = collector_ip = 10.0.0.50:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 1017 calls = 2 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 122086 calls = 13 average_bytes = 9391 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 7 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 224 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 74 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 82 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 27 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 34 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 13 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 1 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 426, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 61, _iter59->first = sent _iter59->second = 15, ] ] ] 2026-06-08 Mon 05:54:22:303.535 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:54:22:303.546 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:54:22:303.570 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: SANDESH: Send FAILED: 1780898062303186 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.38:9041, (*_iter6) = 10.0.0.50:9041, (*_iter6) = 10.0.0.33:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.38:5673, (*_iter6) = 10.0.0.50:5673, (*_iter6) = 10.0.0.33:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-08 Mon 05:54:22:303.581 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-06-08 Mon 05:54:22:303.592 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: Processing scm::EvTcpDeleteSession in state Idle 2026-06-08 Mon 05:54:22:304.127 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:54:22:304.338 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: SANDESH: Send FAILED: 1780898062303310 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3:Control:contrail-dns:0 client_info= [ status = Idle successful_connections = 5 pid = 43016 http_port = 8092 start_time = 1780897899171493 collector_name = collector_ip = 10.0.0.50:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 1017 calls = 2 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 122086 calls = 13 average_bytes = 9391 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 7 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 224 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 74 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 82 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 27 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 34 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 1 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 427, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 61, _iter59->first = sent _iter59->second = 15, ] ] ] 2026-06-08 Mon 05:54:22:304.372 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:54:45:448.381 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:54:45:559.022 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:54:45:559.091 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: SANDESH: Send FAILED: 1780898085558914 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3:Control:contrail-dns:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 6 dequeues = 6 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshMessageRecv enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 79 dequeues = 79 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpClose enqueues = 5 dequeues = 5 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 3 dequeues = 3 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnected enqueues = 5 dequeues = 5 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 8 dequeues = 8 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 109 dequeues = 109 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1780898062303264 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1780898062304120 ] max_sm_queue_count = 5 ] 2026-06-08 Mon 05:54:45:559.103 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:54:45:559.111 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:54:45:559.245 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: SANDESH: Send FAILED: 1780898085558997 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3:Control:contrail-dns:0 client_info= [ status = Idle successful_connections = 5 pid = 43016 http_port = 8092 start_time = 1780897899171493 collector_name = collector_ip = 10.0.0.50:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 1017 calls = 2 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 122086 calls = 13 average_bytes = 9391 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 7 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 225 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 74 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 82 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 28 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 36 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 1 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 428, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 64, _iter59->first = sent _iter59->second = 15, ] ] ] 2026-06-08 Mon 05:54:45:559.261 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:54:45:761.778 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:54:45:995.663 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625440409152, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:54:46:308.443 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:54:46:336.357 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:54:46:673.119 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625440409152, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:54:46:771.600 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625444607552, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:54:46:786.554 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625436210752, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:54:46:972.087 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:54:52:303.406 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625469933120, Pid 43016]: Processing scm::EvIdleHoldTimerExpired in state Idle 2026-06-08 Mon 05:54:52:303.708 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625469933120, Pid 43016]: Connect : Start Connect timer 10.0.0.50:8086 2026-06-08 Mon 05:54:52:303.805 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625469933120, Pid 43016]: Processing scm::EvSandeshSend in state Connect 2026-06-08 Mon 05:54:52:303.854 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625469933120, Pid 43016]: SANDESH: Send FAILED: 1780898092303526 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.50:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.38:9041, (*_iter6) = 10.0.0.50:9041, (*_iter6) = 10.0.0.33:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.38:5673, (*_iter6) = 10.0.0.50:5673, (*_iter6) = 10.0.0.33:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-08 Mon 05:54:52:303.862 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625469933120, Pid 43016]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-06-08 Mon 05:54:52:303.870 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625469933120, Pid 43016]: Processing scm::EvSandeshSend in state Connect 2026-06-08 Mon 05:54:52:303.994 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625469933120, Pid 43016]: SANDESH: Send FAILED: 1780898092303791 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 5 pid = 43016 http_port = 8092 start_time = 1780897899171493 collector_name = collector_ip = 10.0.0.50:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 1017 calls = 2 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 122086 calls = 13 average_bytes = 9391 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 7 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 233 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 74 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 82 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 28 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 38 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 1 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 436, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 66, _iter59->first = sent _iter59->second = 15, ] ] ] 2026-06-08 Mon 05:54:52:304.018 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625469933120, Pid 43016]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:54:52:304.058 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625491276864, Pid 43016]: -(-1) OnSessionEvent TCP Connect Failed 2026-06-08 Mon 05:54:52:304.089 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625491276864, Pid 43016]: -(-1):EvTcpConnectFail 2026-06-08 Mon 05:54:52:304.132 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625474131520, Pid 43016]: Processing scm::EvTcpConnectFail in state Connect 2026-06-08 Mon 05:54:52:304.143 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625474131520, Pid 43016]: Connect : EvTcpConnectFail 2026-06-08 Mon 05:54:52:304.235 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625474131520, Pid 43016]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2026-06-08 Mon 05:54:52:304.271 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625474131520, Pid 43016]: Idle 2026-06-08 Mon 05:54:52:304.328 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625474131520, Pid 43016]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:54:52:304.458 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625474131520, Pid 43016]: SANDESH: Send FAILED: 1780898092304197 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 5 pid = 43016 http_port = 8092 start_time = 1780897899171493 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 1017 calls = 2 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 122086 calls = 13 average_bytes = 9391 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 7 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 233 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 74 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 82 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 29 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 39 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 1 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 436, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 68, _iter59->first = sent _iter59->second = 15, ] ] ] 2026-06-08 Mon 05:54:52:304.475 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625474131520, Pid 43016]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:54:52:304.484 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625474131520, Pid 43016]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:54:52:304.499 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625474131520, Pid 43016]: SANDESH: Send FAILED: 1780898092304218 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.38:9041, (*_iter6) = 10.0.0.50:9041, (*_iter6) = 10.0.0.33:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.38:5673, (*_iter6) = 10.0.0.50:5673, (*_iter6) = 10.0.0.33:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-08 Mon 05:54:52:304.504 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625474131520, Pid 43016]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2026-06-08 Mon 05:54:52:304.523 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625474131520, Pid 43016]: Processing scm::EvTcpDeleteSession in state Idle 2026-06-08 Mon 05:54:52:304.549 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625474131520, Pid 43016]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:54:52:304.656 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625474131520, Pid 43016]: SANDESH: Send FAILED: 1780898092304316 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3:Control:contrail-dns:0 client_info= [ status = Idle successful_connections = 5 pid = 43016 http_port = 8092 start_time = 1780897899171493 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 1017 calls = 2 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 122086 calls = 13 average_bytes = 9391 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 7 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 233 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 74 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 82 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 29 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 39 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 15 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 1 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 437, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 68, _iter59->first = sent _iter59->second = 15, ] ] ] 2026-06-08 Mon 05:54:52:304.676 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625474131520, Pid 43016]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:54:52:398.300 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625474131520, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:54:57:559.497 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:54:57:841.356 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625444607552, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:54:58:150.577 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625491276864, Pid 43016]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.14:8092::10.20.0.17:38974 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2026-06-08 Mon 05:54:58:478.358 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625491276864, Pid 43016]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.14:8092::10.20.0.254:47938 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2026-06-08 Mon 05:54:58:499.588 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625491276864, Pid 43016]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.14:8092::10.20.0.14:37012 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2026-06-08 Mon 05:54:58:739.771 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625474131520, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:54:58:846.708 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:54:58:916.529 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625432012352, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:54:59:016.422 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:54:59:086.991 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625448805952, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:54:59:112.419 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625448805952, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:54:59:200.633 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625474131520, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:54:59:251.916 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625448805952, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:54:59:364.008 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:54:59:429.739 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625448805952, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:54:59:542.620 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625436210752, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:54:59:609.738 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625444607552, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:54:59:719.184 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:54:59:777.770 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:54:59:892.376 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625436210752, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:54:59:932.289 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:54:59:953.750 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625469933120, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:55:00:095.694 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625436210752, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:55:00:127.734 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625436210752, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:55:00:141.055 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:55:00:293.142 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:55:00:318.036 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625474131520, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:55:00:339.872 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625436210752, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:55:00:482.726 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625440409152, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:55:00:485.565 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625440409152, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:55:00:528.783 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:55:00:692.326 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625436210752, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:55:00:719.642 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625448805952, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:55:00:738.153 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625432012352, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:55:00:885.503 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625432012352, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:55:00:909.277 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625474131520, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:55:00:928.562 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625469933120, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:55:01:071.482 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625448805952, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:55:01:093.756 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625469933120, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:55:01:113.514 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:55:01:284.421 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625469933120, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:55:01:297.558 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625432012352, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:55:01:311.629 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625440409152, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:55:01:464.265 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625444607552, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:55:01:472.979 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625432012352, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:55:01:491.275 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:55:01:649.325 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:55:01:678.314 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625469933120, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:55:01:707.087 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625440409152, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:55:01:831.601 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625465734720, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:55:01:856.471 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625432012352, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:55:01:889.317 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625440409152, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:55:02:000.108 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625444607552, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:55:02:039.962 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625440409152, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:55:02:068.668 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625448805952, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:55:02:185.411 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625432012352, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:55:02:227.220 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625440409152, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:55:02:276.390 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625444607552, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:55:02:379.978 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:55:02:419.789 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625432012352, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:55:02:489.802 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:55:02:561.972 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625444607552, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:55:02:604.077 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625432012352, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:55:02:642.515 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625469933120, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:55:02:803.940 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625474131520, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:55:02:847.795 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625432012352, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:55:02:882.664 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:55:02:987.539 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:55:03:037.131 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625457337920, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:55:03:212.598 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625432012352, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:55:03:420.156 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625436210752, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:55:03:583.073 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:55:03:759.439 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625461536320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:55:03:948.866 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:55:04:130.059 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625444607552, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:55:04:155.870 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:55:04:177.597 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625448805952, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:55:05:487.665 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625457337920, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:55:05:510.547 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:55:05:602.958 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625448805952, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:55:06:909.437 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:55:06:971.435 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625448805952, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:55:07:026.861 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625474131520, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:55:07:726.949 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625465734720, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:55:15:559.267 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625440409152, Pid 43016]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:55:15:559.369 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625440409152, Pid 43016]: SANDESH: Send FAILED: 1780898115559146 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3:Control:contrail-dns:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 7 dequeues = 7 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshMessageRecv enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 86 dequeues = 86 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpClose enqueues = 5 dequeues = 5 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 4 dequeues = 4 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnected enqueues = 5 dequeues = 5 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 9 dequeues = 9 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 119 dequeues = 119 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1780898092304270 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1780898092304548 ] max_sm_queue_count = 5 ] 2026-06-08 Mon 05:55:15:559.384 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625440409152, Pid 43016]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:55:15:559.395 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625440409152, Pid 43016]: Processing scm::EvSandeshSend in state Idle 2026-06-08 Mon 05:55:15:559.569 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625440409152, Pid 43016]: SANDESH: Send FAILED: 1780898115559241 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3:Control:contrail-dns:0 client_info= [ status = Idle successful_connections = 5 pid = 43016 http_port = 8092 start_time = 1780897899171493 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 1017 calls = 2 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 122086 calls = 13 average_bytes = 9391 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 7 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 315 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 74 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 82 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 30 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 41 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 18 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 1 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 522, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 71, _iter59->first = sent _iter59->second = 15, ] ] ] 2026-06-08 Mon 05:55:15:559.596 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625440409152, Pid 43016]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:55:22:304.541 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625436210752, Pid 43016]: Processing scm::EvIdleHoldTimerExpired in state Idle 2026-06-08 Mon 05:55:22:304.825 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625436210752, Pid 43016]: Connect : Start Connect timer 10.0.0.38:8086 2026-06-08 Mon 05:55:22:304.943 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625436210752, Pid 43016]: Processing scm::EvSandeshSend in state Connect 2026-06-08 Mon 05:55:22:304.967 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625436210752, Pid 43016]: SANDESH: Send FAILED: 1780898122304609 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.38:9041, (*_iter6) = 10.0.0.50:9041, (*_iter6) = 10.0.0.33:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.38:5673, (*_iter6) = 10.0.0.50:5673, (*_iter6) = 10.0.0.33:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-08 Mon 05:55:22:304.975 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625436210752, Pid 43016]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-06-08 Mon 05:55:22:304.983 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625436210752, Pid 43016]: Processing scm::EvSandeshSend in state Connect 2026-06-08 Mon 05:55:22:305.136 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625436210752, Pid 43016]: SANDESH: Send FAILED: 1780898122304922 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 5 pid = 43016 http_port = 8092 start_time = 1780897899171493 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 1017 calls = 2 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 122086 calls = 13 average_bytes = 9391 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 7 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 315 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 74 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 82 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 30 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 43 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 18 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 1 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 522, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 73, _iter59->first = sent _iter59->second = 15, ] ] ] 2026-06-08 Mon 05:55:22:305.186 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625436210752, Pid 43016]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:55:22:307.168 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625491276864, Pid 43016]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.33:60856::10.0.0.38:8086(-1) < Active session connection complete src/contrail-common/io/tcp_session.cc 281 2026-06-08 Mon 05:55:22:307.198 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625491276864, Pid 43016]: 10.0.0.33:60856::10.0.0.38:8086(-1) OnSessionEvent TCP Connected 2026-06-08 Mon 05:55:22:307.207 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625491276864, Pid 43016]: 10.0.0.33:60856::10.0.0.38:8086(-1):EvTcpConnected 2026-06-08 Mon 05:55:22:307.240 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625457337920, Pid 43016]: Processing scm::EvTcpConnected in state Connect 2026-06-08 Mon 05:55:22:307.249 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625457337920, Pid 43016]: Connect : EvTcpConnected : Cancelling Connect timer 2026-06-08 Mon 05:55:22:307.279 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625457337920, Pid 43016]: ClientInit 2026-06-08 Mon 05:55:22:307.294 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625457337920, Pid 43016]: Sending Ctrl Message for cn-jenkins-deploy-platform-ansible-os-5902-3:contrail-dns:0:Control count 6 2026-06-08 Mon 05:55:22:307.373 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625457337920, Pid 43016]: Processing scm::EvSandeshSend in state ClientInit 2026-06-08 Mon 05:55:22:307.381 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625457337920, Pid 43016]: ClientInit : EvSandeshSend : NodeStatusUVE 2026-06-08 Mon 05:55:22:307.397 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625457337920, Pid 43016]: SANDESH: Send FAILED: 1780898122307274 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.38:9041, (*_iter6) = 10.0.0.50:9041, (*_iter6) = 10.0.0.33:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.38:5673, (*_iter6) = 10.0.0.50:5673, (*_iter6) = 10.0.0.33:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-08 Mon 05:55:22:307.404 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625457337920, Pid 43016]: Received UVE message in wrong state : NodeStatusUVE 2026-06-08 Mon 05:55:22:307.410 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625457337920, Pid 43016]: Processing scm::EvSandeshSend in state ClientInit 2026-06-08 Mon 05:55:22:307.416 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625457337920, Pid 43016]: ClientInit : EvSandeshSend : SandeshCtrlClientToServer 2026-06-08 Mon 05:55:22:307.426 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625457337920, Pid 43016]: Processing scm::EvSandeshSend in state ClientInit 2026-06-08 Mon 05:55:22:307.431 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625457337920, Pid 43016]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2026-06-08 Mon 05:55:22:307.562 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625457337920, Pid 43016]: SANDESH: Send FAILED: 1780898122307361 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3:Control:contrail-dns:0 client_info= [ status = ClientInit successful_connections = 6 pid = 43016 http_port = 8092 start_time = 1780897899171493 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 1017 calls = 2 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 122086 calls = 13 average_bytes = 9391 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 7 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 315 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 74 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 82 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 31 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 44 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 19 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 1 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 523, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 75, _iter59->first = sent _iter59->second = 15, ] ] ] 2026-06-08 Mon 05:55:22:307.597 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625457337920, Pid 43016]: Received UVE message in wrong state : SandeshModuleClientTrace 2026-06-08 Mon 05:55:22:343.558 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625474131520, Pid 43016]: OnMessage control in state: ClientInit 2026-06-08 Mon 05:55:22:346.316 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625474131520, Pid 43016]: Processing scm::EvSandeshMessageRecv in state ClientInit 2026-06-08 Mon 05:55:22:346.334 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625474131520, Pid 43016]: ClientInit : EvSandeshMessageRecv 2026-06-08 Mon 05:55:22:346.383 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625474131520, Pid 43016]: Received Ctrl Message with size 0 2026-06-08 Mon 05:55:22:346.402 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625474131520, Pid 43016]: SyncUVE Syncing ObjectDns val name = cn-jenkins-deploy-platform-ansible-os-5902-3 start_time = 1780897899628794 build_info = {"build-info": [{"build-version": "v0+nightly.r25.1", "build-time": "2026-05-28 00:08:01.734303", "build-user": "root", "build-hostname": "jenkins-build-prepare-sandbox-15063-1.novalocal", "build-id": "v0+nightly.r25.1", "build-number": "202605280007"}]} collector = 10.0.0.38:8086 self_ip_list= [ [ (*_iter12) = 10.20.0.14, ] ] proxy seq 1 2026-06-08 Mon 05:55:22:346.414 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625474131520, Pid 43016]: SyncAllMaps for DnsState:0 period 0 cycle 0 without seqno , total = 1 2026-06-08 Mon 05:55:22:350.836 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625474131520, Pid 43016]: SyncUVE Syncing ObjectGeneratorInfo val name = cn-jenkins-deploy-platform-ansible-os-5902-3:Control:contrail-dns:0 client_info= [ status = ClientInit successful_connections = 6 pid = 43016 http_port = 8092 start_time = 1780897899171493 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 1017 calls = 2 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 122086 calls = 13 average_bytes = 9391 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 7 dequeues = 7 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshMessageRecv enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 86 dequeues = 86 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpClose enqueues = 5 dequeues = 5 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 4 dequeues = 4 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnected enqueues = 5 dequeues = 5 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 9 dequeues = 9 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 119 dequeues = 119 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1780898092304270 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1780898092304548 ] session_stats= [ num_recv_msg = 0 num_recv_msg_fail = 0 num_recv_fail = 0 num_send_msg = 1 num_send_msg_fail = 0 num_send_buffer_fail = 0 num_wait_msgq_enqueue = 0 num_wait_msgq_dequeue = 0 num_write_ready_cb_error = 0 ] session_rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] session_tx_socket_stats= [ bytes = 1610 calls = 1 average_bytes = 1610 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 7 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 315 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 74 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 82 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 31 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 44 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 19 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 1 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 523, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 75, _iter59->first = sent _iter59->second = 15, ] ] max_sm_queue_count = 5 proxy seq 51 2026-06-08 Mon 05:55:22:350.903 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625474131520, Pid 43016]: SyncAllMaps for ModuleClientState:0 period 0 cycle 0 without seqno , total = 1 2026-06-08 Mon 05:55:22:350.927 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625474131520, Pid 43016]: SyncUVE Syncing ObjectDns val name = cn-jenkins-deploy-platform-ansible-os-5902-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.38:9041, (*_iter6) = 10.0.0.50:9041, (*_iter6) = 10.0.0.33:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.38:5673, (*_iter6) = 10.0.0.50:5673, (*_iter6) = 10.0.0.33:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] proxy seq 34 2026-06-08 Mon 05:55:22:350.938 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625474131520, Pid 43016]: SyncAllMaps for NodeStatus:0 period 0 cycle 0 without seqno , total = 1 2026-06-08 Mon 05:55:22:350.945 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625474131520, Pid 43016]: SyncAllMaps for PeerFlapData:0 period 0 cycle 0 without seqno , total = 0 2026-06-08 Mon 05:55:22:350.951 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625474131520, Pid 43016]: SyncAllMaps for PeerStatsData:0 period 0 cycle 0 without seqno , total = 0 2026-06-08 Mon 05:55:22:350.958 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625474131520, Pid 43016]: SyncAllMaps for XmppPeerInfoData:0 period 0 cycle 0 without seqno , total = 0 2026-06-08 Mon 05:55:22:350.993 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625474131520, Pid 43016]: Established 2026-06-08 Mon 05:55:26:635.794 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625491276864, Pid 43016]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.14:8092::10.20.0.14:54030 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2026-06-08 Mon 05:55:27:094.619 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625491276864, Pid 43016]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.14:8092::10.20.0.254:36512 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2026-06-08 Mon 05:55:27:585.962 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625491276864, Pid 43016]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.14:8092::10.20.0.17:53630 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2026-06-08 Mon 05:55:28:110.189 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: OnMessage control in state: Established 2026-06-08 Mon 05:55:28:110.249 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: Processing scm::EvSandeshMessageRecv in state Established 2026-06-08 Mon 05:55:28:110.257 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: Established : EvSandeshMessageRecv 2026-06-08 Mon 05:55:28:110.282 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: Received Ctrl Message with size 3 2026-06-08 Mon 05:55:28:110.297 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: SyncAllMaps for DnsState:0 period 0 cycle 0 with seqno 1, total = 0 2026-06-08 Mon 05:55:28:110.306 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: SyncAllMaps for ModuleClientState:0 period 0 cycle 0 with seqno 52, total = 0 2026-06-08 Mon 05:55:28:110.316 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: SyncAllMaps for NodeStatus:0 period 0 cycle 0 with seqno 36, total = 0 2026-06-08 Mon 05:55:28:110.323 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: SyncAllMaps for PeerFlapData:0 period 0 cycle 0 without seqno , total = 0 2026-06-08 Mon 05:55:28:110.329 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: SyncAllMaps for PeerStatsData:0 period 0 cycle 0 without seqno , total = 0 2026-06-08 Mon 05:55:28:110.335 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: SyncAllMaps for XmppPeerInfoData:0 period 0 cycle 0 without seqno , total = 0 2026-06-08 Mon 05:55:41:234.890 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625491276864, Pid 43016]: XMPP [SYS_DEBUG]: XmppCreateConnection: XMPP Peer cn-jenkins-deploy-platform-ansible-os-5902-3:10.20.0.25 SEND Xmpp creating dynamic channel 10.20.0.14:8093::10.20.0.25:42041 controller/src/xmpp/xmpp_server.cc 577 2026-06-08 Mon 05:55:41:235.006 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625491276864, Pid 43016]: XMPP [SYS_INFO]: XmppConnectionCreate: XMPP Peer cn-jenkins-deploy-platform-ansible-os-5902-3:10.20.0.25 RECV Created Xmpp Server connection from cn-jenkins-deploy-platform-ansible-os-5902-3 To : controller/src/xmpp/xmpp_connection.cc 725 2026-06-08 Mon 05:55:41:235.057 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625491276864, Pid 43016]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.14:8093::10.20.0.25:42041 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2026-06-08 Mon 05:55:41:235.133 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625474131520, Pid 43016]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer cn-jenkins-deploy-platform-ansible-os-5902-3:10.20.0.25 RECV Mode Server: PassiveOpen in state: Idle peer ip: 10.20.0.25 ( ) controller/src/xmpp/xmpp_state_machine.cc 1323 2026-06-08 Mon 05:55:41:235.152 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625448805952, Pid 43016]: [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2026-06-08 Mon 05:55:41:235.441 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625448805952, Pid 43016]: [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active event = EvTcpPassiveOpen local_port = 8093 remote_port = 42041 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2026-06-08 Mon 05:55:41:241.624 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625491276864, Pid 43016]: XMPP [SYS_DEBUG]: XmppCreateConnection: XMPP Peer cn-jenkins-deploy-platform-ansible-os-5902-3:10.20.0.193 SEND Xmpp creating dynamic channel 10.20.0.14:8093::10.20.0.193:45693 controller/src/xmpp/xmpp_server.cc 577 2026-06-08 Mon 05:55:41:241.685 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625491276864, Pid 43016]: XMPP [SYS_INFO]: XmppConnectionCreate: XMPP Peer cn-jenkins-deploy-platform-ansible-os-5902-3:10.20.0.193 RECV Created Xmpp Server connection from cn-jenkins-deploy-platform-ansible-os-5902-3 To : controller/src/xmpp/xmpp_connection.cc 725 2026-06-08 Mon 05:55:41:241.754 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625436210752, Pid 43016]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer cn-jenkins-deploy-platform-ansible-os-5902-3:10.20.0.193 RECV Mode Server: PassiveOpen in state: Idle peer ip: 10.20.0.193 ( ) controller/src/xmpp/xmpp_state_machine.cc 1323 2026-06-08 Mon 05:55:41:241.775 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625478329920, Pid 43016]: [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2026-06-08 Mon 05:55:41:241.872 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625482528320, Pid 43016]: [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active event = EvTcpPassiveOpen local_port = 8093 remote_port = 45693 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2026-06-08 Mon 05:55:41:241.996 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625491276864, Pid 43016]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.14:8093::10.20.0.193:45693 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2026-06-08 Mon 05:55:41:242.253 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625457337920, Pid 43016]: [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.25 state = Active next_state = Established event = EvXmppOpen identifier = an-jenkins-deploy-platform-ansible-os-5902-1/dns ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2026-06-08 Mon 05:55:41:242.563 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625457337920, Pid 43016]: [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.193 state = Active next_state = Established event = EvXmppOpen identifier = an-jenkins-deploy-platform-ansible-os-5902-2/dns ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2026-06-08 Mon 05:56:00:589.509 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625432012352, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:56:32:201.713 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625457337920, Pid 43016]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:56:37:594.793 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 140625432012352, Pid 43016]: DnsAgent [SYS_NOTICE]: DnsOperational: BIND named down; DNS is not operational controller/src/dns/mgr/dns_mgr.cc 745 2026-06-08 Mon 05:57:02:700.321 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622825460800, Pid 74359]: DnsAgent [SYS_INFO]: VersionInfoLog: {"build-info": [{"build-version": "v0+nightly", "build-time": "2026-06-08 00:08:42.213892", "build-user": "root", "build-hostname": "jenkins-build-prepare-sandbox-15167-1.novalocal", "build-id": "v0+nightly", "build-number": "202606080008"}]} src/contrail-common/base/misc_utils.cc 39 2026-06-08 Mon 05:57:02:700.971 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622825460800, Pid 74359]: SANDESH: ROLE : Generator 2026-06-08 Mon 05:57:02:700.985 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622825460800, Pid 74359]: SANDESH: MODULE : contrail-dns 2026-06-08 Mon 05:57:02:700.990 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622825460800, Pid 74359]: SANDESH: SOURCE : cn-jenkins-deploy-platform-ansible-os-5902-3 2026-06-08 Mon 05:57:02:700.995 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622825460800, Pid 74359]: SANDESH: NODE TYPE : Control 2026-06-08 Mon 05:57:02:700.999 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622825460800, Pid 74359]: SANDESH: INSTANCE ID : 0 2026-06-08 Mon 05:57:02:701.005 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622825460800, Pid 74359]: SANDESH: HTTP SERVER PORT : 8092 2026-06-08 Mon 05:57:02:701.011 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622825460800, Pid 74359]: SANDESH: System Log Send Rate Limit: 0 -> 0 2026-06-08 Mon 05:57:02:704.200 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622825460800, Pid 74359]: HTTP Introspect Init 2026-06-08 Mon 05:57:02:704.341 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622825460800, Pid 74359]: TCP [SYS_DEBUG]: TcpServerMessageLog: Server 0.0.0.0:8092 Initialization complete src/contrail-common/io/tcp_server.cc 138 2026-06-08 Mon 05:57:02:704.422 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622825460800, Pid 74359]: Sandesh Http Server Port: 8092 2026-06-08 Mon 05:57:02:704.503 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622825460800, Pid 74359]: SANDESH: NOT Writing http_port 8092TO : /tmp/contrail-dns.74259.http_port 2026-06-08 Mon 05:57:02:704.510 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622825460800, Pid 74359]: SANDESH: CONNECT TO COLLECTOR: 1 2026-06-08 Mon 05:57:02:704.984 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622825460800, Pid 74359]: Idle 2026-06-08 Mon 05:57:02:705.152 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622825460800, Pid 74359]: SANDESH: No Client: 1780898222705088 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3:Control:contrail-dns:0 client_info= [ status = Idle successful_connections = 0 pid = 74359 http_port = 8092 start_time = 1780898222705000 collector_name = collector_ip = 0.0.0.0:0 collector_list= [ [ ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 0, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 0, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-06-08 Mon 05:57:02:706.690 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622825460800, Pid 74359]: SANDESH: Logging: DISABLED -> ENABLED 2026-06-08 Mon 05:57:02:706.710 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622825460800, Pid 74359]: SANDESH: Logging: LEVEL: [ INVALID ] -> [ SYS_DEBUG ] log4level: [ NOTSET ] -> [ DEBUG ] 2026-06-08 Mon 05:57:02:706.865 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622816712256, Pid 74359]: Processing scm::EvStart in state Idle 2026-06-08 Mon 05:57:02:706.923 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622816712256, Pid 74359]: Disconnect 2026-06-08 Mon 05:57:02:707.010 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622816712256, Pid 74359]: Processing scm::EvCollectorUpdate in state Disconnect 2026-06-08 Mon 05:57:02:707.299 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622816712256, Pid 74359]: Connect : Start Connect timer 10.0.0.38:8086 2026-06-08 Mon 05:57:02:710.694 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622816712256, Pid 74359]: Processing scm::EvSandeshSend in state Connect 2026-06-08 Mon 05:57:02:711.785 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622816712256, Pid 74359]: SANDESH: Send FAILED: 1780898222706995 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3:Control:contrail-dns:0 client_info= [ status = Disconnect successful_connections = 0 pid = 74359 http_port = 8092 start_time = 1780898222705000 collector_name = collector_ip = 0.0.0.0:0 collector_list= [ [ ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 0, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-06-08 Mon 05:57:02:711.839 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622816712256, Pid 74359]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:57:02:711.854 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622816712256, Pid 74359]: Processing scm::EvSandeshSend in state Connect 2026-06-08 Mon 05:57:02:711.933 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622816712256, Pid 74359]: SANDESH: Send FAILED: 1780898222707058 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3:Control:contrail-dns:0 client_info= [ status = Disconnect successful_connections = 0 pid = 74359 http_port = 8092 start_time = 1780898222705000 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 0, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-06-08 Mon 05:57:02:711.944 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622816712256, Pid 74359]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:57:02:711.957 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622816712256, Pid 74359]: Processing scm::EvSandeshSend in state Connect 2026-06-08 Mon 05:57:02:712.022 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622816712256, Pid 74359]: SANDESH: Send FAILED: 1780898222710658 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 0 pid = 74359 http_port = 8092 start_time = 1780898222705000 collector_name = collector_ip = 10.0.0.38:8086 collector_list= [ [ (*_iter40) = 10.0.0.38:8086, (*_iter40) = 10.0.0.33:8086, (*_iter40) = 10.0.0.50:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 0, _iter59->first = sent _iter59->second = 0, ] ] ] 2026-06-08 Mon 05:57:02:712.032 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622816712256, Pid 74359]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2026-06-08 Mon 05:57:02:816.144 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622825460800, Pid 74359]: TCP [SYS_DEBUG]: TcpServerMessageLog: Server 10.20.0.14:8093 Initialization complete src/contrail-common/io/tcp_server.cc 138 2026-06-08 Mon 05:57:02:816.235 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622825460800, Pid 74359]: DnsAgent [SYS_NOTICE]: DnsOperational: Bulk sync has been started controller/src/dns/main.cc 402 2026-06-08 Mon 05:57:02:816.275 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622783125056, Pid 74359]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Config Client Mgr SM: Start RabbitMqReader and init Database src/contrail-common/config-client-mgr/config_client_manager.cc 281 2026-06-08 Mon 05:57:02:818.212 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622783125056, Pid 74359]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: RabbitMQ SM: RabbitMQ connection down src/contrail-common/config-client-mgr/config_amqp_client.cc 157 2026-06-08 Mon 05:57:02:818.244 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622783125056, Pid 74359]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: RabbitMQ SM: Connect to Rabbit MQ with queue_delete TRUE src/contrail-common/config-client-mgr/config_amqp_client.cc 167 2026-06-08 Mon 05:57:02:818.356 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622812513856, Pid 74359]: Processing scm::EvSandeshSend in state Connect 2026-06-08 Mon 05:57:02:818.387 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622812513856, Pid 74359]: SANDESH: Send FAILED: 1780898222816335 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Disconnect : EvCollectorUpdate -> Connect ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.38:5673, (*_iter6) = 10.0.0.50:5673, (*_iter6) = 10.0.0.33:5673, ] ] status = Down description = RabbitMQ connection down ], ] ] flag_infos= [ [ ] ] description = Number of connections:2, Expected:3 Missing: Database:Cassandra ], ] ] ] 2026-06-08 Mon 05:57:02:818.395 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622812513856, Pid 74359]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-06-08 Mon 05:57:02:898.283 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622783125056, Pid 74359]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: RabbitMQ SM: RabbitMQ connection established src/contrail-common/config-client-mgr/config_amqp_client.cc 150 2026-06-08 Mon 05:57:02:898.329 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622783125056, Pid 74359]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Config Client Mgr SM: Init Database src/contrail-common/config-client-mgr/config_client_manager.cc 285 2026-06-08 Mon 05:57:02:898.356 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622783125056, Pid 74359]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Lost Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 579 2026-06-08 Mon 05:57:02:898.376 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622783125056, Pid 74359]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db Init src/contrail-common/config-client-mgr/config_cassandra_client.cc 87 2026-06-08 Mon 05:57:02:898.381 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622778926656, Pid 74359]: Processing scm::EvSandeshSend in state Connect 2026-06-08 Mon 05:57:02:898.441 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622778926656, Pid 74359]: SANDESH: Send FAILED: 1780898222898246 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Disconnect : EvCollectorUpdate -> Connect ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.38:5673, (*_iter6) = 10.0.0.50:5673, (*_iter6) = 10.0.0.33:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Number of connections:2, Expected:3 Missing: Database:Cassandra ], ] ] ] 2026-06-08 Mon 05:57:02:898.452 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622778926656, Pid 74359]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-06-08 Mon 05:57:02:898.461 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622778926656, Pid 74359]: Processing scm::EvSandeshSend in state Connect 2026-06-08 Mon 05:57:02:898.487 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622778926656, Pid 74359]: SANDESH: Send FAILED: 1780898222898351 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Disconnect : EvCollectorUpdate -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.38:9041, (*_iter6) = 10.0.0.50:9041, (*_iter6) = 10.0.0.33:9041, ] ] status = Down description = Lost Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.38:5673, (*_iter6) = 10.0.0.50:5673, (*_iter6) = 10.0.0.33:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector, Database:Cassandra connection down ], ] ] ] 2026-06-08 Mon 05:57:02:898.494 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622778926656, Pid 74359]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-06-08 Mon 05:57:02:926.584 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622783125056, Pid 74359]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db SetTableSpace src/contrail-common/config-client-mgr/config_cassandra_client.cc 94 2026-06-08 Mon 05:57:02:950.632 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622783125056, Pid 74359]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db UseColumnFamily uuidTableName src/contrail-common/config-client-mgr/config_cassandra_client.cc 103 2026-06-08 Mon 05:57:02:950.702 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622783125056, Pid 74359]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db UseColumnFamily fqnTableName src/contrail-common/config-client-mgr/config_cassandra_client.cc 109 2026-06-08 Mon 05:57:02:950.783 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622783125056, Pid 74359]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:57:02:950.793 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622783125056, Pid 74359]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: BulkDataSync Started src/contrail-common/config-client-mgr/config_cassandra_client.cc 392 2026-06-08 Mon 05:57:02:951.436 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622770529856, Pid 74359]: Processing scm::EvSandeshSend in state Connect 2026-06-08 Mon 05:57:02:951.482 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622770529856, Pid 74359]: SANDESH: Send FAILED: 1780898222950755 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-5902-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.38:8086, ] ] status = Initializing description = Disconnect : EvCollectorUpdate -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.38:9041, (*_iter6) = 10.0.0.50:9041, (*_iter6) = 10.0.0.33:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.38:5673, (*_iter6) = 10.0.0.50:5673, (*_iter6) = 10.0.0.33:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2026-06-08 Mon 05:57:02:951.492 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622770529856, Pid 74359]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2026-06-08 Mon 05:57:02:953.627 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622783125056, Pid 74359]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:57:02:955.524 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622783125056, Pid 74359]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:57:02:956.975 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622783125056, Pid 74359]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:57:02:959.007 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622783125056, Pid 74359]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:57:02:960.420 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622783125056, Pid 74359]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:57:02:973.791 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622783125056, Pid 74359]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:57:02:976.903 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622783125056, Pid 74359]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:57:02:978.569 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622783125056, Pid 74359]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:57:02:987.401 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622783125056, Pid 74359]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:57:02:988.830 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622783125056, Pid 74359]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:57:02:990.282 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622783125056, Pid 74359]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:57:02:996.085 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622783125056, Pid 74359]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:57:02:997.454 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622783125056, Pid 74359]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:57:02:998.775 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622783125056, Pid 74359]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:57:03:001.283 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622783125056, Pid 74359]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:57:03:005.384 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622783125056, Pid 74359]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:57:03:009.289 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622783125056, Pid 74359]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:57:03:012.869 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622783125056, Pid 74359]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:57:03:014.975 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622783125056, Pid 74359]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:57:03:018.992 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622783125056, Pid 74359]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:57:03:020.212 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622783125056, Pid 74359]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:57:03:021.509 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622783125056, Pid 74359]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:57:03:022.691 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622783125056, Pid 74359]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:57:03:023.493 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622783125056, Pid 74359]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:57:03:024.558 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622783125056, Pid 74359]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:57:03:025.587 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622783125056, Pid 74359]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:57:03:026.521 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622783125056, Pid 74359]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:57:03:027.592 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622783125056, Pid 74359]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:57:03:028.637 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622783125056, Pid 74359]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:57:03:029.720 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622783125056, Pid 74359]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:57:03:030.712 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622783125056, Pid 74359]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:57:03:031.679 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622783125056, Pid 74359]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:57:03:032.843 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622783125056, Pid 74359]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:57:03:036.117 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622783125056, Pid 74359]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:57:03:037.359 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622783125056, Pid 74359]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:57:03:038.176 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622783125056, Pid 74359]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:57:03:039.495 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622783125056, Pid 74359]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:57:03:040.582 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622783125056, Pid 74359]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:57:03:041.778 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622783125056, Pid 74359]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:57:03:042.757 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622783125056, Pid 74359]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:57:03:043.867 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622783125056, Pid 74359]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:57:03:044.764 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622783125056, Pid 74359]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572 2026-06-08 Mon 05:57:03:045.771 UTC cn-jenkins-deploy-platform-ansible-os-5902-3 [Thread 139622783125056, Pid 74359]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 572