10/02/2025 04:51:23.657 7fbd6a987588 [contrail-device-manager] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 10/02/2025 04:51:23.658 7fbd6a987588 [contrail-device-manager] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 10/02/2025 04:51:23.658 7fbd6a987588 [contrail-device-manager] [INFO]: SANDESH: Flow Logging: [None] -> [False] 10/02/2025 04:51:23.658 7fbd620d7048 [contrail-device-manager] [INFO]: Processing event[EvStart] in state[none] 10/02/2025 04:51:23.658 7fbd620d7048 [contrail-device-manager] [DEBUG]: Add uve in the [ObjectConfigNode:NodeStatusUVE] map 10/02/2025 04:51:23.659 7fbd620d7048 [contrail-device-manager] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 10/02/2025 04:51:23.659 7fbd620d7048 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 10/02/2025 04:51:23.659 7fbd620d7048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:51:23.659 7fbd620d7048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 10/02/2025 04:51:23.659 7fbd620d7048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:51:23.660 7fbd620d7048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:51:23.660 7fbd620d7048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 start_time = 1759380683659182 collector_name = collector_ip = collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 10/02/2025 04:51:23.660 7fbd620d7048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:51:23.716 7fbd6a987588 [contrail-device-manager] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.254:5673//', 'pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.48:5673//'] 10/02/2025 04:51:23.727 7fbd6a987588 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection down 10/02/2025 04:51:23.727 7fbd6204b2e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:51:23.728 7fbd6204b2e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 10/02/2025 04:51:23.728 7fbd6204b2e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:51:23.728 7fbd6204b2e8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:51:23.728 7fbd6204b2e8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 10/02/2025 04:51:23.728 7fbd6204b2e8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:51:23.732 7fbd6a987588 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection ESTABLISHED 10/02/2025 04:51:23.732 7fbd6204b828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:51:23.733 7fbd6204b828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 10/02/2025 04:51:23.733 7fbd6204b828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:51:23.764 7fbd6a987588 [contrail-device-manager] [WARNING]: RabbitMQ publish connection down 10/02/2025 04:51:23.765 7fbd620354e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:51:23.765 7fbd620354e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 10/02/2025 04:51:23.765 7fbd620354e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:51:23.768 7fbd6a987588 [contrail-device-manager] [WARNING]: RabbitMQ publish connection ESTABLISHED 10/02/2025 04:51:23.768 7fbd708d7978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:51:23.768 7fbd708d7978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 10/02/2025 04:51:23.769 7fbd708d7978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:51:23.799 7fbd6a987588 [contrail-device-manager] [DEBUG]: Removed remaining AMQP queue from previous run 10/02/2025 04:51:23.799 7fbd6a987588 [contrail-device-manager] [WARNING]: VNCCassandra started with driver cql 10/02/2025 04:51:23.799 7fbd6a987588 [contrail-device-manager] [WARNING]: initializing cassandra cluster 10/02/2025 04:51:23.799 7fbd6a987588 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster before create 10/02/2025 04:51:23.800 7fbd6795e6d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Starting consumer greenlet 10/02/2025 04:51:23.800 7fbd6795e6d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 10/02/2025 04:51:23.800 7fbd6204bac8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Starting publisher greenlet 10/02/2025 04:51:23.800 7fbd6204bac8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Publisher connection 10/02/2025 04:51:23.801 7fbd61fe6048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:51:23.801 7fbd61fe6048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[] connection down >>, ] >> 10/02/2025 04:51:23.801 7fbd61fe6048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:51:23.802 7fbd6a987588 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster created 10/02/2025 04:51:23.806 7fbd6204bac8 [contrail-device-manager] [INFO]: KombuAmqpClient: Publisher connection established 10/02/2025 04:51:23.806 7fbd6795e6d8 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 10/02/2025 04:51:23.806 7fbd6795e6d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Waiting for consumer 10/02/2025 04:51:23.870 7fbd6a987588 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 10/02/2025 04:51:23.872 7fbd61e79b70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:51:23.872 7fbd61e79b70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 10/02/2025 04:51:23.872 7fbd61e79b70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:51:23.877 7fbd6a987588 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster init KS 10/02/2025 04:51:23.878 7fbd61df1048 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:51:23.878 7fbd61df1048 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[] connection down >>, ] >> 10/02/2025 04:51:23.878 7fbd61df1048 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:51:23.884 7fbd6a987588 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster KS are ready 10/02/2025 04:51:23.903 7fbd6a987588 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster CF are ready 10/02/2025 04:51:24.514 7fbd6a987588 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 10/02/2025 04:51:25.550 7fbd6a987588 [contrail-device-manager] [WARNING]: table 'obj_uuid_table', created 10/02/2025 04:51:25.554 7fbd6a987588 [contrail-device-manager] [WARNING]: Zookeeper lock released. 10/02/2025 04:51:27.471 7fbd6a987588 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 10/02/2025 04:51:27.659 7fbd61fe74e0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/02/2025 04:51:27.660 7fbd61fe74e0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 10/02/2025 04:51:27.660 7fbd61fe74e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/02/2025 04:51:27.660 7fbd61fe74e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/02/2025 04:51:27.660 7fbd61fe74e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 10/02/2025 04:51:27.660 7fbd61fe74e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/02/2025 04:51:27.660 7fbd61fe74e0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/02/2025 04:51:27.661 7fbd61fe74e0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/02/2025 04:51:27.661 7fbd61fe74e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:51:27.661 7fbd61fe74e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 10/02/2025 04:51:27.661 7fbd61fe74e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:51:27.661 7fbd61fe74e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:51:27.661 7fbd61fe74e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 start_time = 1759380683659182 collector_name = collector_ip = 10.0.0.48:8086 collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/02/2025 04:51:27.661 7fbd61fe74e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:51:28.763 7fbd6a987588 [contrail-device-manager] [WARNING]: table 'obj_uuid_table' fixed with properties {'gc_grace_seconds': 864000} 10/02/2025 04:51:28.768 7fbd6a987588 [contrail-device-manager] [WARNING]: Zookeeper lock released. 10/02/2025 04:51:29.736 7fbd6a987588 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 10/02/2025 04:51:30.489 7fbd6a987588 [contrail-device-manager] [WARNING]: table 'obj_fq_name_table', created 10/02/2025 04:51:30.494 7fbd6a987588 [contrail-device-manager] [WARNING]: Zookeeper lock released. 10/02/2025 04:51:31.660 7fbd61de2438 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/02/2025 04:51:31.661 7fbd61de2438 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 10/02/2025 04:51:31.661 7fbd61de2438 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/02/2025 04:51:31.661 7fbd61de2438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/02/2025 04:51:31.662 7fbd61de2438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 10/02/2025 04:51:31.662 7fbd61de2438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/02/2025 04:51:31.662 7fbd61de2438 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/02/2025 04:51:31.662 7fbd61de2438 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/02/2025 04:51:31.662 7fbd61de2438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:51:31.662 7fbd61de2438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 10/02/2025 04:51:31.662 7fbd61de2438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:51:31.662 7fbd61de2438 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:51:31.662 7fbd61de2438 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 start_time = 1759380683659182 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/02/2025 04:51:31.662 7fbd61de2438 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:51:32.534 7fbd6a987588 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 10/02/2025 04:51:33.438 7fbd6a987588 [contrail-device-manager] [WARNING]: table 'obj_fq_name_table' fixed with properties {'gc_grace_seconds': 864000} 10/02/2025 04:51:33.445 7fbd6a987588 [contrail-device-manager] [WARNING]: Zookeeper lock released. 10/02/2025 04:51:35.663 7fbd61dbe780 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/02/2025 04:51:35.664 7fbd61dbe780 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 10/02/2025 04:51:35.664 7fbd61dbe780 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/02/2025 04:51:35.664 7fbd61dbe780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/02/2025 04:51:35.665 7fbd61dbe780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 10/02/2025 04:51:35.665 7fbd61dbe780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/02/2025 04:51:35.665 7fbd61dbe780 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/02/2025 04:51:35.665 7fbd61dbe780 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/02/2025 04:51:35.665 7fbd61dbe780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:51:35.665 7fbd61dbe780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 10/02/2025 04:51:35.665 7fbd61dbe780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:51:35.665 7fbd61dbe780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:51:35.665 7fbd61dbe780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 start_time = 1759380683659182 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/02/2025 04:51:35.665 7fbd61dbe780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:51:35.769 7fbd6a987588 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 10/02/2025 04:51:35.773 7fbd6a987588 [contrail-device-manager] [WARNING]: table 'obj_shared_table', created 10/02/2025 04:51:35.776 7fbd6a987588 [contrail-device-manager] [WARNING]: Zookeeper lock released. 10/02/2025 04:51:37.643 7fbd6a987588 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 10/02/2025 04:51:38.560 7fbd6a987588 [contrail-device-manager] [WARNING]: table 'obj_shared_table' fixed with properties {'gc_grace_seconds': 864000} 10/02/2025 04:51:38.565 7fbd6a987588 [contrail-device-manager] [WARNING]: Zookeeper lock released. 10/02/2025 04:51:38.566 7fbd61de1978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:51:38.566 7fbd61de1978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/02/2025 04:51:38.566 7fbd61de1978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:51:38.593 7fbd6a987588 [contrail-device-manager] [ERROR]: Error in db walk read 'type' 10/02/2025 04:51:38.594 7fbd62c74588 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:51:38.594 7fbd62c74588 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Initializing description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Zookeeper:Zookeeper[] connection down >>, ] >> 10/02/2025 04:51:38.594 7fbd62c74588 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:51:38.602 7fbd61dcfc18 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:51:38.602 7fbd61dcfc18 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/02/2025 04:51:38.602 7fbd61dcfc18 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:51:38.603 7fbd6a987588 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: job_status_consumer.dummy 10/02/2025 04:51:38.603 7fbd6a987588 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: job_request_consumer 10/02/2025 04:51:38.603 7fbd6795e6d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 10/02/2025 04:51:38.603 7fbd6a987588 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-4211-3. 10/02/2025 04:51:38.612 7fbd6795e6d8 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 10/02/2025 04:51:38.789 7fbd6795e6d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-4211-3.'] 10/02/2025 04:51:38.810 7fbd6795e6d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 10/02/2025 04:51:38.824 7fbd6795e6d8 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 10/02/2025 04:51:38.879 7fbd6795e6d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-4211-3.'] 10/02/2025 04:51:39.113 7fbd6a987588 [contrail-device-manager] [DEBUG]: Fetching all containers 10/02/2025 04:51:39.153 7fbd6a987588 [contrail-device-manager] [INFO]: Restarting dnsmasq docker: config_dnsmasq_1 10/02/2025 04:51:39.309 7fbd6a987588 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-4211-3..config_queue 10/02/2025 04:51:39.309 7fbd6a987588 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-4211-3..tftp_queue 10/02/2025 04:51:39.665 7fbd61dcccc0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/02/2025 04:51:39.665 7fbd61dcccc0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 10/02/2025 04:51:39.666 7fbd61dcccc0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/02/2025 04:51:39.666 7fbd61dcccc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/02/2025 04:51:39.666 7fbd61dcccc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/02/2025 04:51:39.666 7fbd61dcccc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/02/2025 04:51:39.666 7fbd61dcccc0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/02/2025 04:51:39.666 7fbd61dcccc0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/02/2025 04:51:39.666 7fbd61dcccc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:51:39.666 7fbd61dcccc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/02/2025 04:51:39.666 7fbd61dcccc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:51:39.666 7fbd61dcccc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:51:39.667 7fbd61dcccc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 start_time = 1759380683659182 collector_name = collector_ip = 10.0.0.48:8086 collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/02/2025 04:51:39.667 7fbd61dcccc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:51:39.810 7fbd6a987588 [contrail-device-manager] [WARNING]: Process 140 prepared to run in Full mode... 10/02/2025 04:51:39.810 7fbd6a987588 [contrail-device-manager] [WARNING]: Elected master Device Manager node. Initializing... 10/02/2025 04:51:39.810 7fbd6a987588 [contrail-device-manager] [INFO]: SANDESH: INTROSPECT IS ON: 0.0.0.0:8096 10/02/2025 04:51:39.813 7fbd6a987588 [contrail-device-manager] [DEBUG]: Add [config_updater.xml:/usr/lib/python3.6/site-packages/cfgm_common/uve/config_updater] to home page 10/02/2025 04:51:39.813 7fbd6a987588 [contrail-device-manager] [DEBUG]: Add http request [/Snh_ConfigApiUpdateReq] 10/02/2025 04:51:39.814 7fbd6a987588 [contrail-device-manager] [DEBUG]: Add [greenlets.xml:/usr/lib/python3.6/site-packages/cfgm_common/uve/greenlets] to home page 10/02/2025 04:51:39.814 7fbd6a987588 [contrail-device-manager] [DEBUG]: Add http request [/Snh_GreenletObjectReq] 10/02/2025 04:51:39.815 7fbd6a987588 [contrail-device-manager] [DEBUG]: Add [cpuinfo.xml:/usr/lib/python3.6/site-packages/cfgm_common/uve/nodeinfo/cpuinfo] to home page 10/02/2025 04:51:39.815 7fbd6a987588 [contrail-device-manager] [DEBUG]: Add http request [/Snh_CpuLoadInfoReq] 10/02/2025 04:51:39.818 7fbd6a987588 [contrail-device-manager] [DEBUG]: Add [dm_introspect.xml:/usr/lib/python3.6/site-packages/device_manager/sandesh/dm_introspect] to home page 10/02/2025 04:51:39.819 7fbd6a987588 [contrail-device-manager] [DEBUG]: Add http request [/Snh_RoutingInstanceList] 10/02/2025 04:51:39.819 7fbd6a987588 [contrail-device-manager] [DEBUG]: Add http request [/Snh_VirtualNetworkList] 10/02/2025 04:51:39.819 7fbd6a987588 [contrail-device-manager] [DEBUG]: Add http request [/Snh_VirtualMachineInterfaceList] 10/02/2025 04:51:39.819 7fbd6a987588 [contrail-device-manager] [DEBUG]: Add http request [/Snh_LogicalInterfaceList] 10/02/2025 04:51:39.819 7fbd6a987588 [contrail-device-manager] [DEBUG]: Add http request [/Snh_PhysicalInterfaceList] 10/02/2025 04:51:39.819 7fbd6a987588 [contrail-device-manager] [DEBUG]: Add http request [/Snh_PhysicalRouterList] 10/02/2025 04:51:39.819 7fbd6a987588 [contrail-device-manager] [DEBUG]: Add http request [/Snh_BgpRouterList] 10/02/2025 04:51:39.819 7fbd6a987588 [contrail-device-manager] [DEBUG]: Add http request [/Snh_LogicalRouterList] 10/02/2025 04:51:39.819 7fbd6a987588 [contrail-device-manager] [DEBUG]: Add http request [/Snh_InstanceIpList] 10/02/2025 04:51:39.819 7fbd6a987588 [contrail-device-manager] [DEBUG]: Add http request [/Snh_FloatingIpList] 10/02/2025 04:51:39.819 7fbd6a987588 [contrail-device-manager] [DEBUG]: Add http request [/Snh_PortTupleList] 10/02/2025 04:51:39.819 7fbd6a987588 [contrail-device-manager] [DEBUG]: Add http request [/Snh_FeatureFlagList] 10/02/2025 04:51:39.822 7fbd6a987588 [contrail-device-manager] [DEBUG]: Add [sandesh_alarm.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_alarm] to home page 10/02/2025 04:51:39.822 7fbd6a987588 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshAlarmCacheRequest] 10/02/2025 04:51:39.822 7fbd6a987588 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshAlarmTypesRequest] 10/02/2025 04:51:39.822 7fbd6a987588 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshAlarmAckRequest] 10/02/2025 04:51:39.822 7fbd6a987588 [contrail-device-manager] [DEBUG]: Add [sandesh_ctrl.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_ctrl] to home page 10/02/2025 04:51:39.823 7fbd6a987588 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshCtrlClientToServer] 10/02/2025 04:51:39.823 7fbd6a987588 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshCtrlServerToClient] 10/02/2025 04:51:39.823 7fbd6a987588 [contrail-device-manager] [DEBUG]: Add [sandesh_trace.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_trace] to home page 10/02/2025 04:51:39.823 7fbd6a987588 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceRequest] 10/02/2025 04:51:39.823 7fbd6a987588 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceBufferListRequest] 10/02/2025 04:51:39.824 7fbd6a987588 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceEnableDisableReq] 10/02/2025 04:51:39.824 7fbd6a987588 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceBufStatusReq] 10/02/2025 04:51:39.824 7fbd6a987588 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshTraceBufferEnableDisableReq] 10/02/2025 04:51:39.825 7fbd6a987588 [contrail-device-manager] [DEBUG]: Add [sandesh_uve.xml:/usr/lib/python3.6/site-packages/pysandesh/gen_py/sandesh_uve] to home page 10/02/2025 04:51:39.834 7fbd6a987588 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshMessageStatsReq] 10/02/2025 04:51:39.835 7fbd6a987588 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshLoggingParamsSet] 10/02/2025 04:51:39.835 7fbd6a987588 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshLoggingParamsStatus] 10/02/2025 04:51:39.835 7fbd6a987588 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendingParamsSet] 10/02/2025 04:51:39.835 7fbd6a987588 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendingParamsStatus] 10/02/2025 04:51:39.835 7fbd6a987588 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueParamsSet] 10/02/2025 04:51:39.835 7fbd6a987588 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueParamsReset] 10/02/2025 04:51:39.835 7fbd6a987588 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueParamsStatus] 10/02/2025 04:51:39.835 7fbd6a987588 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueSet] 10/02/2025 04:51:39.836 7fbd6a987588 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshSendQueueStatus] 10/02/2025 04:51:39.837 7fbd6a987588 [contrail-device-manager] [DEBUG]: Add http request [/Snh_CollectorInfoRequest] 10/02/2025 04:51:39.837 7fbd6a987588 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshUVEDSConfReq] 10/02/2025 04:51:39.837 7fbd6a987588 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshUVECacheReq] 10/02/2025 04:51:39.837 7fbd6a987588 [contrail-device-manager] [DEBUG]: Add http request [/Snh_SandeshUVETypesReq] 10/02/2025 04:51:39.841 7fbd6a987588 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Added consumer: device_manager_ztp.ztp_queue 10/02/2025 04:51:39.863 7fbd614e7f60 [contrail-device-manager] [ERROR]: Cannot write http_port 8096 to /tmp/contrail-device-manager.1.http_port 10/02/2025 04:51:39.864 7fbd614e7f60 [contrail-device-manager] [ERROR]: Starting Introspect on HTTP Port 8096 10/02/2025 04:51:39.912 7fbd60cc7f60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:51:39.912 7fbd60cc7f60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Initializing description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 10/02/2025 04:51:39.912 7fbd60cc7f60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:51:39.924 7fbd60c2c390 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:51:39.924 7fbd60c2c390 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Down description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 10/02/2025 04:51:39.924 7fbd60c2c390 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:51:39.928 7fbd6795e6d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Ensuring Consumer connection 10/02/2025 04:51:39.938 7fbd6795e6d8 [contrail-device-manager] [INFO]: KombuAmqpClient: Consumer connection established 10/02/2025 04:51:40.058 7fbd6795e6d8 [contrail-device-manager] [DEBUG]: KombuAmqpClient: Created consumers ['job_status_consumer.dummy', 'job_request_consumer', 'job_abort_consumer.cn-jenkins-deploy-platform-ansible-os-4211-3.', 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-4211-3..config_queue', 'device_manager_ztp.cn-jenkins-deploy-platform-ansible-os-4211-3..tftp_queue', 'device_manager_ztp.ztp_queue'] 10/02/2025 04:51:43.666 7fbd60b69978 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/02/2025 04:51:43.667 7fbd60b69978 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 10/02/2025 04:51:43.667 7fbd60b69978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/02/2025 04:51:43.667 7fbd60b69978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/02/2025 04:51:43.668 7fbd60b69978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Down description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 10/02/2025 04:51:43.668 7fbd60b69978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/02/2025 04:51:43.668 7fbd60b69978 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/02/2025 04:51:43.668 7fbd60b69978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/02/2025 04:51:43.668 7fbd60b69978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:51:43.668 7fbd60b69978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Down description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], ApiServer:ApiServer[ApiServer Connection State updated] connection down >>, ] >> 10/02/2025 04:51:43.669 7fbd60b69978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:51:43.669 7fbd60b69978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:51:43.669 7fbd60b69978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1759380683659182 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/02/2025 04:51:43.669 7fbd60b69978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:51:46.045 7fbd60b58630 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:51:46.045 7fbd60b58630 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/02/2025 04:51:46.045 7fbd60b58630 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:51:47.667 7fbd60b95a20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/02/2025 04:51:47.668 7fbd60b95a20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 10/02/2025 04:51:47.669 7fbd60b95a20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/02/2025 04:51:47.669 7fbd60b95a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/02/2025 04:51:47.669 7fbd60b95a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/02/2025 04:51:47.669 7fbd60b95a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/02/2025 04:51:47.669 7fbd60b95a20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/02/2025 04:51:47.669 7fbd60b95a20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/02/2025 04:51:47.669 7fbd60b95a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:51:47.669 7fbd60b95a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/02/2025 04:51:47.669 7fbd60b95a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:51:47.670 7fbd60b95a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:51:47.670 7fbd60b95a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1759380683659182 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/02/2025 04:51:47.670 7fbd60b95a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:51:51.669 7fbd60a85198 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/02/2025 04:51:51.670 7fbd60a85198 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 10/02/2025 04:51:51.670 7fbd60a85198 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/02/2025 04:51:51.670 7fbd60a85198 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/02/2025 04:51:51.670 7fbd60a85198 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/02/2025 04:51:51.670 7fbd60a85198 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/02/2025 04:51:51.670 7fbd60a85198 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/02/2025 04:51:51.670 7fbd60a85198 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/02/2025 04:51:51.670 7fbd60a85198 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:51:51.671 7fbd60a85198 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/02/2025 04:51:51.671 7fbd60a85198 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:51:51.671 7fbd60a85198 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:51:51.671 7fbd60a85198 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1759380683659182 collector_name = collector_ip = 10.0.0.48:8086 collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/02/2025 04:51:51.671 7fbd60a85198 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:51:55.671 7fbd60a80780 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/02/2025 04:51:55.672 7fbd60a80780 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 10/02/2025 04:51:55.672 7fbd60a80780 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/02/2025 04:51:55.672 7fbd60a80780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/02/2025 04:51:55.672 7fbd60a80780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/02/2025 04:51:55.672 7fbd60a80780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/02/2025 04:51:55.672 7fbd60a80780 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/02/2025 04:51:55.673 7fbd60a80780 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/02/2025 04:51:55.673 7fbd60a80780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:51:55.673 7fbd60a80780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/02/2025 04:51:55.673 7fbd60a80780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:51:55.673 7fbd60a80780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:51:55.673 7fbd60a80780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1759380683659182 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/02/2025 04:51:55.673 7fbd60a80780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:51:59.673 7fbd60a80780 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/02/2025 04:51:59.675 7fbd60a80780 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 10/02/2025 04:51:59.675 7fbd60a80780 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/02/2025 04:51:59.675 7fbd60a80780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/02/2025 04:51:59.675 7fbd60a80780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/02/2025 04:51:59.675 7fbd60a80780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/02/2025 04:51:59.675 7fbd60a80780 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/02/2025 04:51:59.675 7fbd60a80780 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/02/2025 04:51:59.675 7fbd60a80780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:51:59.676 7fbd60a80780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/02/2025 04:51:59.676 7fbd60a80780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:51:59.676 7fbd60a80780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:51:59.676 7fbd60a80780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1759380683659182 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/02/2025 04:51:59.676 7fbd60a80780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:52:03.676 7fbd60a80780 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/02/2025 04:52:03.677 7fbd60a80780 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 10/02/2025 04:52:03.677 7fbd60a80780 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/02/2025 04:52:03.677 7fbd60a80780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/02/2025 04:52:03.677 7fbd60a80780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/02/2025 04:52:03.677 7fbd60a80780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/02/2025 04:52:03.677 7fbd60a80780 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/02/2025 04:52:03.678 7fbd60a80780 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/02/2025 04:52:03.678 7fbd60a80780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:52:03.678 7fbd60a80780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/02/2025 04:52:03.678 7fbd60a80780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:52:03.678 7fbd60a80780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:52:03.678 7fbd60a80780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1759380683659182 collector_name = collector_ip = 10.0.0.48:8086 collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/02/2025 04:52:03.678 7fbd60a80780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:52:07.678 7fbd60a80780 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/02/2025 04:52:07.679 7fbd60a80780 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 10/02/2025 04:52:07.679 7fbd60a80780 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/02/2025 04:52:07.679 7fbd60a80780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/02/2025 04:52:07.680 7fbd60a80780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/02/2025 04:52:07.680 7fbd60a80780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/02/2025 04:52:07.680 7fbd60a80780 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/02/2025 04:52:07.680 7fbd60a80780 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/02/2025 04:52:07.680 7fbd60a80780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:52:07.680 7fbd60a80780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/02/2025 04:52:07.680 7fbd60a80780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:52:07.680 7fbd60a80780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:52:07.680 7fbd60a80780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1759380683659182 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/02/2025 04:52:07.681 7fbd60a80780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:52:11.680 7fbd60a80780 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/02/2025 04:52:11.681 7fbd60a80780 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 10/02/2025 04:52:11.682 7fbd60a80780 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/02/2025 04:52:11.682 7fbd60a80780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/02/2025 04:52:11.682 7fbd60a80780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/02/2025 04:52:11.682 7fbd60a80780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/02/2025 04:52:11.682 7fbd60a80780 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/02/2025 04:52:11.682 7fbd60a80780 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/02/2025 04:52:11.682 7fbd60a80780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:52:11.682 7fbd60a80780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/02/2025 04:52:11.683 7fbd60a80780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:52:11.683 7fbd60a80780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:52:11.683 7fbd60a80780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1759380683659182 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/02/2025 04:52:11.683 7fbd60a80780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:52:15.682 7fbd60a80780 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/02/2025 04:52:15.683 7fbd60a80780 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 10/02/2025 04:52:15.683 7fbd60a80780 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/02/2025 04:52:15.683 7fbd60a80780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/02/2025 04:52:15.684 7fbd60a80780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/02/2025 04:52:15.684 7fbd60a80780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/02/2025 04:52:15.684 7fbd60a80780 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/02/2025 04:52:15.684 7fbd60a80780 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/02/2025 04:52:15.684 7fbd60a80780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:52:15.684 7fbd60a80780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/02/2025 04:52:15.684 7fbd60a80780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:52:15.684 7fbd60a80780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:52:15.684 7fbd60a80780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1759380683659182 collector_name = collector_ip = 10.0.0.48:8086 collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/02/2025 04:52:15.684 7fbd60a80780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:52:19.684 7fbd60a80780 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/02/2025 04:52:19.685 7fbd60a80780 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 10/02/2025 04:52:19.685 7fbd60a80780 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/02/2025 04:52:19.686 7fbd60a80780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/02/2025 04:52:19.686 7fbd60a80780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/02/2025 04:52:19.686 7fbd60a80780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/02/2025 04:52:19.686 7fbd60a80780 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/02/2025 04:52:19.686 7fbd60a80780 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/02/2025 04:52:19.686 7fbd60a80780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:52:19.686 7fbd60a80780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/02/2025 04:52:19.686 7fbd60a80780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:52:19.686 7fbd60a80780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:52:19.686 7fbd60a80780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1759380683659182 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/02/2025 04:52:19.686 7fbd60a80780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:52:23.686 7fbd60a80780 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/02/2025 04:52:23.687 7fbd60a80780 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 10/02/2025 04:52:23.688 7fbd60a80780 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/02/2025 04:52:23.688 7fbd60a80780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/02/2025 04:52:23.688 7fbd60a80780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/02/2025 04:52:23.688 7fbd60a80780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/02/2025 04:52:23.688 7fbd60a80780 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/02/2025 04:52:23.688 7fbd60a80780 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/02/2025 04:52:23.688 7fbd60a80780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:52:23.689 7fbd60a80780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/02/2025 04:52:23.689 7fbd60a80780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:52:23.689 7fbd60a80780 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:52:23.689 7fbd60a80780 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1759380683659182 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/02/2025 04:52:23.689 7fbd60a80780 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:52:27.688 7fbd61e5be10 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/02/2025 04:52:27.689 7fbd61e5be10 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 10/02/2025 04:52:27.689 7fbd61e5be10 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/02/2025 04:52:27.689 7fbd61e5be10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/02/2025 04:52:27.690 7fbd61e5be10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/02/2025 04:52:27.690 7fbd61e5be10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/02/2025 04:52:27.690 7fbd61e5be10 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/02/2025 04:52:27.690 7fbd61e5be10 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/02/2025 04:52:27.690 7fbd61e5be10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:52:27.690 7fbd61e5be10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/02/2025 04:52:27.690 7fbd61e5be10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:52:27.690 7fbd61e5be10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:52:27.690 7fbd61e5be10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1759380683659182 collector_name = collector_ip = 10.0.0.48:8086 collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/02/2025 04:52:27.690 7fbd61e5be10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:52:31.690 7fbd61e23ac8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/02/2025 04:52:31.691 7fbd61e23ac8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 10/02/2025 04:52:31.691 7fbd61e23ac8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/02/2025 04:52:31.692 7fbd61e23ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/02/2025 04:52:31.692 7fbd61e23ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/02/2025 04:52:31.692 7fbd61e23ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/02/2025 04:52:31.692 7fbd61e23ac8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/02/2025 04:52:31.692 7fbd61e23ac8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/02/2025 04:52:31.692 7fbd61e23ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:52:31.692 7fbd61e23ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/02/2025 04:52:31.692 7fbd61e23ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:52:31.692 7fbd61e23ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:52:31.692 7fbd61e23ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1759380683659182 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/02/2025 04:52:31.692 7fbd61e23ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:52:35.692 7fbd61db8f60 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/02/2025 04:52:35.693 7fbd61db8f60 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 10/02/2025 04:52:35.693 7fbd61db8f60 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/02/2025 04:52:35.693 7fbd61db8f60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/02/2025 04:52:35.693 7fbd61db8f60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/02/2025 04:52:35.693 7fbd61db8f60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/02/2025 04:52:35.693 7fbd61db8f60 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/02/2025 04:52:35.694 7fbd61db8f60 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/02/2025 04:52:35.694 7fbd61db8f60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:52:35.694 7fbd61db8f60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/02/2025 04:52:35.694 7fbd61db8f60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:52:35.694 7fbd61db8f60 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:52:35.694 7fbd61db8f60 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1759380683659182 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/02/2025 04:52:35.694 7fbd61db8f60 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:52:39.695 7fbd61df5a20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/02/2025 04:52:39.696 7fbd61df5a20 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 10/02/2025 04:52:39.696 7fbd61df5a20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/02/2025 04:52:39.696 7fbd61df5a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/02/2025 04:52:39.696 7fbd61df5a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/02/2025 04:52:39.696 7fbd61df5a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/02/2025 04:52:39.696 7fbd61df5a20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/02/2025 04:52:39.697 7fbd61df5a20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/02/2025 04:52:39.697 7fbd61df5a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:52:39.697 7fbd61df5a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/02/2025 04:52:39.697 7fbd61df5a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:52:39.697 7fbd61df5a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:52:39.697 7fbd61df5a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1759380683659182 collector_name = collector_ip = 10.0.0.48:8086 collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/02/2025 04:52:39.697 7fbd61df5a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:52:43.697 7fbd60b69eb8 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/02/2025 04:52:43.698 7fbd60b69eb8 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 10/02/2025 04:52:43.698 7fbd60b69eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/02/2025 04:52:43.698 7fbd60b69eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/02/2025 04:52:43.698 7fbd60b69eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/02/2025 04:52:43.698 7fbd60b69eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/02/2025 04:52:43.698 7fbd60b69eb8 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/02/2025 04:52:43.699 7fbd60b69eb8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/02/2025 04:52:43.699 7fbd60b69eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:52:43.699 7fbd60b69eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/02/2025 04:52:43.699 7fbd60b69eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:52:43.699 7fbd60b69eb8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:52:43.699 7fbd60b69eb8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1759380683659182 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/02/2025 04:52:43.699 7fbd60b69eb8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:52:47.698 7fbd60adfcc0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/02/2025 04:52:47.700 7fbd60adfcc0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 10/02/2025 04:52:47.700 7fbd60adfcc0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/02/2025 04:52:47.701 7fbd60adfcc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/02/2025 04:52:47.701 7fbd60adfcc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/02/2025 04:52:47.701 7fbd60adfcc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/02/2025 04:52:47.701 7fbd60adfcc0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/02/2025 04:52:47.701 7fbd60adfcc0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/02/2025 04:52:47.701 7fbd60adfcc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:52:47.701 7fbd60adfcc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/02/2025 04:52:47.701 7fbd60adfcc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:52:47.701 7fbd60adfcc0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:52:47.702 7fbd60adfcc0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1759380683659182 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/02/2025 04:52:47.702 7fbd60adfcc0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:52:51.701 7fbd61df5e10 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/02/2025 04:52:51.702 7fbd61df5e10 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 10/02/2025 04:52:51.702 7fbd61df5e10 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/02/2025 04:52:51.702 7fbd61df5e10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/02/2025 04:52:51.702 7fbd61df5e10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/02/2025 04:52:51.702 7fbd61df5e10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/02/2025 04:52:51.702 7fbd61df5e10 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/02/2025 04:52:51.702 7fbd61df5e10 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/02/2025 04:52:51.702 7fbd61df5e10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:52:51.703 7fbd61df5e10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/02/2025 04:52:51.703 7fbd61df5e10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:52:51.703 7fbd61df5e10 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:52:51.703 7fbd61df5e10 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1759380683659182 collector_name = collector_ip = 10.0.0.48:8086 collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/02/2025 04:52:51.703 7fbd61df5e10 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:52:55.703 7fbd60b22198 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/02/2025 04:52:55.704 7fbd60b22198 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 10/02/2025 04:52:55.704 7fbd60b22198 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/02/2025 04:52:55.704 7fbd60b22198 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/02/2025 04:52:55.704 7fbd60b22198 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/02/2025 04:52:55.704 7fbd60b22198 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/02/2025 04:52:55.704 7fbd60b22198 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/02/2025 04:52:55.705 7fbd60b22198 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/02/2025 04:52:55.705 7fbd60b22198 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:52:55.705 7fbd60b22198 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/02/2025 04:52:55.705 7fbd60b22198 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:52:55.705 7fbd60b22198 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:52:55.705 7fbd60b22198 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1759380683659182 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/02/2025 04:52:55.705 7fbd60b22198 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:52:57.111 7fbd6a987588 [contrail-device-manager] [WARNING]: VNCCassandra started with driver cql 10/02/2025 04:52:57.111 7fbd6a987588 [contrail-device-manager] [WARNING]: initializing cassandra cluster 10/02/2025 04:52:57.111 7fbd6a987588 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster before create 10/02/2025 04:52:57.111 7fbd67b98a20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:52:57.111 7fbd67b98a20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 10/02/2025 04:52:57.112 7fbd67b98a20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:52:57.112 7fbd6a987588 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster created 10/02/2025 04:52:57.159 7fbd6a987588 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 10/02/2025 04:52:57.160 7fbd60b7fb70 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:52:57.161 7fbd60b7fb70 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Initializing description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 10/02/2025 04:52:57.161 7fbd60b7fb70 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:52:57.167 7fbd6a987588 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster init KS 10/02/2025 04:52:57.167 7fbd60b7fa20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:52:57.168 7fbd60b7fa20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 10/02/2025 04:52:57.168 7fbd60b7fa20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:52:57.199 7fbd6a987588 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 10/02/2025 04:52:57.659 7fbd6a987588 [contrail-device-manager] [WARNING]: keyspace 'dm_keyspace', created 10/02/2025 04:52:57.663 7fbd6a987588 [contrail-device-manager] [WARNING]: Zookeeper lock released. 10/02/2025 04:52:57.669 7fbd6a987588 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 10/02/2025 04:52:58.332 7fbd6a987588 [contrail-device-manager] [WARNING]: keyspace 'dm_keyspace' fixed with replication {'class': 'SimpleStrategy', 'replication_factor': None} 10/02/2025 04:52:58.335 7fbd6a987588 [contrail-device-manager] [WARNING]: Zookeeper lock released. 10/02/2025 04:52:58.335 7fbd6a987588 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster KS are ready 10/02/2025 04:52:58.440 7fbd6a987588 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster CF are ready 10/02/2025 04:52:58.448 7fbd6a987588 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 10/02/2025 04:52:59.643 7fbd6a987588 [contrail-device-manager] [WARNING]: table 'dm_pr_vn_ip_table', created 10/02/2025 04:52:59.646 7fbd6a987588 [contrail-device-manager] [WARNING]: Zookeeper lock released. 10/02/2025 04:52:59.651 7fbd6a987588 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 10/02/2025 04:52:59.705 7fbd60c5a978 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/02/2025 04:52:59.706 7fbd60c5a978 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 10/02/2025 04:52:59.706 7fbd60c5a978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/02/2025 04:52:59.706 7fbd60c5a978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/02/2025 04:52:59.706 7fbd60c5a978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 10/02/2025 04:52:59.706 7fbd60c5a978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/02/2025 04:52:59.706 7fbd60c5a978 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/02/2025 04:52:59.706 7fbd60c5a978 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/02/2025 04:52:59.707 7fbd60c5a978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:52:59.707 7fbd60c5a978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 10/02/2025 04:52:59.707 7fbd60c5a978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:52:59.707 7fbd60c5a978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:52:59.707 7fbd60c5a978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1759380683659182 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/02/2025 04:52:59.707 7fbd60c5a978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:53:00.599 7fbd6a987588 [contrail-device-manager] [WARNING]: table 'dm_pr_vn_ip_table' fixed with properties {'gc_grace_seconds': 864000} 10/02/2025 04:53:00.603 7fbd6a987588 [contrail-device-manager] [WARNING]: Zookeeper lock released. 10/02/2025 04:53:00.612 7fbd6a987588 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 10/02/2025 04:53:01.534 7fbd6a987588 [contrail-device-manager] [WARNING]: table 'dm_pr_asn_table', created 10/02/2025 04:53:01.538 7fbd6a987588 [contrail-device-manager] [WARNING]: Zookeeper lock released. 10/02/2025 04:53:01.543 7fbd6a987588 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 10/02/2025 04:53:02.835 7fbd6a987588 [contrail-device-manager] [WARNING]: table 'dm_pr_asn_table' fixed with properties {'gc_grace_seconds': 864000} 10/02/2025 04:53:02.839 7fbd6a987588 [contrail-device-manager] [WARNING]: Zookeeper lock released. 10/02/2025 04:53:02.845 7fbd6a987588 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 10/02/2025 04:53:03.649 7fbd6a987588 [contrail-device-manager] [WARNING]: table 'dm_ni_ipv6_ll_table', created 10/02/2025 04:53:03.654 7fbd6a987588 [contrail-device-manager] [WARNING]: Zookeeper lock released. 10/02/2025 04:53:03.658 7fbd6a987588 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 10/02/2025 04:53:03.706 7fbd60c410f0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/02/2025 04:53:03.707 7fbd60c410f0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 10/02/2025 04:53:03.707 7fbd60c410f0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/02/2025 04:53:03.707 7fbd60c410f0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/02/2025 04:53:03.708 7fbd60c410f0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 10/02/2025 04:53:03.708 7fbd60c410f0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/02/2025 04:53:03.708 7fbd60c410f0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/02/2025 04:53:03.708 7fbd60c410f0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/02/2025 04:53:03.708 7fbd60c410f0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:53:03.708 7fbd60c410f0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 10/02/2025 04:53:03.709 7fbd60c410f0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:53:03.709 7fbd60c410f0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:53:03.709 7fbd60c410f0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1759380683659182 collector_name = collector_ip = 10.0.0.48:8086 collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/02/2025 04:53:03.709 7fbd60c410f0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:53:04.541 7fbd6a987588 [contrail-device-manager] [WARNING]: table 'dm_ni_ipv6_ll_table' fixed with properties {'gc_grace_seconds': 864000} 10/02/2025 04:53:04.547 7fbd6a987588 [contrail-device-manager] [WARNING]: Zookeeper lock released. 10/02/2025 04:53:04.555 7fbd6a987588 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 10/02/2025 04:53:06.309 7fbd6a987588 [contrail-device-manager] [WARNING]: table 'dm_pnf_resource_table', created 10/02/2025 04:53:06.313 7fbd6a987588 [contrail-device-manager] [WARNING]: Zookeeper lock released. 10/02/2025 04:53:06.318 7fbd6a987588 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 10/02/2025 04:53:06.801 7fbd6a987588 [contrail-device-manager] [WARNING]: table 'dm_pnf_resource_table' fixed with properties {'gc_grace_seconds': 864000} 10/02/2025 04:53:06.807 7fbd6a987588 [contrail-device-manager] [WARNING]: Zookeeper lock released. 10/02/2025 04:53:06.807 7fbd67ba9828 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:53:06.808 7fbd67ba9828 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/02/2025 04:53:06.808 7fbd67ba9828 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:53:06.841 7fbd6a987588 [contrail-device-manager] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.254:5673//', 'pyamqp://guest:guest@10.0.0.38:5673//', 'pyamqp://guest:guest@10.0.0.48:5673//'] 10/02/2025 04:53:06.842 7fbd6a987588 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection down 10/02/2025 04:53:06.842 7fbd60a5d4e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:53:06.842 7fbd60a5d4e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Initializing description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:RabbitMQ[] connection down >>, ] >> 10/02/2025 04:53:06.843 7fbd60a5d4e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:53:06.843 7fbd60a5d4e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:53:06.843 7fbd60a5d4e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:RabbitMQ[] connection down >>, ] >> 10/02/2025 04:53:06.843 7fbd60a5d4e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:53:06.846 7fbd6a987588 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection ESTABLISHED 10/02/2025 04:53:06.846 7fbd60a57ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:53:06.846 7fbd60a57ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/02/2025 04:53:06.846 7fbd60a57ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:53:06.872 7fbd6a987588 [contrail-device-manager] [WARNING]: RabbitMQ publish connection down 10/02/2025 04:53:06.873 7fbd60a57ac8 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:53:06.873 7fbd60a57ac8 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:RabbitMQ[] connection down >>, ] >> 10/02/2025 04:53:06.873 7fbd60a57ac8 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:53:06.876 7fbd6a987588 [contrail-device-manager] [WARNING]: RabbitMQ publish connection ESTABLISHED 10/02/2025 04:53:06.876 7fbd60a57240 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:53:06.876 7fbd60a57240 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/02/2025 04:53:06.876 7fbd60a57240 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:53:06.877 7fbd60c41048 [contrail-device-manager] [WARNING]: RabbitMQ publish connection down 10/02/2025 04:53:06.878 7fbd60c5f978 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:53:06.878 7fbd60c5f978 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Down description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:RabbitMQ[] connection down >>, ] >> 10/02/2025 04:53:06.878 7fbd60c5f978 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:53:06.887 7fbd60c41048 [contrail-device-manager] [WARNING]: RabbitMQ publish connection ESTABLISHED 10/02/2025 04:53:06.887 7fbd60c404e0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:53:06.887 7fbd60c404e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/02/2025 04:53:06.887 7fbd60c404e0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:53:07.708 7fbd60a3f0f0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/02/2025 04:53:07.709 7fbd60a3f0f0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 10/02/2025 04:53:07.710 7fbd60a3f0f0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/02/2025 04:53:07.710 7fbd60a3f0f0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/02/2025 04:53:07.710 7fbd60a3f0f0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/02/2025 04:53:07.710 7fbd60a3f0f0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/02/2025 04:53:07.710 7fbd60a3f0f0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/02/2025 04:53:07.710 7fbd60a3f0f0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/02/2025 04:53:07.710 7fbd60a3f0f0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:53:07.710 7fbd60a3f0f0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/02/2025 04:53:07.710 7fbd60a3f0f0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:53:07.711 7fbd60a3f0f0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:53:07.711 7fbd60a3f0f0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1759380683659182 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/02/2025 04:53:07.711 7fbd60a3f0f0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:53:11.710 7fbd60a3f0f0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/02/2025 04:53:11.712 7fbd60a3f0f0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 10/02/2025 04:53:11.712 7fbd60a3f0f0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/02/2025 04:53:11.712 7fbd60a3f0f0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/02/2025 04:53:11.712 7fbd60a3f0f0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/02/2025 04:53:11.712 7fbd60a3f0f0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/02/2025 04:53:11.713 7fbd60a3f0f0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/02/2025 04:53:11.713 7fbd60a3f0f0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/02/2025 04:53:11.713 7fbd60a3f0f0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:53:11.713 7fbd60a3f0f0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/02/2025 04:53:11.713 7fbd60a3f0f0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:53:11.713 7fbd60a3f0f0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:53:11.713 7fbd60a3f0f0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1759380683659182 collector_name = collector_ip = 10.0.0.254:8086 collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/02/2025 04:53:11.713 7fbd60a3f0f0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:53:15.713 7fbd60a3f0f0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/02/2025 04:53:15.714 7fbd60a3f0f0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 10/02/2025 04:53:15.714 7fbd60a3f0f0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/02/2025 04:53:15.714 7fbd60a3f0f0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/02/2025 04:53:15.714 7fbd60a3f0f0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/02/2025 04:53:15.714 7fbd60a3f0f0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/02/2025 04:53:15.714 7fbd60a3f0f0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/02/2025 04:53:15.715 7fbd60a3f0f0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/02/2025 04:53:15.715 7fbd60a3f0f0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:53:15.715 7fbd60a3f0f0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.48:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/02/2025 04:53:15.715 7fbd60a3f0f0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:53:15.715 7fbd60a3f0f0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:53:15.715 7fbd60a3f0f0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1759380683659182 collector_name = collector_ip = 10.0.0.48:8086 collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/02/2025 04:53:15.715 7fbd60a3f0f0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:53:19.715 7fbd60a3f0f0 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/02/2025 04:53:19.717 7fbd60a3f0f0 [contrail-device-manager] [ERROR]: Session Event: TCP Connect Fail 10/02/2025 04:53:19.717 7fbd60a3f0f0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/02/2025 04:53:19.717 7fbd60a3f0f0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/02/2025 04:53:19.717 7fbd60a3f0f0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/02/2025 04:53:19.717 7fbd60a3f0f0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/02/2025 04:53:19.717 7fbd60a3f0f0 [contrail-device-manager] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 10/02/2025 04:53:19.717 7fbd60a3f0f0 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 10/02/2025 04:53:19.717 7fbd60a3f0f0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:53:19.718 7fbd60a3f0f0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.38:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 10/02/2025 04:53:19.718 7fbd60a3f0f0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:53:19.718 7fbd60a3f0f0 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:53:19.718 7fbd60a3f0f0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3.:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 140 http_port = 8096 start_time = 1759380683659182 collector_name = collector_ip = 10.0.0.38:8086 collector_list = [ 10.0.0.48:8086, 10.0.0.38:8086, 10.0.0.254:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 2 >> 10/02/2025 04:53:19.718 7fbd60a3f0f0 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 10/02/2025 04:53:21.002 7fbd61e69978 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4211-1.'], 'obj_dict': {'analytics_node_ip_address': '10.0.0.254', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-4211-1.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4211-1.'], 'id_perms': {'created': '2025-10-02T04:53:20.974759', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-10-02T04:53:20.974759', 'permissions': {'group': 'heat_stack_owner', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13059809695790631189, 'uuid_mslong': 15245672769487193896}}, 'parent_type': 'global-system-config', 'parent_uuid': '4122808e-6277-4889-95ae-9a4eece9dc49', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'd393829b-db21-4728-b53d-c3b977cd8915'}, 'oper': 'CREATE', 'request-id': 'req-1e0ee946-d49d-4f06-ab4e-02aeb11b0015', 'type': 'analytics_node', 'uuid': 'd393829b-db21-4728-b53d-c3b977cd8915'} 10/02/2025 04:53:21.592 7fbd61e69978 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4211-2.'], 'obj_dict': {'analytics_node_ip_address': '10.0.0.38', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-4211-2.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4211-2.'], 'id_perms': {'created': '2025-10-02T04:53:21.581319', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-10-02T04:53:21.581319', 'permissions': {'group': 'heat_stack_owner', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11490827468900345636, 'uuid_mslong': 6707334192768372490}}, 'parent_type': 'global-system-config', 'parent_uuid': '4122808e-6277-4889-95ae-9a4eece9dc49', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '5d153ce9-4dca-4f0a-9f77-9eb3b31b2724'}, 'oper': 'CREATE', 'request-id': 'req-c68ef47a-799c-440b-ac44-ea9598032c2d', 'type': 'analytics_node', 'uuid': '5d153ce9-4dca-4f0a-9f77-9eb3b31b2724'} 10/02/2025 04:53:21.627 7fbd61e69978 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4211-3.'], 'obj_dict': {'analytics_node_ip_address': '10.0.0.48', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-4211-3.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4211-3.'], 'id_perms': {'created': '2025-10-02T04:53:21.612733', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-10-02T04:53:21.612733', 'permissions': {'group': 'heat_stack_owner', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 9259298303084744505, 'uuid_mslong': 12768527574590244677}}, 'parent_type': 'global-system-config', 'parent_uuid': '4122808e-6277-4889-95ae-9a4eece9dc49', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'b132ec2d-b43b-4745-807f-a2bfb0be4f39'}, 'oper': 'CREATE', 'request-id': 'req-f064fe70-53e4-49cd-ad3d-4e92f9f134d2', 'type': 'analytics_node', 'uuid': 'b132ec2d-b43b-4745-807f-a2bfb0be4f39'} 10/02/2025 04:53:23.718 7fbd60c5aa20 [contrail-device-manager] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 10/02/2025 04:53:23.719 7fbd60c5aa20 [contrail-device-manager] [INFO]: Session Event: TCP Connected 10/02/2025 04:53:23.719 7fbd60c5aa20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 10/02/2025 04:53:23.719 7fbd60c5aa20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 10/02/2025 04:53:23.719 7fbd60c5aa20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-ansible-os-4211-3. process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.254:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.254:5673, 10.0.0.38:5673, 10.0.0.48:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.254:9041, 10.0.0.38:9041, 10.0.0.48:9041, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.254:2181, 10.0.0.38:2181, 10.0.0.48:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.254,10.0.0.38,10.0.0.48:8082, ] status = Up description = ApiServer Connection State updated >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 10/02/2025 04:53:23.719 7fbd60c5aa20 [contrail-device-manager] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 10/02/2025 04:53:23.719 7fbd60c5aa20 [contrail-device-manager] [INFO]: Processing event[EvTcpConnected] in state[Connect] 10/02/2025 04:53:23.720 7fbd60c5aa20 [contrail-device-manager] [DEBUG]: Send sandesh control message. uve type count # 19 10/02/2025 04:53:23.720 7fbd60c5aa20 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit] 10/02/2025 04:53:23.720 7fbd60c5aa20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 10/02/2025 04:53:23.720 7fbd60c5aa20 [contrail-device-manager] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit] 10/02/2025 04:53:23.720 7fbd60a596d8 [contrail-device-manager] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 10/02/2025 04:53:23.721 7fbd60a596d8 [contrail-device-manager] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 10/02/2025 04:53:31.884 7fbd60a56630 [contrail-device-manager] [DEBUG]: Extracted sandesh length: 0000001016 10/02/2025 04:53:31.885 7fbd60a56630 [contrail-device-manager] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 10/02/2025 04:53:31.885 7fbd60aba6d8 [contrail-device-manager] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit] 10/02/2025 04:53:31.885 7fbd60aba6d8 [contrail-device-manager] [DEBUG]: Number of uve types in sandesh control message is 0 10/02/2025 04:53:31.885 7fbd60aba6d8 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 10/02/2025 04:53:31.885 7fbd60aba6d8 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 10/02/2025 04:53:31.886 7fbd60aba6d8 [contrail-device-manager] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established] 10/02/2025 04:53:31.886 7fbd60a56780 [contrail-device-manager] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG] 10/02/2025 04:53:31.887 7fbd60a56780 [contrail-device-manager] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID] 10/02/2025 04:53:36.797 7fbd61e69978 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4211-3.'], 'obj_dict': {'analytics_alarm_node_ip_address': '10.0.0.48', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-4211-3.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4211-3.'], 'id_perms': {'created': '2025-10-02T04:53:36.788848', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-10-02T04:53:36.788848', 'permissions': {'group': 'heat_stack_owner', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11445953127560482189, 'uuid_mslong': 16274971023138901404}}, 'parent_type': 'global-system-config', 'parent_uuid': '4122808e-6277-4889-95ae-9a4eece9dc49', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'e1dc4fea-c3f8-4d9c-9ed8-31b9ec038d8d'}, 'oper': 'CREATE', 'request-id': 'req-97d8bd0a-f9f9-42d2-a096-2d058fc0a61f', 'type': 'analytics_alarm_node', 'uuid': 'e1dc4fea-c3f8-4d9c-9ed8-31b9ec038d8d'} 10/02/2025 04:53:36.875 7fbd61e69978 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4211-1.'], 'obj_dict': {'analytics_alarm_node_ip_address': '10.0.0.254', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-4211-1.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4211-1.'], 'id_perms': {'created': '2025-10-02T04:53:36.866069', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-10-02T04:53:36.866069', 'permissions': {'group': 'heat_stack_owner', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13798090884236524317, 'uuid_mslong': 16122989540067920726}}, 'parent_type': 'global-system-config', 'parent_uuid': '4122808e-6277-4889-95ae-9a4eece9dc49', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'dfc05d90-3d16-4b56-bf7c-aa8dc5793b1d'}, 'oper': 'CREATE', 'request-id': 'req-9a3895ed-be2e-4b2b-98b0-41f3d32672b7', 'type': 'analytics_alarm_node', 'uuid': 'dfc05d90-3d16-4b56-bf7c-aa8dc5793b1d'} 10/02/2025 04:53:36.903 7fbd61e69978 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4211-2.'], 'obj_dict': {'analytics_alarm_node_ip_address': '10.0.0.38', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-4211-2.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4211-2.'], 'id_perms': {'created': '2025-10-02T04:53:36.888955', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-10-02T04:53:36.888955', 'permissions': {'group': 'heat_stack_owner', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 9934014411748685218, 'uuid_mslong': 5844272716693784151}}, 'parent_type': 'global-system-config', 'parent_uuid': '4122808e-6277-4889-95ae-9a4eece9dc49', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '511b0712-510a-4657-89dc-b5798de435a2'}, 'oper': 'CREATE', 'request-id': 'req-319e4404-a4a1-4c3f-8dd3-8d69db0c97b2', 'type': 'analytics_alarm_node', 'uuid': '511b0712-510a-4657-89dc-b5798de435a2'} 10/02/2025 04:53:42.219 7fbd60a56630 [contrail-device-manager] [DEBUG]: Extracted sandesh length: 0000001296 10/02/2025 04:53:42.219 7fbd60a56630 [contrail-device-manager] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient] 10/02/2025 04:53:42.219 7fbd60c5f630 [contrail-device-manager] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[Established] 10/02/2025 04:53:42.219 7fbd60c5f630 [contrail-device-manager] [DEBUG]: Number of uve types in sandesh control message is 2 10/02/2025 04:53:42.220 7fbd60c5f630 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 10/02/2025 04:53:42.220 7fbd60c5f630 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 10/02/2025 04:53:44.601 7fbd61e69978 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4211-1.'], 'obj_dict': {'analytics_snmp_node_ip_address': '10.0.0.254', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-4211-1.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4211-1.'], 'id_perms': {'created': '2025-10-02T04:53:44.589743', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-10-02T04:53:44.589743', 'permissions': {'group': 'heat_stack_owner', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11489126847285741721, 'uuid_mslong': 2418592827387168456}}, 'parent_type': 'global-system-config', 'parent_uuid': '4122808e-6277-4889-95ae-9a4eece9dc49', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '2190915c-bcb8-46c8-9f71-93fedfdec899'}, 'oper': 'CREATE', 'request-id': 'req-d7291ab8-f037-47da-9858-53d2bcf8daea', 'type': 'analytics_snmp_node', 'uuid': '2190915c-bcb8-46c8-9f71-93fedfdec899'} 10/02/2025 04:53:45.140 7fbd61e69978 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4211-2.'], 'obj_dict': {'analytics_snmp_node_ip_address': '10.0.0.38', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-4211-2.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4211-2.'], 'id_perms': {'created': '2025-10-02T04:53:45.134412', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-10-02T04:53:45.134412', 'permissions': {'group': 'heat_stack_owner', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13528174294307021075, 'uuid_mslong': 16237498579238932127}}, 'parent_type': 'global-system-config', 'parent_uuid': '4122808e-6277-4889-95ae-9a4eece9dc49', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'e1572eed-8872-4e9f-bbbd-bad867bda113'}, 'oper': 'CREATE', 'request-id': 'req-b5a13d04-a40e-4674-a759-e5022f87a67a', 'type': 'analytics_snmp_node', 'uuid': 'e1572eed-8872-4e9f-bbbd-bad867bda113'} 10/02/2025 04:53:45.164 7fbd61e69978 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4211-3.'], 'obj_dict': {'analytics_snmp_node_ip_address': '10.0.0.48', 'display_name': 'cn-jenkins-deploy-platform-ansible-os-4211-3.', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-ansible-os-4211-3.'], 'id_perms': {'created': '2025-10-02T04:53:45.150123', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-10-02T04:53:45.150123', 'permissions': {'group': 'heat_stack_owner', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11800573769583541864, 'uuid_mslong': 4027775699851558954}}, 'parent_type': 'global-system-config', 'parent_uuid': '4122808e-6277-4889-95ae-9a4eece9dc49', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '37e588a8-b09a-402a-a3c4-0f5241d1be68'}, 'oper': 'CREATE', 'request-id': 'req-fdc69e2c-cf40-45e0-b98b-01102bc25b64', 'type': 'analytics_snmp_node', 'uuid': '37e588a8-b09a-402a-a3c4-0f5241d1be68'} 10/02/2025 04:55:23.432 7fbd61e69978 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config'], 'oper': 'UPDATE', 'request-id': 'req-cadd62f8-4ab9-4434-9c02-b8ed2607a5f9', 'type': 'global_system_config', 'uuid': '4122808e-6277-4889-95ae-9a4eece9dc49'} 10/02/2025 04:55:23.474 7fbd61e69978 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config'], 'oper': 'UPDATE', 'request-id': 'req-315b6d6b-c4a9-4dda-b442-95c1f41d0f80', 'type': 'global_system_config', 'uuid': '4122808e-6277-4889-95ae-9a4eece9dc49'} 10/02/2025 04:55:23.581 7fbd61e69978 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-4211-2'], 'obj_dict': {'display_name': 'an-jenkins-deploy-platform-ansible-os-4211-2', 'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-4211-2'], 'id_perms': {'created': '2025-10-02T04:55:23.574865', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-10-02T04:55:23.574865', 'permissions': {'group': 'heat_stack_owner', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11369208038856148161, 'uuid_mslong': 11046545300672171828}}, 'parent_type': 'global-system-config', 'parent_uuid': '4122808e-6277-4889-95ae-9a4eece9dc49', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '994d366c-81d3-4f34-9dc7-8a79858944c1', 'virtual_router_dpdk_enabled': False, 'virtual_router_ip_address': '10.20.0.193'}, 'oper': 'CREATE', 'request-id': 'req-e01248dc-db05-4a39-9afd-05020ffea8bf', 'type': 'virtual_router', 'uuid': '994d366c-81d3-4f34-9dc7-8a79858944c1'} 10/02/2025 04:55:23.612 7fbd61e69978 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-4211-1'], 'obj_dict': {'display_name': 'an-jenkins-deploy-platform-ansible-os-4211-1', 'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-4211-1'], 'id_perms': {'created': '2025-10-02T04:55:23.600509', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-10-02T04:55:23.600509', 'permissions': {'group': 'heat_stack_owner', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10831938314391680872, 'uuid_mslong': 6950398146198061232}}, 'parent_type': 'global-system-config', 'parent_uuid': '4122808e-6277-4889-95ae-9a4eece9dc49', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '6074c64a-07ab-40b0-9652-c681c2427768', 'virtual_router_dpdk_enabled': False, 'virtual_router_ip_address': '10.20.0.25'}, 'oper': 'CREATE', 'request-id': 'req-3f7f4f1e-adff-4fda-b6b6-486978b5b6a2', 'type': 'virtual_router', 'uuid': '6074c64a-07ab-40b0-9652-c681c2427768'} 10/02/2025 04:55:23.976 7fbd61e69978 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-4211-2', 'vhost0'], 'obj_dict': {'display_name': 'vhost0', 'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-4211-2', 'vhost0'], 'id_perms': {'created': '2025-10-02T04:55:23.933708', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-10-02T04:55:23.933708', 'permissions': {'group': 'heat_stack_owner', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11717393253164551994, 'uuid_mslong': 491075507757599375}}, 'parent_type': 'virtual-router', 'parent_uuid': '994d366c-81d3-4f34-9dc7-8a79858944c1', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '06d0a692-8dc9-4a8f-a29c-8b15146e8f3a', 'virtual_machine_interface_disable_policy': True, 'virtual_machine_interface_mac_addresses': {'mac_address': ['02:06:d0:a6:92:8d']}, 'virtual_network_refs': [{'to': ['default-domain', 'default-project', 'ip-fabric'], 'uuid': 'bf2664b6-446a-4f5c-90f1-1437dc6ae69c'}]}, 'oper': 'CREATE', 'request-id': 'req-19fdaba3-2a5e-441d-88d9-dd4068bf35b7', 'type': 'virtual_machine_interface', 'uuid': '06d0a692-8dc9-4a8f-a29c-8b15146e8f3a'} 10/02/2025 04:55:24.025 7fbd61e69978 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-4211-1', 'vhost0'], 'obj_dict': {'display_name': 'vhost0', 'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-4211-1', 'vhost0'], 'id_perms': {'created': '2025-10-02T04:55:23.968308', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-10-02T04:55:23.968308', 'permissions': {'group': 'heat_stack_owner', 'group_access': 7, 'other_access': 7, 'owner': 'admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12620072937135502679, 'uuid_mslong': 5233346666515024958}}, 'parent_type': 'virtual-router', 'parent_uuid': '6074c64a-07ab-40b0-9652-c681c2427768', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '48a09510-d41a-4c3e-af23-8178e97b7557', 'virtual_machine_interface_disable_policy': True, 'virtual_machine_interface_mac_addresses': {'mac_address': ['02:48:a0:95:10:d4']}, 'virtual_network_refs': [{'to': ['default-domain', 'default-project', 'ip-fabric'], 'uuid': 'bf2664b6-446a-4f5c-90f1-1437dc6ae69c'}]}, 'oper': 'CREATE', 'request-id': 'req-a9d0ecbe-ebd2-446e-9a55-2f9789408c56', 'type': 'virtual_machine_interface', 'uuid': '48a09510-d41a-4c3e-af23-8178e97b7557'} 10/02/2025 04:55:24.053 7fbd61e69978 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-4211-2', 'vhost0'], 'oper': 'UPDATE', 'request-id': 'req-19fdaba3-2a5e-441d-88d9-dd4068bf35b7', 'type': 'virtual_machine_interface', 'uuid': '06d0a692-8dc9-4a8f-a29c-8b15146e8f3a'} 10/02/2025 04:55:24.097 7fbd61e69978 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-ansible-os-4211-1', 'vhost0'], 'oper': 'UPDATE', 'request-id': 'req-a9d0ecbe-ebd2-446e-9a55-2f9789408c56', 'type': 'virtual_machine_interface', 'uuid': '48a09510-d41a-4c3e-af23-8178e97b7557'} 10/02/2025 04:55:24.515 7fbd61e69978 [contrail-device-manager] [DEBUG]: Notification Message: {'fq_name': ['default-domain', 'default-project', 'ip-fabric', 'ip-fabric'], 'oper': 'UPDATE', 'request-id': 'req-91c04edb-d3e4-421c-a148-5a17e05f88ef', 'type': 'access_control_list', 'uuid': '6fbd8731-3032-4f7f-8ea7-4909383572ad'} 10/02/2025 04:55:56.640 7fbd61dfb048 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 10/02/2025 04:56:13.233 7fbd62ceb2e8 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 10/02/2025 04:56:30.585 7fbd62ceb2e8 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 10/02/2025 04:56:53.117 7fbd62ceb2e8 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 10/02/2025 04:58:48.809 7fbd62ceb2e8 [contrail-device-manager] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map