05/15/2025 00:44:15.428 7f0497074898 [contrail-topology] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0]
05/15/2025 00:44:15.428 7f0497074898 [contrail-topology] [INFO]: SANDESH: Trace: PRINT: [None] -> [False]
05/15/2025 00:44:15.428 7f0497074898 [contrail-topology] [INFO]: SANDESH: Flow Logging: [None] -> [False]
05/15/2025 00:44:15.429 7f0497074898 [contrail-topology] [DEBUG]: Add uve <cn-jenkins-deploy-platform-ansible-os-3312-3., 1> in the [ObjectAnalyticsSNMPInfo:NodeStatusUVE] map
05/15/2025 00:44:15.429 7f0497074898 [contrail-topology] [INFO]: Consistent scheduler zk start
05/15/2025 00:44:15.430 7f0490da1d48 [contrail-topology] [ERROR]: Cannot write http_port 5921 to /tmp/contrail-topology.0.http_port
05/15/2025 00:44:15.430 7f0490da1d48 [contrail-topology] [ERROR]: Starting Introspect on HTTP Port 5921
05/15/2025 00:44:15.430 7f048e30b848 [contrail-topology] [INFO]: Processing event[EvStart] in state[none]
05/15/2025 00:44:15.431 7f048e30b848 [contrail-topology] [DEBUG]: Add uve <cn-jenkins-deploy-platform-ansible-os-3312-3.:AnalyticsSNMP:contrail-topology:0, 1> in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map
05/15/2025 00:44:15.431 7f048e30b848 [contrail-topology] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle]
05/15/2025 00:44:15.431 7f048e30b848 [contrail-topology] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/15/2025 00:44:15.431 7f048e30b848 [contrail-topology] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3312-3.  process_status = [ <<  module_id = contrail-topology  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.73:2181, 10.0.0.23:2181, 10.0.0.240:2181,  ]  status = Initializing  description = Connection to Zookeeper initialized  >>,  ]  description = Zookeeper:Zookeeper[Connection to Zookeeper initialized] connection down  >>,  ]  >>  
05/15/2025 00:44:15.431 7f048e30b848 [contrail-topology] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/15/2025 00:44:15.431 7f048e30b848 [contrail-topology] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/15/2025 00:44:15.431 7f048e30b848 [contrail-topology] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3312-3.  process_status = [ <<  module_id = contrail-topology  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.73:2181, 10.0.0.23:2181, 10.0.0.240:2181,  ]  status = Initializing  description = Connection to Zookeeper initialized  >>, <<  type = Collector  name = Collector  server_addrs = [ ,  ]  status = Down  description = none to Idle on EvStart  >>,  ]  description = Zookeeper:Zookeeper[Connection to Zookeeper initialized], Collector:Collector[none to Idle on EvStart] connection down  >>,  ]  >>  
05/15/2025 00:44:15.431 7f048e30b848 [contrail-topology] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/15/2025 00:44:15.431 7f048e30b848 [contrail-topology] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/15/2025 00:44:15.431 7f048e30b848 [contrail-topology] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3312-3.:AnalyticsSNMP:contrail-topology:0  client_info = <<  status = Idle  successful_connections = 0  pid = 1  http_port = 5921  start_time = 1747269855430957  collector_name =   collector_ip =   collector_list = [ 10.0.0.23:8086, 10.0.0.240:8086, 10.0.0.73:8086,  ]  >>  sm_queue_count = 2  max_sm_queue_count = 2  >>  
05/15/2025 00:44:15.431 7f048e30b848 [contrail-topology] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/15/2025 00:44:15.439 7f048e244848 [contrail-topology] [INFO]: Consistent scheduler listen CONNECTED
05/15/2025 00:44:15.439 7f048e30b848 [contrail-topology] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/15/2025 00:44:15.440 7f048e30b848 [contrail-topology] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3312-3.  process_status = [ <<  module_id = contrail-topology  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.73:2181, 10.0.0.23:2181, 10.0.0.240:2181,  ]  status = Up  description = Connection to Zookeeper established  >>, <<  type = Collector  name = Collector  server_addrs = [ ,  ]  status = Down  description = none to Idle on EvStart  >>,  ]  description = Collector:Collector[none to Idle on EvStart] connection down  >>,  ]  >>  
05/15/2025 00:44:15.440 7f048e30b848 [contrail-topology] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/15/2025 00:44:15.531 7f048e244b48 [contrail-topology] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.73:5673//', 'pyamqp://guest:guest@10.0.0.23:5673//', 'pyamqp://guest:guest@10.0.0.240:5673//']
05/15/2025 00:44:15.535 7f048e244b48 [contrail-topology] [WARNING]: RabbitMQ drainer connection down
05/15/2025 00:44:15.536 7f048e244e48 [contrail-topology] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/15/2025 00:44:15.537 7f048e244e48 [contrail-topology] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3312-3.  process_status = [ <<  module_id = contrail-topology  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.73:2181, 10.0.0.23:2181, 10.0.0.240:2181,  ]  status = Up  description = Connection to Zookeeper established  >>, <<  type = Collector  name = Collector  server_addrs = [ ,  ]  status = Down  description = none to Idle on EvStart  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.73:5673, 10.0.0.23:5673, 10.0.0.240:5673,  ]  status = Initializing  description =   >>,  ]  description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down  >>,  ]  >>  
05/15/2025 00:44:15.537 7f048e244e48 [contrail-topology] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/15/2025 00:44:15.537 7f048e244e48 [contrail-topology] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/15/2025 00:44:15.537 7f048e244e48 [contrail-topology] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3312-3.  process_status = [ <<  module_id = contrail-topology  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.73:2181, 10.0.0.23:2181, 10.0.0.240:2181,  ]  status = Up  description = Connection to Zookeeper established  >>, <<  type = Collector  name = Collector  server_addrs = [ ,  ]  status = Down  description = none to Idle on EvStart  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.73:5673, 10.0.0.23:5673, 10.0.0.240:5673,  ]  status = Down  description =   >>,  ]  description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down  >>,  ]  >>  
05/15/2025 00:44:15.537 7f048e244e48 [contrail-topology] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/15/2025 00:44:15.545 7f048e244b48 [contrail-topology] [WARNING]: RabbitMQ drainer connection ESTABLISHED <Connection: amqp://guest:**@10.0.0.73:5673// at 0x7f048c116f28>
05/15/2025 00:44:15.545 7f048e244e48 [contrail-topology] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/15/2025 00:44:15.545 7f048e244e48 [contrail-topology] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3312-3.  process_status = [ <<  module_id = contrail-topology  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.73:2181, 10.0.0.23:2181, 10.0.0.240:2181,  ]  status = Up  description = Connection to Zookeeper established  >>, <<  type = Collector  name = Collector  server_addrs = [ ,  ]  status = Down  description = none to Idle on EvStart  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.73:5673, 10.0.0.23:5673, 10.0.0.240:5673,  ]  status = Up  description =   >>,  ]  description = Collector:Collector[none to Idle on EvStart] connection down  >>,  ]  >>  
05/15/2025 00:44:15.545 7f048e244e48 [contrail-topology] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/15/2025 00:44:15.581 7f048e244a48 [contrail-topology] [DEBUG]: Waiting for allocation...
05/15/2025 00:44:15.681 7f048e244b48 [contrail-topology] [WARNING]: RabbitMQ publish connection down
05/15/2025 00:44:15.682 7f048e244d48 [contrail-topology] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/15/2025 00:44:15.682 7f048e244d48 [contrail-topology] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3312-3.  process_status = [ <<  module_id = contrail-topology  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.73:2181, 10.0.0.23:2181, 10.0.0.240:2181,  ]  status = Up  description = Connection to Zookeeper established  >>, <<  type = Collector  name = Collector  server_addrs = [ ,  ]  status = Down  description = none to Idle on EvStart  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.73:5673, 10.0.0.23:5673, 10.0.0.240:5673,  ]  status = Down  description =   >>,  ]  description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down  >>,  ]  >>  
05/15/2025 00:44:15.682 7f048e244d48 [contrail-topology] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/15/2025 00:44:15.686 7f048e244b48 [contrail-topology] [WARNING]: RabbitMQ publish connection ESTABLISHED <Connection: amqp://guest:**@10.0.0.73:5673// at 0x7f048c11dcf8>
05/15/2025 00:44:15.686 7f048e244d48 [contrail-topology] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/15/2025 00:44:15.686 7f048e244d48 [contrail-topology] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3312-3.  process_status = [ <<  module_id = contrail-topology  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.73:2181, 10.0.0.23:2181, 10.0.0.240:2181,  ]  status = Up  description = Connection to Zookeeper established  >>, <<  type = Collector  name = Collector  server_addrs = [ ,  ]  status = Down  description = none to Idle on EvStart  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.73:5673, 10.0.0.23:5673, 10.0.0.240:5673,  ]  status = Up  description =   >>,  ]  description = Collector:Collector[none to Idle on EvStart] connection down  >>,  ]  >>  
05/15/2025 00:44:15.686 7f048e244d48 [contrail-topology] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/15/2025 00:44:15.687 7f048e244b48 [contrail-topology] [WARNING]: VNCCassandra started with driver cql
05/15/2025 00:44:15.687 7f048e244b48 [contrail-topology] [WARNING]: initializing cassandra cluster
05/15/2025 00:44:15.687 7f048e244b48 [contrail-topology] [WARNING]: CassandraDriverCQL.__init__Cluster before create
05/15/2025 00:44:15.687 7f048e244d48 [contrail-topology] [WARNING]: RabbitMQ publish connection down
05/15/2025 00:44:15.688 7f048e244e48 [contrail-topology] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/15/2025 00:44:15.688 7f048e244e48 [contrail-topology] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3312-3.  process_status = [ <<  module_id = contrail-topology  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.73:2181, 10.0.0.23:2181, 10.0.0.240:2181,  ]  status = Up  description = Connection to Zookeeper established  >>, <<  type = Collector  name = Collector  server_addrs = [ ,  ]  status = Down  description = none to Idle on EvStart  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.73:5673, 10.0.0.23:5673, 10.0.0.240:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.73:9041, 10.0.0.23:9041, 10.0.0.240:9041,  ]  status = Initializing  description =   >>,  ]  description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[] connection down  >>,  ]  >>  
05/15/2025 00:44:15.688 7f048e244e48 [contrail-topology] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/15/2025 00:44:15.688 7f048e244e48 [contrail-topology] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/15/2025 00:44:15.688 7f048e244e48 [contrail-topology] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3312-3.  process_status = [ <<  module_id = contrail-topology  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.73:2181, 10.0.0.23:2181, 10.0.0.240:2181,  ]  status = Up  description = Connection to Zookeeper established  >>, <<  type = Collector  name = Collector  server_addrs = [ ,  ]  status = Down  description = none to Idle on EvStart  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.73:5673, 10.0.0.23:5673, 10.0.0.240:5673,  ]  status = Down  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.73:9041, 10.0.0.23:9041, 10.0.0.240:9041,  ]  status = Initializing  description =   >>,  ]  description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[], Database:Cassandra[] connection down  >>,  ]  >>  
05/15/2025 00:44:15.688 7f048e244e48 [contrail-topology] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/15/2025 00:44:15.691 7f048e244b48 [contrail-topology] [WARNING]: CassandraDriverCQL.__init__Cluster created
05/15/2025 00:44:15.703 7f048e244d48 [contrail-topology] [WARNING]: RabbitMQ publish connection ESTABLISHED <Connection: amqp://guest:**@10.0.0.73:5673// at 0x7f048c11dcf8>
05/15/2025 00:44:15.703 7f048c250d48 [contrail-topology] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/15/2025 00:44:15.703 7f048c250d48 [contrail-topology] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3312-3.  process_status = [ <<  module_id = contrail-topology  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.73:2181, 10.0.0.23:2181, 10.0.0.240:2181,  ]  status = Up  description = Connection to Zookeeper established  >>, <<  type = Collector  name = Collector  server_addrs = [ ,  ]  status = Down  description = none to Idle on EvStart  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.73:5673, 10.0.0.23:5673, 10.0.0.240:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.73:9041, 10.0.0.23:9041, 10.0.0.240:9041,  ]  status = Initializing  description =   >>,  ]  description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[] connection down  >>,  ]  >>  
05/15/2025 00:44:15.704 7f048c250d48 [contrail-topology] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/15/2025 00:44:15.746 7f048e244b48 [contrail-topology] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected
05/15/2025 00:44:15.748 7f048c095b48 [contrail-topology] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/15/2025 00:44:15.749 7f048c095b48 [contrail-topology] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3312-3.  process_status = [ <<  module_id = contrail-topology  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.73:2181, 10.0.0.23:2181, 10.0.0.240:2181,  ]  status = Initializing  description =   >>, <<  type = Collector  name = Collector  server_addrs = [ ,  ]  status = Down  description = none to Idle on EvStart  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.73:5673, 10.0.0.23:5673, 10.0.0.240:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.73:9041, 10.0.0.23:9041, 10.0.0.240:9041,  ]  status = Initializing  description =   >>,  ]  description = Zookeeper:Zookeeper[], Collector:Collector[none to Idle on EvStart], Database:Cassandra[] connection down  >>,  ]  >>  
05/15/2025 00:44:15.749 7f048c095b48 [contrail-topology] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/15/2025 00:44:15.765 7f048e244b48 [contrail-topology] [WARNING]: CassandraDriverCQL.__init__Cluster init KS
05/15/2025 00:44:15.765 7f048c095648 [contrail-topology] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/15/2025 00:44:15.765 7f048c095648 [contrail-topology] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3312-3.  process_status = [ <<  module_id = contrail-topology  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.73:2181, 10.0.0.23:2181, 10.0.0.240:2181,  ]  status = Up  description =   >>, <<  type = Collector  name = Collector  server_addrs = [ ,  ]  status = Down  description = none to Idle on EvStart  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.73:5673, 10.0.0.23:5673, 10.0.0.240:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.73:9041, 10.0.0.23:9041, 10.0.0.240:9041,  ]  status = Initializing  description =   >>,  ]  description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[] connection down  >>,  ]  >>  
05/15/2025 00:44:15.765 7f048c095648 [contrail-topology] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/15/2025 00:44:15.771 7f048e244b48 [contrail-topology] [WARNING]: CassandraDriverCQL.__init__Cluster KS are ready
05/15/2025 00:44:15.814 7f048e244b48 [contrail-topology] [WARNING]: CassandraDriverCQL.__init__Cluster CF are ready
05/15/2025 00:44:15.815 7f047fdddb48 [contrail-topology] [INFO]: Processing event[EvSandeshUVESend] in state[Idle]
05/15/2025 00:44:15.815 7f047fdddb48 [contrail-topology] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3312-3.  process_status = [ <<  module_id = contrail-topology  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.73:2181, 10.0.0.23:2181, 10.0.0.240:2181,  ]  status = Up  description =   >>, <<  type = Collector  name = Collector  server_addrs = [ ,  ]  status = Down  description = none to Idle on EvStart  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.73:5673, 10.0.0.23:5673, 10.0.0.240:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.73:9041, 10.0.0.23:9041, 10.0.0.240:9041,  ]  status = Up  description =   >>,  ]  description = Collector:Collector[none to Idle on EvStart] connection down  >>,  ]  >>  
05/15/2025 00:44:15.815 7f047fdddb48 [contrail-topology] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle]
05/15/2025 00:44:19.431 7f047fdddb48 [contrail-topology] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle]
05/15/2025 00:44:19.433 7f047fdddb48 [contrail-topology] [INFO]: Session Event: TCP Connected
05/15/2025 00:44:19.434 7f047fdddb48 [contrail-topology] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect]
05/15/2025 00:44:19.434 7f047fdddb48 [contrail-topology] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]
05/15/2025 00:44:19.434 7f047fdddb48 [contrail-topology] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = <<  name = cn-jenkins-deploy-platform-ansible-os-3312-3.  process_status = [ <<  module_id = contrail-topology  instance_id = 0  state = Non-Functional  connection_infos = [ <<  type = Zookeeper  name = Zookeeper  server_addrs = [ 10.0.0.73:2181, 10.0.0.23:2181, 10.0.0.240:2181,  ]  status = Up  description =   >>, <<  type = Collector  name = Collector  server_addrs = [ 10.0.0.23:8086,  ]  status = Initializing  description = Idle to Connect on EvIdleHoldTimerExpired  >>, <<  type = Database  name = RabbitMQ  server_addrs = [ 10.0.0.73:5673, 10.0.0.23:5673, 10.0.0.240:5673,  ]  status = Up  description =   >>, <<  type = Database  name = Cassandra  server_addrs = [ 10.0.0.73:9041, 10.0.0.23:9041, 10.0.0.240:9041,  ]  status = Up  description =   >>,  ]  description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down  >>,  ]  >>  
05/15/2025 00:44:19.434 7f047fdddb48 [contrail-topology] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect]
05/15/2025 00:44:19.434 7f047fdddb48 [contrail-topology] [INFO]: Processing event[EvTcpConnected] in state[Connect]
05/15/2025 00:44:19.434 7f047fdddb48 [contrail-topology] [DEBUG]: Send sandesh control message. uve type count # 5
05/15/2025 00:44:19.434 7f047fdddb48 [contrail-topology] [INFO]: Sandesh Client: Event[EvTcpConnected] => State[Connect] -> State[ClientInit]
05/15/2025 00:44:19.434 7f047fdddb48 [contrail-topology] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit]
05/15/2025 00:44:19.435 7f047fdddb48 [contrail-topology] [INFO]: Processing event[EvSandeshUVESend] in state[ClientInit]
05/15/2025 00:44:19.435 7f047fdddd48 [contrail-topology] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG]
05/15/2025 00:44:19.436 7f047fdddd48 [contrail-topology] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID]
05/15/2025 00:44:19.440 7f047fdddc48 [contrail-topology] [DEBUG]: Extracted sandesh length: 0000001017
05/15/2025 00:44:19.441 7f047fdddc48 [contrail-topology] [DEBUG]: Received sandesh control message [SandeshCtrlServerToClient]
05/15/2025 00:44:19.441 7f047fdddd48 [contrail-topology] [INFO]: Processing event[EvSandeshCtrlMessageRecv] in state[ClientInit]
05/15/2025 00:44:19.441 7f047fdddd48 [contrail-topology] [DEBUG]: Number of uve types in sandesh control message is 0
05/15/2025 00:44:19.441 7f047fdddd48 [contrail-topology] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-3312-3.: 16> in the [ObjectAnalyticsSNMPInfo:NodeStatusUVE] map
05/15/2025 00:44:19.441 7f047fdddd48 [contrail-topology] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-3312-3.:AnalyticsSNMP:contrail-topology:0: 2> in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map
05/15/2025 00:44:19.442 7f047fdddd48 [contrail-topology] [INFO]: Sandesh Client: Event[EvSandeshCtrlMessageRecv] => State[ClientInit] -> State[Established]
05/15/2025 00:44:19.442 7f048e244948 [contrail-topology] [INFO]: Sandesh Send Level [INVALID] -> [SYS_DEBUG]
05/15/2025 00:44:19.443 7f048e244948 [contrail-topology] [INFO]: Sandesh Send Level [SYS_DEBUG] -> [INVALID]
05/15/2025 00:44:45.543 7f048e30b848 [contrail-topology] [INFO]: members: ['cn-jenkins-deploy-platform-ansible-os-3312-3.-1', 'cn-jenkins-deploy-platform-ansible-os-3312-1.-1', 'cn-jenkins-deploy-platform-ansible-os-3312-2.-1']
05/15/2025 00:44:45.544 7f048e30b848 [contrail-topology] [INFO]: partitions: ['11', '12', '19', '21', '23', '24', '25', '30', '36', '37', '38', '42', '43', '45']
05/15/2025 00:44:46.597 7f048e244a48 [contrail-topology] [DEBUG]: got work:  ['11', '12', '19', '21', '23', '24', '25', '30', '36', '37', '38', '42', '43', '45']
05/15/2025 00:44:46.597 7f048e244a48 [contrail-topology] [DEBUG]: @populate_work_items(): done!
05/15/2025 00:44:46.597 7f048e244a48 [contrail-topology] [DEBUG]: work items:  [] from the list []
05/15/2025 00:44:46.597 7f048e244a48 [contrail-topology] [DEBUG]: Add uve <cn-jenkins-deploy-platform-ansible-os-3312-3., 1> in the [ObjectAnalyticsSNMPInfo:TopologyUVE] map
05/15/2025 00:45:51.630 7f048e244c48 [contrail-topology] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config'],
 'oper': 'UPDATE',
 'request-id': 'req-1880ab7c-1d57-4acc-b2f6-113e19e156fe',
 'type': 'global_system_config',
 'uuid': '879f3268-0db5-40db-9548-dcd7934cd654'}
05/15/2025 00:45:51.685 7f048e244c48 [contrail-topology] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config'],
 'oper': 'UPDATE',
 'request-id': 'req-9d87db9a-2b08-4a86-9f80-51905d2f631a',
 'type': 'global_system_config',
 'uuid': '879f3268-0db5-40db-9548-dcd7934cd654'}
05/15/2025 00:45:51.770 7f048e244c48 [contrail-topology] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config',
             'an-jenkins-deploy-platform-ansible-os-3312-1'],
 'obj_dict': {'display_name': 'an-jenkins-deploy-platform-ansible-os-3312-1',
              'fq_name': ['default-global-system-config',
                          'an-jenkins-deploy-platform-ansible-os-3312-1'],
              'id_perms': {'created': '2025-05-15T00:45:51.762710',
                           'creator': None,
                           'description': None,
                           'enable': True,
                           'last_modified': '2025-05-15T00:45:51.762710',
                           'permissions': {'group': 'reader',
                                           'group_access': 7,
                                           'other_access': 7,
                                           'owner': 'admin',
                                           'owner_access': 7},
                           'user_visible': True,
                           'uuid': {'uuid_lslong': 11126166362920955668,
                                    'uuid_mslong': 3091933792204374264}},
              'parent_type': 'global-system-config',
              'parent_uuid': '879f3268-0db5-40db-9548-dcd7934cd654',
              'perms2': {'global_access': 0,
                         'owner': 'cloud-admin',
                         'owner_access': 7,
                         'share': []},
              'uuid': '2ae8c166-f95e-40f8-9a68-155badcdaf14',
              'virtual_router_dpdk_enabled': False,
              'virtual_router_ip_address': '10.20.0.17'},
 'oper': 'CREATE',
 'request-id': 'req-0ba0b4d0-4c27-4749-8134-efbf0dc6fa29',
 'type': 'virtual_router',
 'uuid': '2ae8c166-f95e-40f8-9a68-155badcdaf14'}
05/15/2025 00:45:51.859 7f048e244c48 [contrail-topology] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config',
             'an-jenkins-deploy-platform-ansible-os-3312-2'],
 'obj_dict': {'display_name': 'an-jenkins-deploy-platform-ansible-os-3312-2',
              'fq_name': ['default-global-system-config',
                          'an-jenkins-deploy-platform-ansible-os-3312-2'],
              'id_perms': {'created': '2025-05-15T00:45:51.850893',
                           'creator': None,
                           'description': None,
                           'enable': True,
                           'last_modified': '2025-05-15T00:45:51.850893',
                           'permissions': {'group': 'reader',
                                           'group_access': 7,
                                           'other_access': 7,
                                           'owner': 'admin',
                                           'owner_access': 7},
                           'user_visible': True,
                           'uuid': {'uuid_lslong': 9758855119373212497,
                                    'uuid_mslong': 6446139933397896931}},
              'parent_type': 'global-system-config',
              'parent_uuid': '879f3268-0db5-40db-9548-dcd7934cd654',
              'perms2': {'global_access': 0,
                         'owner': 'cloud-admin',
                         'owner_access': 7,
                         'share': []},
              'uuid': '59754a1d-c6a4-4ee3-876e-6b06c4be1b51',
              'virtual_router_dpdk_enabled': False,
              'virtual_router_ip_address': '10.20.0.193'},
 'oper': 'CREATE',
 'request-id': 'req-add4aec5-5719-4e56-a37b-2507365c02db',
 'type': 'virtual_router',
 'uuid': '59754a1d-c6a4-4ee3-876e-6b06c4be1b51'}
05/15/2025 00:45:52.125 7f048e244c48 [contrail-topology] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config',
             'an-jenkins-deploy-platform-ansible-os-3312-1',
             'vhost0'],
 'obj_dict': {'display_name': 'vhost0',
              'fq_name': ['default-global-system-config',
                          'an-jenkins-deploy-platform-ansible-os-3312-1',
                          'vhost0'],
              'id_perms': {'created': '2025-05-15T00:45:52.109908',
                           'creator': None,
                           'description': None,
                           'enable': True,
                           'last_modified': '2025-05-15T00:45:52.109908',
                           'permissions': {'group': 'reader',
                                           'group_access': 7,
                                           'other_access': 7,
                                           'owner': 'admin',
                                           'owner_access': 7},
                           'user_visible': True,
                           'uuid': {'uuid_lslong': 9868465466631269728,
                                    'uuid_mslong': 17274529245943383843}},
              'parent_type': 'virtual-router',
              'parent_uuid': '2ae8c166-f95e-40f8-9a68-155badcdaf14',
              'perms2': {'global_access': 0,
                         'owner': 'cloud-admin',
                         'owner_access': 7,
                         'share': []},
              'uuid': 'efbb74d3-268d-4723-88f3-d50e78c42d60',
              'virtual_machine_interface_disable_policy': True,
              'virtual_machine_interface_mac_addresses': {'mac_address': ['02:ef:bb:74:d3:26']},
              'virtual_network_refs': [{'to': ['default-domain',
                                               'default-project',
                                               'ip-fabric'],
                                        'uuid': '045a9397-b709-4d22-8b57-b32e04b8b530'}]},
 'oper': 'CREATE',
 'request-id': 'req-99e0b514-0bf7-4de3-9710-bc696da622f3',
 'type': 'virtual_machine_interface',
 'uuid': 'efbb74d3-268d-4723-88f3-d50e78c42d60'}
05/15/2025 00:45:52.173 7f048e244c48 [contrail-topology] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config',
             'an-jenkins-deploy-platform-ansible-os-3312-1',
             'vhost0'],
 'oper': 'UPDATE',
 'request-id': 'req-99e0b514-0bf7-4de3-9710-bc696da622f3',
 'type': 'virtual_machine_interface',
 'uuid': 'efbb74d3-268d-4723-88f3-d50e78c42d60'}
05/15/2025 00:45:52.241 7f048e244c48 [contrail-topology] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config',
             'an-jenkins-deploy-platform-ansible-os-3312-2',
             'vhost0'],
 'obj_dict': {'display_name': 'vhost0',
              'fq_name': ['default-global-system-config',
                          'an-jenkins-deploy-platform-ansible-os-3312-2',
                          'vhost0'],
              'id_perms': {'created': '2025-05-15T00:45:52.223249',
                           'creator': None,
                           'description': None,
                           'enable': True,
                           'last_modified': '2025-05-15T00:45:52.223249',
                           'permissions': {'group': 'reader',
                                           'group_access': 7,
                                           'other_access': 7,
                                           'owner': 'admin',
                                           'owner_access': 7},
                           'user_visible': True,
                           'uuid': {'uuid_lslong': 12396442995029178916,
                                    'uuid_mslong': 1302353362405247641}},
              'parent_type': 'virtual-router',
              'parent_uuid': '59754a1d-c6a4-4ee3-876e-6b06c4be1b51',
              'perms2': {'global_access': 0,
                         'owner': 'cloud-admin',
                         'owner_access': 7,
                         'share': []},
              'uuid': '1212e37b-a233-4a99-ac09-03398d17fa24',
              'virtual_machine_interface_disable_policy': True,
              'virtual_machine_interface_mac_addresses': {'mac_address': ['02:12:12:e3:7b:a2']},
              'virtual_network_refs': [{'to': ['default-domain',
                                               'default-project',
                                               'ip-fabric'],
                                        'uuid': '045a9397-b709-4d22-8b57-b32e04b8b530'}]},
 'oper': 'CREATE',
 'request-id': 'req-d5cb8dfd-09d6-4df2-be40-5cf155bf270d',
 'type': 'virtual_machine_interface',
 'uuid': '1212e37b-a233-4a99-ac09-03398d17fa24'}
05/15/2025 00:45:52.279 7f048e244c48 [contrail-topology] [DEBUG]: Notification Message: {'fq_name': ['default-global-system-config',
             'an-jenkins-deploy-platform-ansible-os-3312-2',
             'vhost0'],
 'oper': 'UPDATE',
 'request-id': 'req-d5cb8dfd-09d6-4df2-be40-5cf155bf270d',
 'type': 'virtual_machine_interface',
 'uuid': '1212e37b-a233-4a99-ac09-03398d17fa24'}
05/15/2025 00:45:52.666 7f048e244c48 [contrail-topology] [DEBUG]: Notification Message: {'fq_name': ['default-domain', 'default-project', 'ip-fabric', 'ip-fabric'],
 'oper': 'UPDATE',
 'request-id': 'req-48521a5e-fd49-4c56-9d27-06f5861e9229',
 'type': 'access_control_list',
 'uuid': '166e53c1-2131-46cd-b7e4-91cd2efd3116'}
05/15/2025 00:46:15.461 7f048e30b848 [contrail-topology] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-3312-3.: 17> in the [ObjectAnalyticsSNMPInfo:NodeStatusUVE] map
05/15/2025 00:46:45.512 7f048e30b848 [contrail-topology] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-3312-3.: 17> in the [ObjectAnalyticsSNMPInfo:NodeStatusUVE] map
05/15/2025 00:54:46.611 7f048e244a48 [contrail-topology] [DEBUG]: got work:  ['11', '12', '19', '21', '23', '24', '25', '30', '36', '37', '38', '42', '43', '45']
05/15/2025 00:54:46.611 7f048e244a48 [contrail-topology] [DEBUG]: @populate_work_items(): done!
05/15/2025 00:54:46.611 7f048e244a48 [contrail-topology] [DEBUG]: work items:  [] from the list []
05/15/2025 01:04:46.632 7f048e244a48 [contrail-topology] [DEBUG]: got work:  ['11', '12', '19', '21', '23', '24', '25', '30', '36', '37', '38', '42', '43', '45']
05/15/2025 01:04:46.632 7f048e244a48 [contrail-topology] [DEBUG]: @populate_work_items(): done!
05/15/2025 01:04:46.632 7f048e244a48 [contrail-topology] [DEBUG]: work items:  [] from the list []
05/15/2025 01:14:46.645 7f048e244a48 [contrail-topology] [DEBUG]: got work:  ['11', '12', '19', '21', '23', '24', '25', '30', '36', '37', '38', '42', '43', '45']
05/15/2025 01:14:46.645 7f048e244a48 [contrail-topology] [DEBUG]: @populate_work_items(): done!
05/15/2025 01:14:46.645 7f048e244a48 [contrail-topology] [DEBUG]: work items:  [] from the list []
05/15/2025 01:24:46.658 7f048e244a48 [contrail-topology] [DEBUG]: got work:  ['11', '12', '19', '21', '23', '24', '25', '30', '36', '37', '38', '42', '43', '45']
05/15/2025 01:24:46.658 7f048e244a48 [contrail-topology] [DEBUG]: @populate_work_items(): done!
05/15/2025 01:24:46.658 7f048e244a48 [contrail-topology] [DEBUG]: work items:  [] from the list []
05/15/2025 01:34:46.671 7f048e244a48 [contrail-topology] [DEBUG]: got work:  ['11', '12', '19', '21', '23', '24', '25', '30', '36', '37', '38', '42', '43', '45']
05/15/2025 01:34:46.671 7f048e244a48 [contrail-topology] [DEBUG]: @populate_work_items(): done!
05/15/2025 01:34:46.671 7f048e244a48 [contrail-topology] [DEBUG]: work items:  [] from the list []
05/15/2025 01:38:43.429 7f048e30b848 [contrail-topology] [DEBUG]: send sync_uve <cn-jenkins-deploy-platform-ansible-os-3312-3.: 17> in the [ObjectAnalyticsSNMPInfo:NodeStatusUVE] map