2025-08-04 02:32:54,119 - INFO - Using existing project ['default-domain', 'admin'](b12c869f-cf68-4ba1-84ef-c49e0683dbcf)
2025-08-04 02:32:54,130 - INFO - ================================================================================
2025-08-04 02:32:54,131 - INFO - STARTING TEST : test_db_nodemgr_status
2025-08-04 02:32:54,131 - INFO - TEST DESCRIPTION : Test to verify db nodemgr status
2025-08-04 02:32:54,787 - DEBUG - Nothing to compare xmpp stats {'10.0.0.41': {'10.20.0.18': '0', '10.20.0.129': '0'}, '10.0.0.57': {'10.20.0.193': '0', '10.20.0.18': '0'}} with
2025-08-04 02:32:54,787 - INFO - Initial checks done. Running the testcase now
2025-08-04 02:32:54,787 - INFO -
2025-08-04 02:32:54,787 - DEBUG - Verifying if db node_mgr running...
2025-08-04 02:32:54,787 - INFO - Verifying through collector 10.0.0.61 for db node cn-jenkins-deploy-platform-ansible-os-3750-1.
2025-08-04 02:32:54,787 - DEBUG - Requesting: http://10.0.0.61:8081/analytics/uves/database-node/cn-jenkins-deploy-platform-ansible-os-3750-1.?flat
2025-08-04 02:32:55,198 - DEBUG - Requesting: http://10.0.0.61:8081/analytics/uves/database-node/cn-jenkins-deploy-platform-ansible-os-3750-1.?flat
2025-08-04 02:32:55,279 - INFO - Verifying through collector 10.0.0.61 for db node cn-jenkins-deploy-platform-ansible-os-3750-2.
2025-08-04 02:32:55,279 - DEBUG - Requesting: http://10.0.0.61:8081/analytics/uves/database-node/cn-jenkins-deploy-platform-ansible-os-3750-2.?flat
2025-08-04 02:32:55,378 - INFO - Verifying through collector 10.0.0.61 for db node cn-jenkins-deploy-platform-ansible-os-3750-3.
2025-08-04 02:32:55,378 - DEBUG - Requesting: http://10.0.0.61:8081/analytics/uves/database-node/cn-jenkins-deploy-platform-ansible-os-3750-3.?flat
2025-08-04 02:32:56,114 - DEBUG - No XMPP flaps were noticed during the test
2025-08-04 02:32:56,114 - INFO - END TEST : test_db_nodemgr_status : PASSED[0:00:02]
2025-08-04 02:32:56,115 - INFO - --------------------------------------------------------------------------------
2025-08-04 02:32:56,118 - INFO - ================================================================================
2025-08-04 02:32:56,118 - INFO - STARTING TEST : test_verify_generator_collector_connections
2025-08-04 02:32:56,118 - INFO - TEST DESCRIPTION :
Description: Verify generator:module connections to collector
1.Verify all generators connected to collector - fails otherwise
2.Get the xmpp peers in vrouter uve and get the active xmpp peer out of it
3.Verify from agent introspect that active xmpp matches with step 2 - fails otherwise
4.Get bgp peers from bgp-peer uve and verify from control node introspect that that matches - fails otherwise
Maintainer: sandipd@juniper.net
2025-08-04 02:32:56,762 - DEBUG - Nothing to compare xmpp stats {'10.0.0.41': {'10.20.0.18': '0', '10.20.0.129': '0'}, '10.0.0.57': {'10.20.0.193': '0', '10.20.0.18': '0'}} with
2025-08-04 02:32:56,762 - INFO - Initial checks done. Running the testcase now
2025-08-04 02:32:56,762 - INFO -
2025-08-04 02:32:56,772 - INFO - IP 10.0.0.27 port 8083 connected to collector 10.0.0.61
2025-08-04 02:32:56,780 - INFO - IP 10.0.0.38 port 8083 connected to collector 10.0.0.61
2025-08-04 02:32:56,788 - INFO - IP 10.0.0.61 port 8083 connected to collector 10.0.0.61
2025-08-04 02:32:56,794 - INFO - IP 10.0.0.27 port 8084 connected to collector 10.0.0.38
2025-08-04 02:32:56,800 - INFO - IP 10.0.0.38 port 8084 connected to collector 10.0.0.38
2025-08-04 02:32:56,805 - INFO - IP 10.0.0.61 port 8084 connected to collector 10.0.0.61
2025-08-04 02:32:56,813 - INFO - IP 10.0.0.38 port 8087 connected to collector 10.0.0.38
2025-08-04 02:32:56,821 - INFO - IP 10.0.0.38 port 8088 connected to collector 10.0.0.38
2025-08-04 02:32:56,827 - INFO - IP 10.0.0.27 port 8090 connected to collector 10.0.0.61
2025-08-04 02:32:56,833 - INFO - IP 10.0.0.38 port 8090 connected to collector 10.0.0.27
2025-08-04 02:32:56,841 - INFO - IP 10.0.0.61 port 8090 connected to collector 10.0.0.61
2025-08-04 02:32:56,849 - INFO - IP 10.0.0.27 port 8091 connected to collector 10.0.0.27
2025-08-04 02:32:56,857 - INFO - IP 10.0.0.38 port 8091 connected to collector 10.0.0.38
2025-08-04 02:32:56,864 - INFO - IP 10.0.0.61 port 8091 connected to collector 10.0.0.27
2025-08-04 02:32:56,865 - DEBUG - Verifying through opserver in 10.0.0.27
2025-08-04 02:32:57,002 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3750-1:Control:contrail-control:0 is connected to collector 10.0.0.27
2025-08-04 02:32:57,002 - DEBUG - Verifying through opserver in 10.0.0.27
2025-08-04 02:32:57,093 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3750-1:Control:contrail-dns:0 is connected to collector 10.0.0.27
2025-08-04 02:32:57,093 - DEBUG - Verifying through opserver in 10.0.0.27
2025-08-04 02:32:57,147 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3750-2:Control:contrail-control:0 is connected to collector 10.0.0.27
2025-08-04 02:32:57,147 - DEBUG - Verifying through opserver in 10.0.0.27
2025-08-04 02:32:57,192 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3750-2:Control:contrail-dns:0 is connected to collector 10.0.0.27
2025-08-04 02:32:57,192 - DEBUG - Verifying through opserver in 10.0.0.27
2025-08-04 02:32:57,254 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3750-3:Control:contrail-control:0 is connected to collector 10.0.0.27
2025-08-04 02:32:57,254 - DEBUG - Verifying through opserver in 10.0.0.27
2025-08-04 02:32:57,293 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3750-3:Control:contrail-dns:0 is connected to collector 10.0.0.27
2025-08-04 02:32:57,293 - DEBUG - Verifying through opserver in 10.0.0.27
2025-08-04 02:32:57,340 - INFO - Validated that an-jenkins-deploy-platform-ansible-os-3750-1:Compute:contrail-vrouter-agent:0 is connected to collector 10.0.0.27
2025-08-04 02:32:57,340 - DEBUG - Verifying through opserver in 10.0.0.27
2025-08-04 02:32:57,380 - INFO - Validated that an-jenkins-deploy-platform-ansible-os-3750-2:Compute:contrail-vrouter-agent:0 is connected to collector 10.0.0.27
2025-08-04 02:32:57,381 - DEBUG - Verifying through opserver in 10.0.0.27
2025-08-04 02:32:57,404 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3750-1.:Config:contrail-schema:0 is connected to collector 10.0.0.27
2025-08-04 02:32:57,404 - DEBUG - Verifying through opserver in 10.0.0.27
2025-08-04 02:32:57,433 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3750-1.:Config:contrail-svc-monitor:0 is connected to collector 10.0.0.27
2025-08-04 02:32:57,452 - DEBUG - Verifying through opserver in 10.0.0.27
2025-08-04 02:32:57,476 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3750-1.:Config:contrail-api:0 is connected to collector 10.0.0.27
2025-08-04 02:32:57,476 - DEBUG - Verifying through opserver in 10.0.0.27
2025-08-04 02:32:57,499 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3750-2.:Config:contrail-api:0 is connected to collector 10.0.0.27
2025-08-04 02:32:57,499 - DEBUG - Verifying through opserver in 10.0.0.27
2025-08-04 02:32:57,522 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3750-3.:Config:contrail-api:0 is connected to collector 10.0.0.27
2025-08-04 02:32:57,545 - DEBUG - Verifying through opserver in 10.0.0.27
2025-08-04 02:32:57,567 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3750-1.:Analytics:contrail-analytics-api:0 is connected to collector 10.0.0.27
2025-08-04 02:32:57,567 - DEBUG - Verifying through opserver in 10.0.0.27
2025-08-04 02:32:57,590 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3750-1.:Analytics:contrail-analytics-api:0 is connected to collector 10.0.0.27
2025-08-04 02:32:57,591 - DEBUG - Verifying through opserver in 10.0.0.27
2025-08-04 02:32:57,612 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3750-1.:Analytics:contrail-analytics-api:0 is connected to collector 10.0.0.27
2025-08-04 02:32:57,612 - DEBUG - Verifying through opserver in 10.0.0.27
2025-08-04 02:32:57,636 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3750-2.:Analytics:contrail-analytics-api:0 is connected to collector 10.0.0.27
2025-08-04 02:32:57,636 - DEBUG - Verifying through opserver in 10.0.0.27
2025-08-04 02:32:57,658 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3750-2.:Analytics:contrail-analytics-api:0 is connected to collector 10.0.0.27
2025-08-04 02:32:57,658 - DEBUG - Verifying through opserver in 10.0.0.27
2025-08-04 02:32:57,681 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3750-2.:Analytics:contrail-analytics-api:0 is connected to collector 10.0.0.27
2025-08-04 02:32:57,681 - DEBUG - Verifying through opserver in 10.0.0.27
2025-08-04 02:32:57,701 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3750-3.:Analytics:contrail-analytics-api:0 is connected to collector 10.0.0.27
2025-08-04 02:32:57,701 - DEBUG - Verifying through opserver in 10.0.0.27
2025-08-04 02:32:57,724 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3750-3.:Analytics:contrail-analytics-api:0 is connected to collector 10.0.0.27
2025-08-04 02:32:57,725 - DEBUG - Verifying through opserver in 10.0.0.27
2025-08-04 02:32:57,750 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3750-3.:Analytics:contrail-analytics-api:0 is connected to collector 10.0.0.27
2025-08-04 02:32:57,751 - DEBUG - Verifying through opserver in 10.0.0.27
2025-08-04 02:32:57,786 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3750-1.:Analytics:contrail-collector:0 is connected to collector 10.0.0.27
2025-08-04 02:32:57,786 - DEBUG - Verifying through opserver in 10.0.0.27
2025-08-04 02:32:57,818 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3750-1.:Database:contrail-query-engine:0 is connected to collector 10.0.0.27
2025-08-04 02:32:57,819 - DEBUG - Verifying through opserver in 10.0.0.27
2025-08-04 02:32:57,854 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3750-2.:Analytics:contrail-collector:0 is connected to collector 10.0.0.27
2025-08-04 02:32:57,854 - DEBUG - Verifying through opserver in 10.0.0.27
2025-08-04 02:32:57,886 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3750-2.:Database:contrail-query-engine:0 is connected to collector 10.0.0.27
2025-08-04 02:32:57,886 - DEBUG - Verifying through opserver in 10.0.0.27
2025-08-04 02:32:57,921 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3750-3.:Analytics:contrail-collector:0 is connected to collector 10.0.0.27
2025-08-04 02:32:57,921 - DEBUG - Verifying through opserver in 10.0.0.27
2025-08-04 02:32:57,956 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3750-3.:Database:contrail-query-engine:0 is connected to collector 10.0.0.27
2025-08-04 02:32:58,067 - INFO - Active xmpp peer in an-jenkins-deploy-platform-ansible-os-3750-1 vrouter uve is 10.20.0.18
2025-08-04 02:32:58,103 - DEBUG - Requesting: http://10.20.0.192:8085/Snh_AgentXmppConnectionStatusReq?
2025-08-04 02:32:58,116 - INFO - Active xmpp peer in an-jenkins-deploy-platform-ansible-os-3750-1 agent introspect is 10.20.0.18
2025-08-04 02:32:58,159 - DEBUG - Requesting: http://10.0.0.61:8081/analytics/uves/vrouter/an-jenkins-deploy-platform-ansible-os-3750-2?flat
2025-08-04 02:32:58,202 - INFO - Active xmpp peer in an-jenkins-deploy-platform-ansible-os-3750-2 vrouter uve is 10.20.0.193
2025-08-04 02:32:58,238 - DEBUG - Requesting: http://10.20.0.201:8085/Snh_AgentXmppConnectionStatusReq?
2025-08-04 02:32:58,252 - INFO - Active xmpp peer in an-jenkins-deploy-platform-ansible-os-3750-2 agent introspect is 10.20.0.193
2025-08-04 02:32:58,342 - INFO - Vrouter an-jenkins-deploy-platform-ansible-os-3750-1 connected to {'10.20.0.129', '10.20.0.18'} xmpp peers out of {'10.20.0.129', '10.20.0.18', '10.20.0.193'}
2025-08-04 02:32:58,342 - INFO - xmpp peer correctly displayed as ['10.20.0.18', '10.20.0.129'] for vrouter an-jenkins-deploy-platform-ansible-os-3750-1
2025-08-04 02:32:58,401 - DEBUG - Requesting: http://10.0.0.61:8081/analytics/uves/vrouter/an-jenkins-deploy-platform-ansible-os-3750-2?flat
2025-08-04 02:32:58,446 - INFO - Vrouter an-jenkins-deploy-platform-ansible-os-3750-2 connected to {'10.20.0.18', '10.20.0.193'} xmpp peers out of {'10.20.0.129', '10.20.0.18', '10.20.0.193'}
2025-08-04 02:32:58,446 - INFO - xmpp peer correctly displayed as ['10.20.0.193', '10.20.0.18'] for vrouter an-jenkins-deploy-platform-ansible-os-3750-2
2025-08-04 02:32:58,446 - INFO - Verifying through opserver in 10.0.0.61
2025-08-04 02:32:58,447 - DEBUG - Requesting: http://10.0.0.61:8081/analytics/uves/control-node/cn-jenkins-deploy-platform-ansible-os-3750-1?flat
2025-08-04 02:32:58,477 - DEBUG - Requesting: http://10.0.0.61:8081/analytics/uves/control-node/cn-jenkins-deploy-platform-ansible-os-3750-2?flat
2025-08-04 02:32:58,513 - DEBUG - Requesting: http://10.0.0.61:8081/analytics/uves/control-node/cn-jenkins-deploy-platform-ansible-os-3750-3?flat
2025-08-04 02:32:58,543 - DEBUG - Requesting: http://10.0.0.61:8081/analytics/uves/control-node/cn-jenkins-deploy-platform-ansible-os-3750-1?flat
2025-08-04 02:32:58,582 - DEBUG - Requesting: http://10.0.0.61:8081/analytics/uves/control-node/cn-jenkins-deploy-platform-ansible-os-3750-2?flat
2025-08-04 02:32:58,608 - DEBUG - Requesting: http://10.0.0.61:8081/analytics/uves/control-node/cn-jenkins-deploy-platform-ansible-os-3750-3?flat
2025-08-04 02:32:58,634 - DEBUG - Verifying for cn-jenkins-deploy-platform-ansible-os-3750-1 bgp-router uve
2025-08-04 02:32:58,634 - INFO - Bgp peers = 2
2025-08-04 02:32:58,634 - DEBUG - Verifying for cn-jenkins-deploy-platform-ansible-os-3750-2 bgp-router uve
2025-08-04 02:32:58,635 - INFO - Bgp peers = 2
2025-08-04 02:32:58,635 - DEBUG - Verifying for cn-jenkins-deploy-platform-ansible-os-3750-3 bgp-router uve
2025-08-04 02:32:58,635 - INFO - Bgp peers = 2
2025-08-04 02:32:58,635 - INFO - Total xmpp peer connections found = 4
2025-08-04 02:32:59,273 - DEBUG - No XMPP flaps were noticed during the test
2025-08-04 02:32:59,273 - INFO - END TEST : test_verify_generator_collector_connections : PASSED[0:00:03]
2025-08-04 02:32:59,274 - INFO - --------------------------------------------------------------------------------
2025-08-04 02:32:59,276 - INFO - ================================================================================
2025-08-04 02:32:59,277 - INFO - STARTING TEST : test_verify_generator_connections_to_collector_node
2025-08-04 02:32:59,277 - INFO - TEST DESCRIPTION : Test to validate generator connections
2025-08-04 02:32:59,922 - DEBUG - Nothing to compare xmpp stats {'10.0.0.41': {'10.20.0.18': '0', '10.20.0.129': '0'}, '10.0.0.57': {'10.20.0.193': '0', '10.20.0.18': '0'}} with
2025-08-04 02:32:59,922 - INFO - Initial checks done. Running the testcase now
2025-08-04 02:32:59,922 - INFO -
2025-08-04 02:32:59,922 - DEBUG - Verifying through opserver in 10.0.0.27
2025-08-04 02:32:59,968 - INFO - Validated that an-jenkins-deploy-platform-ansible-os-3750-1:Compute:contrail-vrouter-agent:0 is connected to collector 10.0.0.27
2025-08-04 02:32:59,968 - DEBUG - Verifying through opserver in 10.0.0.27
2025-08-04 02:32:59,989 - INFO - Validated that an-jenkins-deploy-platform-ansible-os-3750-1:Compute:contrail-vrouter-nodemgr:0 is connected to collector 10.0.0.27
2025-08-04 02:32:59,989 - DEBUG - Verifying through opserver in 10.0.0.27
2025-08-04 02:33:00,028 - INFO - Validated that an-jenkins-deploy-platform-ansible-os-3750-2:Compute:contrail-vrouter-agent:0 is connected to collector 10.0.0.27
2025-08-04 02:33:00,028 - DEBUG - Verifying through opserver in 10.0.0.27
2025-08-04 02:33:00,055 - INFO - Validated that an-jenkins-deploy-platform-ansible-os-3750-2:Compute:contrail-vrouter-nodemgr:0 is connected to collector 10.0.0.27
2025-08-04 02:33:00,055 - DEBUG - Verifying through opserver in 10.0.0.27
2025-08-04 02:33:00,078 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3750-1.:Analytics:contrail-analytics-nodemgr:0 is connected to collector 10.0.0.27
2025-08-04 02:33:00,079 - DEBUG - Verifying through opserver in 10.0.0.27
2025-08-04 02:33:00,118 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3750-1.:Analytics:contrail-collector:0 is connected to collector 10.0.0.27
2025-08-04 02:33:00,119 - DEBUG - Verifying through opserver in 10.0.0.27
2025-08-04 02:33:00,144 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3750-1.:Analytics:contrail-analytics-api:0 is connected to collector 10.0.0.27
2025-08-04 02:33:00,144 - DEBUG - Verifying through opserver in 10.0.0.27
2025-08-04 02:33:00,165 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3750-2.:Analytics:contrail-analytics-nodemgr:0 is connected to collector 10.0.0.27
2025-08-04 02:33:00,165 - DEBUG - Verifying through opserver in 10.0.0.27
2025-08-04 02:33:00,207 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3750-2.:Analytics:contrail-collector:0 is connected to collector 10.0.0.27
2025-08-04 02:33:00,207 - DEBUG - Verifying through opserver in 10.0.0.27
2025-08-04 02:33:00,236 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3750-2.:Analytics:contrail-analytics-api:0 is connected to collector 10.0.0.27
2025-08-04 02:33:00,236 - DEBUG - Verifying through opserver in 10.0.0.27
2025-08-04 02:33:00,261 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3750-3.:Analytics:contrail-analytics-nodemgr:0 is connected to collector 10.0.0.27
2025-08-04 02:33:00,261 - DEBUG - Verifying through opserver in 10.0.0.27
2025-08-04 02:33:00,298 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3750-3.:Analytics:contrail-collector:0 is connected to collector 10.0.0.27
2025-08-04 02:33:00,298 - DEBUG - Verifying through opserver in 10.0.0.27
2025-08-04 02:33:00,333 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3750-3.:Analytics:contrail-analytics-api:0 is connected to collector 10.0.0.27
2025-08-04 02:33:00,333 - DEBUG - Verifying through opserver in 10.0.0.27
2025-08-04 02:33:00,368 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3750-1.:Database:contrail-query-engine:0 is connected to collector 10.0.0.27
2025-08-04 02:33:00,368 - DEBUG - Verifying through opserver in 10.0.0.27
2025-08-04 02:33:00,396 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3750-1.:Database:contrail-database-nodemgr:0 is connected to collector 10.0.0.27
2025-08-04 02:33:00,397 - DEBUG - Verifying through opserver in 10.0.0.27
2025-08-04 02:33:00,429 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3750-2.:Database:contrail-query-engine:0 is connected to collector 10.0.0.27
2025-08-04 02:33:00,429 - DEBUG - Verifying through opserver in 10.0.0.27
2025-08-04 02:33:00,459 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3750-2.:Database:contrail-database-nodemgr:0 is connected to collector 10.0.0.27
2025-08-04 02:33:00,459 - DEBUG - Verifying through opserver in 10.0.0.27
2025-08-04 02:33:00,505 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3750-3.:Database:contrail-query-engine:0 is connected to collector 10.0.0.27
2025-08-04 02:33:00,505 - DEBUG - Verifying through opserver in 10.0.0.27
2025-08-04 02:33:00,541 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3750-3.:Database:contrail-database-nodemgr:0 is connected to collector 10.0.0.27
2025-08-04 02:33:00,541 - DEBUG - Verifying through opserver in 10.0.0.27
2025-08-04 02:33:00,568 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3750-1.:Config:contrail-api:0 is connected to collector 10.0.0.27
2025-08-04 02:33:00,569 - DEBUG - Verifying through opserver in 10.0.0.27
2025-08-04 02:33:00,604 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3750-2.:Config:contrail-api:0 is connected to collector 10.0.0.27
2025-08-04 02:33:00,604 - DEBUG - Verifying through opserver in 10.0.0.27
2025-08-04 02:33:00,635 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3750-3.:Config:contrail-api:0 is connected to collector 10.0.0.27
2025-08-04 02:33:00,635 - DEBUG - Verifying through opserver in 10.0.0.27
2025-08-04 02:33:00,707 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3750-1:Control:contrail-control:0 is connected to collector 10.0.0.27
2025-08-04 02:33:00,707 - DEBUG - Verifying through opserver in 10.0.0.27
2025-08-04 02:33:00,737 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3750-1:Control:contrail-control-nodemgr:0 is connected to collector 10.0.0.27
2025-08-04 02:33:00,738 - DEBUG - Verifying through opserver in 10.0.0.27
2025-08-04 02:33:00,789 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3750-1:Control:contrail-dns:0 is connected to collector 10.0.0.27
2025-08-04 02:33:00,789 - DEBUG - Verifying through opserver in 10.0.0.27
2025-08-04 02:33:00,846 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3750-2:Control:contrail-control:0 is connected to collector 10.0.0.27
2025-08-04 02:33:00,847 - DEBUG - Verifying through opserver in 10.0.0.27
2025-08-04 02:33:00,879 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3750-2:Control:contrail-control-nodemgr:0 is connected to collector 10.0.0.27
2025-08-04 02:33:00,879 - DEBUG - Verifying through opserver in 10.0.0.27
2025-08-04 02:33:00,926 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3750-2:Control:contrail-dns:0 is connected to collector 10.0.0.27
2025-08-04 02:33:00,926 - DEBUG - Verifying through opserver in 10.0.0.27
2025-08-04 02:33:01,029 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3750-3:Control:contrail-control:0 is connected to collector 10.0.0.27
2025-08-04 02:33:01,029 - DEBUG - Verifying through opserver in 10.0.0.27
2025-08-04 02:33:01,053 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3750-3:Control:contrail-control-nodemgr:0 is connected to collector 10.0.0.27
2025-08-04 02:33:01,053 - DEBUG - Verifying through opserver in 10.0.0.27
2025-08-04 02:33:01,088 - INFO - Validated that cn-jenkins-deploy-platform-ansible-os-3750-3:Control:contrail-dns:0 is connected to collector 10.0.0.27
2025-08-04 02:33:01,757 - DEBUG - No XMPP flaps were noticed during the test
2025-08-04 02:33:01,757 - INFO - END TEST : test_verify_generator_connections_to_collector_node : PASSED[0:00:02]
2025-08-04 02:33:01,758 - INFO - --------------------------------------------------------------------------------
2025-08-04 02:33:01,761 - INFO - ================================================================================
2025-08-04 02:33:01,761 - INFO - STARTING TEST : test_verify_process_status_agent
2025-08-04 02:33:01,761 - INFO - TEST DESCRIPTION : Test to validate process_status
2025-08-04 02:33:02,405 - DEBUG - Nothing to compare xmpp stats {'10.0.0.41': {'10.20.0.18': '0', '10.20.0.129': '0'}, '10.0.0.57': {'10.20.0.193': '0', '10.20.0.18': '0'}} with
2025-08-04 02:33:02,405 - INFO - Initial checks done. Running the testcase now
2025-08-04 02:33:02,405 - INFO -
2025-08-04 02:33:02,494 - WARNING - an-jenkins-deploy-platform-ansible-os-3750-1:contrail-vrouter-agent module connection to ['10.20.0.193:5269'] servers NOT UP
2025-08-04 02:33:02,494 - INFO - an-jenkins-deploy-platform-ansible-os-3750-1:contrail-vrouter-agent module connection to ['10.20.0.129:5269'] servers UP
2025-08-04 02:33:02,495 - INFO - an-jenkins-deploy-platform-ansible-os-3750-1 connected to xmpp 10.20.0.129
2025-08-04 02:33:02,495 - INFO - an-jenkins-deploy-platform-ansible-os-3750-1:contrail-vrouter-agent module connection to ['10.20.0.18:5269'] servers UP
2025-08-04 02:33:02,495 - INFO - an-jenkins-deploy-platform-ansible-os-3750-1 connected to xmpp 10.20.0.18
2025-08-04 02:33:02,495 - INFO - an-jenkins-deploy-platform-ansible-os-3750-1:contrail-vrouter-agent module connection to ['10.20.0.193:53'] servers UP
2025-08-04 02:33:02,495 - INFO - an-jenkins-deploy-platform-ansible-os-3750-1 connected to dns 10.20.0.193
2025-08-04 02:33:02,495 - INFO - an-jenkins-deploy-platform-ansible-os-3750-1:contrail-vrouter-agent module connection to ['10.20.0.129:53'] servers UP
2025-08-04 02:33:02,495 - INFO - an-jenkins-deploy-platform-ansible-os-3750-1 connected to dns 10.20.0.129
2025-08-04 02:33:02,495 - WARNING - an-jenkins-deploy-platform-ansible-os-3750-1:contrail-vrouter-agent module connection to ['10.20.0.18:53'] servers NOT UP
2025-08-04 02:33:02,495 - INFO - an-jenkins-deploy-platform-ansible-os-3750-1:contrail-vrouter-agent module connection to ['10.0.0.27:8086'] servers UP
2025-08-04 02:33:02,495 - INFO - an-jenkins-deploy-platform-ansible-os-3750-1 connected to collector 10.0.0.27
2025-08-04 02:33:02,496 - WARNING - an-jenkins-deploy-platform-ansible-os-3750-1:contrail-vrouter-agent module connection to ['10.0.0.38:8086'] servers NOT UP
2025-08-04 02:33:02,496 - WARNING - an-jenkins-deploy-platform-ansible-os-3750-1:contrail-vrouter-agent module connection to ['10.0.0.61:8086'] servers NOT UP
2025-08-04 02:33:03,168 - DEBUG - No XMPP flaps were noticed during the test
2025-08-04 02:33:03,168 - INFO - END TEST : test_verify_process_status_agent : PASSED[0:00:02]
2025-08-04 02:33:03,168 - INFO - --------------------------------------------------------------------------------
2025-08-04 02:33:03,171 - INFO - ================================================================================
2025-08-04 02:33:03,171 - INFO - STARTING TEST : test_verify_process_status_analytics_node
2025-08-04 02:33:03,171 - INFO - TEST DESCRIPTION : Test to validate process_status-Analytics-Node
2025-08-04 02:33:03,832 - DEBUG - Nothing to compare xmpp stats {'10.0.0.41': {'10.20.0.18': '0', '10.20.0.129': '0'}, '10.0.0.57': {'10.20.0.193': '0', '10.20.0.18': '0'}} with
2025-08-04 02:33:03,832 - INFO - Initial checks done. Running the testcase now
2025-08-04 02:33:03,832 - INFO -
2025-08-04 02:33:04,032 - INFO - cn-jenkins-deploy-platform-ansible-os-3750-1.:contrail-collector module connection to ['10.0.0.27:8086'] servers UP
2025-08-04 02:33:04,032 - WARNING - cn-jenkins-deploy-platform-ansible-os-3750-1.:contrail-collector module connection to ['10.0.0.27:6379'] servers NOT UP
2025-08-04 02:33:04,032 - WARNING - cn-jenkins-deploy-platform-ansible-os-3750-1.:contrail-collector module connection to ['10.0.0.38:6379'] servers NOT UP
2025-08-04 02:33:04,032 - WARNING - cn-jenkins-deploy-platform-ansible-os-3750-1.:contrail-collector module connection to ['10.0.0.61:6379'] servers NOT UP
2025-08-04 02:33:04,032 - INFO - cn-jenkins-deploy-platform-ansible-os-3750-1.:contrail-collector module connection to ['127.0.0.1:6379'] servers UP
2025-08-04 02:33:04,032 - INFO - cn-jenkins-deploy-platform-ansible-os-3750-1.:contrail-collector module connection to ['10.0.0.27:9042'] servers UP
2025-08-04 02:33:04,076 - WARNING - cn-jenkins-deploy-platform-ansible-os-3750-2.:contrail-collector module connection to ['10.0.0.27:8086'] servers NOT UP
2025-08-04 02:33:04,076 - INFO - cn-jenkins-deploy-platform-ansible-os-3750-2.:contrail-collector module connection to ['10.0.0.38:8086'] servers UP
2025-08-04 02:33:04,076 - WARNING - cn-jenkins-deploy-platform-ansible-os-3750-2.:contrail-collector module connection to ['10.0.0.27:6379'] servers NOT UP
2025-08-04 02:33:04,076 - WARNING - cn-jenkins-deploy-platform-ansible-os-3750-2.:contrail-collector module connection to ['10.0.0.38:6379'] servers NOT UP
2025-08-04 02:33:04,076 - WARNING - cn-jenkins-deploy-platform-ansible-os-3750-2.:contrail-collector module connection to ['10.0.0.61:6379'] servers NOT UP
2025-08-04 02:33:04,076 - INFO - cn-jenkins-deploy-platform-ansible-os-3750-2.:contrail-collector module connection to ['127.0.0.1:6379'] servers UP
2025-08-04 02:33:04,076 - INFO - cn-jenkins-deploy-platform-ansible-os-3750-2.:contrail-collector module connection to ['10.0.0.27:9042'] servers UP
2025-08-04 02:33:04,134 - WARNING - cn-jenkins-deploy-platform-ansible-os-3750-3.:contrail-collector module connection to ['10.0.0.27:8086'] servers NOT UP
2025-08-04 02:33:04,134 - WARNING - cn-jenkins-deploy-platform-ansible-os-3750-3.:contrail-collector module connection to ['10.0.0.38:8086'] servers NOT UP
2025-08-04 02:33:04,134 - INFO - cn-jenkins-deploy-platform-ansible-os-3750-3.:contrail-collector module connection to ['10.0.0.61:8086'] servers UP
2025-08-04 02:33:04,135 - WARNING - cn-jenkins-deploy-platform-ansible-os-3750-3.:contrail-collector module connection to ['10.0.0.27:6379'] servers NOT UP
2025-08-04 02:33:04,135 - WARNING - cn-jenkins-deploy-platform-ansible-os-3750-3.:contrail-collector module connection to ['10.0.0.38:6379'] servers NOT UP
2025-08-04 02:33:04,135 - WARNING - cn-jenkins-deploy-platform-ansible-os-3750-3.:contrail-collector module connection to ['10.0.0.61:6379'] servers NOT UP
2025-08-04 02:33:04,135 - INFO - cn-jenkins-deploy-platform-ansible-os-3750-3.:contrail-collector module connection to ['127.0.0.1:6379'] servers UP
2025-08-04 02:33:04,135 - INFO - cn-jenkins-deploy-platform-ansible-os-3750-3.:contrail-collector module connection to ['10.0.0.27:9042'] servers UP
2025-08-04 02:33:04,177 - WARNING - cn-jenkins-deploy-platform-ansible-os-3750-1.:contrail-analytics-api module connection to ['10.0.0.27:8086'] servers NOT UP
2025-08-04 02:33:04,178 - WARNING - cn-jenkins-deploy-platform-ansible-os-3750-1.:contrail-analytics-api module connection to ['10.0.0.38:8086'] servers NOT UP
2025-08-04 02:33:04,178 - INFO - cn-jenkins-deploy-platform-ansible-os-3750-1.:contrail-analytics-api module connection to ['10.0.0.61:8086'] servers UP
2025-08-04 02:33:04,230 - INFO - cn-jenkins-deploy-platform-ansible-os-3750-2.:contrail-analytics-api module connection to ['10.0.0.27:8086'] servers UP
2025-08-04 02:33:04,281 - WARNING - cn-jenkins-deploy-platform-ansible-os-3750-3.:contrail-analytics-api module connection to ['10.0.0.27:8086'] servers NOT UP
2025-08-04 02:33:04,281 - WARNING - cn-jenkins-deploy-platform-ansible-os-3750-3.:contrail-analytics-api module connection to ['10.0.0.38:8086'] servers NOT UP
2025-08-04 02:33:04,281 - INFO - cn-jenkins-deploy-platform-ansible-os-3750-3.:contrail-analytics-api module connection to ['10.0.0.61:8086'] servers UP
2025-08-04 02:33:04,335 - WARNING - cn-jenkins-deploy-platform-ansible-os-3750-1.:contrail-analytics-nodemgr module connection to ['10.0.0.27:8086'] servers NOT UP
2025-08-04 02:33:04,335 - WARNING - cn-jenkins-deploy-platform-ansible-os-3750-1.:contrail-analytics-nodemgr module connection to ['10.0.0.38:8086'] servers NOT UP
2025-08-04 02:33:04,335 - INFO - cn-jenkins-deploy-platform-ansible-os-3750-1.:contrail-analytics-nodemgr module connection to ['10.0.0.61:8086'] servers UP
2025-08-04 02:33:04,378 - WARNING - cn-jenkins-deploy-platform-ansible-os-3750-2.:contrail-analytics-nodemgr module connection to ['10.0.0.27:8086'] servers NOT UP
2025-08-04 02:33:04,379 - WARNING - cn-jenkins-deploy-platform-ansible-os-3750-2.:contrail-analytics-nodemgr module connection to ['10.0.0.38:8086'] servers NOT UP
2025-08-04 02:33:04,379 - INFO - cn-jenkins-deploy-platform-ansible-os-3750-2.:contrail-analytics-nodemgr module connection to ['10.0.0.61:8086'] servers UP
2025-08-04 02:33:04,423 - INFO - cn-jenkins-deploy-platform-ansible-os-3750-3.:contrail-analytics-nodemgr module connection to ['10.0.0.27:8086'] servers UP
2025-08-04 02:33:05,084 - DEBUG - No XMPP flaps were noticed during the test
2025-08-04 02:33:05,084 - INFO - END TEST : test_verify_process_status_analytics_node : PASSED[0:00:02]
2025-08-04 02:33:05,084 - INFO - --------------------------------------------------------------------------------
2025-08-04 02:33:05,087 - INFO - ================================================================================
2025-08-04 02:33:05,087 - INFO - STARTING TEST : test_verify_process_status_config
2025-08-04 02:33:05,087 - INFO - TEST DESCRIPTION : Test to validate process_status-Config
2025-08-04 02:33:05,745 - DEBUG - Nothing to compare xmpp stats {'10.0.0.41': {'10.20.0.18': '0', '10.20.0.129': '0'}, '10.0.0.57': {'10.20.0.193': '0', '10.20.0.18': '0'}} with
2025-08-04 02:33:05,745 - INFO - Initial checks done. Running the testcase now
2025-08-04 02:33:05,745 - INFO -
2025-08-04 02:33:05,774 - WARNING - No object found for module contrail-device-manager
2025-08-04 02:33:05,835 - WARNING - No object found for module contrail-device-manager
2025-08-04 02:33:05,864 - WARNING - cn-jenkins-deploy-platform-ansible-os-3750-1.:contrail-api module connection to 10.0.0.27:8086 servers NOT UP
2025-08-04 02:33:05,864 - INFO - cn-jenkins-deploy-platform-ansible-os-3750-1.:contrail-api module connection to 10.0.0.38:8086 servers UP
2025-08-04 02:33:05,864 - INFO - cn-jenkins-deploy-platform-ansible-os-3750-1.:contrail-api module connection to 10.0.0.27:2181 servers UP
2025-08-04 02:33:05,865 - WARNING - cn-jenkins-deploy-platform-ansible-os-3750-1.:contrail-api module connection to 10.0.0.27:9161 servers NOT UP
2025-08-04 02:33:05,865 - WARNING - cn-jenkins-deploy-platform-ansible-os-3750-1.:contrail-api module connection to 10.0.0.38:9161 servers NOT UP
2025-08-04 02:33:05,865 - WARNING - cn-jenkins-deploy-platform-ansible-os-3750-1.:contrail-api module connection to 10.0.0.61:9161 servers NOT UP
2025-08-04 02:33:05,865 - WARNING - cn-jenkins-deploy-platform-ansible-os-3750-1.:contrail-api module connection to cn-jenkins-deploy-platform-ansible-os-3750-1.*:9161 servers NOT UP
2025-08-04 02:33:05,866 - WARNING - cn-jenkins-deploy-platform-ansible-os-3750-1.:contrail-api module connection to cn-jenkins-deploy-platform-ansible-os-3750-2.*:9161 servers NOT UP
2025-08-04 02:33:05,866 - WARNING - cn-jenkins-deploy-platform-ansible-os-3750-1.:contrail-api module connection to cn-jenkins-deploy-platform-ansible-os-3750-3.*:9161 servers NOT UP
2025-08-04 02:33:06,505 - ERROR - AssertionError
Python 3.6.8: /usr/bin/python3
Mon Aug 4 02:33:05 2025
A problem occurred in a Python script. Here is the sequence of
function calls leading up to the error, in the order they occurred.
/contrail-test/tcutils/wrappers.py in wrapper(self=, *args=(), **kwargs={})
78 log.info('Initial checks done. Running the testcase now')
79 log.info('')
80 result = function(self, *args, **kwargs)
81 if self.inputs.upgrade:
82 pid = os.getpid()
result = None
function =
self =
args = ()
kwargs = {}
/contrail-test/scripts/analytics/test_analytics.py in test_verify_process_status_config(self=)
333
334 '''
335 self.analytics_obj.verify_process_and_connection_infos_config()
336
337 @test.attr(type=['cb_sanity', 'sanity', 'dev_sanity_dpdk'])
self =
self.analytics_obj =
self.analytics_obj.verify_process_and_connection_infos_config = >
/contrail-test/tcutils/collector/analytics_tests.py in verify_process_and_connection_infos_config(self=)
4218 'contrail-api',\
4219 server,node = cfgm)
4220 assert result
4221 result = False
4222
result = False
AssertionError:
__cause__ = None
__class__ =
__context__ = None
__delattr__ =
__dict__ = {}
__dir__ =
__doc__ = 'Assertion failed.'
__eq__ =
__format__ =
__ge__ =
__getattribute__ =
__gt__ =
__hash__ =
__init__ =
__init_subclass__ =
__le__ =
__lt__ =
__ne__ =
__new__ =
__reduce__ =
__reduce_ex__ =
__repr__ =
__setattr__ =
__setstate__ =
__sizeof__ =
__str__ =
__subclasshook__ =
__suppress_context__ = False
__traceback__ =
args = ()
with_traceback =
The above is a description of an error in a Python program. Here is
the original traceback:
Traceback (most recent call last):
File "/contrail-test/tcutils/wrappers.py", line 80, in wrapper
result = function(self, *args, **kwargs)
File "/contrail-test/scripts/analytics/test_analytics.py", line 335, in test_verify_process_status_config
self.analytics_obj.verify_process_and_connection_infos_config()
File "/contrail-test/tcutils/collector/analytics_tests.py", line 4220, in verify_process_and_connection_infos_config
assert result
AssertionError
2025-08-04 02:33:06,521 - DEBUG - No XMPP flaps were noticed during the test
2025-08-04 02:33:06,521 - INFO -
2025-08-04 02:33:06,521 - INFO - END TEST : test_verify_process_status_config : FAILED[0:00:01]
2025-08-04 02:33:06,522 - INFO - --------------------------------------------------------------------------------
2025-08-04 02:33:06,525 - INFO - ================================================================================
2025-08-04 02:33:06,525 - INFO - STARTING TEST : test_verify_process_status_control_node
2025-08-04 02:33:06,525 - INFO - TEST DESCRIPTION : Test to validate process_status-Control-Node
2025-08-04 02:33:07,186 - DEBUG - Nothing to compare xmpp stats {'10.0.0.41': {'10.20.0.18': '0', '10.20.0.129': '0'}, '10.0.0.57': {'10.20.0.193': '0', '10.20.0.18': '0'}} with
2025-08-04 02:33:07,186 - INFO - Initial checks done. Running the testcase now
2025-08-04 02:33:07,186 - INFO -
2025-08-04 02:33:07,218 - INFO - cn-jenkins-deploy-platform-ansible-os-3750-1:contrail-control module connection to 10.0.0.27:9041 servers UP
2025-08-04 02:33:07,218 - INFO - cn-jenkins-deploy-platform-ansible-os-3750-1:contrail-control module connection to 10.0.0.27:5673 servers UP
2025-08-04 02:33:07,218 - WARNING - cn-jenkins-deploy-platform-ansible-os-3750-1:contrail-control module connection to ['10.0.0.27:8086'] servers NOT UP
2025-08-04 02:33:07,219 - WARNING - cn-jenkins-deploy-platform-ansible-os-3750-1:contrail-control module connection to ['10.0.0.38:8086'] servers NOT UP
2025-08-04 02:33:07,219 - INFO - cn-jenkins-deploy-platform-ansible-os-3750-1:contrail-control module connection to ['10.0.0.61:8086'] servers UP
2025-08-04 02:33:07,251 - INFO - cn-jenkins-deploy-platform-ansible-os-3750-2:contrail-control module connection to 10.0.0.27:9041 servers UP
2025-08-04 02:33:07,251 - INFO - cn-jenkins-deploy-platform-ansible-os-3750-2:contrail-control module connection to 10.0.0.27:5673 servers UP
2025-08-04 02:33:07,251 - WARNING - cn-jenkins-deploy-platform-ansible-os-3750-2:contrail-control module connection to ['10.0.0.27:8086'] servers NOT UP
2025-08-04 02:33:07,255 - WARNING - cn-jenkins-deploy-platform-ansible-os-3750-2:contrail-control module connection to ['10.0.0.38:8086'] servers NOT UP
2025-08-04 02:33:07,256 - INFO - cn-jenkins-deploy-platform-ansible-os-3750-2:contrail-control module connection to ['10.0.0.61:8086'] servers UP
2025-08-04 02:33:07,288 - INFO - cn-jenkins-deploy-platform-ansible-os-3750-3:contrail-control module connection to 10.0.0.27:9041 servers UP
2025-08-04 02:33:07,288 - INFO - cn-jenkins-deploy-platform-ansible-os-3750-3:contrail-control module connection to 10.0.0.27:5673 servers UP
2025-08-04 02:33:07,288 - WARNING - cn-jenkins-deploy-platform-ansible-os-3750-3:contrail-control module connection to ['10.0.0.27:8086'] servers NOT UP
2025-08-04 02:33:07,288 - WARNING - cn-jenkins-deploy-platform-ansible-os-3750-3:contrail-control module connection to ['10.0.0.38:8086'] servers NOT UP
2025-08-04 02:33:07,288 - INFO - cn-jenkins-deploy-platform-ansible-os-3750-3:contrail-control module connection to ['10.0.0.61:8086'] servers UP
2025-08-04 02:33:07,946 - DEBUG - No XMPP flaps were noticed during the test
2025-08-04 02:33:07,946 - INFO - END TEST : test_verify_process_status_control_node : PASSED[0:00:01]
2025-08-04 02:33:07,946 - INFO - --------------------------------------------------------------------------------