2025-10-16 16:36:15,165 - INFO - Domain Default found not creating
2025-10-16 16:36:15,432 - INFO - Project ctest-TestRouters-01222227 not found, creating it
2025-10-16 16:36:16,080 - INFO - Created Project:ctest-TestRouters-01222227, ID : 807e3fe8-3af3-4ee9-8b25-339d20d29b41 
2025-10-16 16:36:17,750 - INFO - ================================================================================
2025-10-16 16:36:17,751 - INFO - STARTING TEST    : test_basic_router_behavior
2025-10-16 16:36:17,751 - INFO - TEST DESCRIPTION : Validate a router is able to route packets between two VNs
        Create a router
        Create 2 VNs, and a VM in each
        Add router port from each VN
        Ping between VMs
        
2025-10-16 16:36:18,013 - DEBUG - Nothing to compare xmpp stats {'10.0.0.141': {'10.20.0.17': '0'}} with
2025-10-16 16:36:18,014 - INFO - Initial checks done. Running the testcase now
2025-10-16 16:36:18,014 - INFO - 
2025-10-16 16:36:18,665 - DEBUG - Response for create_network : {'network': {'id': '0559fb77-05df-449e-971e-0663545dcdfc', 'name': 'ctest-vn1-62275531', 'tenant_id': '807e3fe83af34ee98b25339d20d29b41', 'project_id': '807e3fe83af34ee98b25339d20d29b41', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestRouters-01222227', 'ctest-vn1-62275531'], 'port_security_enabled': True, 'description': ''}}
2025-10-16 16:36:18,878 - DEBUG - Response for create_subnet : {'subnet': {'id': '0835643e-93aa-4f15-8a22-eec1978e98dc', 'name': '', 'tenant_id': '807e3fe83af34ee98b25339d20d29b41', 'network_id': '0559fb77-05df-449e-971e-0663545dcdfc', 'ip_version': 4, 'cidr': '89.202.221.64/26', 'allocation_pools': [{'start': '89.202.221.66', 'end': '89.202.221.126'}], 'gateway_ip': '89.202.221.65', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '89.202.221.66', 'tags': [], 'project_id': '807e3fe83af34ee98b25339d20d29b41'}}
2025-10-16 16:36:18,902 - INFO - Created VN ctest-vn1-62275531
2025-10-16 16:36:18,959 - DEBUG - VN ctest-vn1-62275531 UUID is 0559fb77-05df-449e-971e-0663545dcdfc
2025-10-16 16:36:19,215 - ERROR - Some exception while doing Neutron net-list
Traceback (most recent call last):
  File "/contrail-test/fixtures/quantum_test.py", line 276, in get_vn_obj_if_present
    net_rsp = self.obj.list_networks(**query_dict)['networks']
  File "/usr/local/lib/python3.9/site-packages/neutronclient/v2_0/client.py", line 817, in list_networks
    return self.list('networks', self.networks_path, retrieve_all,
  File "/usr/local/lib/python3.9/site-packages/neutronclient/v2_0/client.py", line 369, in list
    for r in self._pagination(collection, path, **params):
  File "/usr/local/lib/python3.9/site-packages/neutronclient/v2_0/client.py", line 384, in _pagination
    res = self.get(path, params=params)
  File "/usr/local/lib/python3.9/site-packages/neutronclient/v2_0/client.py", line 353, in get
    return self.retry_request("GET", action, body=body,
  File "/usr/local/lib/python3.9/site-packages/neutronclient/v2_0/client.py", line 330, in retry_request
    return self.do_request(method, action, body=body,
  File "/usr/local/lib/python3.9/site-packages/neutronclient/v2_0/client.py", line 294, in do_request
    self._handle_fault_response(status_code, replybody, resp)
  File "/usr/local/lib/python3.9/site-packages/neutronclient/v2_0/client.py", line 269, in _handle_fault_response
    exception_handler_v20(status_code, error_body)
  File "/usr/local/lib/python3.9/site-packages/neutronclient/v2_0/client.py", line 91, in exception_handler_v20
    raise client_exc(message=error_message,
neutronclient.common.exceptions.InternalServerError: The server has either erred or is incapable of performing the requested operation.<br /><br />



Neutron server returns request_ids: ['req-bd08588e-adf4-407d-8ac4-0d257fa7ea8e']
2025-10-16 16:36:19,216 - ERROR - Got exception as "The server has either erred or is incapable of performing the requested operation.<br /><br />\n\n\n\nNeutron server returns request_ids: ['req-bd08588e-adf4-407d-8ac4-0d257fa7ea8e']" while creating ctest-vn2-72976137
Traceback (most recent call last):
  File "/contrail-test/fixtures/quantum_test.py", line 276, in get_vn_obj_if_present
    net_rsp = self.obj.list_networks(**query_dict)['networks']
  File "/usr/local/lib/python3.9/site-packages/neutronclient/v2_0/client.py", line 817, in list_networks
    return self.list('networks', self.networks_path, retrieve_all,
  File "/usr/local/lib/python3.9/site-packages/neutronclient/v2_0/client.py", line 369, in list
    for r in self._pagination(collection, path, **params):
  File "/usr/local/lib/python3.9/site-packages/neutronclient/v2_0/client.py", line 384, in _pagination
    res = self.get(path, params=params)
  File "/usr/local/lib/python3.9/site-packages/neutronclient/v2_0/client.py", line 353, in get
    return self.retry_request("GET", action, body=body,
  File "/usr/local/lib/python3.9/site-packages/neutronclient/v2_0/client.py", line 330, in retry_request
    return self.do_request(method, action, body=body,
  File "/usr/local/lib/python3.9/site-packages/neutronclient/v2_0/client.py", line 294, in do_request
    self._handle_fault_response(status_code, replybody, resp)
  File "/usr/local/lib/python3.9/site-packages/neutronclient/v2_0/client.py", line 269, in _handle_fault_response
    exception_handler_v20(status_code, error_body)
  File "/usr/local/lib/python3.9/site-packages/neutronclient/v2_0/client.py", line 91, in exception_handler_v20
    raise client_exc(message=error_message,
neutronclient.common.exceptions.InternalServerError: The server has either erred or is incapable of performing the requested operation.<br /><br />



Neutron server returns request_ids: ['req-bd08588e-adf4-407d-8ac4-0d257fa7ea8e']

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/contrail-test/fixtures/vn_test.py", line 295, in _create_vn_orch
    self.obj = self.orchestrator.get_vn_obj_if_present(self.vn_name,
  File "/contrail-test/fixtures/openstack.py", line 184, in get_vn_obj_if_present
    return self.quantum_h.get_vn_obj_if_present(vn_name, **kwargs)
  File "/contrail-test/fixtures/quantum_test.py", line 282, in get_vn_obj_if_present
    raise NetworkClientException(message=str(e))
quantum_test.NetworkClientException: "The server has either erred or is incapable of performing the requested operation.<br /><br />\n\n\n\nNeutron server returns request_ids: ['req-bd08588e-adf4-407d-8ac4-0d257fa7ea8e']"
2025-10-16 16:36:19,220 - INFO - Deleting VN ctest-vn1-62275531 
2025-10-16 16:36:19,364 - DEBUG - Response for deleting network ()
2025-10-16 16:36:19,630 - ERROR - NetworkClientException
Python 3.9.21: /usr/bin/python3
Thu Oct 16 16:36:19 2025

A problem occurred in a Python script.  Here is the sequence of
function calls leading up to the error, in the order they occurred.

 /contrail-test/tcutils/wrappers.py in wrapper(self=<scripts.neutron.test_routers.TestRouters.test_b...sanity,dev_sanity_dpdk,sanity] id=0x7f4d6ca95c70>, *args=(), **kwargs={})
   78             log.info('Initial checks done. Running the testcase now')
   79             log.info('')
   80             result = function(self, *args, **kwargs)
   81             if self.inputs.upgrade:
   82                 pid = os.getpid()
result = None
function = <function TestRouters.test_basic_router_behavior>
self = <scripts.neutron.test_routers.TestRouters.test_b...sanity,dev_sanity_dpdk,sanity] id=0x7f4d6ca95c70>
args = ()
kwargs = {}

 /contrail-test/scripts/neutron/test_routers.py in test_basic_router_behavior(self=<scripts.neutron.test_routers.TestRouters.test_b...sanity,dev_sanity_dpdk,sanity] id=0x7f4d6ca95c70>)
   48         router_name = get_random_name('router1')
   49         vn1_fixture = self.create_vn(vn1_name, vn1_subnets)
   50         vn2_fixture = self.create_vn(vn2_name, vn2_subnets)
   51         vn1_vm1_fixture = self.create_vm(vn1_fixture, vn1_vm1_name,
   52                                          image_name='cirros')
vn2_fixture undefined
self = <scripts.neutron.test_routers.TestRouters.test_b...sanity,dev_sanity_dpdk,sanity] id=0x7f4d6ca95c70>
self.create_vn = <bound method GenericTestBase.create_vn of <scri...anity,dev_sanity_dpdk,sanity] id=0x7f4d6ca95c70>>
vn2_name = 'ctest-vn2-72976137'
vn2_subnets = ['17.197.214.128/26']

 /contrail-test/common/base.py in create_vn(self=<scripts.neutron.test_routers.TestRouters.test_b...sanity,dev_sanity_dpdk,sanity] id=0x7f4d6ca95c70>, vn_name='ctest-vn2-72976137', vn_subnets=['17.197.214.128/26'], cleanup=True, **kwargs={})
  189 
  190     def create_vn(self, vn_name=None, vn_subnets=None, cleanup=True, **kwargs):
  191         vn_fixture = self.create_only_vn(vn_name=vn_name,
  192                                      vn_subnets=vn_subnets,
  193                                      **kwargs)
vn_fixture undefined
self = <scripts.neutron.test_routers.TestRouters.test_b...sanity,dev_sanity_dpdk,sanity] id=0x7f4d6ca95c70>
self.create_only_vn = <bound method GenericTestBase.create_only_vn of <class 'scripts.neutron.test_routers.TestRouters'>>
vn_name = 'ctest-vn2-72976137'
vn_subnets = ['17.197.214.128/26']
kwargs = {}

 /contrail-test/common/base.py in create_only_vn(cls=<class 'scripts.neutron.test_routers.TestRouters'>, vn_name='ctest-vn2-72976137', vn_subnets=['17.197.214.128/26'], **kwargs={})
  184                       subnets=vn_subnets,
  185                       **kwargs)
  186         vn_fixture.setUp()
  187         return vn_fixture
  188     # end create_only_vn
vn_fixture = <vn_test.VNFixture object>
vn_fixture.setUp = <bound method VNFixture.setUp of <vn_test.VNFixture object>>

 /contrail-test/fixtures/vn_test.py in setUp(self=<vn_test.VNFixture object>)
  437     def setUp(self):
  438         super(VNFixture, self).setUp()
  439         self.create()
  440 
  441     def create(self):
self = <vn_test.VNFixture object>
self.create = <bound method VNFixture.create of <vn_test.VNFixture object>>

 /contrail-test/fixtures/vn_test.py in create(self=<vn_test.VNFixture object>)
  450             self._create_vn_api(self.vn_name, self.project_obj)
  451         else:
  452             self._create_vn_orch()
  453         if not self.obj:
  454              self.logger.debug('VN %s not present' % (self.vn_name))
self = <vn_test.VNFixture object>
self._create_vn_orch = <bound method VNFixture._create_vn_orch of <vn_test.VNFixture object>>

 /contrail-test/fixtures/vn_test.py in _create_vn_orch(self=<vn_test.VNFixture object>)
  335             if '503' in str(e) or '504' in str(e):
  336                 return False
  337             raise NetworkClientException(message=str(e))
  338 
  339     def get_vn_list_in_project(self, project_uuid):
global NetworkClientException = <class 'quantum_test.NetworkClientException'>
message undefined
builtinstr = <class 'str'>
e undefined
NetworkClientException: '"The server has either erred or is incapable of performing the requested operation.<br /><br />\\n\\n\\n\\nNeutron server returns request_ids: [\'req-bd08588e-adf4-407d-8ac4-0d257fa7ea8e\']"'
    __cause__ = None
    __class__ = <class 'quantum_test.NetworkClientException'>
    __context__ = NetworkClientException()
    __delattr__ = <method-wrapper '__delattr__' of NetworkClientException object>
    __dict__ = {'_error_string': '"The server has either erred or is incapable of ...ds: [\'req-bd08588e-adf4-407d-8ac4-0d257fa7ea8e\']"', 'message': '"The server has either erred or is incapable of ...ds: [\'req-bd08588e-adf4-407d-8ac4-0d257fa7ea8e\']"', 'request_ids': None, 'status_code': 0}
    __dir__ = <built-in method __dir__ of NetworkClientException object>
    __doc__ = None
    __eq__ = <method-wrapper '__eq__' of NetworkClientException object>
    __format__ = <built-in method __format__ of NetworkClientException object>
    __ge__ = <method-wrapper '__ge__' of NetworkClientException object>
    __getattribute__ = <method-wrapper '__getattribute__' of NetworkClientException object>
    __gt__ = <method-wrapper '__gt__' of NetworkClientException object>
    __hash__ = <method-wrapper '__hash__' of NetworkClientException object>
    __init__ = <bound method NetworkClientException.__init__ of NetworkClientException()>
    __init_subclass__ = <built-in method __init_subclass__ of type object>
    __le__ = <method-wrapper '__le__' of NetworkClientException object>
    __lt__ = <method-wrapper '__lt__' of NetworkClientException object>
    __module__ = 'quantum_test'
    __ne__ = <method-wrapper '__ne__' of NetworkClientException object>
    __new__ = <built-in method __new__ of type object>
    __reduce__ = <built-in method __reduce__ of NetworkClientException object>
    __reduce_ex__ = <built-in method __reduce_ex__ of NetworkClientException object>
    __repr__ = <method-wrapper '__repr__' of NetworkClientException object>
    __setattr__ = <method-wrapper '__setattr__' of NetworkClientException object>
    __setstate__ = <built-in method __setstate__ of NetworkClientException object>
    __sizeof__ = <built-in method __sizeof__ of NetworkClientException object>
    __str__ = <bound method NetworkClientException.__str__ of NetworkClientException()>
    __subclasshook__ = <built-in method __subclasshook__ of type object>
    __suppress_context__ = False
    __traceback__ = <traceback object>
    __weakref__ = None
    _error_string = '"The server has either erred or is incapable of ...ds: [\'req-bd08588e-adf4-407d-8ac4-0d257fa7ea8e\']"'
    args = ()
    message = '"The server has either erred or is incapable of ...ds: [\'req-bd08588e-adf4-407d-8ac4-0d257fa7ea8e\']"'
    req_ids_msg = 'Neutron server returns request_ids: %s'
    request_ids = None
    status_code = 0
    with_traceback = <built-in method with_traceback of NetworkClientException object>

The above is a description of an error in a Python program.  Here is
the original traceback:

Traceback (most recent call last):
  File "/contrail-test/fixtures/quantum_test.py", line 276, in get_vn_obj_if_present
    net_rsp = self.obj.list_networks(**query_dict)['networks']
  File "/usr/local/lib/python3.9/site-packages/neutronclient/v2_0/client.py", line 817, in list_networks
    return self.list('networks', self.networks_path, retrieve_all,
  File "/usr/local/lib/python3.9/site-packages/neutronclient/v2_0/client.py", line 369, in list
    for r in self._pagination(collection, path, **params):
  File "/usr/local/lib/python3.9/site-packages/neutronclient/v2_0/client.py", line 384, in _pagination
    res = self.get(path, params=params)
  File "/usr/local/lib/python3.9/site-packages/neutronclient/v2_0/client.py", line 353, in get
    return self.retry_request("GET", action, body=body,
  File "/usr/local/lib/python3.9/site-packages/neutronclient/v2_0/client.py", line 330, in retry_request
    return self.do_request(method, action, body=body,
  File "/usr/local/lib/python3.9/site-packages/neutronclient/v2_0/client.py", line 294, in do_request
    self._handle_fault_response(status_code, replybody, resp)
  File "/usr/local/lib/python3.9/site-packages/neutronclient/v2_0/client.py", line 269, in _handle_fault_response
    exception_handler_v20(status_code, error_body)
  File "/usr/local/lib/python3.9/site-packages/neutronclient/v2_0/client.py", line 91, in exception_handler_v20
    raise client_exc(message=error_message,
neutronclient.common.exceptions.InternalServerError: The server has either erred or is incapable of performing the requested operation.<br /><br />



Neutron server returns request_ids: ['req-bd08588e-adf4-407d-8ac4-0d257fa7ea8e']

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/contrail-test/fixtures/vn_test.py", line 295, in _create_vn_orch
    self.obj = self.orchestrator.get_vn_obj_if_present(self.vn_name,
  File "/contrail-test/fixtures/openstack.py", line 184, in get_vn_obj_if_present
    return self.quantum_h.get_vn_obj_if_present(vn_name, **kwargs)
  File "/contrail-test/fixtures/quantum_test.py", line 282, in get_vn_obj_if_present
    raise NetworkClientException(message=str(e))
quantum_test.NetworkClientException: "The server has either erred or is incapable of performing the requested operation.<br /><br />\n\n\n\nNeutron server returns request_ids: ['req-bd08588e-adf4-407d-8ac4-0d257fa7ea8e']"

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/contrail-test/tcutils/wrappers.py", line 80, in wrapper
    result = function(self, *args, **kwargs)
  File "/contrail-test/scripts/neutron/test_routers.py", line 50, in test_basic_router_behavior
    vn2_fixture = self.create_vn(vn2_name, vn2_subnets)
  File "/contrail-test/common/base.py", line 191, in create_vn
    vn_fixture = self.create_only_vn(vn_name=vn_name,
  File "/contrail-test/common/base.py", line 186, in create_only_vn
    vn_fixture.setUp()
  File "/contrail-test/fixtures/vn_test.py", line 439, in setUp
    self.create()
  File "/contrail-test/fixtures/vn_test.py", line 452, in create
    self._create_vn_orch()
  File "/contrail-test/fixtures/vn_test.py", line 337, in _create_vn_orch
    raise NetworkClientException(message=str(e))
quantum_test.NetworkClientException: '"The server has either erred or is incapable of performing the requested operation.<br /><br />\\n\\n\\n\\nNeutron server returns request_ids: [\'req-bd08588e-adf4-407d-8ac4-0d257fa7ea8e\']"'



2025-10-16 16:36:19,637 - DEBUG - No XMPP flaps were noticed during the test
2025-10-16 16:36:19,637 - INFO - 
2025-10-16 16:36:19,637 - INFO - END TEST : test_basic_router_behavior : FAILED[0:00:02]
2025-10-16 16:36:19,637 - INFO - --------------------------------------------------------------------------------
2025-10-16 16:36:20,446 - INFO - Deleted project: ctest-TestRouters-01222227, ID : 807e3fe8-3af3-4ee9-8b25-339d20d29b41