2025-08-18 15:35:39,287 - INFO - Using existing project ['default-domain', 'admin'](424872b8-35a4-403e-b480-f7f7a3ba4af3) 2025-08-18 15:35:39,296 - INFO - ================================================================================ 2025-08-18 15:35:39,296 - INFO - STARTING TEST : test_db_manage 2025-08-18 15:35:39,297 - INFO - TEST DESCRIPTION : Test db_manage.py tool 2025-08-18 15:35:39,425 - DEBUG - Nothing to compare xmpp stats {'10.0.0.254': {'10.0.0.254': '0'}} with 2025-08-18 15:35:39,425 - INFO - Initial checks done. Running the testcase now 2025-08-18 15:35:39,425 - INFO - 2025-08-18 15:35:43,981 - DEBUG - 2025-08-18 15:35:41,972 INFO: (v1.41) Checker check_zk_mode_and_node_count: Success 2025-08-18 15:35:41,990 INFO: (v1.41) Checker check_cassandra_keyspace_replication: Success 2025-08-18 15:35:42,358 INFO: (v1.41) Checker check_obj_mandatory_fields: Success 2025-08-18 15:35:42,874 INFO: (v1.41) Checker check_orphan_resources: Success 2025-08-18 15:35:43,373 INFO: (v1.41) Checker check_fq_name_uuid_match: Success 2025-08-18 15:35:43,373 WARNING: Be careful, that check can return false positive errors if stale FQ names and stale resources were not cleaned before. Run at least commands 'clean_obj_missing_mandatory_fields', 'clean_orphan_resources' and 'clean_stale_fq_names' before. 2025-08-18 15:35:43,608 INFO: (v1.41) Checker check_duplicate_fq_name: Success 2025-08-18 15:35:43,621 INFO: (v1.41) Checker check_route_targets_routing_instance_backrefs: Success 2025-08-18 15:35:43,633 INFO: (v1.41) Checker check_subnet_uuid: Success 2025-08-18 15:35:43,673 ERROR: (v1.41) Checker check_subnet_addr_alloc: Exception, NameError Python 3.9.21: /usr/bin/python3 Mon Aug 18 15:35:43 2025 A problem occurred in a Python script. Here is the sequence of function calls leading up to the error, in the order they occurred. /usr/local/lib/python3.9/site-packages/vnc_cfg_api_server/db_manage.py in wrapper(*args=(,), **kwargs={}) 1547 self = args[0] 1548 try: 1549 errors = func(*args, **kwargs) 1550 if not errors: 1551 self._logger.info('(v%s) Checker %s: Success', __version__, errors undefined func = args = (,) kwargs = {} /usr/local/lib/python3.9/site-packages/vnc_cfg_api_server/db_manage.py in check_subnet_addr_alloc(self=) 1861 # whether ip allocated in subnet in zk match iip+fip in cassandra 1862 (zk_all_vns, cassandra_all_vns, duplicate_ips, 1863 ret_errors, stale_zk_path) = self.audit_subnet_addr_alloc() 1864 1865 # check stale ZK subnet path ret_errors undefined stale_zk_path undefined self = self.audit_subnet_addr_alloc = > /usr/local/lib/python3.9/site-packages/vnc_cfg_api_server/db_manage.py in audit_subnet_addr_alloc(self=) 1179 self.base_subnet_zk_path) 1180 num_addrs = 0 1181 for subnet_path in self._subnet_path_discovery( 1182 ret_errors, stale_zk_path): 1183 if subnet_path.startswith("%s/" % self.base_subnet_zk_path): subnet_path undefined self = self._subnet_path_discovery = > ret_errors = [] stale_zk_path = [] /usr/local/lib/python3.9/site-packages/vnc_cfg_api_server/db_manage.py in _subnet_path_discovery(self=, ret_errors=[], stale_zk_path=[]) 1167 subnet_paths.add(path) 1168 1169 deep_path_discovery(self.base_subnet_zk_path) 1170 return subnet_paths 1171 deep_path_discovery = .deep_path_discovery> self = self.base_subnet_zk_path = '/api-server/subnets' /usr/local/lib/python3.9/site-packages/vnc_cfg_api_server/db_manage.py in deep_path_discovery(path='/api-server/subnets') 1154 except AddrFormatError: 1155 try: 1156 suffixes = zk_children_with_retry(path) 1157 except kazoo.exceptions.NoNodeError: 1158 self._logger.debug("ZK subnet path '%s' does not exits" % suffixes undefined zk_children_with_retry = .zk_children_with_retry> path = '/api-server/subnets' /usr/local/lib/python3.9/site-packages/vnc_cfg_api_server/db_manage.py in zk_children_with_retry(path='/api-server/subnets', retries=5, delay=0.2) 1147 if kids or attempt == retries - 1: 1148 return kids 1149 time.sleep(delay) 1150 1151 def deep_path_discovery(path): time undefined delay = 0.2 NameError: name 'time' is not defined __cause__ = None __class__ = __context__ = AddrFormatError('invalid IPNetwork /api-server/subnets') __delattr__ = __dict__ = {} __dir__ = __doc__ = 'Name not found globally.' __eq__ = __format__ = __ge__ = __getattribute__ = __gt__ = __hash__ = __init__ = __init_subclass__ = __le__ = __lt__ = __ne__ = __new__ = __reduce__ = __reduce_ex__ = __repr__ = __setattr__ = __setstate__ = __sizeof__ = __str__ = __subclasshook__ = __suppress_context__ = False __traceback__ = args = ("name 'time' is not defined",) with_traceback = The above is a description of an error in a Python program. Here is the original traceback: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/netaddr/strategy/ipv4.py", line 129, in str_to_int return _struct.unpack('>I', _inet_pton(AF_INET, addr))[0] OSError: illegal IP address string passed to inet_pton During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/netaddr/ip/__init__.py", line 380, in __init__ self._value = self._module.str_to_int(addr, flags) File "/usr/local/lib/python3.9/site-packages/netaddr/strategy/ipv4.py", line 133, in str_to_int raise AddrFormatError('%r is not a valid IPv4 address string!' % (addr,)) netaddr.core.AddrFormatError: '' is not a valid IPv4 address string! During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/netaddr/ip/__init__.py", line 962, in parse_ip_network ip = IPAddress(val1, module.version, flags=INET_PTON) File "/usr/local/lib/python3.9/site-packages/netaddr/ip/__init__.py", line 382, in __init__ raise AddrFormatError('base address %r is not IPv%d' netaddr.core.AddrFormatError: base address '' is not IPv4 During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/netaddr/strategy/ipv4.py", line 269, in expand_partial_address tokens = ['%d' % int(addr)] ValueError: invalid literal for int() with base 10: '' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/netaddr/ip/__init__.py", line 1116, in __init__ value, prefixlen = parse_ip_network(module, addr, File "/usr/local/lib/python3.9/site-packages/netaddr/ip/__init__.py", line 966, in parse_ip_network expanded_addr = _ipv4.expand_partial_address(val1) File "/usr/local/lib/python3.9/site-packages/netaddr/strategy/ipv4.py", line 271, in expand_partial_address raise error netaddr.core.AddrFormatError: invalid partial IPv4 address: ''! During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/vnc_cfg_api_server/db_manage.py", line 1153, in deep_path_discovery IPNetwork(path.split(':', 3)[-1]) File "/usr/local/lib/python3.9/site-packages/netaddr/ip/__init__.py", line 1127, in __init__ raise AddrFormatError('invalid IPNetwork %s' % (addr,)) netaddr.core.AddrFormatError: invalid IPNetwork /api-server/subnets During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/vnc_cfg_api_server/db_manage.py", line 1549, in wrapper errors = func(*args, **kwargs) File "/usr/local/lib/python3.9/site-packages/vnc_cfg_api_server/db_manage.py", line 1863, in check_subnet_addr_alloc ret_errors, stale_zk_path) = self.audit_subnet_addr_alloc() File "/usr/local/lib/python3.9/site-packages/vnc_cfg_api_server/db_manage.py", line 1181, in audit_subnet_addr_alloc for subnet_path in self._subnet_path_discovery( File "/usr/local/lib/python3.9/site-packages/vnc_cfg_api_server/db_manage.py", line 1169, in _subnet_path_discovery deep_path_discovery(self.base_subnet_zk_path) File "/usr/local/lib/python3.9/site-packages/vnc_cfg_api_server/db_manage.py", line 1156, in deep_path_discovery suffixes = zk_children_with_retry(path) File "/usr/local/lib/python3.9/site-packages/vnc_cfg_api_server/db_manage.py", line 1149, in zk_children_with_retry time.sleep(delay) NameError: name 'time' is not defined Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/netaddr/strategy/ipv4.py", line 129, in str_to_int return _struct.unpack('>I', _inet_pton(AF_INET, addr))[0] OSError: illegal IP address string passed to inet_pton During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/netaddr/ip/__init__.py", line 380, in __init__ self._value = self._module.str_to_int(addr, flags) File "/usr/local/lib/python3.9/site-packages/netaddr/strategy/ipv4.py", line 133, in str_to_int raise AddrFormatError('%r is not a valid IPv4 address string!' % (addr,)) netaddr.core.AddrFormatError: '' is not a valid IPv4 address string! During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/netaddr/ip/__init__.py", line 962, in parse_ip_network ip = IPAddress(val1, module.version, flags=INET_PTON) File "/usr/local/lib/python3.9/site-packages/netaddr/ip/__init__.py", line 382, in __init__ raise AddrFormatError('base address %r is not IPv%d' netaddr.core.AddrFormatError: base address '' is not IPv4 During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/netaddr/strategy/ipv4.py", line 269, in expand_partial_address tokens = ['%d' % int(addr)] ValueError: invalid literal for int() with base 10: '' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/netaddr/ip/__init__.py", line 1116, in __init__ value, prefixlen = parse_ip_network(module, addr, File "/usr/local/lib/python3.9/site-packages/netaddr/ip/__init__.py", line 966, in parse_ip_network expanded_addr = _ipv4.expand_partial_address(val1) File "/usr/local/lib/python3.9/site-packages/netaddr/strategy/ipv4.py", line 271, in expand_partial_address raise error netaddr.core.AddrFormatError: invalid partial IPv4 address: ''! During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/vnc_cfg_api_server/db_manage.py", line 1153, in deep_path_discovery IPNetwork(path.split(':', 3)[-1]) File "/usr/local/lib/python3.9/site-packages/netaddr/ip/__init__.py", line 1127, in __init__ raise AddrFormatError('invalid IPNetwork %s' % (addr,)) netaddr.core.AddrFormatError: invalid IPNetwork /api-server/subnets During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/vnc_cfg_api_server/db_manage.py", line 1549, in wrapper errors = func(*args, **kwargs) File "/usr/local/lib/python3.9/site-packages/vnc_cfg_api_server/db_manage.py", line 1863, in check_subnet_addr_alloc ret_errors, stale_zk_path) = self.audit_subnet_addr_alloc() File "/usr/local/lib/python3.9/site-packages/vnc_cfg_api_server/db_manage.py", line 1181, in audit_subnet_addr_alloc for subnet_path in self._subnet_path_discovery( File "/usr/local/lib/python3.9/site-packages/vnc_cfg_api_server/db_manage.py", line 1169, in _subnet_path_discovery deep_path_discovery(self.base_subnet_zk_path) File "/usr/local/lib/python3.9/site-packages/vnc_cfg_api_server/db_manage.py", line 1156, in deep_path_discovery suffixes = zk_children_with_retry(path) File "/usr/local/lib/python3.9/site-packages/vnc_cfg_api_server/db_manage.py", line 1149, in zk_children_with_retry time.sleep(delay) NameError: name 'time' is not defined NameError Python 3.9.21: /usr/bin/python3 Mon Aug 18 15:35:43 2025 A problem occurred in a Python script. Here is the sequence of function calls leading up to the error, in the order they occurred. /usr/local/bin/db-manage in 3 from vnc_cfg_api_server.db_manage import main 4 if __name__ == '__main__': 5 if sys.argv[0].endswith('.exe'): 6 sys.argv[0] = sys.argv[0][:-4] 7 sys.exit(main()) sys = sys.exit = main = /usr/local/lib/python3.9/site-packages/vnc_cfg_api_server/db_manage.py in main() 3452 verb = args.operation 3453 if 'db_%s' % (verb) in globals(): 3454 return globals()['db_%s' % (verb)](args, api_args) 3455 3456 if getattr(DatabaseChecker, verb, None): builtinglobals = verb = 'check' args = Namespace(operation='check', api_conf='/etc/cont...1024, log_file='/var/log/contrail/db_manage.log') api_args = Namespace(conf_file=['/etc/contrail/contrail-api...esh_base.SandeshConfig object at 0x7fc1ab3861f0>) /usr/local/lib/python3.9/site-packages/vnc_cfg_api_server/db_manage.py in db_check(args=Namespace(operation='check', api_conf='/etc/cont...1024, log_file='/var/log/contrail/db_manage.log'), api_args=Namespace(conf_file=['/etc/contrail/contrail-api...esh_base.SandeshConfig object at 0x7fc1ab3861f0>)) 3369 # ID allocation inconsistencies 3370 db_checker.check_subnet_uuid() 3371 db_checker.check_subnet_addr_alloc() 3372 db_checker.check_route_targets_id() 3373 db_checker.check_virtual_networks_id() db_checker = db_checker.check_subnet_addr_alloc = > /usr/local/lib/python3.9/site-packages/vnc_cfg_api_server/db_manage.py in wrapper(*args=(,), **kwargs={}) 1547 self = args[0] 1548 try: 1549 errors = func(*args, **kwargs) 1550 if not errors: 1551 self._logger.info('(v%s) Checker %s: Success', __version__, errors undefined func = args = (,) kwargs = {} /usr/local/lib/python3.9/site-packages/vnc_cfg_api_server/db_manage.py in check_subnet_addr_alloc(self=) 1861 # whether ip allocated in subnet in zk match iip+fip in cassandra 1862 (zk_all_vns, cassandra_all_vns, duplicate_ips, 1863 ret_errors, stale_zk_path) = self.audit_subnet_addr_alloc() 1864 1865 # check stale ZK subnet path ret_errors undefined stale_zk_path undefined self = self.audit_subnet_addr_alloc = > /usr/local/lib/python3.9/site-packages/vnc_cfg_api_server/db_manage.py in audit_subnet_addr_alloc(self=) 1179 self.base_subnet_zk_path) 1180 num_addrs = 0 1181 for subnet_path in self._subnet_path_discovery( 1182 ret_errors, stale_zk_path): 1183 if subnet_path.startswith("%s/" % self.base_subnet_zk_path): subnet_path undefined self = self._subnet_path_discovery = > ret_errors = [] stale_zk_path = [] /usr/local/lib/python3.9/site-packages/vnc_cfg_api_server/db_manage.py in _subnet_path_discovery(self=, ret_errors=[], stale_zk_path=[]) 1167 subnet_paths.add(path) 1168 1169 deep_path_discovery(self.base_subnet_zk_path) 1170 return subnet_paths 1171 deep_path_discovery = .deep_path_discovery> self = self.base_subnet_zk_path = '/api-server/subnets' /usr/local/lib/python3.9/site-packages/vnc_cfg_api_server/db_manage.py in deep_path_discovery(path='/api-server/subnets') 1154 except AddrFormatError: 1155 try: 1156 suffixes = zk_children_with_retry(path) 1157 except kazoo.exceptions.NoNodeError: 1158 self._logger.debug("ZK subnet path '%s' does not exits" % suffixes undefined zk_children_with_retry = .zk_children_with_retry> path = '/api-server/subnets' /usr/local/lib/python3.9/site-packages/vnc_cfg_api_server/db_manage.py in zk_children_with_retry(path='/api-server/subnets', retries=5, delay=0.2) 1147 if kids or attempt == retries - 1: 1148 return kids 1149 time.sleep(delay) 1150 1151 def deep_path_discovery(path): time undefined delay = 0.2 NameError: name 'time' is not defined __cause__ = None __class__ = __context__ = AddrFormatError('invalid IPNetwork /api-server/subnets') __delattr__ = __dict__ = {} __dir__ = __doc__ = 'Name not found globally.' __eq__ = __format__ = __ge__ = __getattribute__ = __gt__ = __hash__ = __init__ = __init_subclass__ = __le__ = __lt__ = __ne__ = __new__ = __reduce__ = __reduce_ex__ = __repr__ = __setattr__ = __setstate__ = __sizeof__ = __str__ = __subclasshook__ = __suppress_context__ = False __traceback__ = args = ("name 'time' is not defined",) with_traceback = The above is a description of an error in a Python program. Here is the original traceback: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/netaddr/strategy/ipv4.py", line 129, in str_to_int return _struct.unpack('>I', _inet_pton(AF_INET, addr))[0] OSError: illegal IP address string passed to inet_pton During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/netaddr/ip/__init__.py", line 380, in __init__ self._value = self._module.str_to_int(addr, flags) File "/usr/local/lib/python3.9/site-packages/netaddr/strategy/ipv4.py", line 133, in str_to_int raise AddrFormatError('%r is not a valid IPv4 address string!' % (addr,)) netaddr.core.AddrFormatError: '' is not a valid IPv4 address string! During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/netaddr/ip/__init__.py", line 962, in parse_ip_network ip = IPAddress(val1, module.version, flags=INET_PTON) File "/usr/local/lib/python3.9/site-packages/netaddr/ip/__init__.py", line 382, in __init__ raise AddrFormatError('base address %r is not IPv%d' netaddr.core.AddrFormatError: base address '' is not IPv4 During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/netaddr/strategy/ipv4.py", line 269, in expand_partial_address tokens = ['%d' % int(addr)] ValueError: invalid literal for int() with base 10: '' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/netaddr/ip/__init__.py", line 1116, in __init__ value, prefixlen = parse_ip_network(module, addr, File "/usr/local/lib/python3.9/site-packages/netaddr/ip/__init__.py", line 966, in parse_ip_network expanded_addr = _ipv4.expand_partial_address(val1) File "/usr/local/lib/python3.9/site-packages/netaddr/strategy/ipv4.py", line 271, in expand_partial_address raise error netaddr.core.AddrFormatError: invalid partial IPv4 address: ''! During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/vnc_cfg_api_server/db_manage.py", line 1153, in deep_path_discovery IPNetwork(path.split(':', 3)[-1]) File "/usr/local/lib/python3.9/site-packages/netaddr/ip/__init__.py", line 1127, in __init__ raise AddrFormatError('invalid IPNetwork %s' % (addr,)) netaddr.core.AddrFormatError: invalid IPNetwork /api-server/subnets During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/bin/db-manage", line 7, in sys.exit(main()) File "/usr/local/lib/python3.9/site-packages/vnc_cfg_api_server/db_manage.py", line 3454, in main return globals()['db_%s' % (verb)](args, api_args) File "/usr/local/lib/python3.9/site-packages/vnc_cfg_api_server/db_manage.py", line 3371, in db_check db_checker.check_subnet_addr_alloc() File "/usr/local/lib/python3.9/site-packages/vnc_cfg_api_server/db_manage.py", line 1549, in wrapper errors = func(*args, **kwargs) File "/usr/local/lib/python3.9/site-packages/vnc_cfg_api_server/db_manage.py", line 1863, in check_subnet_addr_alloc ret_errors, stale_zk_path) = self.audit_subnet_addr_alloc() File "/usr/local/lib/python3.9/site-packages/vnc_cfg_api_server/db_manage.py", line 1181, in audit_subnet_addr_alloc for subnet_path in self._subnet_path_discovery( File "/usr/local/lib/python3.9/site-packages/vnc_cfg_api_server/db_manage.py", line 1169, in _subnet_path_discovery deep_path_discovery(self.base_subnet_zk_path) File "/usr/local/lib/python3.9/site-packages/vnc_cfg_api_server/db_manage.py", line 1156, in deep_path_discovery suffixes = zk_children_with_retry(path) File "/usr/local/lib/python3.9/site-packages/vnc_cfg_api_server/db_manage.py", line 1149, in zk_children_with_retry time.sleep(delay) NameError: name 'time' is not defined 2025-08-18 15:35:43,981 - WARNING - Errors are still present: 2025-08-18 15:35:43,673 ERROR: (v1.41) Checker check_subnet_addr_alloc: Exception, NameError 2025-08-18 15:35:43,981 - WARNING - Ignore some errors untill fix 2025-08-18 15:35:44,103 - DEBUG - No XMPP flaps were noticed during the test
2025-08-18 15:35:44,103 - INFO - END TEST : test_db_manage : PASSED[0:00:05]
2025-08-18 15:35:44,103 - INFO - --------------------------------------------------------------------------------