INFO: =================== Mon Sep 29 01:19:28 AM UTC 2025 =================== /contrail-functions.sh: line 167: /etc/contrail/vnc_api_lib.ini.tmp: No such file or directory /contrail-functions.sh: line 202: /etc/contrail/vnc_api_lib.ini.tmp: No such file or directory mv: cannot stat '/etc/contrail/vnc_api_lib.ini.tmp': No such file or directory [DEFAULTS] http_server_ip=0.0.0.0 log_file=/var/log/contrail/database-nodemgr/contrail-database-nodemgr.log log_level=SYS_NOTICE log_local=1 hostip=10.0.0.48 db_port=9042 db_jmx_port=7200 db_use_ssl=False [COLLECTOR] server_list=10.0.0.48:8086 [SANDESH] introspect_ssl_enable=False sandesh_ssl_enable=False [DEFAULTS] minimum_diskGB = 2 09/29/2025 01:19:30.303 7fa409fba100 [contrail-database-nodemgr] [INFO]: SANDESH: CONNECT TO COLLECTOR: True 09/29/2025 01:19:30.423 7fa409fba100 [contrail-database-nodemgr] [ERROR]: Failed to import package "database.sandesh" 09/29/2025 01:19:30.431 7fa409fba100 [contrail-database-nodemgr] [INFO]: SANDESH: INTROSPECT IS ON: 0.0.0.0:8103 09/29/2025 01:19:30.435 7fa409fba100 [contrail-database-nodemgr] [ERROR]: Failed to import package "database.sandesh" 09/29/2025 01:19:30.440 7fa409fba100 [contrail-database-nodemgr] [INFO]: SANDESH: Logging: LEVEL: [SYS_INFO] -> [SYS_NOTICE] CRITICAL:root:Exit data: {'ID': '0fbeb12050ccbda874411bcb89bef05fe70d29a6758dd6d940541f36d028f4e3', 'Running': False, 'ExitCode': 1, 'ProcessConfig': {'tty': True, 'entrypoint': 'cqlsh', 'arguments': ['10.0.0.48', '9042', '-e', 'quit'], 'privileged': False, 'user': '0'}, 'OpenStdin': False, 'OpenStderr': True, 'OpenStdout': True, 'CanRemove': False, 'ContainerID': 'baa20a654d406b3db5b8d47eaa0834ca701754510375b68125b23982559b0f53', 'DetachKeys': '', 'Pid': 71127} ERROR:contrail-database-nodemgr:Failed to connect to database by CQL and socket: Result: 1 CRITICAL:root:Exit data: {'ID': '835ea6cb7f8c935dffa83f82519771600dcde61fee806eb6a55fb522cfe12cc9', 'Running': False, 'ExitCode': 1, 'ProcessConfig': {'tty': True, 'entrypoint': 'nodetool', 'arguments': ['-p', '7200', 'compactionstats'], 'privileged': False, 'user': '0'}, 'OpenStdin': False, 'OpenStderr': True, 'OpenStdout': True, 'CanRemove': False, 'ContainerID': 'baa20a654d406b3db5b8d47eaa0834ca701754510375b68125b23982559b0f53', 'DetachKeys': '', 'Pid': 71145} ERROR:contrail-database-nodemgr:Failed to get nodetool compactionstats: Result: 1 ERROR:contrail-database-nodemgr:Failed to get core files: [Errno 2] No such file or directory: '/var/crashes' [Introspect:8103]10.0.0.48 - - [2025-09-29 01:20:53] "GET /Snh_SandeshUVECacheReq?x=NodeStatus HTTP/1.1" 200 6749 0.002245 [Introspect:8103]10.0.0.48 - - [2025-09-29 01:21:16] "GET /Snh_SandeshUVECacheReq?x=NodeStatus HTTP/1.1" 200 6749 0.001056 [Introspect:8103]10.0.0.48 - - [2025-09-29 01:21:29] "GET /Snh_SandeshUVECacheReq?x=NodeStatus HTTP/1.1" 200 6749 0.001112 CRITICAL:root:Exit data: {'ID': 'c6a8e937a025d982092c4be6d4d97b1af111cf87273c8e26f786b6ac0fd8c1dc', 'Running': False, 'ExitCode': 1, 'ProcessConfig': {'tty': True, 'entrypoint': 'cqlsh', 'arguments': ['10.0.0.48', '9042', '-e', 'quit'], 'privileged': False, 'user': '0'}, 'OpenStdin': False, 'OpenStderr': True, 'OpenStdout': True, 'CanRemove': False, 'ContainerID': 'baa20a654d406b3db5b8d47eaa0834ca701754510375b68125b23982559b0f53', 'DetachKeys': '', 'Pid': 73407} WARNING:contrail-database-nodemgr:CQLSH failed, but Cassandra native transport port is open CRITICAL:root:Exit data: {'ID': '7eb318f5aad429a8bca14ea802106a135dd68308bff5345e423a6b1edfbcfb6c', 'Running': False, 'ExitCode': 1, 'ProcessConfig': {'tty': True, 'entrypoint': 'nodetool', 'arguments': ['-p', '7200', 'compactionstats'], 'privileged': False, 'user': '0'}, 'OpenStdin': False, 'OpenStderr': True, 'OpenStdout': True, 'CanRemove': False, 'ContainerID': 'baa20a654d406b3db5b8d47eaa0834ca701754510375b68125b23982559b0f53', 'DetachKeys': '', 'Pid': 73425} ERROR:contrail-database-nodemgr:Failed to get nodetool compactionstats: Result: 1 ERROR:contrail-database-nodemgr:Failed to get core files: [Errno 2] No such file or directory: '/var/crashes' [Introspect:8103]10.0.0.48 - - [2025-09-29 01:21:42] "GET /Snh_SandeshUVECacheReq?x=NodeStatus HTTP/1.1" 200 6716 0.001386 [Introspect:8103]10.0.0.48 - - [2025-09-29 01:21:44] "GET /Snh_SandeshUVECacheReq?x=NodeStatus HTTP/1.1" 200 6716 0.001038 CRITICAL:root:Exit data: {'ID': '3b5f5814590bcc1ba740cdacc459dff4dde00a3350ce6e5282738c50b205a66e', 'Running': False, 'ExitCode': 1, 'ProcessConfig': {'tty': True, 'entrypoint': 'cqlsh', 'arguments': ['10.0.0.48', '9042', '-e', 'quit'], 'privileged': False, 'user': '0'}, 'OpenStdin': False, 'OpenStderr': True, 'OpenStdout': True, 'CanRemove': False, 'ContainerID': 'baa20a654d406b3db5b8d47eaa0834ca701754510375b68125b23982559b0f53', 'DetachKeys': '', 'Pid': 75234} WARNING:contrail-database-nodemgr:CQLSH failed, but Cassandra native transport port is open CRITICAL:root:Exit data: {'ID': '1a544b5c00cd86421e0d6a7bf54801b1254d724971df0abccce8e6d19e877a96', 'Running': False, 'ExitCode': 1, 'ProcessConfig': {'tty': True, 'entrypoint': 'nodetool', 'arguments': ['-p', '7200', 'compactionstats'], 'privileged': False, 'user': '0'}, 'OpenStdin': False, 'OpenStderr': True, 'OpenStdout': True, 'CanRemove': False, 'ContainerID': 'baa20a654d406b3db5b8d47eaa0834ca701754510375b68125b23982559b0f53', 'DetachKeys': '', 'Pid': 75261} ERROR:contrail-database-nodemgr:Failed to get nodetool compactionstats: Result: 1 ERROR:contrail-database-nodemgr:Failed to get core files: [Errno 2] No such file or directory: '/var/crashes' CRITICAL:root:Exit data: {'ID': 'c2fe869b39fac6323d19bba1a45b876ff61d5bd8d88e85bc89a4cc77d31c3515', 'Running': False, 'ExitCode': 1, 'ProcessConfig': {'tty': True, 'entrypoint': 'cqlsh', 'arguments': ['10.0.0.48', '9042', '-e', 'quit'], 'privileged': False, 'user': '0'}, 'OpenStdin': False, 'OpenStderr': True, 'OpenStdout': True, 'CanRemove': False, 'ContainerID': 'baa20a654d406b3db5b8d47eaa0834ca701754510375b68125b23982559b0f53', 'DetachKeys': '', 'Pid': 77974} WARNING:contrail-database-nodemgr:CQLSH failed, but Cassandra native transport port is open CRITICAL:root:Exit data: {'ID': '2d2220186eb95756ce7addb0a47c78d33a4ba5caa854799960c258cd535e6788', 'Running': False, 'ExitCode': 1, 'ProcessConfig': {'tty': True, 'entrypoint': 'nodetool', 'arguments': ['-p', '7200', 'compactionstats'], 'privileged': False, 'user': '0'}, 'OpenStdin': False, 'OpenStderr': True, 'OpenStdout': True, 'CanRemove': False, 'ContainerID': 'baa20a654d406b3db5b8d47eaa0834ca701754510375b68125b23982559b0f53', 'DetachKeys': '', 'Pid': 78068} ERROR:contrail-database-nodemgr:Failed to get nodetool compactionstats: Result: 1 ERROR:contrail-database-nodemgr:Failed to get core files: [Errno 2] No such file or directory: '/var/crashes' CRITICAL:root:Exit data: {'ID': '720724d1f10435d652fe1c2f2b74d40eb83dab4ce42df648c9f2b847c42ef760', 'Running': False, 'ExitCode': 1, 'ProcessConfig': {'tty': True, 'entrypoint': 'cqlsh', 'arguments': ['10.0.0.48', '9042', '-e', 'quit'], 'privileged': False, 'user': '0'}, 'OpenStdin': False, 'OpenStderr': True, 'OpenStdout': True, 'CanRemove': False, 'ContainerID': 'baa20a654d406b3db5b8d47eaa0834ca701754510375b68125b23982559b0f53', 'DetachKeys': '', 'Pid': 81165} WARNING:contrail-database-nodemgr:CQLSH failed, but Cassandra native transport port is open CRITICAL:root:Exit data: {'ID': '269202bdc1f4f835fed4a0fabea3610200f9c62d6be287e0678b38ed6734d960', 'Running': False, 'ExitCode': 1, 'ProcessConfig': {'tty': True, 'entrypoint': 'nodetool', 'arguments': ['-p', '7200', 'compactionstats'], 'privileged': False, 'user': '0'}, 'OpenStdin': False, 'OpenStderr': True, 'OpenStdout': True, 'CanRemove': False, 'ContainerID': 'baa20a654d406b3db5b8d47eaa0834ca701754510375b68125b23982559b0f53', 'DetachKeys': '', 'Pid': 81184} ERROR:contrail-database-nodemgr:Failed to get nodetool compactionstats: Result: 1 ERROR:contrail-database-nodemgr:Failed to get core files: [Errno 2] No such file or directory: '/var/crashes' CRITICAL:root:Exit data: {'ID': 'ac3a535919b89d2dc7fcd1bb19a2e5241be30cd8b32dc8c3e75389ffd356ada9', 'Running': False, 'ExitCode': 1, 'ProcessConfig': {'tty': True, 'entrypoint': 'cqlsh', 'arguments': ['10.0.0.48', '9042', '-e', 'quit'], 'privileged': False, 'user': '0'}, 'OpenStdin': False, 'OpenStderr': True, 'OpenStdout': True, 'CanRemove': False, 'ContainerID': 'baa20a654d406b3db5b8d47eaa0834ca701754510375b68125b23982559b0f53', 'DetachKeys': '', 'Pid': 82330} WARNING:contrail-database-nodemgr:CQLSH failed, but Cassandra native transport port is open CRITICAL:root:Exit data: {'ID': '442189076e54684226b43e4570a2cbe9928b2fce2dee9e43995ac29267270fd4', 'Running': False, 'ExitCode': 1, 'ProcessConfig': {'tty': True, 'entrypoint': 'nodetool', 'arguments': ['-p', '7200', 'compactionstats'], 'privileged': False, 'user': '0'}, 'OpenStdin': False, 'OpenStderr': True, 'OpenStdout': True, 'CanRemove': False, 'ContainerID': 'baa20a654d406b3db5b8d47eaa0834ca701754510375b68125b23982559b0f53', 'DetachKeys': '', 'Pid': 82349} ERROR:contrail-database-nodemgr:Failed to get nodetool compactionstats: Result: 1 ERROR:contrail-database-nodemgr:Failed to get core files: [Errno 2] No such file or directory: '/var/crashes' CRITICAL:root:Exit data: {'ID': 'acfd687fba2480bfe7e75e15c4380957bb1eb392b699b52e5cff0434a5a67778', 'Running': False, 'ExitCode': 1, 'ProcessConfig': {'tty': True, 'entrypoint': 'cqlsh', 'arguments': ['10.0.0.48', '9042', '-e', 'quit'], 'privileged': False, 'user': '0'}, 'OpenStdin': False, 'OpenStderr': True, 'OpenStdout': True, 'CanRemove': False, 'ContainerID': 'baa20a654d406b3db5b8d47eaa0834ca701754510375b68125b23982559b0f53', 'DetachKeys': '', 'Pid': 83399} WARNING:contrail-database-nodemgr:CQLSH failed, but Cassandra native transport port is open CRITICAL:root:Exit data: {'ID': 'd159fe3fb8a1c553383877d65ab6d683450cbc19578a8c0fb80f040799d19967', 'Running': False, 'ExitCode': 1, 'ProcessConfig': {'tty': True, 'entrypoint': 'nodetool', 'arguments': ['-p', '7200', 'compactionstats'], 'privileged': False, 'user': '0'}, 'OpenStdin': False, 'OpenStderr': True, 'OpenStdout': True, 'CanRemove': False, 'ContainerID': 'baa20a654d406b3db5b8d47eaa0834ca701754510375b68125b23982559b0f53', 'DetachKeys': '', 'Pid': 83418} ERROR:contrail-database-nodemgr:Failed to get nodetool compactionstats: Result: 1 ERROR:contrail-database-nodemgr:Failed to get core files: [Errno 2] No such file or directory: '/var/crashes' CRITICAL:root:Exit data: {'ID': '06a3f506624a7ee45b41c1d912b7112538efffc12cc30b2f8eabe96df8a6ef3e', 'Running': False, 'ExitCode': 1, 'ProcessConfig': {'tty': True, 'entrypoint': 'cqlsh', 'arguments': ['10.0.0.48', '9042', '-e', 'quit'], 'privileged': False, 'user': '0'}, 'OpenStdin': False, 'OpenStderr': True, 'OpenStdout': True, 'CanRemove': False, 'ContainerID': 'baa20a654d406b3db5b8d47eaa0834ca701754510375b68125b23982559b0f53', 'DetachKeys': '', 'Pid': 85169} WARNING:contrail-database-nodemgr:CQLSH failed, but Cassandra native transport port is open CRITICAL:root:Exit data: {'ID': 'a5a54e8a27433f0cdc82626a8cf88775b4d4a7e00f8cf5072a7a918c04cba52d', 'Running': False, 'ExitCode': 1, 'ProcessConfig': {'tty': True, 'entrypoint': 'nodetool', 'arguments': ['-p', '7200', 'compactionstats'], 'privileged': False, 'user': '0'}, 'OpenStdin': False, 'OpenStderr': True, 'OpenStdout': True, 'CanRemove': False, 'ContainerID': 'baa20a654d406b3db5b8d47eaa0834ca701754510375b68125b23982559b0f53', 'DetachKeys': '', 'Pid': 85206} ERROR:contrail-database-nodemgr:Failed to get nodetool compactionstats: Result: 1 ERROR:contrail-database-nodemgr:Failed to get core files: [Errno 2] No such file or directory: '/var/crashes' CRITICAL:root:Exit data: {'ID': '8b0a480cd89ac9c8f427846439324767b4ba3484ac672c1e2d0ce1ee2758ac69', 'Running': False, 'ExitCode': 1, 'ProcessConfig': {'tty': True, 'entrypoint': 'cqlsh', 'arguments': ['10.0.0.48', '9042', '-e', 'quit'], 'privileged': False, 'user': '0'}, 'OpenStdin': False, 'OpenStderr': True, 'OpenStdout': True, 'CanRemove': False, 'ContainerID': 'baa20a654d406b3db5b8d47eaa0834ca701754510375b68125b23982559b0f53', 'DetachKeys': '', 'Pid': 87418} WARNING:contrail-database-nodemgr:CQLSH failed, but Cassandra native transport port is open CRITICAL:root:Exit data: {'ID': 'db7d36ef8313dee5b8e89648e1fd65b011da89f414bef573f0444b92006aff5a', 'Running': False, 'ExitCode': 1, 'ProcessConfig': {'tty': True, 'entrypoint': 'nodetool', 'arguments': ['-p', '7200', 'compactionstats'], 'privileged': False, 'user': '0'}, 'OpenStdin': False, 'OpenStderr': True, 'OpenStdout': True, 'CanRemove': False, 'ContainerID': 'baa20a654d406b3db5b8d47eaa0834ca701754510375b68125b23982559b0f53', 'DetachKeys': '', 'Pid': 87436} ERROR:contrail-database-nodemgr:Failed to get nodetool compactionstats: Result: 1 ERROR:contrail-database-nodemgr:Failed to get core files: [Errno 2] No such file or directory: '/var/crashes' [Introspect:8103]10.0.0.48 - - [2025-09-29 01:29:00] "GET /Snh_SandeshUVECacheReq?x=NodeStatus HTTP/1.1" 200 6717 0.001478 CRITICAL:root:Exit data: {'ID': 'b2f82e08fd1c1ca4eb41ca05b9dfc0af6ab3ce4df57a65b9ebd6ac7d45ff974c', 'Running': False, 'ExitCode': 1, 'ProcessConfig': {'tty': True, 'entrypoint': 'cqlsh', 'arguments': ['10.0.0.48', '9042', '-e', 'quit'], 'privileged': False, 'user': '0'}, 'OpenStdin': False, 'OpenStderr': True, 'OpenStdout': True, 'CanRemove': False, 'ContainerID': 'baa20a654d406b3db5b8d47eaa0834ca701754510375b68125b23982559b0f53', 'DetachKeys': '', 'Pid': 92460} WARNING:contrail-database-nodemgr:CQLSH failed, but Cassandra native transport port is open CRITICAL:root:Exit data: {'ID': '25a72834200600a5b5cf7eba8fcd822e0942c1c64e2eb02916c9ffa8c173f341', 'Running': False, 'ExitCode': 1, 'ProcessConfig': {'tty': True, 'entrypoint': 'nodetool', 'arguments': ['-p', '7200', 'compactionstats'], 'privileged': False, 'user': '0'}, 'OpenStdin': False, 'OpenStderr': True, 'OpenStdout': True, 'CanRemove': False, 'ContainerID': 'baa20a654d406b3db5b8d47eaa0834ca701754510375b68125b23982559b0f53', 'DetachKeys': '', 'Pid': 92498} ERROR:contrail-database-nodemgr:Failed to get nodetool compactionstats: Result: 1 ERROR:contrail-database-nodemgr:Failed to get core files: [Errno 2] No such file or directory: '/var/crashes'