Summary


This is a troubleshooting guide that will assist you with resolving database constraint related error messages that occur during a Aqua 5.0 deployment and prevents the Aqua Database from accepting any traffic from the Aqua Console or Gateway.


Environment


Symptoms

 

Why is the Aqua Console constantly in a "CrashLoopBackOff" Status?

  • In your Kubernetes cluster where you are deploying the Aqua and you are using the database as a pod for testing or POC purposes, you may notice that your Aqua Console/Web pod is in a "CrashLoopBackOff" and is periodically trying to restart - as demonstrated below in the screenshot;


Why does the Aqua Database have `constraint` errors messages in its logs?

  • You may also notice that in your Aqua Database pod, there are constraint related error messages which look like the following log snippet below;
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.

The database cluster will be initialized with locale "en_US.utf8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".

Data page checksums are disabled.

fixing permissions on existing directory /var/lib/postgresql/data ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok


Success. You can now start the database server using:

pg_ctl -D /var/lib/postgresql/data -l logfile start

initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
waiting for server to start....2020-08-19 14:32:06.731 UTC [46] LOG: starting PostgreSQL 12.4 (Debian 12.4-1.pgdg100+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 8.3.0-6) 8.3.0, 64-bit
2020-08-19 14:32:06.772 UTC [46] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2020-08-19 14:32:06.819 UTC [47] LOG: database system was shut down at 2020-08-19 14:32:05 UTC
2020-08-19 14:32:06.831 UTC [46] LOG: database system is ready to accept connections
done
server started

/usr/local/bin/docker-entrypoint.sh: ignoring /docker-entrypoint-initdb.d/*

2020-08-19 14:32:06.911 UTC [46] LOG: received fast shutdown request
waiting for server to shut down....2020-08-19 14:32:06.920 UTC [46] LOG: aborting any active transactions
2020-08-19 14:32:06.922 UTC [46] LOG: background worker "logical replication launcher" (PID 53) exited with exit code 1
2020-08-19 14:32:06.923 UTC [48] LOG: shutting down
2020-08-19 14:32:06.981 UTC [46] LOG: database system is shut down
done
server stopped

PostgreSQL init process complete; ready for start up.

2020-08-19 14:32:07.034 UTC [1] LOG: starting PostgreSQL 12.4 (Debian 12.4-1.pgdg100+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 8.3.0-6) 8.3.0, 64-bit
2020-08-19 14:32:07.034 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
2020-08-19 14:32:07.034 UTC [1] LOG: listening on IPv6 address "::", port 5432
2020-08-19 14:32:07.066 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2020-08-19 14:32:07.107 UTC [55] LOG: database system was shut down at 2020-08-19 14:32:06 UTC
2020-08-19 14:32:07.119 UTC [1] LOG: database system is ready to accept connections
2020-08-19 14:40:08.278 UTC [107] ERROR: constraint "profiler_sessions_repository_fkey" of relation "profiler_sessions" does not exist
2020-08-19 14:40:08.278 UTC [107] STATEMENT:
ALTER TABLE profiler_sessions
DROP CONSTRAINT profiler_sessions_repository_fkey

2020-08-19 14:40:11.335 UTC [111] ERROR: constraint "profiler_sessions_repository_fkey" of relation "profiler_sessions" does not exist
2020-08-19 14:40:11.335 UTC [111] STATEMENT:
ALTER TABLE profiler_sessions
DROP CONSTRAINT profiler_sessions_repository_fkey

2020-08-19 14:40:29.112 UTC [114] ERROR: constraint "profiler_sessions_repository_fkey" of relation "profiler_sessions" does not exist
2020-08-19 14:40:29.112 UTC [114] STATEMENT:
ALTER TABLE profiler_sessions
DROP CONSTRAINT profiler_sessions_repository_fkey

2020-08-19 14:40:57.017 UTC [118] ERROR: constraint "profiler_sessions_repository_fkey" of relation "profiler_sessions" does not exist
2020-08-19 14:40:57.017 UTC [118] STATEMENT:
ALTER TABLE profiler_sessions
DROP CONSTRAINT profiler_sessions_repository_fkey

Cause

The cause of this issue is likely due to the version of PostgreSQL (v12.4) used as the base image for the Aqua database in 5.0 - which is incompatible on this specific major release of Aqua


Solution

Use the officially supported versions of PostgreSQL for Aqua 5.0

  • Check the status of the Aqua database and console pod to see if the console is in a `Running` status;

  • Lastly, check to see if you can successfully access the Aqua UI;



Related Information