Check out our Resources section: Read through our eBooks, compare us to other solutions, and learn how customers are using pganalyze. See all resources.

Enterprise Server: Object Storage Setup (Amazon S3)

1. Pre-requisites

pganalyze Enterprise Server currently requires the following when storing statistics snapshots and logs in Amazon S3:

  • An Amazon S3 bucket to be used for temporary statistics data
  • An Amazon S3 bucket to be used for log data (this is where encrypted log data will be stored)
  • An Amazon KMS key to be used for log text encryption (you can find KMS under "Encryption Keys" in the IAM console)
  • An Amazon IAM user that can access the S3 bucket and KMS key (we need the keys in the next steps)

The setup for these should be straightforward - we recommend choosing a region thats close to your database servers, but latency shouldn't be too much of an issue in general.

If you are not using Amazon Web Services, the local storage setup might be a better fit.

2. Setup IAM user

The pganalyze Docker container needs to have authenticated access to the S3 bucket and KMS key. You can create a regular user, save the access keys, and then assign the following policy:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket"
            ],
            "Resource": [
                "arn:aws:s3:::mydb-logs"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:PutObject",
                "s3:GetObject",
                "s3:DeleteObject"
            ],
            "Resource": [
                "arn:aws:s3:::mydb-logs/*"
            ]
        },
        {
            "Sid": "Stmt1440353940000",
            "Effect": "Allow",
            "Action": [
                "s3:*"
            ],
            "Resource": [
                "arn:aws:s3:::mydb-snapshots/*",
                "arn:aws:s3:::mydb-snapshots"
            ]
        }
    ]
}

You will also need to assign this IAM user as a "Key User" to the KMS key.

3. S3 bucket retention policy and CORS settings

Please ensure that S3 buckets don't have public access enabled. All S3 objects created by pganalyze are created as private only.

To avoid keeping data forever, we recommend configuring the S3 buckets to expire objects after 35 days using lifecycle rules.

For the logs S3 bucket you also need to configure this CORS policy (under the Permissions tab) for the Log Insights functionality:

<?xml version="1.0" encoding="UTF-8"?>
<CORSConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
<CORSRule>
    <AllowedOrigin>*</AllowedOrigin>
    <AllowedMethod>GET</AllowedMethod>
    <ExposeHeader>x-amz-meta-x-amz-iv</ExposeHeader>
    <ExposeHeader>x-amz-meta-x-amz-unencrypted-content-length</ExposeHeader>
    <ExposeHeader>x-amz-meta-x-amz-wrap-alg</ExposeHeader>
    <ExposeHeader>x-amz-meta-x-amz-cek-alg</ExposeHeader>
    <ExposeHeader>x-amz-meta-x-amz-key-v2</ExposeHeader>
    <ExposeHeader>x-amz-meta-x-amz-matdesc</ExposeHeader>
    <AllowedHeader>Authorization</AllowedHeader>
    <AllowedHeader>Range</AllowedHeader>
</CORSRule>
</CORSConfiguration>

4. Configuration on pganalyze Docker container

You will need to add the following configuration to the Docker container:

AWS_REGION=us-east-1
AWS_S3_SNAPSHOTS_BUCKET=mydb-snapshots
AWS_S3_LOGS_BUCKET=mydb-logs
AWS_KMS_LOGS_CMK=...

By default pganalyze will attempt IAM authentication using the AWS metadata service / instance roles. If you want to use static keys you can set the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY variables.

The AWS_KMS_LOGS_CMK is the "ARN" thats listed on the encryption key.

After a pganalyze container restart you can continue with the general steps.


Couldn't find what you were looking for or want to talk about something specific?
Start a conversation with us →