pganalyze Log Insights Limits

Log Insights has a rate limit of 1 GB of log data per day for each server. Limits for each server in your organization are tracked separately. Additional log quota can be added to your plan by contacting pganalyze Support.

Log Insights limits are not present in the Enterprise version.

In addition, certain values of some Postgres log settings are not supported for Log Insights (see below).

How Log Insights rate limiting works

As the collector sends Log Insights data to pganalyze for processing, the service tracks the total number of bytes received for each server. Each server starts with a 1 GB quota that is continuously refilled at the rate of 1 GB per day.

Effectively, each server has a 1 GB allotment of log data over a rolling 1 day period. This allows for sending bursts of log data as long as the average rate is under 1 GB per day.

When incoming log data exceeds the limit for a particular server, the affected log lines are not recorded by pganalyze and will not show up in Log Insights. EXPLAIN plans and query samples present in affected lines are still recorded. The amount of log lines dropped, in bytes, is recorded whenever the rate limit is exceeded. This is visible as a warning when you visit the Log Insights page for an affected server.

All logs processed by the collector count toward this limit.

Settings to adjust to stay under the rate limit

There are a number of Postgres log settings that can be adjusted to help you stay under this threshold:

• Disable log_duration if set
• Set log_statement to 'none' or 'ddl'
• Ensure log_min_messages is not set below the default level of 'warning'
• Raise auto_explain.log_min_duration (if using auto_explain) or log_min_duration_statement
• Use auto_explain.sample_rate (if using auto_explain)
• Use log_min_duration_sample and log_statement_sample_rate instead of log_min_duration_statement
• Disable log_disconnections and log_connections if you do not need these for auditing
• Increase deadlock_timeout or disable log_lock_waits

Unsupported log settings

Certain Postgres log settings tend to cause excessive log volume and are not supported for Log Insights. If an unsupported configuration is found, the collector will not process log data and Log Insights will not be available for that server. A warning will be logged. Other pganalyze features will continue to work normally.

The following settings are not supported:

• log_min_duration_statement set to less than 10ms
• log_duration set to 'on'
• log_statement set to 'all'
• log_error_verbosity set to 'verbose'