TABLE OF CONTENTS

Introduction

In the process of running security scans using the Trivy Premium scanner, users may encounter rate-limiting errors when downloading the Trivy-DB vulnerability database from GitHub Container Registry (ghcr.io). These errors can interrupt scans and cause delays, especially in environments that require frequent or automated scanning. This article explains how Trivy-DB works, describes the common rate-limiting issues, and provides solutions to mitigate these problems by utilizing authentication and other best practices.


What is Trivy-DB?

Trivy-DB is a database of security vulnerabilities that is maintained and updated by the Trivy project. It includes vulnerability data from trusted sources like:


  • The National Vulnerability Database (NVD)
  • Operating system repositories (Debian, Ubuntu, Alpine, etc.)
  • Language-specific vulnerability databases (e.g., Python's PyPI, Java's Maven)


When Trivy performs a scan, it compares the components of your software (such as Docker images or file systems) against the vulnerabilities stored in Trivy-DB. If it finds a match, it alerts you about potential security risks and suggests remediation strategies.


How Does Trivy Use Trivy-DB?

  • Download the DB: When Trivy is run, it first checks whether its vulnerability database is up to date. If an update is needed, it fetches the latest version from the GitHub Container Registry (ghcr.io) where Trivy-DB is hosted.
  • Scan the Target: Once the database is downloaded, Trivy scans the target (such as a Docker image or a file system) for vulnerabilities by comparing the packages and dependencies with the data in Trivy-DB.
  • Report Vulnerabilities: If Trivy finds a vulnerability, it generates a detailed report specifying the affected package, the severity of the vulnerability, and whether a fix is available.


For example, when scanning a Docker image, Trivy inspects the installed libraries and compares them with Trivy-DB. Any vulnerabilities present in the image will be flagged in the report.


Applicability

This article applies to both Aqua Trivy OSS scanner and commercial version via the use of TRIVY_RUN_AS_PLUGIN "aqua" during CI/CD repository/artifacts scanning.


P. S. As of the publication of this article, the current Trivy version was v0.55.


Symptoms

When running Trivy scans, users may encounter the following error messages indicating that GitHub’s rate limits have been exceeded:

  1. TOOMANYREQUESTS Error:
    trivy execution failed - init error: DB error: failed to download vulnerability DB: database download error: 
    OCI repository error: 1 error occurred: * GET https://ghcr.io/v2/aquasecurity/trivy-db/manifests/2: 
    TOOMANYREQUESTS: retry-after: 36.507µs, allowed: 44000/minute
  2. API Rate Limit Exceeded:
    $ trivy image ...
    API rate limit exceeded for xxx.xxx.xxx.xxx.
    Specify GITHUB_TOKEN for authentication https://developer.github.com/v3/#rate-limiting

These errors prevent the vulnerability database from being updated, which can interrupt ongoing scans and potentially cause incomplete or outdated results.


Causes

The rate-limiting errors occur when too many requests are made to GitHub's servers in a short period. The main causes include:

  1. Unauthenticated requests: Unauthenticated requests to GitHub’s API or container registry (ghcr.io) are subject to stricter rate limits. This is often the root cause of the errors.
  2. Frequent DB updates: Trivy checks for updates to Trivy-DB each time a scan is initiated. Multiple scans running in parallel or across multiple machines can lead to exceeding the allowed request limits.
  3. Shared IP address: Multiple users or systems behind a shared IP (e.g., corporate networks or CI/CD pipelines) can quickly hit the limit if they make frequent requests, since the rate limit is often applied per IP address for unauthenticated users.


Solutions

To resolve the rate-limiting issues and ensure smoother operations, follow these step-by-step solutions:


(A) Point Trivy to consume its databases from registry.aquasec.com

Trivy-DB are also available in the main Aqua registry. By pulling the databases from this location we will de-facto bypass the ghr.io registry and the TOOMANYREQUESTS error will be resolved. Upgrade to the  latest scanner if and as needed.


Steps:

  1. Since the Aqua registry is private you need to use your Aqua credentials (the ones used to access the Aqua Success portal with) to access it. Therefore you need to create the relevant secrets to pass it to Trivy as variables:
    TRIVY_USERNAME=$AQUA_USERNAME
TRIVY_PASSWORD=$AQUA_PASSWORD
  2. Add the following arguments to your scan command:
    --db-repository=registry.aquasec.com/trivy-db:2
--checks-bundle-repository=registry.aquasec.com/trivy-checks:1
--java-db-repository=registry.aquasec.com/trivy-java-db:1
    For example your scan command should look like or be similar to:
  3. trivy fs --sast --reachability --scanners misconfig,vuln,secret --db-repository=registry.aquasec.com/trivy-db:2 --checks-bundle-repository=registry.aquasec.com/trivy-checks:1 --java-db-repository=registry.aquasec.com/trivy-java-db:1 .

GitHub Actions example:

      # Step 4: Run Trivy scan
      - name: Run Trivy scan
        uses: docker://aquasec/aqua-scanner
        with:
          args: trivy fs --sast --reachability --scanners misconfig,vuln,secret --db-repository=registry.aquasec.com/trivy-db:2 --checks-bundle-repository=registry.aquasec.com/trivy-checks:1 --java-db-repository=registry.aquasec.com/trivy-java-db:1 /github/workspace
        env:
          AQUA_KEY: ${{ secrets.AQUA_KEY }}
          AQUA_SECRET: ${{ secrets.AQUA_SECRET }}
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
          AQUA_URL: ${{ vars.AQUA_URL }}
          CSPM_URL: ${{ vars.CSPM_URL }}
          TRIVY_RUN_AS_PLUGIN: "aqua"
          TRIVY_USERNAME: ${{ secrets.TRIVY_USERNAME }}
          TRIVY_PASSWORD: ${{ secrets.TRIVY_PASSWORD }}

We also suggest adding a volume to the Trivy cache folder to avoid downloading the database for every scan.

-v $(PWD):"/scanning" -v $HOME/Library/Caches:/tmp/.cache/

GitHub Actions example:

# Step 4: Run Trivy scan
      - name: Run Trivy scan
        run: |
          docker run \
          -e AQUA_KEY="${{ secrets.AQUA_KEY }}" \
          -e AQUA_SECRET="${{ secrets.AQUA_SECRET }}" \
          -e GITHUB_TOKEN="${{ secrets.GITHUB_TOKEN }}" \
          -e AQUA_URL="${{ vars.AQUA_URL }}" \
          -e CSPM_URL="${{ vars.CSPM_URL }}" \
          -e TRIVY_RUN_AS_PLUGIN="aqua" \
          -e TRIVY_USERNAME="${{ secrets.TRIVY_USERNAME }}" \
          -e TRIVY_PASSWORD="${{ secrets.TRIVY_PASSWORD }}" \
          -v ${{ github.workspace }}:/scanning \
          -v ${{ runner.temp }}/trivy_cache:/tmp/.cache/ \
          aquasec/aqua-scanner:latest \
          trivy fs --sast --reachability --scanners misconfig,vuln,secret --db-repository=registry.aquasec.com/trivy-db:2 --checks-bundle-repository=registry.aquasec.com/trivy-checks:1 --java-db-repository=registry.aquasec.com/trivy-java-db:1 /scanning
(B) Authenticate Using a GitHub Personal Access Token (PAT)

By authenticating with GitHub, users can significantly increase their rate limits. This method is especially useful in environments with frequent scans.


Steps:

  1. Create a GitHub Personal Access Token (PAT) by following GitHub’s guide. Ensure the token has the necessary scope, such as read:packages.

  2. Export the token as an environment variable:

    export GITHUB_TOKEN=XXXXXXXXXX
  3. Use the token when running a Trivy scan. For example:
    GITHUB_TOKEN=XXXXXXXXXX trivy image alpine:3.10

This method authenticates your requests and helps prevent the API rate limit exceeded error.

(C) Authenticate for Trivy-DB downloads


For frequent database downloads from ghcr.io, using a GitHub username and token helps bypass the TOOMANYREQUESTS error.


Steps:

  1. Set your GitHub credentials in the environment variables:
    export TRIVY_AUTH_URL="ghcr.io"
    export TRIVY_USERNAME="<your_github_username>"
    export TRIVY_PASSWORD="<your_github_pat>"
  2. Run your scans normally, ensuring the vulnerability database downloads are authenticated.

(D) Mirror Trivy-DB Locally

In larger environments, such as enterprise CI/CD pipelines or where multiple systems require frequent scans, you can host a local mirror of Trivy-DB to avoid making multiple requests to GitHub.


Steps:

  1. Pull the images from ghcr.io or from the registry.aquasec.com registry e.g.
    docker pull ghcr.io/aquasecurity/trivy-db:2
  2. Set up a private registry on your infrastructure to host the mirrored database. Tag and push e.g.
    docker tag ghcr.io/aquasecurity/trivy-db:2 <my-registry-url>/<my-repo>/trivy-db:2
docker push <my-registry-url>/<my-repo>/trivy-db:2
  3. Configure your scan command to pull the database from your local mirror, reducing dependencies on external rate-limited services. e.g.
    TRIVY_USERNAME=<registry-usr> TRIVY_PASSWORD=<registry=pwd> trivy image --db-<my-registry-url>/<my-repo>/trivy-db  --java-db-repository <my-registry-url>/<my-repo>/trivy-java-db <image-to-scan>


Tips and Tricks

  • Use authentication for all scans: Always use a GitHub Personal Access Token to authenticate your scans, especially if you expect to run multiple scans or download the database frequently.

  • Monitor your requests: GitHub provides tools to check the current rate limit status. You can use this information to monitor and manage your request usage.


Conclusion

By understanding the cause of rate-limiting errors and applying these solutions -- such as authenticating with GitHub, using cached versions of Trivy-DB, or scheduling updates -- you can ensure smoother and uninterrupted vulnerability scanning with Trivy. These steps help mitigate the impact of GitHub’s rate limits and allow for more efficient use of Trivy in both Aqua Self-Hosted and Aqua SaaS environments.


Additional Resources

GitHub Rate Limiting Guide

Running Trivy Advanced

Troubleshooting - Trivy (aquasecurity.github.io)

Aqua trivy-db


image