Private PyPI with Docker Compose on RHEL 8.10

Posted on by Thiago Crepaldi

Last Updated on April 23, 2026 by Thiago Crepaldi

For developers managing internal libraries or AI infrastructure, a private PyPI server is essential for hosting nightly builds without polluting public registries. devpi is the gold standard for this, specifically for its “volatile” index feature which allows for rapid iteration.

In this guide, we’ll deploy it on RHEL 8.10 using Docker Compose, rooting everything in /opt/local for clean service isolation.

Prerequisites

  • RHEL 8.10 with rootful Docker installed.
  • Docker Compose V2.
  • Sudo access for filesystem and firewall management.

Prepare the Host Filesystem

We will store both our configuration (docker-compose.yml) and the actual package data under /opt/local/devpi.

# Create the project structure
sudo mkdir -p /opt/local/devpi/data
sudo chmod 700 /opt/local/devpi/data
cd /opt/local/devpi

Configure the RHEL Firewall

By default, RHEL will block external access. Open 3141 persistently:

sudo firewall-cmd --permanent --add-port=3141/tcp
sudo firewall-cmd --reload

Create the Docker Compose Configuration

Create a docker-compose.yml file in /opt/local/devpi.

Critical RHEL Note: We use the :Z flag on the volume mount. This tells Docker to automatically relabel the host directory for SELinux compliance, preventing “Permission Denied” errors.

services:
  devpi:
    image: jonasal/devpi-server:latest
    container_name: devpi-server
    restart: always
    ports:
      - "3141:3141"
    volumes:
      - ./data:/data:Z
    environment:
      - DEVPI_PASSWORD=SetYourAdminPassword

Deploy the Service

sudo docker compose up -d

Configuring the “Nightly” Index

Standard PyPI indexes don’t allow version overwrites. For nightlies, we want a volatile index.

From your workstation (with devpi-client installed):

# 1. Connect and login
devpi use http://<your-server-ip>:3141
devpi login root --password=SetYourAdminPassword

# 2. Create a CI/CD user
devpi user -c build_bot password=bot-secret-pass

# 3. Create the volatile index
devpi index -c build_bot/nightly volatile=True

Enabling/Disabling PyPI.org Caching

One of devpi’s best features is its ability to act as a caching proxy for the official PyPI.

To Enable Caching (Inherit from PyPI):

If you want your index to find and cache public packages when they aren’t found locally:

devpi index build_bot/nightly bases=root/pypi

To Disable Caching (Private Only):

If you want to isolate your index so it only contains your internal packages (improving security by preventing dependency confusion attacks):

devpi index build_bot/nightly bases=""

Create a Dummy Package for Testing

Create a small internal utility library to verify the setup.

pyproject.toml:

[build-system]
requires = ["setuptools>=68.0", "wheel"]
build-backend = "setuptools.build_meta"

[project]
name = "geek-utils"
version = "0.1.0.dev1"
description = "Internal test library for GeekIsTheWay"
requires-python = ">=3.9"
dependencies = ["requests>=2.28"]

src/geek_utils/core.py:

import secrets

def get_geek_token():
    return f"GEEK-{secrets.token_hex(16)}"

Build and Publish Dummy Package for testing

Build the Package

pip install build
python -m build

Uploading via Twine

Option A: Explicit Command

twine upload --repository-url http://<your-server-ip>:3141/build_bot/nightly/ \
  -u build_bot -p bot-secret-pass dist/*

Option B: Using .pypirc (Recommended) Add this to your ~/.pypirc:

[nightly]
repository = http://<your-server-ip>:3141/build_bot/nightly/
username = build_bot
password = bot-secret-pass

Then run: twine upload -r nightly dist/*

Client-Side Configuration (pip & requirements.txt)

Set Global Configuration

pip config set global.index-url http://<your-server-ip>:3141/build_bot/nightly/+simple/
pip config set global.trusted-host <your-server-ip>

Using Requirements Files

requirements.txt (Mixed Mode)

--extra-index-url http://<your-server-ip>:3141/build_bot/nightly/+simple/
--trusted-host <your-server-ip>

geek-utils==0.1.0.dev1
requests>=2.28.0

Install Command Examples

Mixed Mode (devpi + PyPI.org): pip install --extra-index-url http://<your-server-ip>:3141/build_bot/nightly/+simple/ --trusted-host <your-server-ip> geek-utils

Strict Mode (Only use devpi): pip install --index-url http://<your-server-ip>:3141/build_bot/nightly/+simple/ --trusted-host <your-server-ip> geek-utils

Deleting Packages from the Registry

If a nightly build is corrupted or no longer needed, use the following commands from your workstation.

Delete a Specific Version

This is the most common task for cleaning up specific failed builds:

# Connect and login if not already
devpi use http://<your-server-ip>:3141/build_bot/nightly
devpi login build_bot --password bot-secret-pass

# Delete version 0.1.0.dev1 of geek-utils
devpi remove geek-utils==0.1.0.dev1

Delete an Entire Project

To remove every version of a project from the index:

devpi remove geek-utils

Bulk Cleanup (Pro-Tip)

While devpi doesn’t have a built-in “TTL” (Time To Live) for packages, you can use a simple bash loop with the client:

# Example: List versions and remove them (logic can be expanded for date filtering)
devpi list geek-utils

Promoting Packages Between Indexes

A common “Geek” workflow is to upload a package to a nightly index for testing, and once verified, promote it to a stable index. In devpi, this is done using the push command.

Understanding Source vs. Target

The devpi-client works based on your current session context.

  • Source Index: This is the index you are currently “using” via the devpi use command.
  • Target Index: This is the index path you specify at the end of the push command.

Step 1: Create the Target Index

If you haven’t already, create the stable (target) index. For stable releases, we typically set volatile=False to prevent accidental overwrites.

# Create a stable index
devpi index -c build_bot/stable volatile=False

Step 2: Set the Source Context

Tell the client which index you want to pull from.

# Switch your current context to the nightly index (Source)
devpi use http://<your-server-ip>:3141/build_bot/nightly
devpi login build_bot --password bot-secret-pass

Step 3: Execute the Promotion (Push)

Now that the client is “pointing” at nightly, you can push a specific package to the stable index.

# Syntax: devpi push <package_name>==<version> <target_index_path>
devpi push geek-utils==0.1.0.dev1 build_bot/stable

What happened here? The client took the wheel for geek-utils==0.1.0.dev1 from your current context (build_bot/nightly) and copied it into build_bot/stable. The binary hash remains identical, ensuring that what you tested is exactly what you promoted.

Uploading and Deleting Packages

Uploading Pre-Built Wheels

If your CI/CD system already built a wheel and you just need to upload the file directly without using twine:

# Ensure you are using the correct index
devpi use http://<your-server-ip>:3141/build_bot/nightly

# Upload the file directly
devpi upload dist/geek_utils-0.1.0.dev1-py3-none-any.whl

Deleting a Specific Index

If an experimental index is no longer needed, you can delete it entirely. This will remove all packages stored within that specific index.

# You must be logged in as the owner or root
devpi index --delete build_bot/experimental-feature

Troubleshooting & Maintenance

  • Docker Pull Limits: If you hit rate limits, run sudo docker login on the RHEL host.
  • SELinux: Verify labels with ls -Zd /opt/local/devpi/data.
  • Backup: sudo tar -czvf devpi-backup.tar.gz /opt/local/devpi.
  • Preventing Dependency Confusion
    • If a package exists in both your private index and root/pypi, devpi will prefer the local version if the versions are identical. However, pip will always try to download the highest version number it can find across all available indexes.
    • To fix this for your nightly builds:
      • Always use a higher version/suffix: Tag your nightlies with a suffix that PyPI.org wouldn’t have (e.g., 1.2.0.dev20260416).
      • Explicit Bases: If you are worried about a specific project, you can create an index with no bases specifically for that project, ensuring 0% chance of leaking to or from the public internet.
  • Check Disk Usage: Run sudo du -sh /opt/local/devpi/data/server/* on the RHEL host to see which users or indexes are the heaviest.
  • The “Inheritance” Strategy: To save space and maintain order, you can make a prod index that simply inherits from stable:Bashdevpi index -c build_bot/prod bases=build_bot/stable This allows prod to see all stable packages without physically duplicating the files.
  • Error: “403 Forbidden”: You are likely trying to push to an index owned by another user. Ensure you are logged in as root or the owner of the target index.
  • Error: “409 Conflict”: You are trying to push to a volatile=False index (stable) where that version already exists. You must delete the version in the target index first if you truly intended to replace it.

Leave a Reply