shipping production AI · since 2026 NAICS 541330 / 541511 / 541512 / 541519  ·  CMMC-aware
Refinery Report / AI Security / post · romise
AI SecuritySupply Chain SecurityCredential TheftIncident Response

LiteLLM 1.107.0 Compromise: Pin, Purge and Rotate

LiteLLM release 1.107.0 shipped a credential stealer on PyPI and in Docker tags for under three hours. Find every host that ran it, pin to 1.106.2 and rotate the keys it could read.

E
By Ernest Provo
Founder & CEO · how we research & review
October 23, 2026
9 min · 1,987 words

By Ernest Provo · published October 23, 2026 · reviewed October 23, 2026

LiteLLM release 1.107.0 contained a credential stealer and a remote-code loader. The malicious files were published to PyPI and to LiteLLM’s Docker tags on October 21, 2026. The official source tag was clean, but the published artifacts were built from a lookalike repository. Pin to 1.106.2 or any version released before the incident, rebuild every image and virtual environment that installed the bad release, and rotate every key the host could read, starting with the provider keys stored in the proxy.

What happened

LiteLLM’s incident report gives this sequence for October 21, 2026, in UTC:

Time Event
10:08 A credential tied to a privileged developer account overrode branch protections and applied a modified release workflow. The report says the attacker then force-pushed a change after a merge attempt was blocked.
11:11 A credential tied to a second developer account satisfied the production approval gate.
11:39 to 13:23 Seven malicious wheels were live on PyPI.
11:46 to 14:20 Docker Hub tags v1.107.0, latest, main-latest and main-stable pointed to the compromised image. On GitHub Container Registry the same tags were affected until 13:51.
20:30 Last status update in the report: incident contained, investigation in progress.

The release was built from Berri-AI/litellm, a lookalike of the official BerriAI/litellm repository. The sequence matters for your own pipeline. A privileged credential bypassed review controls, and a second credential satisfied the approval gate. Both were developer accounts, so the attacker held credentials that looked legitimate at every step. The report says the attacker held three sets of developer GitHub CLI credentials, and the timeline shows two used at the gates, so review all three accounts.

What the payload does

The malicious build added three files to the Python site-packages directory: cook.pth, cook.py and sitecustomize.py. Python executes .pth files and sitecustomize.py each time an interpreter starts in that environment. The payload therefore runs whenever Python starts in that environment, including when pip runs, even if nothing imports LiteLLM.

LiteLLM’s report and the analyses from Hot Open Lab, Cycode and Corgea describe the following behavior:

Removing the package does not undo what already ran. Treat any host that executed 1.107.0 as compromised until you have checked it.

Who is exposed

You are exposed if any of the following happened during the windows above:

LiteLLM states that all versions before 1.107.0 are safe. Its public statement also says enterprise deployments that pull only from its private registry are unaffected. Confirm the source and digest of your own images before you rely on that.

Triage without starting Python

Do not run pip show litellm, python -c or any other Python command on a host you suspect. Starting an interpreter re-runs the payload. Check with tools that do not execute code.

  1. Search for the dropped files. On Linux, run find / \( -name 'cook.pth' -o -name 'cook.py' -o -name 'sitecustomize.py' -o -name 'cook*.pyc' -o -name 'sitecustomize*.pyc' \) 2>/dev/null. Search for the compiled .pyc files too, because a .pyc file can run without its source. Then inspect sitecustomize.py in each site-packages directory. Some environments ship one legitimately, so compare it with a known-good copy from before October 21.
  2. Check for the lock file and daemon. Look for /tmp/remote_code_loader.lock and for a detached Python process holding it.
  3. Check image digests, not tags. Compare the digests of running and cached images against a pre-incident digest you recorded, or against a freshly rebuilt image. A tag such as latest tells you nothing once it has been repointed.
  4. Search build records. Search requirements files, lockfiles and CI logs for litellm==1.107.0, and for unpinned litellm installs that ran on October 21.
  5. Review cloud audit logs. LiteLLM recommends checking CloudTrail for GetCallerIdentity calls from the instance role around install time. Separately, review for use of those credentials from outside your infrastructure. Equivalent logs exist on other clouds.

Keep the output of each check. It becomes the evidence file for your incident record.

Remediation order

  1. Stop before installing anything. Do not run pip install -U inside an infected environment. The install itself starts Python and re-runs the payload.
  2. Remove the payload. Delete cook.pth, cook.py, sitecustomize.py and any matching __pycache__ entries from every affected site-packages directory. Stop the detached process that holds the lock file.
  3. Rebuild instead of patching in place. Rebuild images and virtual environments from scratch, pinned to litellm==1.106.2 or another explicit pre-incident version. For containers, rebuild from a pre-incident base and pin the rebuilt image by digest.
  4. Rotate what the host could reach. LiteLLM’s list covers the instance or task IAM role, every provider key in the proxy database, the master and virtual keys, the DATABASE_URL password, and any secrets in the environment of running processes. Beyond that list, rotate any other secret the host held, such as SSH keys, CI and registry tokens, and Kubernetes service account tokens. That is our recommendation, not part of LiteLLM’s list.
  5. Assume exfiltration if the host had outbound access. LiteLLM says a marker file confirms exfiltration, but its absence proves nothing. As of its October 21 update, the report does not name the marker file, so do not rely on a clean marker check. Rotation is the control that matters.

If a live service still depends on a critical key, rotate that key first so a working credential is not left exposed for the length of the rebuild. Otherwise, follow the order above.

Controls that limit the next incident

The attack passed through a release workflow, a credential and a floating tag. Each control below closes one of those paths. They apply to any AI gateway you run, not only LiteLLM.

Control Why it matters here
Pin dependencies with hashes An unpinned install picks up a new release the moment it is published.
Pin images by digest The latest tag moved to the compromised image during the window.
Protect release workflows The report says a modified workflow was applied by overriding branch protections.
Require reviewers, not just credentials, for release approval The approval gate was satisfied by a second developer credential. An attacker holding two credentials can clear a gate that counts credentials, not people.
Keep tokens out of logs LiteLLM’s report believes developer GitHub CLI tokens reached internal deployment logs. Treat that as a hypothesis until it is confirmed.
Restrict outbound traffic from proxy hosts The stealer sends data out. A default-deny egress rule limits what it can reach.
Reference provider keys from a secrets manager If your setup allows it, the proxy database should hold references, not raw keys. One database dump then yields less.

What is still unknown

A clean result from any single check is not proof of safety while the investigation is open. Recheck when LiteLLM publishes its indicators.

Why a vulnerability scanner will not flag this

A vulnerability scanner matches installed packages against known CVEs. Hot Open Lab’s analysis states that there is no CVE for this incident yet: “It is a supply-chain compromise, not a code bug.” A scanner can help you find where LiteLLM is installed, but detection here depends on file, digest and log checks, and on the rotation list above.

Evidence to keep

Keep these records for the incident file. They show what you checked and when.

Next step · check your AI gateway

Find every host that could have pulled LiteLLM 1.107.0.

Bring your list of AI gateways, CI runners and container registries. We will tell you which need rebuilds, which need rotation and what evidence to keep.

Questions buyers ask

Did we run this if we never imported LiteLLM? Possibly, yes. The .pth file runs on every Python start in an environment where the package was installed, so an import is not required. Check the environment, not your code.

We pull a floating Docker tag. Are we exposed? You are exposed if a pull or restart happened from Docker Hub between 11:46 and 14:20 UTC on October 21, or from GitHub Container Registry between 11:46 and 13:51 UTC. Restarting a container from an image that already ran the payload does not clean it. Rebuild from a digest pinned to a pre-incident version.

Is the private registry safe? LiteLLM says enterprise deployments that pull only from its private registry are unaffected. Confirm your own source and digest before you rely on that statement.

Does a vulnerability scan catch this? No. There is no CVE for this incident yet. Use the file, digest and log checks above.

Do we have to notify anyone? Counsel decides. The answer depends on whether the host could reach personal or regulated data and whether that access is confirmed. Some notification clocks start at discovery, so raise it early.

For the wider program, see the AI security assessment. To see how this differs from a scan, read penetration test vs vulnerability scan. For another supply chain risk in agent tooling, read MCP supply chain security.

Key facts

Read next · AI Security & Governance

P
Founder · Principal Engineer
Data & AI engineer · 10+ yrs hands-on

Writes most of the long-form here. Lives in the codebase. Active on GitHub and LinkedIn.

§ Next step

Not sure which of these is you?

Tell us what's broken in a paragraph and a principal reads it directly, or walk the ladder from a low-commitment first engagement up to retained work.

One long-form a week.

Subscribe to the Refinery Report. Practitioner deep-dives on AI engineering, security, and the realities of running production systems. Unsubscribe in one click.

~12 issues / quarter