Python Infrastructure
All Systems Operational
python.org   Operational
PyPI Operational
pypi.org   ? Operational
files.pythonhosted.org - Files   ? Operational
files.pythonhosted.org - Redirects   ? Operational
docs.python.org   ? Operational
hg.python.org   Operational
bugs.python.org   Operational
wiki.python.org   Operational
mail.python.org Operational
Mailing Lists and Archives   Operational
Message Handling Services   Operational
pypy.org   Operational
speed.pypy.org   Operational
psfmember.org   Operational
us.pycon.org   Operational
Content Delivery Network Operational
Fastly US East (JFK)   Operational
Fastly Asia/Pacific (HK)   Operational
Fastly US East (IAD)   Operational
Fastly US East (ATL)   Operational
Fastly US East (MIA)   Operational
Fastly US Central (DEN)   Operational
Fastly US Central (DFW)   Operational
Fastly US Central (ORD)   Operational
Fastly US West (LAX)   Operational
Fastly US West (SEA)   Operational
Fastly US West (SJC)   Operational
Fastly Europe (FRA)   Operational
Fastly Europe (AMS)   Operational
Fastly Europe (LCY)   Operational
Fastly Europe (LHR)   Operational
Fastly Asia/Pacific (SYD)   Operational
Fastly Asia/Pacific (SIN)   Operational
Fastly Asia/Pacific (TYO)   Operational
Fastly Asia/Pacific (NZ)   Operational
Operational
Degraded Performance
Partial Outage
Major Outage
Maintenance
System Metrics Month Week Day
PyPI Errors ?
Fetching
PyPI CDN Miss Latency ?
Fetching
Past Incidents
May 5, 2018

No incidents reported today.

May 4, 2018
CDN Anycast issues
Resolved - The initial reports have been resolved. We're tracking a few other issues that are not likely related directly.
May 4, 12:18 UTC
Monitoring - We have not heard any new reports of access issues and are continuing to monitor. If you do experience access issues to files.pythonhosted.org, please follow the instructions found here: https://github.com/pypa/warehouse/issues/3885#issuecomment-386438449
May 3, 22:22 UTC
Update - We've been notified by our provider that they have deployed changes that may improve the situation and have requested updates from users who reported ongoing issues.
May 3, 21:44 UTC
Identified - We're still receiving reports of Anycast IP access issues for files.pythonhosted.org.
May 3, 20:59 UTC
Monitoring - We are monitoring for CDN issues, but our reproducible case has stopped failing.
May 3, 19:35 UTC
Investigating - We are investigating reports of issues accessing services hosted by our CDN when accessing via Anycast IPs.

Anycast is used for apex domains of python.org, pythonhosted.org, and pypi.org. Additionally users who query DNS from Google DNS and OpenDNS receive Anycast IPs in response for other services which are hosted by our CDN, as the provider cannot localize the result.

We are working with the CDN provider to determine what is causing this, and the scope of the impact is unclear.
May 3, 15:43 UTC
PyPI: Backends down
Resolved - Our search cluster provider has confirmed their fix and are verifying. We'll be resolving ours and will open a new incident if further disruptions occur.
May 4, 02:16 UTC
Update - Our search cluster has recovered. We have completed work to finalize safeguards against this kind of outage in the future as well.

We'll continue to monitor for a period before resolving this incident.
May 4, 01:57 UTC
Monitoring - Interim fix is deployed. Search queries in the UI and XMLRPC API are not operational.
May 4, 01:35 UTC
Update - We're preparing a short term workaround while our search provider recovers.
May 4, 01:25 UTC
Update - We've confirmed that uploads, simple index, JSON API, and file downloads are not affected.
May 4, 01:17 UTC
Update - We've confirmed the issue with our hosted search provider and are working to resolve.
May 4, 01:14 UTC
Identified - Our search cluster is currently down. Due to timeouts being set too high in our client libraries, the frequency of search requests is tying up all of our backend resources leading to health check timeouts.
May 4, 01:11 UTC
Investigating - We are investigating a down situation for the PyPI backends. Web UI, Uploads, and API access may be impacted.
May 4, 01:08 UTC
May 2, 2018

No incidents reported.

May 1, 2018
Python.org service disruption
Resolved - Python.org is back online. Due to the previous incident related to the PyPI transition, some cron jobs were left disabled which resulted in expired internal certificates.
May 1, 10:37 UTC
Investigating - Python.org is unavailable at this time. We are currently investigating the cause.
May 1, 10:20 UTC
PyPI.org final migration
Resolved - This incident has been resolved.
May 1, 01:59 UTC
Monitoring - We've completed the migration and noted a brief interruption for XMLRPC requests. We're monitoring and will resolve this when we're confident all is complete.

If you experience any issues as a result of this maintenance, let us know at https://github.com/pypa/warehouse/issues
May 1, 00:16 UTC
Identified - We are executing the final migration for the pypi.org rollout. This will bring the pypi.python.org CDN configuration into the pypi.org service.

This migration has a minor risk of interrupting some requests to the legacy domain.
Apr 30, 23:04 UTC
Apr 30, 2018
python.org outage
Resolved - We're solid for now!
Apr 30, 19:24 UTC
Monitoring - We've determined that the code used to display recent blog posts on the python.org homepage was attempting to call out to resolve `legacy.pypi.org`, which was included in a recent blog.python.org post... however that service was shutdown this morning.

The blog post has been pulled to keep it from being rendered and we're working to determine what codepath was causing the timeouts.
Apr 30, 18:36 UTC
Update - We are working on determining the issue more closely. A networked resource loaded on the homepage is causing timeouts.
Apr 30, 17:55 UTC
Identified - The internal network that connects our load balancers to the servers running python.org's code is experiencing an outage. We are working to resolve the issue.
Apr 30, 16:56 UTC
Investigating - Python.org is erroring for all users, we are investigating.
Apr 30, 16:41 UTC
PyPI Legacy Sunsetting!
Completed - The backends for the previous generation PyPI have been taken down. This maintenance is complete.

From everyone on the PyPI team, thank you for working with us through this process.

PyPI is dead, long live PyPI.
Apr 30, 15:10 UTC
Verifying - All work to disable access to PyPI 1.0 has been completed.

All requests to `pypi.python.org` will now see a redirect to the appropriate URL for the new service.

We are going to spin down the backends over the next hour.

Any remaining impact can be reported at https://github.com/pypa/warehouse/issues and we'll work to assist you as best we can.
Apr 30, 14:26 UTC
Update - We are deploying our CDN configuration to redirect all traffic to pypi.python.org to pypi.org! This will impact Artifactory users who are not running latest releases from JFrog. Please see https://jfrog.com/knowledge-base/why-am-i-not-able-to-connect-to-pypi-python-org/ and https://github.com/pypa/warehouse/issues/3275 for mitigation steps and discussion.
Apr 30, 14:19 UTC
In progress - We're commencing the maintenance! The DNS name `legacy.pypi.org` will be disabled, and DNS should propagate over the next 5-10 minutes.
Apr 30, 14:10 UTC
Scheduled - Today we'll be sunsetting the original PyPI service.

## Impacted users include:
- Those using `legacy.pypi.org` directly
- Artifactory users not running the latest releases from JFrog

### `legacy.pypi.org` users

You should use `pypi.org`! If there is a feature that the new codebase does not support, you should file an issue at https://github.com/pypa/warehouse/issues as soon as possible!

### Artifactory Users who have not upgraded

Please see the [guidance from JFrog](https://jfrog.com/knowledge-base/why-am-i-not-able-to-connect-to-pypi-python-org/) and [full discussion of the issue](https://github.com/pypa/warehouse/issues/3275).

### Any other impacted users

Let us know! File an issue with our [issue tracker](https://github.com/pypa/warehouse/issues).
Apr 30, 13:42 UTC
Apr 29, 2018

No incidents reported.

Apr 28, 2018

No incidents reported.

Apr 27, 2018

No incidents reported.

Apr 26, 2018

No incidents reported.

Apr 25, 2018
PyPI.org Elevated Error Rates
Resolved - PyPI.org: From 13:14:00 UTC to 13:35:00 UTC we experienced an overload in the web backend that serves traffic to PyPI's web user interface. We have moved this web traffic over to another backend and service is restored.
Apr 25, 13:55 UTC
Investigating - PyPI.org is currently experiencing elevated error rates.
Apr 25, 13:28 UTC
Apr 24, 2018

No incidents reported.

Apr 23, 2018

No incidents reported.

Apr 22, 2018

No incidents reported.

Apr 21, 2018

No incidents reported.

← Incident History Powered by Statuspage