Python Infrastructure Status

Python Infrastructure

PyPI Service Degradation

Monitoring - With the temporary disabling of XMLRPC we are hoping that the mass consumer that is causing us trouble will make contact. Due to the huge swath of IPs we were unable to make a more targeted block without risking more severe disruption, and were not able to receive a response from their abuse contact or direct outreach in an actionable time frame.
Dec 14, 17:46 UTC

Update - Due to the overwhelming surges of inbound XMLRPC search requests (and growing) we will be temporarily disabling the XMLRPC search endpoint until further notice.
Dec 14, 17:30 UTC

Identified - We've identified that the issue is with excess volume to our XLMRPC search endpoint that powers `pip search` among other tools. We are working to try to identify patterns and prohibit abusive clients to retain service health.
Dec 14, 15:09 UTC

Investigating - PyPI's search backends are experiencing an outage causing the backends to timeout and fail, leading to degradation of service for the web app. Uploads and installs are currently unaffected but logged in actions and search via the web app and API access via XMLRPC are currently experiencing partial outages.
Dec 14, 09:41 UTC

python.org Operational

python.org - CDN Operational

python.org - Backends Operational

PyPI Partial Outage

pypi.org - CDN Partial Outage

pypi.org - Backends Partial Outage

files.pythonhosted.org - Files Operational

files.pythonhosted.org - Redirects Operational

files.pythonhosted.org - Redirects Backends Operational

PyPI Hosting Platforms Operational

AWS elasticache-us-east-2 Operational

AWS elb-us-east-2 Operational

AWS ec2-us-east-2 Operational

AWS rds-us-east-2 Operational

Google Cloud Platform Google Cloud Storage Operational

docs.python.org Operational

docs.python.org - Backends Operational

docs.python.org - CDN Operational

bugs.python.org Operational

wiki.python.org Operational

mail.python.org Operational

Message Handling Services Operational

Mailing Lists and Archives - Mailman Operational

Mailing Lists and Archives - Mailman 3 Operational

psfmember.org Operational

us.pycon.org Operational

PyPy Operational

pypy.org Operational

speed.pypy.org Operational

Content Delivery Network Under Maintenance

Fastly US East (JFK) Operational

Fastly Asia/Pacific (HK) Operational

Fastly US East (IAD) Operational

Fastly US East (ATL) Operational

Fastly US East (MIA) Operational

Fastly US Central (DEN) Operational

Fastly US Central (DFW) Operational

Fastly US Central (ORD) Operational

Fastly US West (LAX) Operational

Fastly US West (SEA) Operational

Fastly US West (SJC) Operational

Fastly Europe (FRA) Operational

Fastly Europe (AMS) Operational

Fastly Europe (LCY) Operational

Fastly Europe (LHR) Operational

Fastly Asia/Pacific (SYD) Operational

Fastly Asia/Pacific (SIN) Operational

Fastly Asia/Pacific (NZ) Operational

Fastly Brisbane (BNE) Operational

Fastly Dubai (FJR) Under Maintenance

Fastly Melbourne (MEL) Operational

Fastly Osaka (ITM) Operational

Fastly Perth (PER) Operational

Fastly Singapore (QPG) Operational

Fastly Singapore (XSP) Operational

Fastly Tokyo (HND) Operational

Fastly Tokyo (TYO) Operational

Fastly Wellington (WLG) Operational

Fastly Dublin (DUB) Operational

Fastly Copenhagen (CPH) Operational

Fastly Frankfurt (HHN) Operational

Fastly Helsinki (HEL) Operational

Fastly London (LON) Operational

Fastly Madrid (MAD) Operational

Fastly Manchester (MAN) Operational

Fastly Milan (MXP) Operational

Fastly Oslo (OSL) Operational

Fastly Buenos Aires (EZE) Operational

Fastly Bogota (BOG) Operational

Fastly Curitiba (CWB) Operational

Fastly Rio de Janeiro (GIG) Operational

Fastly Santiago (SCL) Operational

Fastly Sāo Paulo (GRU) Operational

Fastly Johannesburg (JNB) Operational

Fastly Cape Town (CPT) Operational

Fastly Vancouver (YVR) Operational

Fastly Toronto (YYZ) Operational

Fastly St. Louis (STL) Operational

Fastly Palo Alto (PAO) Operational

Fastly Newark (EWR) Operational

Fastly New York (LGA) Operational

Fastly Montreal (YUL) Operational

Fastly Minneapolis (STP) Operational

Fastly Minneapolis (MSP) Operational

Fastly Los Angeles (BUR) Operational

Fastly Kansas City (MCI) Operational

Fastly Jacksonville (JAX) Operational

Fastly Houston (IAH) Operational

Fastly Dallas (DAL) Operational

Fastly Columbus (CMH) Operational

Fastly Chicago (PWK) Operational

Fastly Chicago (MDW) Operational

Fastly Chicago (CHI) Operational

Fastly Boston (BOS) Operational

Fastly Atlanta (PDK) Operational

Fastly Atlanta (FTY) Operational

Fastly Ashburn (WDC) Operational

Fastly Ashburn (DCA) Operational

Fastly Ashburn (BWI) Operational

Fastly New Delhi (DEL) Operational

Fastly Mumbai (BOM) Operational

Fastly Chennai (MAA) Operational

Fastly Vienna (VIE) Operational

Fastly Stockholm (BMA) Operational

Fastly Paris (CDG) Operational

Operational

Degraded Performance

Partial Outage

Major Outage

Maintenance

System Metrics Month Week Day

PyPI CDN Edge Errors

Fetching

PyPI Files CDN Edge Errors

Fetching

PyPI CDN Miss Times

Fetching

PyPI Files CDN Miss Times

Fetching

Past Incidents

Dec 14, 2020

Unresolved incident: PyPI Service Degradation.

Dec 13, 2020

No incidents reported.

Dec 12, 2020

No incidents reported.

Dec 11, 2020

No incidents reported.

Dec 10, 2020

No incidents reported.

Dec 9, 2020

No incidents reported.

Dec 8, 2020

No incidents reported.

Dec 7, 2020

No incidents reported.

Dec 6, 2020

No incidents reported.

Dec 5, 2020

No incidents reported.

Dec 4, 2020

No incidents reported.

Dec 3, 2020

No incidents reported.

Dec 2, 2020

No incidents reported.

Dec 1, 2020

No incidents reported.

Nov 30, 2020

No incidents reported.

Subscribe to Incident