GitHub CTO Vlad Fedorov reveals what caused the 7-hour outage, from capacity limits and retry bugs to GitHub’s plans to prevent another major disruption.

GitHub’s growth exposed a serious capacity problem. The GitHub CTO, Vlad Fedorov, has revealed why GitHub suffered one of its longest outages of August, lasting 7 hours and 47 minutes on August 17. The cause was not a faulty code push or configuration change. Instead, GitHub simply hit a level of traffic that parts of its infrastructure could not handle.

The outage began after traffic reached an extreme peak. A critical infrastructure component in GitHub’s Central US data center failed to scale fast enough, creating capacity pressure that first affected authentication and then spread across github.com, Actions, APIs, pull requests, issues, and Copilot. The problem became worse because an Istio sidecar reached its concurrency limit while the autoscaling system was monitoring the wrong component. Load balancers in the Central US eventually became saturated.

A retry bug in Visual Studio Code also amplified traffic to one internal endpoint by around 10 times. Engineers had to limit gateway retries and reject some Copilot token requests with HTTP 403 errors before recovery could continue.

“If you were trying to ship software that day, we let you down,” Fedorov said in GitHub’s August 20 post.

Why did recovery take so long?

Most services returned during the day, with Actions recovering by 1803 UTC. However, the Copilot Token Service took until 2102 UTC to fully recover. Scraping attacks targeting codeload endpoints added another challenge.

At the peak, error rates reached nearly 20% for web and API traffic, while archive and raw repository downloads saw errors approaching 50%.

The incident also highlighted how quickly GitHub is growing. Monthly commits have increased from 1.4 billion in April to 2.9 billion. The company has added more than 3 million CPU cores and 120 petabytes of high-speed storage, while Azure now handles about 58% of GitHub’s platform load, compared with 12% in May.

GitHub is changing how its infrastructure scales

As Business Fortune observes, GitHub plans to introduce consistent retry limits, retry budgets, variable timeouts, stronger monitoring and better isolation between critical systems. It is also developing architecture that can scale read capacity with the number of users, starting with its largest monorepos.

With this being GitHub’s second major August incident, reliability has become an urgent priority. The company’s future will depend on whether it can match its infrastructure growth with the explosive demand from developers, repositories, Actions, and AI-powered tools such as Copilot.

 

FAQs

What caused the GitHub outage?

A critical infrastructure component could not scale with unprecedented traffic.

How long did the outage last?

The August 17 outage lasted 7 hours and 47 minutes.

Did a code change cause the outage?

No. GitHub said the incident was fundamentally a capacity failure.

How much has GitHub’s workload grown?

Monthly commits increased from 1.4 billion in April to 2.9 billion.

What is GitHub doing to prevent another outage?

It is adding capacity, improving retry controls, strengthening monitoring, isolating systems, and redesigning infrastructure to scale more effectively.