What's the point here? That everyone should use the n-nines notation? Sure. However, companies have no interest in doing anything that makes them look worse.

Also, is anyone else getting the bitter taste of AI writing from this page?

Services can have a 50% uptime (or a 50% downtime if you prefer) as long as it's the time when I need it to be up (or don't need it.)

Which is to say that the significance of downtime depends on the user. Talking about nines only makes sense internally when you are evaluating your infrastructure and operations. It doesn't tell you squat about impact to your customer.

Can we post uptime stats instead? I just axed two of my servers from colocation a couple of days ago. Sad to see them go, six years of FreeBSD.

    root@vixen:/fountain/crystals #                                                         
    *** FINAL System shutdown message from dblrabbit@  ***                                     
    System going down IMMEDIATELY                                                  
    System shutdown time has arrived
    root@vixen:/fountain/crystals # uptime
     3:05PM  up 1931 days, 18:13, 0 users, load averages: 1.01, 1.03, 1.41

    root@cookie:/srv/users/dblrabbit # uptime
     3:07PM  up 1931 days, 16:59, 1 user, load averages: 1.76, 1.17, 1.06
    root@cookie:/srv/users/dblrabbit # poweroff
    Shutdown NOW!
    poweroff: [pid 47177]

No since kernel live patching is not universal it promotes bad security practices to maximize the uptime of a single server.

We say something like:

GitHub Actions: 12 hours affected in the last 30 days (98.31% uptime).

This is trying to shine the most favorable possible light onto a deteriorating situation. It doesn't take away from the fact that most businesses have measurable missed revenue in downtime. Customers that shop somewhere else, ads that were never severed, leads that grew a little colder. 12 hours of downed GitHub results in millions of dollars of lost developer productivity that was externalized by Microsoft to other companies.

We shouldn't be trying to spin downtime as "just a few hours a month." Those hours cost real dollars.

I don’t see why you think “12 hours affected in the last 30 days (98.31% uptime)”

Is trying to spin anything. It’s making easier to see the impact over the last 30 days. I agree with the article

One thing I would like to see is how many of those hours are during normal business hours in my country.

Not all hours are created equal when it comes to downtime and my intuition is that most of these 12 landed within my working hours.

In terms of impact that then might mean they were down for 7.5% of the time I needed them, or had business hours uptime of 92.5% which is… both not very good and very disruptive.

On the other hand, downtime at 4AM would be much less impactful even if it happened every day and added up to more overall downtime.

they were down for 7.5% of the time I needed them, or had business hours uptime of 92.5%

You can obviously compute this for a particular customer, but being a global service, it's pretty much guaranteed that someone somewhere experienced the worse of those numbers

You can compute them for the average. In other words, the total customer impact is the number of business-hours of downtime across all customers divided by the total number of business hours of all customers.

This is important because it's quite possible that the downtime is biased toward the times they have the most active users.

These metrics tend to be bullshit in contracts.

For example we had a 6 9 (99.9999%) requirement from a customer for any given 3-6 month period. If we violated that, we owed them their money back (baring the outage wasn’t caused by us - I.e our cloud provider shit the bed).

That’s something like 7.5 seconds. For a contract over $1.5M. Am I the only one who thinks that’s outrageous expectations?

EDIT: The web app was for generating SBOMs of static assets.

I worked in realtime trading. No. Not at all outrageous. Quite reasonable actually. If that’s what we agreed and I need you to be reliable I will charge you back for being unreliable. I’m happy to pay top and extra dollar for the SLA but that means it needs to be acted on.

It's a contract, you're free to negotiate?

If you agree to those terms knowing it's unrealistic, you're agreeing to give away your service for free.

Are you sure it wasn't prorated?

If it wasn't prorated anyone who approved the contract needs training and/or firing. If it is prorated, that is generally not a problem. Small outages aren't even worth the effort of trying to get the money back, and if you have a large enough one to make it worthwhile it is likely the prorated refund is still going to be laughably small.

Until there is an industry wide definition of outage, degraded performance, etc then it's all moot

Separately from how you present the number, the very concept of "uptime" as a single number is a bit muddy in the context of a distributed system, where different components can be differently available for different users.

Also, 0.1% downtime in the form of a 45-minute outage per month is very different from 0.1% of requests failing in brief bursts. You often see downtime reported as "increased error rates" which is so vague as to be meaningless.

Google's "windowed user-uptime" attempts to deal with this a bit better, by exposing different views of the data instead of trying to condense uptime into a single number: https://www.usenix.org/system/files/nsdi20-paper-hauer.pdf

Uptime % has the benefit that it's easy to understand.

Wider audience started to use status pages because the service unreliability became so much more noticeable than before and not the other way around. I never had to use a status page for bear blog or protonmail because i never had and issue with it or just i never noticed.

I am now _required_ to consult status page of github, circleci or MS services etc because i need to know why a build is not passing, why i cannot open a repo, why is my work stalling.

Percentages matter, it is just so much more obvious why they matter when it comes down to important pieces of the internet like github. And i highly doubt the number of 12 hours in the last month. MS has been downplaying the issues they have with GH performance for a while now and i don't think it is time to start to believe them yet. Maintaining these pieces of infrastructure is responsibility and a burden.

Overall i would be careful with "nonlinear significance of numbers near 100%" we are talking gh being well into the 90's this year and one number that infra people are also often being reminded about is that "1% is 3.5 days".

Things are tough for gh people and i feel for them but they are not a startup or a underdog of some sort to receive sympathy in that case.

I usually tell people you don't need as much reliability as you think.

Three nines reliability is great for most purposes. 8 hours downtime a year.

If your system produces money at a constant rate, it captures 99.9% of the available money. Even two nines or one nine might be pretty good on that basis, when the alternative is spending 2x or 10x as much - let's build another unreliable system with that money that captures some other independent market opportunity.

Poor reliability is a problem where you need to chain many systems together, or where the cost of a single failure is very large compared to a success. Or - as happens commonly because of load - if your periods of unreliability are correlated with periods of maximum opportunity, like an e-commerce site failing on Black Friday or a trading system failing when the market is most busy. But if you don't have one of those cases, evaluate whether investing in reliability is actually worth it to you.

GitHub is an example where two nines of reliability ought to be OK. The argument against it is that it's bad marketing to have an unreliable service, especially one aimed at software engineers. And if GitHub is largely a marketing play by Microsoft anyway (do they really make back its cost in enterprise subscriptions?) then marketing considerations need to drive its reliability.

One thing that is I feel missed about uptime percentage when compared to on premise uptime is when the downtime occurs. Its far more impactful if its in the middle of the working day or during the busy period of shopping. A store that goes offline in the middle of black friday or in the run up to Christmas is harmed a lot more than some down time on a Sunday night/Monday morning at 3am.

One thing I have noted over time is a lot of these AWS, Azure et el downtimes is they occur in the middle of everyones day, millions of people are impacted by them. Same with github its getting in the way of work. Whereas when we hosted services on our own equipment the downtime was usually out of main hours. The percentages are in many ways the wrong measure of downtime because hours aren't equal in impact to businesses.

These service status pages have so many other problems, I can't really get excited about this article's point. As a user of a service, I don't really care that an additional 0.09% of uptime is any more or less difficult to achieve than an additional 0.9%, even if you describe it in terms of fractions of time. I only care about two things: what the service status is right now and your service reliability's impact to me over the long term (get out of here with your 30-day crap).

In the electric utility world we have a few IEEE standardized metrics (with appropriately IEEE'd acronyms) for tracking service reliability that I like much better and always wish for when I'm looking at a status page. Pie in the sky stuff for sure, nobody wants to do this analysis and publish the results without a regulator telling them have to, but c'est la vie.

SAIDI - System Average Interruption Duration Index. How many minutes an average customer experienced service interruption in a year. This is the big one I'd want to see on your service status page IMHO. For the power grid, we consider any outage longer than five minutes to be an interruption ("non-momentary outage").

SAIFI - System Average Interruption Frequency Index. How many total periods of interruption occurred for the average customer in a year.

CAIDI - Customer Average Interruption Duration Index. How long it takes service to be restored for the average customer when there is an interruption.

For the US, here is what these numbers look like: https://www.eia.gov/electricity/annual/html/epa_11_03.html. If you're outside the US look up yours and have a good laugh at us. :)

For the "right now" aspect you have probably visited your utility's outage map, but here I would say we do much better than most utilities. The level of detail on the investigation and resolution is often more detailed, and we usually know better than to bother providing much in the way of a concrete estimate for restoration time of a current outage (though this is getting better in the utility space).