staging-devopsy.kinsta.cloud

Category: DevOps Toolbox

  • Optimizing Cloud Costs for DevOps With AI-Assisted Kubernetes

    Optimizing Cloud Costs for DevOps With AI-Assisted Kubernetes

    As indicated in my prior blog, Optimizing Cloud Costs for DevOps With AI-Assisted Orchestration, an AI-assisted Kubernetes orchestrator is needed to optimize cloud costs for DevOps, DevSecOps and SRE. This blog describes the landscape for cloud-native optimization approaches and a features requirement blueprint for an AI-assisted Kubernetes orchestrator. The blog also describes a roadmap for implementing an AI-assisted Kubernetes orchestrator and the benefits of a solution using an AI-assisted Kubernetes orchestrator.

    Cloud-Native Optimization Approaches Landscape

    As indicated in the article Tricks for Cloud Cost Optimization, “Cloud cost optimization (CCO) is a technique of running cloud-native applications effectively, at minimal possible cost, without affecting the health of the services.” The most typical approaches are variants of the following:

    • Rightsizing pre-paid, fixed and reserved instances: Cloud service pricing models are intended to fit your application and business needs with the hundreds of choices available from cloud service providers. This approach can be cost-efficient when application and business needs are constant over time. However, this approach is not sufficiently cost-efficient for many applications and business needs that are constantly changing, such as use cases for DevOps, DevSecOps and SRE.

    • Spot instances can offer low cost-per-instance, but reliability is not guaranteed because the cloud provider can interrupt these services anytime. Unless the use of spot instances is limited to workloads that do not require a high level of reliability, this approach is limited. This approach does have the potential to have a broader application if a smart management layer is implemented to orchestrate the use of the spot instances.

    • Pay-as-you-go offerings by cloud service providers, used together with infrastructure-as-code (IaC) and auto-scaling, can help save on cloud costs. This approach facilitates the automatic spin-up of resources when needed and automatic shutdown when not needed. However, to realize the benefits of this approach requires constant supervision of cloud usage and manual adjustments to realize the benefits. Businesses may not even be aware of the resource scaling until the monthly bill arrives.

    • Cloud auto-stopping of entire applications and clusters can help to manage cloud resources automatically to ensure that they run only when needed, never when idle. This approach is not granular enough to sufficiently reduce costs for rapidly fluctuating resource needs for DevOps, DevSecOps and SRE use cases.

    • Manual cloud costs management, in which team members are taught to understand their own cloud spend and cloud cost management practices, can help reduce costs. However, this approach is subject to human involvement, is slow and generally does not provide granular controls needed to sufficiently deliver cloud cost savings 24/7.

    None of the traditional cloud cost optimization approaches described above have the requisite smarts, speed or granularity to sufficiently optimize the highly dynamic requirements for DevOps, DevSecOps and SRE use cases. An AI-assisted orchestration approach, when supported with the right features, can make up for the deficiencies of the typical optimization approaches.

    Features for AI-assisted Kubernetes Orchestrator

    As indicated by CAST AI an AI-Assisted Kubernetes orchestrator needs the following features:

    • Easy to start – It should take less than five minutes to fully onboard a cluster.
    • Easy to use – The orchestrator should abstract layers of complexity, so little knowledge is needed to use it other than a basic knowledge of running Kubernetes clusters in a public cloud. For example, a working knowledge of the kubectl command line utility for creating and managing Kubernetes clusters.
    • SaaS configuration – Avoids the need to host and manage the orchestrator itself.
    • Autoscaler policies – Define threshold values and upper bounds so instances are automatically added or removed as per the policy.
    • Cloud Support for AWS EKS, GCS GKS, Azure AKS and kOps clusters
    • Reports to monitor and track cost savings for each cluster.
    • Features to optimize each cluster.
    • High availability to ensure the orchestrator itself is not a failure point.
    • Collect and analyze each cluster configuration to provide the most optimal setup along with a savings estimation for the current cloud environment.
    • Security – Detect and monitor K8s vulnerabilities and configuration issues.

    Roadmap for Implementing AI-Assisted Kubernetes Orchestrator

    According to the article Roadmap To Cloud Cost Optimization, organizations need to understand that they need to deploy cloud computing the right way. This not only paves way for the actual digital transformation but is even instrumental to cloud cost management.”

    The article Top 3 Issues in Kubernetes Cost Management indicates, “Teams cannot manage and monitor Kubernetes’ costs without having the right tools in hand. No doubt, you can collect all the data yourself and create the right solutions. It wouldn’t be easy to modify these solutions if changes are required.” The speed of change with DevOps, DevSecOps and SRE is not compatible with the typical tools.

    A good example AI tool, CAST AI, indicates how easy it can be to onboard a Kubernetes cluster. First, connect your Kubernetes cluster to CAST AI and see how much you can save by optimizing cluster configuration. After exploring the available savings and cost reports, you can onboard the cluster into CAST AI and set the Autoscaler policies, which will manage the cluster for you. It takes less than five minutes to fully onboard a cluster.

    Benefits of Solutions Using AI-Assisted Kubernetes Orchestrator

    An AI-assisted cloud orchestrator can provide the following benefits.

    • Spot instance automation
    • Spot instance fallback
    • Real-time autoscaling
    • Instant rebalancing
    • Visibility into cloud usage and costs
    • Avoids over-provisioning
    • Cloud resources are not left idle
    • A consistent approach between teams and departments
    • Reduce monthly average cloud spend by up to 60%

    Summary

    The prior blog Optimizing Cloud Costs for DevOps With AI-Assisted Orchestration explained why an AI-assisted Kubernetes orchestrator is needed to optimize cloud costs for DevOps, DevSecOps and SRE. In this blog, specific requirements for such an AI-assisted Kubernetes orchestrator was explained using CAST AI as an example. The steps to onboard a cluster to the AI tool was explained.

  • Fiberplane Adds Wasm Plug-in Framework for DevOps Notebook

    Fiberplane Adds Wasm Plug-in Framework for DevOps Notebook

    Fiberplane today announced it is extending the reach of a real-time collaboration notebook designed for DevOps teams by making available tools for building plug-ins based on the WebAssembly (Wasm) framework.

    Micha Hernandez van Leuffen, Fiberplane CEO, said these plug-ins, dubbed providers, will make it much simpler for anyone to connect the Fiberplane notebook to other DevOps tools and platforms.

    The Fiberplane notebook uses the same operational transformation algorithm used in Google Docs to enable teams to easily collaborate around a document or spreadsheet. The core platform itself is built in Rust and WebAssembly. That approach enables a DevOps team to, for example, write and run PromQL queries and display Prometheus charts alongside other data presented via an integrated graph. Wasm makes it possible to run those providers both in the browser and on a backend platform.

    Fiberplane also announced it is open sourcing its Provider Development Kit (PDK), written in Rust, to enable organizations or third-party developers to create additional plug-ins. A Fiberplane Daemon then executes the plug-ins. The overall goal is to provide an open format based on Wasm for consuming DevOps data, noted Hernandez van Leuffen.

    The full open source Fiberplane WebAssembly-based Plugin System includes a Fp-bindgen to provide the bindings generator for building plug-ins, the Provider Development Kit (PDK), a toolchain for building plug-ins, a set of pre-configured plug-ins for Prometheus, ElasticSearch, Sentry, Grafana Loki, Cloudwatch and a generic HTTP provider, the Fiberplane command line interface and the Fiberplane Daemon.

    Fiberplane is essentially taking the concept of notebooks used for collaboration by data science teams and applying it to DevOps workflows. The goal is to improve DevOps teams’ productivity by reducing the amount of time it takes to sort through multiple types and classes of DevOps data to resolve an issue.

    As IT environments become more distributed in the age of the cloud and microservices, troubleshooting issues now invariably spans the responsibilities of multiple members of an IT team. Fiberplane, in effect, makes it possible to spin up a virtual war room that enables the members of an IT organization to collaborate until the issue of the moment is resolved.

    The speed at which any IT issue is resolved has become a crucial metric. Given the complexity of IT environments, it’s inevitable that incidents will arise. Ideally, IT teams would discover those issues before they disrupt the business. However, in reality, the metrics organizations use to evaluate the effectiveness of a DevOps team are often how quickly IT issues are resolved once they are discovered. The longer it takes for a distributed DevOps team to marshal its resources, the more likely it becomes that team will be perceived as being inefficient—regardless of what level of application availability is actually achieved.

    Most DevOps teams have, of course, implemented a wide range of incident management best practices that create a level of muscle memory for responding to a wide range of issues. The trouble is there will always be unforeseen issues that arise. DevOps teams, as a result, need to find ways to quickly focus on an unpredictable issue that they didn’t anticipate.

  • Understanding the Infrastructure Automation Maturity Model

    Understanding the Infrastructure Automation Maturity Model

    Nearly every company wants to evolve its digital transformation and increase the pace of software delivery and operations. And infrastructure automation is a new means to achieve these ends. It supports the emerging discipline of platform engineering and can increase scalability and reactivity to unforeseen events, enabling software ecosystems to be more anti-fragile. Increased infrastructure automation also helps free up engineers from focusing too much on managing DevOps when they really should be building out new features.

    But introducing more infrastructure abstraction can be met with certain challenges. This is often because departments have a wide range of cultures, processes and tooling already in place. With varying maturity levels across teams, it can be challenging to gauge where a company is before attempting to introduce new automation across the board. And as further evolution may require significant changes, having a baseline is key to planning for the future.

    I recently chatted with David Williams, senior vice president of market strategy at Quali, about an internal technique they’ve been using at Quali to gauge the maturity of their clients’ infrastructure automation. Their infrastructure automation maturity model breaks down the journey into four unique stages, moving from the least-mature ad-hoc automation to the most-mature autonomous solutions. Below, we’ll review this model and consider what questions need to be asked to ascertain where your team sits on the spectrum. We’ll see how enterprises can leverage such a model to hone in on improving their software operations.

    Four Questions to Determine Infrastructure Maturity

    DevOps is still a wild west in terms of how people approach it, said Williams. So before attempting to mandate sweeping cultural changes, it helps to first determine the base level of DevOps maturity within an organization. Here are four questions that can quickly assess the state of infrastructure automation within an organization.

    1. What Culture Already Exists?

    First, it’s good to evaluate the culture that already exists. How do teams deliver value and how is this value measured? Perhaps throughput or code quality is measured with specific metrics or SLAs. Understanding the DevOps attitude (and aptitude) within the organization will highlight pre-existing capabilities and show how teams collaborate.

    2. How Are People Organized?

    The genetic makeup of teams can dictate much about development and operations. How are teams organized? Are they large, siloed teams? Or are they two-pizza-sized teams? Understanding team organization and communication patterns are necessary to identify bottlenecks.

    3. What Processes Are Already in Place?

    Next, consider the development timelines and procedures already established and in use. Are teams applying SAFe, Lean, Agile or Scrum methodologies? What are these exact processes and how mature are they? Answers to these questions will demonstrate general release cadences and how teams drive productization.

    4. What Infrastructure Automation is Already Being Used?

    Lastly, you’ll need a window into what level of infrastructure automation, if any, is already in use today. How is it delivered? How is it managed? How is the success of this automation measured, and what is the overall outcome? You will likely find that automation is typically quite siloed and teams have differing approaches. This auditing is vital to identify what works well and what doesn’t.

    The Four Levels of the Infrastructure Automation Maturity Model

    Asking the above questions will quickly determine an organization’s “readiness state,” said Williams. But how do these answers map to the overall maturity of the organization’s infrastructure automation? Well, he shared four basic levels of infrastructure automation maturity that are commonly found in practice throughout today’s software development environments, ranging from bespoke scripts to more robust self-defining automation.

    Level 1: Ad-hoc

    The most basic level of infrastructure automation is ad-hoc integrations involving a mix of old and new technology. This level is chaotic and fragmented, with varying processes used throughout the organization. These bespoke integrations and scripts also may not have excellent visibility, limiting the potential for reusability within an organization.

    Level 2: Automation

    Next is having automation that is specifically tied to a certain technology stack. Nowadays, organizations are commonly hybrid multi-cloud, having adopted automation piecemeal with each new stack. This often results in multiple silos of automation stacks that are blind to each other. At this maturity level, one automation might fire off another automated procedure and so on, but engineers aren’t 100% aware of how they influence each other.

    Level 3: Frictionless

    The next level of infrastructure maturity begins to take the reins with more intentional automation adoption. At this stage, organizations embrace an internal developer platform, similar to Spotify’s Backstage, that helps engineers deliver and manage infrastructure more seamlessly. This tooling is more advanced than other levels integrating key commands into the developer’s IDE. Doing so eliminates the need to switch between environments and reduces friction.

    Level 4: Self-Defining

    Finally, the most advanced form of automation involves deeper intelligence within application operations, including the ability to dynamically optimize infrastructure. At this stage, infrastructure is self-defining, meaning production applications can automatically adapt, be self-governed and enable bursting and non-bursting capabilities. Such intelligence is typically proprietary to cloud providers, said Williams, yet companies often desire this intelligence inherent at the application level. Williams describes this stage as the “intelligent allocation of resources in line with application life cycle requirements.”

    Benefits of Self-Defining Infrastructure Automation

    Self-defining infrastructure automation has many benefits for companies, including having the infrastructure in line with application needs, scalability and measuring usage to optimize cloud spending. Many points within the software development life cycle could warrant increased automation, such as infrastructure-as-code (IaC), deployment release engines, configuration, traffic scalability, cloud management, containerization and documentation. But it’s not until you open conversations around infrastructure automation maturity and understand who is responsible for managing it that you can hone in on what areas can be improved.

  • Lineaje Unfurls Platform for Creating and Managing SBOMs

    Lineaje Unfurls Platform for Creating and Managing SBOMs

    Lineaje this week unfurled a platform for creating and managing the software bills of materials (SBOMs) that are increasingly mandated by enterprise IT organizations and government agencies.

    Fresh from raising $7 million in seed funding, Lineaje CEO Javed Hasan said the SBOM360 platform provides a simpler way to manage all the SBOMs that will soon inundate organizations.

    The SBOM360 platform makes it possible to use a search engine to discover all the components of an application and all the dependencies that exist within it, including vulnerabilities, and indicators of compromise (IOCs) in seconds, said Hasan. The company then adds fingerprinting technology that enables the platform to attest to the authenticity of the entire software supply chain, he added.

    While there is no shortage of tools for creating SBOMs, there are not as many platforms that enable DevOps teams to both create and manage them. In the wake of an executive order issued by the Biden administration requiring federal agencies to have an SBOM for all the software they use, interest in SBOMs to better secure software supply chains has significantly increased. However, very few organizations have a process in place that enables them to operationalize all the SBOMs that might be created.

    More challenging still, the pace at which software is being updated has accelerated to the point where many SBOMs are likely to be outdated. The only way to keep pace with that rate of change is to rely on a platform designed from the ground up to dynamically create SBOMs in seconds, noted Hasan.

    Finally, IT organizations are now being held more accountable for legacy software often polluted with vulnerabilities that need to be remediated, he added. As such, organizations need a tool that makes it simpler to identify vulnerabilities within the context of an SBOM.

    It’s not clear whether greater awareness of the need for SBOMs is translating into concrete action, but the expectation is that more organizations will soon be following the lead of the federal government by requiring them. That shift will naturally bring a lot of additional pressure to bear on DevOps teams. Historically, organizations tended to allow developers to aggregate software components without always checking to see if they are compromised. Cybercriminals are becoming increasingly more adept at injecting malware into software components in the hope that malware will find its way into any number of downstream applications.

    Regardless of the approach an organization takes to better secure software supply chains, more rigor is being applied to how software is built and deployed. The days when developers were free to aggregate software components as they saw fit by, for example, relying on open source software downloaded from a repository, are now at an end. The expectation now is the developer will be held accountable for the integrity of all components used within an application as organizations move to implement additional processes to more rigorously review code before it is deployed in a production environment.

  • Report Affirms Continued AWS Cloud Dominance

    Report Affirms Continued AWS Cloud Dominance

    A report published this week by Wiz, a provider of a cloud security platform, found more than half of organizations (57%) are using multiple clouds, with 22% currently using three or more platforms.

    However, the report also noted that 72% of all workloads are running on the Amazon Web Services (AWS) cloud. Among organizations that have more than one cloud platform, Microsoft Azure (41%) is the most common secondary platform (41%), while Google Cloud Platform (GCP) is the most used tertiary platform among organizations employing two or more clouds (44%).

    The report also noted 9% of AWS customers use managed database servers, compared to 87% in GCP and 82% in Azure. However, over 91% of companies also have non-managed database servers running in a cloud. In total, only 6% of companies use managed database servers exclusively, compared to 90% using a mix of managed and non-managed database servers, the report finds.

    The most widely deployed databases are PostgreSQL, Redis and MySQL, with 90% of organizations running at least one instance of a PostgreSQL server.

    Scott Piper, principal cloud security researcher for Wiz, said the report makes it clear that despite gains by Microsoft and Google, AWS remains the most dominant cloud service provider. Even after a decade, rival cloud service providers have not been able to overcome the first-mover advantage that AWS continues to enjoy. In many cases, a second or third cloud is being employed only because a single development team preferred it or an IT team assumed responsibility for it when their company acquired another organization using it, noted Piper.

    However, while most organizations have multiple accounts per cloud service provider, the report also found that the bulk of the application workloads are managed via only a handful of accounts, he added. That suggests more organizations are subject to higher levels of disruption in the event of an outage than many of them may realize, said Piper.

    One of the reasons so many organizations have a narrow range of accounts, of course, is they simply don’t have enough personnel. In fact, a general skills shortage also tends to limit the number of cloud platforms organizations might otherwise employ.

    Multiple cloud platforms also tend to exacerbate cybersecurity challenges. With the introduction of each additional cloud service, the overall size of the attack surface that needs to be defended expands. The Wiz report found 47% of organizations have at least one database or storage bucket publicly exposed to the internet. It can take less than 13 hours for cybercriminals to discover and access an exposed bucket with a guessable name, noted Piper.

    More challenging still are the number of application programming interfaces (APIs) that need to be secured as cloud service providers continue to add new services, he added.

    Ideally, organizations should be adopting a framework to harden privileges, encryption, authentication and connectivity and improve hygiene, otherwise known as PEACH, said Piper.

    Regardless of the approach to cloud computing, it’s apparent there’s still a long way to go before organizations are ready to embrace hybrid cloud computing spanning multiple clouds.

  • Voice.ai ‘Stole’ Code ¦ AWS Gets Filthier

    Voice.ai ‘Stole’ Code ¦ AWS Gets Filthier

    Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.

    This week: Alleged theft of GPL code, and Amazon will run its data centers on gas. (more…)

  • Blameless Integrates Incident Management Platform With Opsgenie

    Blameless Integrates Incident Management Platform With Opsgenie

    Blameless this week announced it has integrated its namesake incident management solution with the Opsgenie alerting platform from Atlassian.

    Aaron Lober, director of product marketing for Blameless, said the Blameless platform is now integrated with both Opsgenie and PagerDuty, the two most widely used platforms for managing alerts.

    Blameless provides an incident management platform that normalizes data collected from a wide range of DevOps tools and platforms to determine how best to respond to different incident classes. Armed with those insights, it then becomes possible to quickly assemble the best team to respond to an event, said Lober.

    That capability reduces stress because Blameless also surfaces recommendations for resolving incidents based on the data it has collected, he added.

    The Blameless platform is tailored to organizations that have hired site reliability engineers (SREs) to manage IT operations using DevOps best practices, noted Lober. The overarching goal is to eliminate rote processes that, over time, conspire to burn out incident management teams that often experience high turnover rates, he added.

    Opsgenie has been connected to the Blameless platform via the service catalog and alerting functions. Blameless users responding to or managing an incident can escalate issues to any teams defined within Opsgenie. The service catalog and alerting data generated throughout the life cycle of an incident is automatically captured and stored on the Blameless Incident Timeline. That capability makes it possible to identify appropriate service owners and the escalation protocol that was followed by that team. Blameless can also display the actual user name of each responder that has been notified.

    When accessing Blameless via Slack or Microsoft Teams, Opsgenie can automate both when an incident channel is created and pre-select relevant services that should be made available. Alerts can also be manually triggered using commands from the Blameless chatbot for those who prefer to trigger alerts from within Slack or Teams themselves.

    As IT environments become increasingly more complex, more organizations are making a concerted effort to apply DevOps best  practices to incident management. The goal is to reduce downtime and increase overall resiliency at a time when organizations are more dependent on software than ever. As part of that effort, IT teams should identify the repetitive processes that span multiple incidents and then move to automate as many as possible.

    In the longer term, it’s not clear what impact artificial intelligence (AI) will have on incident management. At some point, though, IT teams should expect their efforts to be augmented by machine learning algorithms capable of identifying issues before they balloon into a major crisis. However, there will always be some human intervention required to manage incidents for the foreseeable future, noted Lober.

    In the meantime, most IT teams should continue to expect the unexpected at a time when IT environments have never been more dynamic and change multiple times a day.

  • New Relic Bolsters Observability Platform

    New Relic Bolsters Observability Platform

    New Relic has updated its observability platform to include the ability to better detect and track errors in addition to monitoring vulnerabilities.

    At the same time, an instance of the New Relic observability platform that runs natively on the Microsoft Azure cloud platform is now also generally available.

    Manav Khurana, general manager for observability product and chief growth office for New Relic, said the alliance will enable Microsoft Azure customers to consume New Relic as part of their existing Microsoft Azure services contract.

    New Relic already provides an errors inbox to track issues, but is now adding a tool to track the impact an error might have by correlating data across traces and logs. The company has also integrated its platform with Slack to make it simpler for IT teams to generate alerts when vulnerabilities are introduced into codebases and manage errors more collaboratively.

    Similarly, New Relic is also looking to foster greater collaboration between software engineering and cybersecurity teams. A New Relic Vulnerability Management tool with no additional configuration required captures security signals generated via the company’s Telemetry Data Platform in a way that can be analyzed using an interactive application security testing (IAST) tool that New Relic gained with its acquisition of K2 Cyber Security. Additional IAST capabilities available in preview include the ability to leverage a patented deterministic technique to identify and provide automated vulnerability validation that highlights a proof-of-exploit.

    The overall goal is to provide more visibility and context for organizations that are implementing best DevSecOps practices, said Khurana.

    It’s still relatively early days as far as the integration of observability platforms into DevOps workflows is concerned. A recent survey conducted by New Relic found most organizations don’t expect to have robust observability practices in place by 2025. While most organizations have tools to monitor predefined metrics today, there is a significant gap between observability platforms that enable IT teams to determine the root cause of an IT issue and monitoring platforms that generate an alert whenever a threshold is exceeded.

    In fact, many organizations have too many monitoring tools that often surface conflicting or redundant alerts, noted Khurana.

    One way or another, more codebases are being instrumented. As that process continues to evolve, the agents embedded in that code make it simpler to collect the logs and traces needed to advance observability. The challenge, as always, is that it takes time for the number of fully instrumented applications to exceed the number of legacy applications that are either partially instrumented or not instrumented at all.

    As IT environments become more complex, the need for observability is becoming much more pressing at a time when most IT teams are more short-handed than ever, noted Khurana.

    Regardless of approach, there’s no doubt that observability, with patience, will impact the management of IT for the better.

  • Five Great DevOps Job Opportunities

    Five Great DevOps Job Opportunities

    staging-devopsy.kinsta.cloud is now providing a weekly DevOps jobs report through which opportunities for DevOps professionals will be highlighted as part of an effort to better serve our audience.

    Our goal in these challenging economic times is to make it easier for DevOps professionals to advance their careers.

    Of course, the pool of available DevOps talent is still relatively constrained, so when one DevOps professional takes on a new role it tends to create an opportunity for others.

    The five job postings shared this week are selected based on the company looking to hire, the vertical industry segment and, naturally, the pay scale being offered.

    We’re also committed to providing additional insights into the state of the DevOps job market. In the meantime, for your consideration:

    LinkedIn

    SiriusXM
    Washington, D.C.
    Director – Client Build and Release Management
    $166,000 to $300,000

    SimplyHired

    Dataflix
    Remote
    DevOps Engineer
    $140,000 to $190,000

    ZipRecruiter

    Bay Systems
    Mountain View, California
    Senior Full Stack Developer
    $120,000 to $174,000

    Monster.com

    GlobalLogic
    Remote
    Senior DevOps Engineer
    $120,000 to $130,000

    Indeed.com

    Leidos
    Ashburn, Virginia
    DevSecOps Engineer
    $97,000 to $202,000

  • Microsoft Outage Outrage: Was it BGP or DNS?

    Microsoft Outage Outrage: Was it BGP or DNS?

    Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.

    This week: All of Microsoft’s cloud services go down, everywhere except China. Redmond’s IaaS, PaaS and SaaS—including GitHub—were dead for several hours, and are still running unreliably—despite Microsoft saying it’s fixed the problem. (more…)