staging-devopsy.kinsta.cloud

Tag: devops

  • Operations Model: DevOps, NoOps, AIOps or None of the Above Ops?

    Operations Model: DevOps, NoOps, AIOps or None of the Above Ops?

    Within an organization, an operations model must be fit-for-purpose and be able to adapt as business needs change. At the same time, changing technology necessitates an advancement in how an organization operates. Small and medium-sized enterprises are faced with the difficult choice of how much to invest in operations and which approach to choose. In light of the numerous possibilities, companies need to identify the right upgrades for their needs. For instance, should a company now using a traditional operations model invest significantly to reach DevOps? Or would AIOps better fit customers’ needs? Is NoOps an option?

    There is no “correct” answer. Each option comes with pros and cons and important budget considerations. That is why every business needs to evaluate the appropriate system for its products and its customers with three concepts in mind: cost, goals and experimentation.

    Operations Models

    Traditional IT operations (ITOps) is the process of designing, developing, deploying and maintaining the infrastructure and software components of a specific product or application. It also ensures the customer experience is delivered through traditional means of support such as tickets and escalated paths for resolutions.

    DevOps is a process for accelerating the delivery of features to the application or product infrastructure in a consistent model with less human intervention, while also providing a better quality product through automation in the software development life cycle.

    An AIOps process introduces data science into the operating model by learning the behavior of the systems, scaling according to the needs of a platform (both infrastructure and customer usage of the platform). It expands the horizon of DevOps through the introduction of machine learning, focusing on data generated from the hardware and software systems, and allows organizations to grow organically based on demand.

    NoOps, although the name suggests otherwise, is an advanced approach to managing IT operations through the mindset that everything is derived as development. This model is best applied with startups and companies with high technological maturity.

    Typically, these companies have advanced skill levels to deliver the best experience by adapting to the new industry trends of both software and hardware maintenance practices. NoOps is at the opposite end of the spectrum from traditional operations, and it’s largely a theoretical concept at this point.

    NoOps is a fully automated, cloud-based approach that eliminates the operations team, aside from an integrated team focused on life cycle. The self-managed system requires resiliency and a high level of maturity with fail points that have already been addressed. NoOps, or high maturity structures, will also provide quick learning on drawbacks, customer and product needs. However, moving toward a self-maintained system requires upkeep for advances and innovation.

    Invest in an Operations Model to Reach Higher Levels of Maturity

    The first factor to consider in evaluating an operational approach is cost. Each level of IT maturation is gradual, incremental and expensive. Costs vary depending on the size of the company and the type and number of products. A small company that seeks to move from a level 2 to 4 maturity would need to make a significant financial investment.

    Some questions to consider when looking at operations model upgrades include:

    • Does this make sense for the business?
    • Will the investment allow for scaling?
    • Will it increase customer loyalty and trust in the brand?

    Next, it’s important to consider overarching goals and brand identity. The appropriate level of maturity might depend on the company’s value proposition. If, in the company’s vision, technology is at the heart of its journey, the company needs to invest, at a minimum, in a DevOps operating model to have the competitive edge to grow the business and meet its needs. If the company’s vision is to retain existing business and gain customers through continued investment in innovation based on the trend analysis of their core business, then AIOps will have the best ROI. If the company’s vision, culture and mission are intertwined with technology, adapting to NoOps is the right move.

    Know the Product, Customer and When to Halt

    Still, it’s important not to overreach with an operations model that might not fit the customer. To make a sound decision for the organization, it is important to understand the model, the product, customer base and when to halt an operational model and invest in a new one. The leadership team needs to ask how adaptable its end users are before implementing changes. Take, for instance, the poor consumer response to replacing live customer service representatives with automated features. Some end users were not prepared to adapt to the massive change in the platform or service offerings this move entailed.

    Product-driven businesses often don’t see investing in new operations models as worthwhile. This mindset can result in continual churn that creates a backlog and increased operating costs for maintaining systems and services. Another important consideration in deciding which operating model meets the company’s needs comes through experimentation. That is because finding the right operating model for an enterprise can’t be done with a cookie-cutter approach. The point is to maintain the existing quality and productivity while investing in outcomes that will garner more trust and confidence in the company. If you do this right, the end user will appreciate the changes, and the company will outperform the competition.

    Allow for Operations Failure

    Enterprises should expect failures during experimentation with different models and plan to pressure-test an advancing model for the cracks that will emerge. It is important to invest in failure to understand why and how the model did not perform to expectations. Those failures provide valuable insight and important data, result in significant learning about products and customers and help you know what to improve or modify going forward.

    An operations model serves as the backbone for any business that wants to grow. Smart operations investments will ensure adaptability, competitive advantage and customer loyalty. But there’s no one-size-fits-all, easy-to-follow recipe for each enterprise. Whether the investment results in AIOps, NoOps, DevOps or traditional operations is unique to each organization and needs to be calculated with the brand, customer and end goal in mind.

  • 4 Ways to Scale DevOps in 2021

    4 Ways to Scale DevOps in 2021

    January is a month often dedicated to planning, setting intentions and preparing ourselves professionally and personally for a fresh year. In my own capacity as a DevOps and SRE leader, many of the clients I work with will reinvigorate internal efforts to streamline how they work in Q1.

    I often advise clients to make very concrete goals and milestones for their technology organizations to become more agile, streamlined and modernized. Setting these goals is helpful to articulate to the wider business where you want to go and how you plan to get there. Here are a few DevOps-related initiatives your team may want to tackle in the first half of this year.

    #1: Tighten Security through DevOps

    Securing your organization’s infrastructure is an evergreen priority for technology leaders. When done properly, security should be baked into all DevOps processes (sometimes called DevSecOps) from the design phase all the way to application maintenance. To be effective and not slow down development, security processes need to move at the speed of today’s fast-moving environments, which is near impossible, or, at least very challenging, unless security is well automated.

    Automated security measures can be applied to the full life cycle of product development and should make development time faster and smoother in the long run if you can catch vulnerabilities early. Identify automation tools that work for your team and experiment with what works best.

    Similarly, in fast-moving environments, it’s tempting to play fast and loose with privileged access to unblock individuals and teams. Lax policies for user rights exposes the organization to too much risk from potential hackers. Implement a solution or protocol to keep access controlled. Policy and governance are critical for well-rounded DevSecOps. Communicate policies regularly with the entire organization. Ensure that all engineers know how to comply and enforce them.

    #2: Make DevOps an Organizational Culture

    Much like security, DevOps needs to evolve from an individual role to a cross-functional, holistic, cultural approach to developing software. Organizations should foster a DevOps culture rather than identifying specific individuals in DevOps roles.

    So how do you know when this culture shift has happened? A good pulse check is to see how quickly your company can adapt to change. How mired is your team in legacy code or infrastructure? Can individual teams operate autonomously? Teams and individuals that are independent and accountable are able to continuously experiment and learn at a much faster rate.

    DevOps cultural transformation won’t happen overnight – but it can be expedited if embraced by the entire company. I typically advise companies to focus on making incremental, consistent progress that builds autonomy, trust and transparency. These are the building blocks of a thriving DevOps culture.

    #3: Simplify Your Infrastructure

    Complexity, over time, creates duplication, unnecessary systems and technical debt. All of these have a cost, and create security vulnerabilities. Create organizational goals to clean up and reduce complexity. I often suggest teams create a roadmap to move toward self-service and building platforms. As you focus on building a DevOps organizational culture, you will likely notice that teams will identify opportunities to simplify and standardize, which will also remove complexity.

    #4 Review system SLAs and SLOs

    Establishing and monitoring key metrics are a fundamental part of any SRE role. Service level agreements (SLAs) and service level objectives (SLOs) need to be defined and monitored to ensure your team is fulfilling them. System availability (uptime, latency, etc) is core to any cloud SRE team. Define what reliability means for your organization. January is a good time of year to start fresh and articulate these metrics, how they relate to your core business and what investments or changes need to be made to achieve them.

  • How MLOps Can Help Get AI Projects to Deployment

    How MLOps Can Help Get AI Projects to Deployment

    Did you know that most AI projects never get fully deployed? In fact, a recent survey by NewVantage Partners revealed that only 15% of leading enterprises have gotten any AI into production at all. Unfortunately, many models get built and trained, but never make it to business scenarios where they can provide insights and value. This gap – deemed the production gap – leaves models unable to be used, wastes resources and stops AI ROI in its tracks. But it’s not the technology that is holding things back. In most cases, the barriers to businesses and organizations becoming data-driven can be reduced to three things: people, process and culture. So, the question is, how can we overcome these challenges and start getting real value from AI? To overcome this production gap and finally get ROI from their AI, enterprises must consider formalizing an MLOps strategy.

    MLOps, or machine learning operations, refers to the culmination of people, processes, practices and underpinning technologies that automate the deployment, monitoring and management of machine learning models into production in a scalable, fully governed way. Laying an MLOps foundation allows data, development and production teams to work collaboratively and leverage automation to deploy, monitor, manage and govern machine learning services and initiatives within an organization. In short, MLOps can help enterprises see AI projects through to deployment at scale by addressing the three previously mentioned barriers: people, process and culture.

    People

    There’s a skill, motivation and incentive gap between teams developing machine learning models (data scientists), operators of those models (DevOps, software developers, IT, etc.), and the business leaders who look for the results of that work. By providing actionable visibility into the different steps of the machine learning model life cycle for these different roles, MLOps can increase collaboration across departments so that enterprises can streamline the process of getting models operational and finally meet strategic goals.

    Process

    Operations teams are geared towards ensuring overall quality of service, optimizing runtime environments on their cloud, resource management and role-based access, among many other tasks. Data science teams are often unaware of the considerations these dependencies require, and, hence, models they create do not take these tasks into consideration. Additionally, lack of a proper native governance structure as it pertains to machine learning models – with system, performance, life cycle, and user logs – stifles troubleshooting, as well as hinders legal and regulatory reporting. Organizations that don’t properly monitor their models end up introducing potential risk to their organizations. When production models don’t reflect the ever-changing patterns in data and user behavior, these factors may affect the accuracy and overall performance of the model. Machine learning-based applications are far more sensitive to production environments than typical software applications. Therefore, machine learning models need to be validated and updated over time to ensure that they’re performing properly and leveraging the most accurate and recent data. As part of implementing an MLOps strategy, by having the right monitoring processes in place to detect drift – and many other machine learning-specific metrics – enterprises can ensure their models are delivering the most accurate results, and that their projects are more likely to move forward.

    Culture

    Fear of failure (or worse) can delay organizations from collaborating to operationalize machine learning models, or prevent this entirely. Leadership needs to be clear about their commitment to the process and fully support teams in enduring the hardships involved. This is only compounded by the fact that machine learning applications typically tend to involve processes that might introduce material risk to the company. To stay on top of such transformations while still producing trustworthy value, enterprises must keep careful tabs on all model deployment processes and the updates they entail. Many operations professionals are not aware of the unique characteristics and sensitivities of machine learning. As their role is focused on managing mission-critical production environments, they are often solely concerned with the implications on their work that deploying machine learning into production might have. While these are justifiable concerns, in the final analysis, they serve as trivial reasons for significant delays in progress and deployment. The way to mitigate all these risks and get all teams aligned around these common goals is to implement a solid MLOps strategy that matches the goals and unique position of the organization and its various teams’ responsibilities.

    MLOps: All Systems Go

    MLOps allows organizations to alleviate the people, process and culture issues posed by machine learning. MLOps provides a strategy and a set of processes and best practices, a technological backbone, for managing and scaling machine learning-based services through automation. It also provides for seamless collaboration between the data science teams responsible for generating models with the teams that are accountable for running the services in production environments. This combination of process automation with role-based collaboration and governance capabilities helps pave the way toward strategic AI goals. Specifically, MLOps can help organizations move from anticipating the management and scaling of machine learning services into production and turn that into reality.

  • DevOps for the Development and Delivery of High-Performance Applications

    DevOps for the Development and Delivery of High-Performance Applications

    Engineering teams that successfully implement DevOps practices often deliver software with higher business value. These organizations can easily identify areas where problematic code exists, quickly eliminate issues and deliver high-performance applications more quickly than their competitors that use traditional delivery methodologies. DevOps practitioners understand that creating an efficient model for performance testing goes hand-in-hand with four key objectives of digital transformation: extending performance testing to new roles; integration into continuous integration/continuous delivery (CI/CD) processes; end-to-end performance monitoring; and continuous optimization.

    Reduce Cycle Time and Time to Market

    In order to meet the requirements of rapid, high-quality application delivery, modern software teams need an approach that goes beyond conventional performance testing. This is where DevOps-centric approaches can provide faster, more efficient development and help ensure the delivery of high-quality software. Building the components of an application using agile approaches and validating features as they are created ensures that applications can be tested and verified more quickly, with no increase in associated costs or resources.

    By adopting DevOps practices, individual steps within the application delivery process can be automated as part of a DevOps tool-chain. For example, automating quality assurance test runs can eliminate – or significantly reduce – time-consuming human or analog searches for defects in code. Test automation can lead to an increased volume and expanded scope of testing, which, in turn, leads to a reduction in the defect escape rate. That saves resources, time and money.

    Shift Left to Increase Agility

    To mitigate performance errors and to address issues as early as possible in the application delivery cycle, tests should be executed during the build stage of the development process. By shifting left – running the tests as early as possible, ideally as part of a code commit – enterprises reduce the risk of discovering defects or issues with the product after significant amounts of development work have been completed. It is widely accepted that addressing defects later in development cycles takes significantly more time and money versus identifying and remediating defects early in the development cycle.

    As developers perform rapid, high quality, incremental changes, business analysts or product owners can more easily share changes, concepts or suggestions with their development teams. This continuous flow of communication can significantly reduce feedback time, increasing an organization’s ability to react to changing market requirements and allowing a level of flexibility within the development process that drives competitive advantage.

    DevOps in a Hybrid Landscape

    By supporting a wide range of tools and technologies, organizations that adopt DevOps practices can support applications that run in a local network environment, in cloud-based environments or, more commonly, in hybrid IT environments. As companies continue cloud migrations and transformations, well-designed DevOps tools can help simplify this transition while continuing to provide support for the legacy applications that provide core capabilities and processing for many enterprises. These hybrid tools are fully integrated into existing application development processes, allowing all application delivery teams to successfully transition to DevOps-centric practices without leaving any teams behind.

    One particularly important DevOps practice is performance optimization through continuous analysis. By using a centralized store and a common approach for quality-centric data collation, this practice helps analyze, enforce and improve the connections between software developer, CI Build and end-to-end performance testing to ensure a “quality first” mentality and higher-value delivery outcomes. The application of data visualization techniques allows teams to access code – build – test results in real-time and make decisions based on up-to-date insights.

    If DevOps practices are successfully established within a team or organization, they not only support more efficient software development and delivery but also are cost-effective. However, in order to successfully implement DevOps practices, enterprises need to understand and implement both technological and cultural changes. Through a combination of process, tooling and cultural simplification, and aligning teams toward common business objectives, organizations can deliver their end goal of creating high-quality, high-value software that satisfies the needs of the end user. By focusing on the common goal, and understanding the journey necessary to get there, enterprises can do so more quickly and at less cost – a win-win!

  • Set DevOps Free With Modern Application Security

    Set DevOps Free With Modern Application Security

    Development teams increasingly use technologies like cloud computing and microservices, in conjunction with DevOps principles, to innovate faster and remain competitive.

    But such progress comes at a cost. The fast pace of DevOps has left security teams scrambling to keep pace and install appropriate security guardrails. Traditional security practices used to occur in the final stages before release. The rapid release cycles of agile software development often make this difficult, if not impossible.

    The speed of development is outpacing the rate at which security teams can check configurations and scan for vulnerabilities, especially as developers now outnumber security professionals 500 to 1 and modern application environments represent a vast attack surface.

    The pressure to keep up, and do so securely, has led to a gradual reframing of DevOps as DevSecOps, where security is “shifted left” and introduced earlier in the software development life cycle.

    Unfortunately, while most businesses understand the intent of DevSecOps, they are still unsure how to translate that intent into action. In fact, just 14% fully integrate security throughout the software development life cycle.

    Recent research also shows that, while 65% of security teams report shifting left, less than one fifth are doing the scans necessary to verify that shift is actually happening. The same report suggests that most security teams don’t have the processes in place to monitor and protect cutting-edge application technologies, such as microservices, APIs and cloud native/serverless.

    This lack of visibility leaves security organizations blind when problems surface in production. The longer vulnerabilities go undiscovered in the development cycle, the more costly they are to fix. A study by IBM System Science Institute suggests that fixing a defect found during implementation can cost six times as much as one identified during early design phases. A defect uncovered in production can cost 100 times as much. Even more disturbing, nearly half of enterprises admit to deploying vulnerable applications, just to meet tight deadlines.

    It’s clear that compliance and security oversight are often overlooked, and even deliberately avoided, in favor of faster, more frequent deployments.

    The Great DevOps-DevSecOps Divide

    One of the most significant issues lies with developers’ ingrained perceptions. In the past, teams worked independently, with rigidly scheduled handoffs between distinct phases of development. Security operations (SecOps) teams often introduced security functions during the final stages of the development and release process, which resulted in delays.

    Successfully putting the “Sec” into DevSecOps depends on changing older cultural biases, reinforcing the need to embrace security and empowering teams with the right tools and automation to make smarter decisions without slowing down the entire organization. In essence, DevOps and security teams are all aiming for the same goal – a high-quality, timely product. The difference lies in how they measure and define “high quality” and “timely”.

    The truth is, development won’t be slowing down any time soon. Thirty eight percent of developers now release monthly or faster, and 54% of containers live for five minutes or less. Viewing security as a separate entity bolted onto the code, instead of a key feature that must be embedded end to end, slows down development processes and reduces efficiency.

    Delivering Speed and Security

    The core questions organizations must answer to improve application security are: What makes it easier for DevOps teams and application security teams to collaborate? And what does built-in security really look like?

    For organizations that want to improve their DevOps security, here are some tips:

    • Automate security as much as possible. Invest in security solutions that can be automated and embedded directly into the CI/CD pipeline. This makes it easier to secure applications without sacrificing development speed for the sake of security. Adopting technologies like static code analysis, dynamic analysis and penetration testing reduces risks and alerts developers to potential problems. There will never be enough security professionals to handle all security issues on their own, so use automation whenever possible.
    • Build security as a guardrail, not a gate. Provide appropriate guidance and tooling, so that security becomes a guardrail rather than a gate. For example, ensuring that DevOps teams have access to templated policies enables them to align applications with security requirements from the outset, without adding unnecessary time to development. Applications and security policies can also be tested as part of the CI/CD pipeline, so they are checked like any other functional specification. In short, it’s important to provide developers with everything they need to create and test applications with the appropriate security controls applied, or they won’t do it. Empowering development teams with a better understanding of security and self-service compliance reduces vulnerabilities and risks to the organization. Developers always want to move faster, so make it possible for them to “speed” safely.
    • Enable more proactive responsibility. The average developer cannot be expected to have the level of expertise necessary to stay current with all the latest security trends—they have enough trouble keeping their programming skills up to date. Reduce complexity and make it easier to gain developer buy-in by selecting solutions that provide simplified, easy-to-understand insights within the CI/CD feedback loop.
    • Make security adaptable, scalable and reliable. Secure applications with solutions that offer consistent, centralized and self-service security for any environment. For example, AI-driven security policy engines are one way to enable the adaptability necessary to support rapid change in applications resulting from CI/CD methodologies. Being able to adapt security policies in response to the latest attacks and identify dependencies makes it easier to assess risk and take action faster.

    The dawn of frictionless and adaptable security

    Security can no longer be an afterthought, bolted on to a process. Today, integrated security must become a normal part of any DevOps implementation. Making security frictionless and adaptable enables development teams to power ahead without fear. Modern application security can be a robust support system that empowers organizations to reach their business goals securely.

  • Cloudflare Lays Out GitOps Ambitions for CDN Platform

    Cloudflare Lays Out GitOps Ambitions for CDN Platform

    Cloudflare’s acquisition of Linc is the latest move in the hosting provider’s move to better align its content delivery network (CDN) service with the tools and frameworks organizations rely on to build modern applications using GitOps processes, according to CEO Matthew Prince. Cloudflare also announced other updates and automation-related features, including a way to automatically deploy JavaScript applications using Linc’s technology.

    Prince said these latest moves are part of an ongoing effort to simplify integration of front-end applications with back-end services.

    Cloudflare Pages, for example, enables developers to select their preferred repository, identify what types of frameworks they are using, and then automatically deploy their applications on the Cloudflare CDN. The goal is to automate the continuous integration process in a way that enables developers to focus more of their time and effort on developing code, said Prince.

    Linc’s technology, meanwhile, will make back-end services more accessible to front-end applications using Front-end Application Bundles (FABs) that accelerate server-side rendering by combining front-end and back-end logic. The result is better performance across any type of application that relies on JavaScript, application programming interfaces (APIs) and markup languages, otherwise known as JAMstack, a framework for building complex applications.

    Prince said JAMstack adoption is accelerating because it allows developers to continuously modify the user experience (UX) without the need to modify back-end services. The CDN, meanwhile, has emerged as a platform for continuously deploying applications closer to the point where they are consumed by the end user, said Prince.

    That’s become especially critical at a time when digital business processes require data to be processed in near-time, and as closely as possible to the point where it is generated, said Prince. That’s not possible when application logic runs on a public cloud, Prince added.

    Cloudflare currently has points of presence in 200 cities worldwide, and is actively courting developers as part of an effort to usurp CDN rivals. Most recently, Cloudflare added a Durable Objects capability that makes it possible to run stateful applications across a distributed, serverless Cloudflare Workers service.

    Ultimately, Prince said the goal is to automate the majority of the DevOps process, as code moves from a Git repository to the network edge. A CDN will also provide the means to monitor and secure applications running at the network edge, Prince added.

    It’s not clear how much application logic will be pushed to the network edge via CDNs; however, it’s apparent CDNs are emerging as an important deployment platform that should be incorporated with DevOps processes. How providers achieve that goal is, in part, dictated by the framework used to build an application.

    Cloudflare’s moves are indicative of the larger trends of automating DevOps processes, a development that should make DevOps best practices more widely accessible to many more developers.

  • How Healthcare Companies Can Optimize Software for Improved Patient Outcomes

    How Healthcare Companies Can Optimize Software for Improved Patient Outcomes

    Healthcare companies are more reliant than ever on software to help them achieve good patient outcomes. Recent research reveals that the use of software as a service (SaaS) solutions is growing exponentially in health care, covering a wide range of areas including:

    • Patient diagnosis
    • Medical imaging
    • Therapy and patient self-care
    • Data collection and analysis
    • Hospital management

    Due to the increased use of digital products in patient care, healthcare providers recognize the need to speed innovation and fundamentally rethink how they build software. This has led many engineering teams to move toward continuous integration and continuous delivery (CI/CD) when building and deploying software.

    Considering the industry’s emphasis on improving patient experiences, it makes sense that healthcare providers are keener than ever to optimize their systems. In fact, around 90% of healthcare providers are implementing or already using customized software solutions and development.

    As teams look to improve these critical processes in software development, feature flagging is one way to speed up development and optimize software performance in the healthcare industry.

    CI/CD Challenges for Healthcare Companies

    The healthcare industry faces unique challenges when applying DevOps strategies such as CI/CD.

    Intense Regulation

    The main problem is that, for obvious reasons, health care faces stringent regulation and oversight. A high level of scrutiny is necessary in an industry in which one small mistake or misjudgment can cost patient lives; however, in terms of DevOps, waiting for regulatory evaluation and approval can slow down the entire software development process, which defeats the core purpose of CI/CD.

    Security

    Data security poses another challenge for healthcare providers, especially when dealing with cloud-based software solutions. Cloud-based systems will always have potential vulnerabilities, as they rely on infrastructure that connects a variety of devices and smart technology equipment. There will always be inherent weaknesses in remote systems, as opposed to on-premises, too. A recent survey revealed that 42% of respondents said their major security concern was unauthorized access to cloud systems through insecure APIs. Network configuration problems were a close second, with 40% of respondents citing those as a concern.

    In a healthcare setting, another issue is the high volume and high velocity of data. Even a small, specialist clinic that may only see 150 patients per day will still generate vast amounts of data over the course of a month or year. Most of the data is private and sensitive, which means that data privacy and cybersecurity are top priorities for healthcare companies.

    The Impact on DevOps

    At first glance, because of the high level of regulatory oversight and cybersecurity concerns, it might seem that health care is a less-than-ideal environment for agile DevOps processes. How can you adopt fast-paced CI/CD cadences and innovate rapidly when there are necessary, opposing forces?

    The answer is to leverage innovative software development solutions such as on-premises feature flagging.

    Feature Flagging and Its Impact on Software Innovation

    Feature flags are, essentially, toggles or switches that allow software development teams to turn features on or off. The main advantage of feature flagging is that it allows for faster software deployment. One or more features can be coded into the software during development and made ready to go, but it is turned off for the user and/or patient on initial deployment.

    The ability to embed features within software that can be toggled on or off is perfectly suited for the healthcare industry since that approach allows for the necessary regulatory approval(s) before full deployment.

    For example, look at the rapid rise of telemedicine during the COVID-19 pandemic. Doctors, hospitals and patients alike were forced, along with the rest of the world, into the “new normal” of virtual interaction to mitigate the spread of disease while still providing quality treatment.

    For most industries, adding video chat/conferencing capabilities to existing applications was relatively easy. But for health care, the approval process, privacy and security concerns slowed down the adoption of these features. In our experience, typical healthcare teams are, best-case scenario, running on monthly—or, most likely, quarterly—deployment cycles to align their deployment efforts with compliance and governance review boards. Product and engineering teams work to get their updates and improvements in line with this cadence, which can make innovation difficult.

    If teams wanted to deliver this capability quickly, they could adopt feature flagging to decouple the concepts of “deploy,” “approve” and “release.” To use the video chat example, here are the steps they’d be able to take:

    1) Build the videoconferencing capability into their app and deploy the feature to production with the capability switched on for internal users (IP targeting, specific email addresses, etc.) only.

    2) Now that the feature is in live production, they can perform actual field testing to ensure that the functionality is working well, while simultaneously moving through internal and external approval processes.

    3) Once the review is complete, move to the release stage by switching the feature(s) on and allowing (external) end users to access the capability.

    All this can be achieved without needing to redeploy the software, and the developers are still free to make iterative improvements to the application.

    Now that the regulatory hurdles are addressed, it’s important to acknowledge that security is still an issue for healthcare companies that employ a SaaS feature flagging platform.

    While most providers focus solely on SaaS, the ability to deploy feature flagging software on proprietary cloud infrastructure is important in the healthcare space. This model enables both extra security and HIPAA compliance, and also opens up important capabilities and performance results that SaaS-only providers have a hard time matching. Since this technology typically becomes deeply embedded in the fabric of an organization’s product and technology, feature flagging is well-suited to an on-premises implementation.

    The implementation of on-premises feature flagging is a great way for healthcare companies to increase deployment velocity from monthly to daily. It allows their software development teams to adopt CI/CD cadences and innovate more rapidly, without the risk of breaching regulations or making sensitive data easily available to hackers.

  • AWS Looks to Meld MLOps and DevOps

    AWS Looks to Meld MLOps and DevOps

    The rate at which machine learning operations (MLOps) and DevOps best practices are converging is poised for rapid acceleration in 2021. At the recent online AWS re:Invent conference, Amazon Web Services (AWS) announced it is adding a bevy of capabilities to Amazon SageMaker, a managed MLOps service, that includes a continuous integration/continuous delivery (CI/CD) service for MLOps it is calling Amazon SageMaker Pipelines.

    Amazon SageMaker Pipelines enables developers to define a machine learning workflow that encompasses everything from loading data to training configuration; algorithm set up and debugging algorithms. Rather than having to acquire and deploy a standalone CI/CD platform for building artificial intelligence (AI) models, AWS is making the case for using a managed service that enables developers to manage workflows from within the Amazon SageMaker Studio tools it already provides for building AI models.

    Workflows can be shared and reused between teams to either re-create a model or can serve as a launch point from which to create another AI model.

    Amazon SageMaker Pipelines also logs each event in Amazon SageMaker Experiments, which enables IT teams to organize and track machine learning experiments and versions of AI models. In addition, a Deep Profiling for Amazon SageMaker Debugger now makes it possible for developers to more quickly train AI models by automatically monitoring system resource utilization and generating alerts whenever bottlenecks are detected.

    AWS has also added a Distributed Training capability to Amazon SageMaker that it claims makes it possible to train large, complex deep learning models up to two times faster than current AWS approaches by enabling data and AI parallelism across its cloud services.

    The cloud services provider has also added an Amazon SageMaker Edge Manager service that enables developers to optimize, secure, monitor and maintain machine learning models deployed on fleets of edge computing devices. The Amazon SageMaker JumpStart service enables developers to search for AI models, algorithms and sample notebooks.

    Finally, AWS has added additional data preparation tools that come pre-configured with support of AWS and third-party data sources, a repository for storing AI models, a preview of additional tools for automatically matching AI models to data sources such as the Amazon RedShift data lake, the ability to match an AI model to a graph database and a tool that detects bias within AI models.

    Bratin Saha, vice president and general manager for machine learning services at AWS, said as usage of AI models increases, the scope of these initiatives naturally lends itself more to cloud platforms. Many applications are starting to employ multiple classes of AI models in combination with each other; Saha noted this requires massive amounts of data and compute resources to train those models that are only readily available in the cloud.

    Overall, AWS claims it already has more than 100,000 customers building AI applications on its platform using various tools and processor classes. AWS said in 2021 it will add G4ad Graphics Processing Unit (GPU) instances based on processors from AMD and revealed its intention to build its own GPUs next year. The company also plans to add an Amazon EC2 instance based on up to eight of Intel’s Habana GAUDI accelerators to deliver up to 40% better price-performance than current GPU-based EC2 instances.

    At the same time, AWS has extended its alliance with NVIDIA to make software modules for building AI models available on the AWS Marketplace.

    In effect, Saha said AWS is inviting IT teams to either build AI models on AWS or bring their own AI models, encapsulated in containers, to its platform.

    AWS is clearly aiming to dominate AI, regardless of how such capabilities are infused into enterprise applications. In fact, in many cases, AWS will automatically construct the AI model for customers, or simply return the results generated by an AI model via an application programming interface (API). It will be up to each organization to determine when building their own AI model makes the most sense, and to what degree they want to rely on AWS to automate the deployment process.

    What’s certain is that many of the same DevOps principles—created long before AI models ever came along—will still be relevant as the applications that use those models are continuously updated for years to come.

  • Rapid Strategic SRE Assessments Accelerate IT Transformations

    Rapid Strategic SRE Assessments Accelerate IT Transformations

    A Strategic SRE Assessment is critical in helping organizations successfully transform traditional IT operations to SRE

    Site reliability engineering (SRE) is rapidly becoming more important to organizations that need to scale IT operations while maintaining or improving reliability and security, and to modernize operations to match the growth of DevOps in the organization. Without sufficiently thorough consideration given to best practices for people, processes and technologies, the transformation from traditional IT operations to SRE is unlikely to succeed. Despite challenges, the transformation to SRE is compelling because it provides an opportunity to reduce total operations costs and improve security and reliability while scaling services.

    To be successful, IT leaders need a comprehensive understanding of SRE best practices and need a well-engineered, yet affordable strategy and road map that effectively implements best practices and balances the many aspects needed to transform people, proces and technology. A Strategic SRE Assessment approach, based on best practices, is a key tool to help leaders understand, obtain the strategy and create a road map that will successfully transform traditional IT operations to SRE.

    There is no “standard” SRE assessment approach in the industry. This article enumerates a practical and proven Strategic SRE Assessment approach. The heart of the Strategic SRE Assessment approach is a survey of SRE best practices categorized under nine pillars: Culture, Work Sharing, Toil Reduction, SLAs/SLOs/SLIs, Measurements, Anti-Fragility, Deployments, Performance Management and Incidents Management. The survey is used to collect importance and capability scores for each of the SRE practices. As part of the assessment, a survey determines the current state and preferences of SRE practices for an organization for each of the nine SRE pillars.

    The following are examples of best practices for each pillar that are part of the Strategic SRE assessment survey:

    Culture – Leaders need to understand and sponsor a clear vision for SRE.

    • The culture embraces working practices to proactively shifts left the “Wisdom of Production.”
    • The culture encourages learning from failure and continuous improvement.
    • The SRE role is clearly defined and understood and SREs are considered a vital part of the organization.
    • Rules of engagement and principles for how SRE teams interact with their environment are codified—not only the production environment but also the product development teams, testing teams, users and so on. Those rules and work practices help to maintain SRE focus on engineering work as opposed to operations work.

    Work Sharing – Successful SRE requires that work be shared between SRE teams and development.

    • SRE teams are composed of software developers with strong operations knowledge or IT operations people with strong software development skills.
    • SRE workloads are managed to a budget with typically 50% or less spent on operations work.
    • SRE on-call work budgets are typically 25% or less. Developers share on-call work.
    • Technical debt is worked in small increments.

    Toil Reduction – SREs work to reduce non-value-added work through automation and standards.

    • Work standardization and automation—of both tools and processes—is employed to improve scalability, repeatability and other important goals.
    • Work standardization is used to reduce time spent on non-value-added work.
    • Automation is used to reduce time spent on non-value-added work.

    SLAs/SLOs/SLIs – Service level metrics, based on user perspectives, combined with error budget policies, are critical to managing flow and reliability.

    • SLOs are selected to measure availability, performance and other metrics in terms that matter to an end user.
    • An error budget (= 1-availability) internal service level objective (SLO) is established for each service. SLOs are set to be more stringent than external service level agreements (SLAs) promised to customers. The SLO budget considers what level users will accept, alternatives for users who are dissatisfied, and usage patterns of the product at different availability levels.
    • An error budget policy prescribes consequences if an error budget is spent. For example, the service freezes changes (except urgent security and bug fixes addressing any cause of the increased errors) until either the service has earned back room in the budget or the count period resets.

    Measurements – Smart monitoring implements observability to accelerate response times to incidents.

    • Monitoring solutions are designed to produce service level indicators (SLIs) from which SLOs can be calculated automatically.
    • SLIs do not require a human to interpret an alerting domain. Instead, software observability algorithms do the interpreting, and humans are notified only when they need to act.
    • Monitoring systems use telemetry and instrumentation to ensure the data being monitored is accurate, relevant and available in real-time.

    Anti-Fragility – Systems of people, processes and technologies are continuously tested and improved to assure they are resilient enough to serve applications as they scale.

    • Anti-fragility strategies are practiced to proactively assure the resilience of applications, infrastructures and pipeline services.
    • Fire drills are periodically conducted to proactively determine weaknesses in systems of people, processes and technologies that could potentially impact SLOs.
    • Chaos monkey or equivalent infrastructure testing strategies are used to proactively determine weaknesses in infrastructure systems that could potentially impact SLOs.
    • Security testing and DevSecOps practices reduce the risk of security vulnerabilities.

    Deployments – Deployment strategies reduce the risk of deployment failures.

    • Non-emergency deployments use progressive rollouts in which changes are applied to small fractions of traffic and capacity at one time. If unexpected behavior is detected, the changes are rolled back first and diagnosed afterward to minimize mean time to recovery.
    • Automation is used to implementing progressive rollouts, detect problems and implement rollback changes safely when problems arise, because outages are primarily caused by changes in a live system.
    • Deployment strategies such as blue-green, feature flag rollouts, A/B testing and canary rollouts are used to reduce the blast radius of failed deployments.

    Performance Monitoring – Proactive testing and monitoring ensure applications and infrastructures have the capacity necessary to flex and scale when needed.

    • The SRE team is responsible for provisioning and monitoring utilization because utilization is a function of how a given service works and how it is provisioned.
    • The SRE team is responsible for capacity planning and provisioning. Regular load testing of the system is used to correlate raw capacity (servers, disks and so on) to service capacity.
    • Provision to handle a simultaneous planned and unplanned outage without making the user experience unacceptable. This results in an “N + 2” configuration, in which peak traffic can be handled by N instances (possibly in degraded mode) while two instances are unavailable.

    Incident Management – Incidents are managed in an efficient, blameless manner with a goal to use incidents to improve responses.

    • Playbooks record the best practices for human response actions because playbooks produce roughly a 3x improvement in mean time to recovery.
    • Emergency response performance is measured using mean time to repair (MTTR) metrics instead of mean time to failure (MTTF) because the most relevant metric in evaluating the effectiveness of emergency response is how quickly the response team can bring the system back to health—that is, the MTTR.
    • Postmortems are blameless and focus on process and technology, not people.
    • Postmortems are conducted for significant incidents regardless of whether they resulted in a page; postmortems that did not trigger a page are even more valuable, as they likely point to clear monitoring gaps. Find all root causes of the event and assign actions to correct the problem or improve how it is addressed next time.

    SRE Assessment Surveys Are Not Enough

    The survey alone is an essential tool to collect information about the current state and preferences for each of the SRE practices. However, this is only part of the comprehensive Strategic SRE Assessment approach.

    The Strategic SRE Assessment approach starts with workshops to get alignment between leaders and teams regarding big-picture goals prior to conducting the survey. This is necessary so that everyone who answers the survey has a common perspective and can better respond with considered views of importance and capability scores for each of the practices. The results of the survey include an analysis that points out specific pillars and individual practices that have a high gap score in which the practices are scored as important but not yet practiced well. The high-priority gaps are then used to organize a value-stream map working session, that is designed to find the highest priority bottlenecks in people, processes and technology bottlenecks in the value steam, relative to the organization’s goals and gaps. With the goals, gaps, and data from the value stream map in hand, the next step is to formulate an implementation road map that directly addresses the goals, gaps and bottlenecks in the value stream.

    The final step is to obtain strategic alignment of the big-picture plan by reviewing the findings and road map with the leadership and implementation teams. This entire Strategic SRE Assessment approach, from goal alignment through surveys, value stream mapping, road map creation and strategic alignment, varies depending on several factors such as the size of the organization, number of applications and size of the deployment domains to be included in the assessment.

    A Rapid Strategic SRE Assessment for a sample set of model applications typically is conducted in 21 days. More complex Strategic SRE assessments involving many applications may take up to 90 days.

    In most cases the Rapid Strategic SRE Assessment is the best place to start. Most often, it is not a good idea and is too expensive to “boil the ocean” and make changes across all applications and deployment domains at once. By starting with a small number of model applications, it is easier to implement, measure and demonstrate success and use the results at each step to justify ongoing investments needed to complete the transformation across the organization. Another advantage of the Rapid Strategic SRE Assessment is that it is a quick and affordable approach to check-point progress as SRE is implemented progressively across applications and deployment domains.

    Key Takeaways for SRE Transformation Leaders

    SRE offers immense value for modernizing and scaling IT operations reliably and securely and to complement DevOps transformation. Adherence to SRE best practices is essential to reduce risk and assure security as the transformation is implemented.

    Each organization is different and has different requirements and priorities for its applications and deployment domains. The specific SRE practices that are the highest priority vary according to the priorities of applications and deployment domains. The Rapid Strategic SRE assessment approach, based on nine pillars of SRE best practices, is quick, effective and affordable.

    The Rapid Strategic SRE Assessment is a preferred approach to kick-start the SRE transformation. This approach is also a preferred and affordable tool to reassess the direction as SRE is implemented progressively across the organization.

    Summary

    There is no “standard” SRE assessment approach in the industry. The Rapid Strategic SRE Assessment approach described in this article has been tried and proven with large and smaller organizations. The author invites your comments regarding this approach and the SRE best practices.

  • Up Your DevOps Game: It’s Time for NoOps

    Up Your DevOps Game: It’s Time for NoOps

    NoOps is here and it’s more than just a buzzword or an ideal. The next generation of platform-as-a-service (PaaS) platforms are evolving to minimize the developers’ operational burden and reduce dependency on DevOps for deployment. Dev teams that don’t want to deal with operations or don’t have in-house operations resources should consider deploying applications on a NoOps platform.

    To understand the concept of NoOps, we need to start with an understanding of how deployment management has evolved over the decades.

    From Hardware and Software and Back Again

    It’s been a long trend of back and forth between developers and “deployers.”

    It was once mandatory for software developers to be fluent in the hardware for which they were creating software. They wrote the code and then worked with the operations team to deploy it on in-house servers. Everything was tightly coupled. The first mainframes—IBMs and SiliconGraphics machines—ran on custom-built software, and the developers were responsible for the deployment and management. The developers had the machine first, and then had to create the code to make them run—assembly or low-level language or even hardware code; rope memory, in some cases.

    Then came the x86 revolution, when hardware became more standardized and was decoupled from the complexities of deployment. Developers started focusing on the software and organizations had separate teams that would manage the deployments.

    With the advent of virtualization and infrastructure as a service (IaaS), the deployment of software now depended on APIs for infrastructure provisioning, and we returned to an integrated development and deployment function—in other words, DevOps.

    The Problems DevOps Created

    The problem then became one of choice: As APIs multiplied, developers had to decide which APIs would be most relevant, efficient and useful to deploy their software—and on what infrastructure: bare metal, virtual machines such as EC2 and VMWare, Kubernetes, Swan, OpenShift, serverless, Lambda, Knative, etc.

    Not only did the teams have to decide on what IaaS to deploy the actual software, but they needed to also choose how and where the layers and logs would be monitored—and even where those logs would be stored. They need to consider things such as traceability and horizontal or vertical scaling issues, to name a few.

    This has resulted in the creation of bespoke permutations and combinations that work great … until vendors’ upgrades require them to keep up with tens or hundreds of configuration changes. Most teams are keeping it together, but it definitely can be done more efficiently.

    Now It’s time to Up the Game to NoOps

    It’s time for the next approach: Limit the number of choices to create standard best-in-class operations that deliver economies of scale and easily evolve with minimal hassle.

    NoOps simplifies cloud operations—everyone can do things the same way. NoOps aims to “completely automate the deployment, monitoring and management of applications and the infrastructure on which they run,” according to Forrester, which coined the term.

    NoOps is about standardizing the approach to deployments and reducing the number of variables, bringing simplicity. At its core, NoOps is focused on automating deployments and executions that are predictable and repeatable.

    NoOps & Standardization

    The development and increasing adoption of containers are critical to the entire NoOps philosophy. Containers provide the ability to independently deploy services and applications, automating and standardizing the process to deploy anything, anywhere. Using containers delivers the tremendous portability that hasn’t been seen since the development of generic hardware.

    With encapsulation within the container, whatever is running inside will behave the same no matter where it is deployed.

    The NoOps-containers movement will transform the entire DevOps industry.

    Let’s go back even further. In the 1800s, few homes had electricity. They hooked up whatever appliance they had to whatever generator they were running and kept their fingers crossed. Once governments took over the role of bringing electricity to every home with standardized voltages and outlets, the appliance industry took off.

    A true NoOps platform will provide a standard “socket” that works for everything. The developer will simply build the app, and it will “plug in” anywhere. With NoOps, developers don’t have to worry about how the application will work, deploy or scale, nor do they have to consider reliability or elasticity. The platform will deliver it automatically.

    The “Gift” of Containers

    Containers free developers from managing infrastructure and reduce the operational burden associated with building and deploying applications. They also minimize application conflicts among different environments and improve portability and team collaboration.

    Containers make application deployment easier than ever before. Continuous integration doesn’t change as part of this process. Adopting containers as part of NoOps simply automates continuous delivery—developers write the code, commit the changes and push it to the container’s remote repository. Based on the Docker file, an image will be built and the containers that run your application will be deployed. Since all deployments are rolling, no downtime occurs.

    Leveraging containers provides speed, agility and flexibility to any infrastructure, thus bringing cost savings and dynamism. Containers can flexibly host microservices, which can be scaled individually, as necessary. Using CPU, memory or even custom business rules, developers can set auto-scaling policies so applications are ready to serve an increased number of requests whenever needed. As the platform monitors application performance, it can use these insights to intelligently architect a right-sized infrastructure.

    Putting NoOps Into Play

    If you’re considering adopting NoOps, look for a PaaS provider with the expertise to make the right platform choices for you based on your application development language of choice. For example, if you are looking to host a Ruby application, ensure that the provider has the requisite Ruby expertise.

    A true NoOps platform will streamline deployment, automatically scale based on your business requirements and provide outstanding performance. The automation dramatically boosts productivity and reduces costs associated with deploying and managing applications.

    NoOps platforms monitor operations and automatically resolve issues using built-in observability mechanisms and intelligent alerts. Centralized logging, metrics and traceability make it easy to identify issues and initiate fixes without expensive downtime. Moreover, the platform should ensure strong security boundaries and adhere to applicable compliance regulations.

    When “going NoOps,” also make sure that the solution provides comprehensive documentation and 24×7 support for emergencies that would normally be handled by an in-house DevOps team.

    Wrapping It Up

    NoOps can eliminate much of the work associated with deploying and monitoring applications by simplifying and streamlining operations and reducing the need for extensive in-house DevOps support. While it’s not a one-size-fits-all solution, it delivers great value helping companies of all sizes from start-ups to enterprise teams save time and money and better redirect their resources toward high-value activities, creating the best code to deliver the highest quality products.