staging-devopsy.kinsta.cloud

Category: DevOps Toolbox

  • No-Code Automated Software Solves a Host of Problems

    No-Code Automated Software Solves a Host of Problems

    There’s a huge disconnect in corporate America, and it’s costing companies big time. The disconnect involves software testing, or, more precisely, the lack of it. More than nine out of 10 CEOs are confident that their company’s software is tested regularly, yet roughly eight out of 10 product testers reported that up to 40% of their software goes out the door with insufficient testing. This dichotomy came to light in the latest Leapwork survey of 500 CEOs and software testers.

    The consequences of untested software are potentially dire and far from hypothetical. Flawed software that delayed the launch of new car models contributed to the dismissal of Volkswagen CEO Herbert Diess, while Facebook saw its market cap value plummet by billions in 2021 when a software glitch came to light. 

    It’s not as if anyone is unaware of such risks: A whopping 95% of CEOs surveyed admitted they’re concerned about losing their jobs if a software failure occurs, while 76% of testers shared that concern. Those feelings are certainly valid, as more than three out of four CEOs reported that software failures damaged their companies’ reputations during the past five years.  

    Despite these realizations, CEOs seem willing to gamble. A full 85% said they considered it acceptable to release software that wasn’t adequately tested, as long as it can be patched later. That attitude has made patching standard operating procedure. More than half the testers said their teams spend five to 10 days a year patching software. 

    Why are organizations willing to take unnecessary risks by releasing software that’s potentially not ready for prime time? There are several common reasons—and a solution that many are overlooking. 

    The Problem with Manual Testing

    One of the major problems is the continued reliance on manual testing, which requires time and resources companies can’t afford. Only 43% of testers said they use automation to some extent in their testing, proving that the majority of testing is still a manual process.  

    That significantly slows product development, an issue for businesses anxious to get their products to market before the competition. Nearly 40% of CEOs said the time-consuming nature of manual testing is one of the primary reasons they release programs that haven’t been properly vetted. Testers clearly feel the crunch, with about one in three saying faster development cycles don’t allow enough time for testing. 

    Given the choice between rushing through testing or delaying a release, companies often roll the dice and opt for the former.  

    Two recent pandemic-related trends have made the situation even more difficult. More organizations are embracing digital platforms to cope with the effects of COVID-19 on their businesses, increasing the need for IT talent. At the same time, upheaval in the labor market has resulted in widespread resignations and job hopping, contributing to a serious shortage of skilled IT personnel. 

    The Coding Conundrum

    Even in this chaotic business climate, experienced software developers remain a hot commodity, with many career options at their fingertips. The competition for talent has left many IT teams scrambling and understaffed, making it difficult to produce high-quality software at a breakneck pace.  

    Eliminating the need for manual testing could relieve the pressure to a great extent. By automating testing, organizations can reduce the burden on staff and minimize the risk of errors caused by deadline pressures. That would allow more testing of more software in less time, which would help move product out the door and into the hands of customers. 

    But automated testing isn’t without its challenges, as many organizations have discovered. Many solutions that are billed as “low code” still demand coding skills beyond the capability of typical business users and testers. That need for coding is an obstacle that can drag down the pace of testing and make it difficult for business users to participate in the process.    

    That challenge grows as the need for ongoing maintenance expands. The test engineers who came on board to set up an automation solution can find themselves bogged down in maintaining code. As a result, speed to market continues to suffer. 

    Democratizing the Test Process

    There is a solution: true no-code test platforms. This type of solution allows both testers and business users to design, run and maintain tests without the need for coding. Business users who understand how the software deliverables are supposed to grow business can be intimately involved in testing, which enhances the quality and functionality of the product and the satisfaction of customers. 

    It also promotes speed. When testing is largely automated and more people are involved, it’s possible to test more software in less time. This eliminates bottlenecks and addresses the issue of scale. The use of automation on a no-code platform has been shown to speed time to market by a factor of 10—an improvement that largely eliminates the need to rush potentially flawed software to market without adequate testing.   

    Organizations can position themselves to meet the growing demand for digital solutions by revamping the way they test software. By eliminating manual testing and adopting an automated platform that doesn’t require coding skills, they can afford to test all their products thoroughly. The end result: They can minimize the risk of bugs that could undermine their success and threaten job security.  

  • Best of 2022: Agile/Scrum is a Failure – Here’s Why

    Best of 2022: Agile/Scrum is a Failure – Here’s Why

    As we close out 2022, we at staging-devopsy.kinsta.cloud wanted to highlight the most popular articles of the year. Following is the latest in our series of the Best of 2022.

    Welcome to The Long View—where we peruse the news and strip it to the essentials. Let’s work out what really matters.

    This week, something a bit different: I’m devoting my entire column to Agile and Scrum. They’re increasingly getting a bad reputation, being associated with the worst aspects of toxic workplace culture: Sexism, racism, burnout, micromanagement. (more…)

  • Best of 2022: Microsoft Adds Raft of Tools and Azure Cloud Extensions

    Best of 2022: Microsoft Adds Raft of Tools and Azure Cloud Extensions

    As we close out 2022, we at staging-devopsy.kinsta.cloud wanted to highlight the most popular articles of the year. Following is the latest in our series of the Best of 2022.

    At its Build 2022 conference this week, Microsoft added a bevy of tools and platform updates as part of its effort to improve the productivity of developers and the DevOps teams that support them.

    A Microsoft Dev Box offering, for example, provides access to a set of pre-configured environments on the Azure cloud that enables developers to be onboarded in seconds, while a previously announced GitHub Copilot tool that makes use of artificial intelligence to make writing code faster will be made generally available this summer.

    Donovan Brown, a partner program manager in the Azure CTO Incubations team, told conference attendees the overarching goal was to enable developers to stay agile while they continue to work remotely in the wake of the COVID-19 pandemic.

    For DevOps teams, a Smart Maps capability was added to the Application Insights module within Azure Monitor. Smart Maps leverages machine learning algorithms to surface performance bottlenecks or failure hotspots and is also available as a technical preview.

    There is also a tool that discovers and groups dependent Hyper-V virtual machines (VMs) and physical servers to ensure all required components are identified and included during an Azure migration.

    Other updates include an App Service Landing Zone Accelerator that provides a combination of artifacts and documentation to automate application deployment using the Azure App Service 3.0 and preview tools for migrating ASP.NET web applications to the platform. In addition, Microsoft is previewing support for Google Remote Procedure Call (gRPC) on the Microsoft Azure App Service. A .NET Multi-Platform App UI framework allows for building natively compiled applications for iOS, Android, macOS and Windows using C# and XAML in a single codebase.

    Microsoft has also extended the reach of its cloud platform for managing application programming interfaces (APIs). The Azure API Management service now supports the Azure Private Link virtual private network (VPN) along with a capability that automatically renews certificates. In addition, Microsoft is previewing support for the GraphQL query language for APIs.

    In addition, a GitHub OpenID Connect (OIDC) with Azure Active Directory (Azure AD) workload identity federation capability has also been added to eliminate the need to store and access secrets outside of the Azure cloud platform.

    Microsoft, in partnership with VMware, is adding a managed Azure instance of the Spring framework for building Java applications and making previews of the Dynatrace observability platform and NGINX proxy software available as software-as-a-service (SaaS) applications on the Azure cloud.

    Finally, Microsoft has added a raft of tools to make it simpler to build cloud-native applications on Kubernetes clusters.

    It’s not clear how quickly developers are adopting these tools, but DevOps teams can expect to see the rate at which applications are developed on the Azure cloud increase in the months ahead. The challenge, of course, will be managing all those applications—both as they are being built and after they are deployed either in Azure or some other compatible IT environment.

  • AWS Saves Ukraine’s Data | WPF ‘is not Dead’ (yet) | Devs Quit for Cash

    AWS Saves Ukraine’s Data | WPF ‘is not Dead’ (yet) | Devs Quit for Cash

    Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.

    This week: Amazon S3 is keeping Ukraine’s data safe, we ask if Windows Presentation Foundation is dead, and developers tell us why they switch jobs. (more…)

  • Rezilion Updates Open Source MI-X Tool to Better Secure App Development

    Rezilion Updates Open Source MI-X Tool to Better Secure App Development

    Rezilion has updated its open source MI-X vulnerability discovery tool to include mitigation and remediation recommendations.

    In addition, the tool can now produce machine-readable output in either a JSON or CSV format.

    Finally, the company added Windows support for Heartbleed and SpookySSL vulnerabilities in Windows environments.

    MI-X makes it possible to use a command line interface (CLI) tool to identify and establish the exploitability of more than 20 well-known common vulnerabilities and exposures (CVEs).

    Yotam Perkal, director of vulnerability research for Rezilion, said Rezilion created the tool as part of its efforts to create a platform that enables organizations to better secure their software supply chains. The primary purpose of the platform is to prevent vulnerabilities from being introduced into applications in the first place by providing developers with the context required to understand the potential severity of a vulnerability, he added.

    There’s no doubt that developers need tools that enable them to easily identify vulnerabilities while writing code. However, shifting responsibility for vulnerabilities entirely left toward developers is not a panacea for application security. DevOps teams need to embrace DevSecOps best practices to ensure that the build created by multiple developers is secure and that the runtime environment where applications run is also free of malware.

    The challenge is that achieving that goal needs to go beyond adding additional tools to a DevOps workflow. The biggest challenge is narrowing the historic cultural divide between developers and cybersecurity teams tasked with discovering vulnerabilities. Most cybersecurity professionals don’t understand how applications are developed, so every instance of vulnerability found in an application looks the same to them—regardless of whether the actual code is externally accessible.

    DevOps teams, meanwhile, are trying to strike a delicate balance between patching innumerable vulnerabilities and building and deploying additional applications. More than anything, those teams need to be able to prioritize their patching efforts based on the actual versus theoretical severity of a vulnerability, noted Perkal.

    It may never be possible to address all the application security debt application environments have accrued over the years. However, as apps are built and application security issues are addressed more proactively, the state of cybersecurity should improve. The issue, of course, is that cybercriminals are now targeting software supply chains; inserting malware into a DevOps workflow creates a potential opportunity to compromise any number of downstream applications. Organizations are now racing against time to secure their software supply chains.

    The good news is that no developer intentionally sets out to build an insecure application. It’s just that far too many of them have been conditioned to think of security as an obstacle they need to overcome rather than address. However, the more tools they are provided to remediate as many vulnerabilities as possible, the better off everyone concerned will be.

  • EU Data Privacy: Try Again | SkyPilot: Choose Cheapest Cloud | WFH: Better Meetings

    EU Data Privacy: Try Again | SkyPilot: Choose Cheapest Cloud | WFH: Better Meetings

    Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.

    This week: EU-U.S. Data Privacy Framework reemerges, Berkeley brings cloud selection box, and proof that remote work is good work. (more…)

  • New Relic Extends Observability Reach to More DevOps Platforms

    New Relic Extends Observability Reach to More DevOps Platforms

    New Relic today expanded its ecosystem by adding integrations between its observability platform and a wide variety of DevOps and cybersecurity tools. The New Relic observability platform now integrates with offerings from Atlassian, Amazon Web Services (AWS), CircleCI, Confluent, GitHub, JFrog, Lacework and Snyk and the open source Jenkins continuous integration/continuous delivery (CI/CD) platform.

    Peter Pezaris, senior vice president for strategy and experience at New Relic, said the goal is to make it simple to share analytics based on telemetry data collected from IT operations environments with the tools organizations rely on to build and deploy applications.

    In total, there are now more than 500 integrations with the New Relic observability platform, he noted.

    Only 27% of respondents to a recent New Relic survey reported that they have achieved full-stack observability and only 5% claimed they have a mature observability practice in place. A third (33%) of respondents also said they still primarily detect outages manually or based on complaints from end users. More than half (52%) of respondents said they experienced high-business-impact outages once per week or more and 29% said they take more than an hour to resolve those outages.

    On the plus side, nearly three-quarters (72%) of respondents expected to maintain or increase their observability budgets next year, with more than half (52%) expecting observability budgets to increase over the next year.

    Observability, in one form or another, has always been a tenet of any DevOps practice. For the most part, the best that could be achieved until recently was continuous monitoring of a set of pre-defined metrics. With the arrival of observability platforms, it’s becoming much simpler to not only monitor metrics but also launch queries that surface issues long before an IT service can be disrupted.

    An observability platform achieves that goal by making it possible to query logs, metrics and distributed traces in a way that surfaces patterns that have not been pre-defined. In effect, an observability platform makes it simpler to debug application environments where the number of dependencies that exist between services are becoming too complex to track or decipher manually.

    Of course, those applications will need to be instrumented using agent software capable of generating the data observability platforms require. Fortunately, open source projects such as OpenTelemetry are helping to drive down the total cost of instrumentation by making agent software more accessible to a wider range of application developers.

    It’s not clear how quickly IT teams will be embracing observability platforms, but there is now a clear opportunity to narrow the divide between IT operations teams and application developers, noted Pezaris. As each DevOps platform makes it easier to collect data via application programming interfaces (APIs), richer telemetry data can be collected, he added.

    In the meantime, as application environments continue to become more complex in the cloud-native era, most DevOps teams will need additional tools. Existing legacy monitoring tools don’t make it easy to discover, for example, which microservice might be adversely impacting the overall performance of an application because calls are being rerouted. The frustrating thing, as always, is that it can take days or weeks to pinpoint an issue that might take only a few minutes to fix.

  • Get Comfortable with Shifting Right to Improve Resiliency

    Get Comfortable with Shifting Right to Improve Resiliency

    The operations team receives a page at 2:00 a.m. Something isn’t working correctly. Do they know how to fix it? Perhaps more importantly, do they know whether it matters to customers or how it impacts the business? There are different techniques that companies implement to maximize the quality of their software. We talk about shifting left or right, chaos engineering and others. While they each have their own goals to improve quality, there’s an underlying common goal as well: All three techniques increase your team’s knowledge of the system and their ability to resolve problems quickly.

    Here, we’ll walk through the software life cycle and discuss the common challenges to software quality, running complex systems and managing dependencies between teams. The common thread is a proper observability strategy which can help software quality, hand-offs between different teams and resolve problems quickly by ensuring whole-system resiliency from design to maintenance.

    Techniques for Improving Quality

    Shift left is a concept that has attracted much attention recently. The practice revolves around the idea that detecting and correcting problems sooner minimizes their impact. As the thinking goes, they’re easier to fix, they wind up costing less and there are fewer ongoing issues with bad code.

    The rise of DevOps and DevSecOps has propelled the popularity of shift left. Yet, there’s a fact that development teams often miss: no matter how early an organization spots errors, problems will inevitably remain. Early testing can’t catch everything. As a result, some organizations have begun to rebel and move to a shift-right framework. It normalizes errors and aims to better resolve issues across the development process, including post-production.

    Shift right is the perhaps less-extreme cousin to its rebellious relative, chaos engineering. With chaos engineering, team members cause problems on purpose to keep everyone on their toes and test the limits of the system. They shut down servers, disconnect wires, delete files, etc. The goal is to confirm system resilience–from a user perspective, there should be no impact even with all of this chaos. A resilient system means that teams can fix failed components at their leisure during the day. With a resilient system, critical failures are less likely to happen and it becomes second nature to fix them when the dreaded 2:00 a.m. page does occur. Unlike chaos engineering, shift right doesn’t explicitly break things–it just makes the assumption that broken things will exist so teams need to be ready to handle them.

    Both concepts (and yes, even chaos engineering) are valid—but each company needs to carefully analyze which is right for them. One variable to consider is the complexity of the system you’re building. Working on a rocket launch or a surgery robot? Probably should shift left as much as possible. Changing the color of a button? Maybe it’s not the end of the world to verify the fix in production. For most of us, projects fall somewhere between these two extremes–so sometimes shifting left is necessary, and other times shifting right is pragmatic. A careful analysis of the risks should be performed to determine what kind of testing is needed.

    Although teams tend to be pretty good at analyzing the complexity of their software and making development process decisions, they often forget the complexity of other companies’ code which they depend on. Even a simple web application deployed today has tons of dependencies: The server it’s running on, the local ISPs, transit ISPs, load balancer, DNS provider, TLS certificate, CDN and browser, just to name a few! After analyzing the complexities of all of the dependencies across the internet, it becomes clear that finding every possible issue that might impact their application is a daunting task.

    The goal, again, is to find any issues before they reach production. Or if they reach production, fix them before they impact customers. Or if they impact customers, fix them as soon as possible after that. To do any of these things quickly, the level of observability of the overall system is paramount. With simple systems—or even complex systems that are entirely under one company’s control—tools such as application performance monitoring (APM) are common. But when the scope of the system is as big as the internet, internet performance monitoring (IPM) tools are necessary instead.

    With both APM and IPM tools, the goal is to know as much about the system as possible – enabling teams to identify problems quickly, pinpoint the root cause, and fix it. When an organization is equipped to monitor development and operational processes effectively from application to delivery, there’s a shift toward higher quality code, fewer errors, improved security and lower costs for the enterprise. There are also fewer disruptions and distractions for engineers, DevOps teams and others.

    Dealing With Disconnects

    Whether shift left or shift right is used, there will be problems that enter production. There is an invisible wall in many organizations–where engineering teams “throw code over the wall” to the team which is now responsible for running it. In some companies, this is the operations team; in others, it’s site reliability engineering or production engineering–the common trend is that in most companies, it’s not the same team who created the software.

    These operations teams often lack the nuanced information they need to resolve production issues. They might spend hours or even days attempting to track down the source of the problem. There’s often finger-pointing between different teams who created or own different portions of the system. Things can quickly devolve into chaos—and tempers can fray. After all, these kinds of problems always happen at 2:00 a.m., right?

    That’s where end-to-end observability enters the picture. Imagine a framework with proof that the development team has performed sufficient knowledge transfer to the operations team. That the operations team has asked all of the relevant questions about running and maintaining the application. Using the right tools, engineering and operations teams need to monitor every aspect and every component of the system. Creating this monitoring strategy means that everyone involved also builds an understanding of the system–everyone is on the same page. All of a sudden, there is a common language to discuss problems, and teams operating the software in production can take debugging a lot further before they need to ask for help.

    Ultimately, shifting left or shifting right is a personal choice for every organization. Even if every possible test case is executed before deployment to production though, issues will happen–so take the opportunity to implement a robust system-level monitoring strategy that can be used to solve technical questions during that 2:00 a.m. call.

  • Shift Left Testing in Microservices Environments

    Shift Left Testing in Microservices Environments

    By now, it’s common knowledge that the later a bug is detected in the software development life cycle (SDLC), the longer it takes and the more expensive it is to fix that bug.

    In 2017, the Ponemon Institute found that it cost around $80 on average to fix a defect detected early in the SDLC and around $7,600 to fix the same defect if it was detected in the production stage.

    Relative Cost of Fixing a Defect Based on Development Stage

    Design

    Implementation

    Testing

    Maintenance

    Cost of Fixing Defects

    1x

    6.5x

    15x

    100x

    While these numbers can be debated, it’s clear that early detection of defects can save organizations significant time and money while enabling those organizations to deliver higher quality software more rapidly. For these reasons, many organizations are implementing shift left testing practices, including in microservices environments.

    Shift left is a set of practices intended to find and prevent defects early in the software delivery process instead of during the testing and maintenance stages. The idea behind shift left testing is to improve software quality by making actionable test signals and feedback available as early in the SDLC as possible.

    Here, we’ll highlight how shift left testing, in combination with test automation, can better enable both innovation and fast delivery of high-quality software. We’ll also address some of the challenges involved in implementing shift left testing in microservices environments, as well as introduce a valuable strategy for successful microservices implementation.

    Shift Left Testing Benefits

    The waterfall model of development involved the explicit passing of responsibilities between highly specialized design, development, QA, and release teams. It also involved lengthy feedback loops.

    Scrum and agile methodologies made the entire SDLC more flexible and nimble by introducing sprints and allowing more frequent iterative development and delivery. Further, DevOps and DevSecOps focus on removing the silos between development, operations, and security through tooling and automation. As a result, the time to market and quality have improved dramatically. Adding shift left testing into the mix better positions teams to handle the broad range of responsibilities from the design stage through the maintenance stage as effectively as possible.

    Shift left testing focuses on prevention rather than detection. Shift left benefits include the following:

    • Increase efficiency by eliminating bugs earlier in the SDLC.
    • Reduce human errors and associated costs.
    • Increase delivery speed and reduce the time between releases.
    • Improve the quality of software.
    • Gain a competitive advantage.

    It may be helpful to think of shift left testing like this: Test early and in parallel with development, instead of the traditional waterfall model of development followed by QA testing. 

    Shift Left Testing Challenges in a Microservices Environment

    By structuring an application as a collection of loosely coupled, independently deployable services, a microservices architecture enables frequent delivery of large, complex applications.

    While shift left testing practices bring many benefits, shift left testing also comes with three challenges that organizations must address when implementing it across a decentralized, microservices-based stack:

    1. Building and managing each microservice separately reduces development time. However, implementing shift left testing at the microservices level means shift left practices must be applied to each individual microservice.
    2. Microservices developers can choose the programming language and service dependencies best suited to their development needs. However, as a result, shift left testing practices need to be implemented and managed across different programming languages and dependencies.
    3. To ensure that microservices function as expected, teams usually run integration tests, end-to-end tests, and performance tests that assess functionality in the context of the larger stack. Shift left testing is especially challenging in the microservices context because of the complexity of testing against the stack.

    Microservices, each with their own SDLC pipeline

    Pipeline Scaling Considerations

    Several classes of automated shift left tests can be integrated into the DevOps pipeline, including unit tests, integration tests, performance tests, and security tests. Different microservices will have different testing needs.

    The key is to apply shift left testing practices to each microservice while maintaining the flexibility and decentralization of those microservices. The ideal workflow system and DevOps pipeline implementation must simplify pipeline creation while enabling development teams to manage the pipeline and requiring minimal input from DevOps and platform teams. In other words, shift left practices must be tailored to the specific needs of the microservice in question to make the testing process scalable and easy for development teams to manage on a continuous basis.

    As just one example, GitHub Actions and similar CI/CD pipeline tools enable development teams to use GitOps-based practices to configure, manage, and administer the entire pipeline for their microservice while leaving infrastructure testing and testing that relies on or affects other parts of the system to DevOps teams.

    Test Environment Considerations

    Test environments are environments where unit tests, integration tests, performance tests and security tests can be run on an isolated collection of microservices and configurations. The nature and quality of these test environments directly correlate with the effectiveness of shift left testing.

    A cloned production or staging environment can be used for testing when the number of microservices is low and overall complexity is relatively low. However, once an organization reaches a threshold number of microservices or a threshold level of complexity, runaway infrastructure costs and operational burdens make such environments untenable.

    In response, some organizations rely upon smoke testing (minimal viable stability testing) and mock-based integration testing early in the SDLC. While such tests can prove useful in the near term, deferring essential testing to a later stage in the SDLC often introduces test signal and validity problems in the staging environment and can lead to delivery delays and serious issues with software quality.

    So how can shift left testing be used successfully in a microservices environment? The solution involves the use of scalable, ephemeral test environments.

    Ephemeral Shift Left Testing Environments

    The availability of high-quality test environments is a common bottleneck for organizations looking to scale shift left testing practices. Using a few high-fidelity staging environments for signal testing can slow the release process. More to the point, it tends to slow down the entire SDLC and increase stack instability over time.

    Ephemeral test environments offer a fundamentally different approach. These sandbox environments use isolated environments that enable users to test independently without impacting other users. In particular, they enable isolation while sharing resources (e.g., external cloud dependencies and hosted databases) using application-level multitenancy instead of the resource duplication approach in traditional environments.

    The idea is that microservice test environments can be used to test changes against a much larger stack of services. Because multiple tests can be conducted at the same time, test coverage and resource efficiency vastly increase.

    High-quality sandbox environments can spin up in seconds and scale to tens of thousands of microservices. In fact, each pull request, branch, or commit can have its own test environment without significantly increasing infrastructure and operational costs. Any change, no matter how small, can be tested in isolation to ensure performance and stability against the entire stack.

    Shift Left Testing Solutions for Microservices

    A microservices-based organization looking to scale shift left testing across the organization must first address several challenges. Considerations include the need to apply testing practices to separate microservices, the need to accommodate different programming languages and service dependencies and the need to test against the stack.

    The right tooling and automation are needed before shift left testing practices are scaled across the organization. In addition, a cultural shift may be required in order to help developers understand the ways in which shift left testing can expedite and improve the core functionality that they are developing.

    To apply valuable shift left testing practices in a microservices environment, organizations should consider using ephemeral sandbox environments. Using the right set of tools in a sandbox environment can enable development teams to test microservices throughout the SDLC. In doing so, organizations can ensure those services follow security best practices and perform exactly as intended, both independently and against the stack–all while reducing costs, speeding time to market and improving software quality and user satisfaction.

  • 5 Tips to Consider When Introducing Infrastructure Automation

    5 Tips to Consider When Introducing Infrastructure Automation

    Although we discuss infrastructure automation often, the fact is that a lot of manual processes still exist throughout DevOps. Whether it’s rewriting scripts to spin up new servers, updating cloud-native policies or configuring computing environments, manual toil is an all-too-common reality. To make matters worse, engineers commonly create lengthy scripts that work in isolation but are too bulky to be reusable by other developers working with slightly different constraints.

    More infrastructure automation is the key to increasing deployment agility, but setting aside time to automate your daily workflow can easily take a back burner, especially when developers are already behind on their long list of to-dos. So, for the time-pressed, where should they start? What exactly should be automated? And how can we effectively reuse and share automation to help other areas of the business?

    I recently connected with John Laffey, technical product marketing director, Puppet. Laffey has thirty years’ worth of tech support and engineering operations knowledge under his belt and was “doing DevOps before it was called DevOps.” He shared with me five helpful tips to think about when introducing more infrastructure automation into your workflows. Below, we’ll expand on each of those points and consider other best practices around automation as the industry moves toward platform engineering.

    1. Automate Things That Interrupt Your Day

    First, Laffey recommended starting with the things that are easy to fix. These are the tedious actions that interrupt your day. Perhaps a disk runs out of space and troubleshooting takes 15 or 20 minutes. Minor interruptions like this might seem like part of the job, but they definitely add up over time.

    2. Automate Things You Can Reuse

    Secondly, try not to automate things for an ultra-specific purpose, said Laffey. Instead, focus on building automation that is more generic and can have more value to others. This follows the DevOps mantra of teamwork and collaboration. One-off automation that is too verbose or specific to a certain condition won’t apply well across groups.

    3. Automate Stuff That Doesn’t Change Much

    Next, automate the infrastructure that isn’t prone to change. These are fairly reliable things that you’re sure will be part of your systems for the foreseeable future. Because if a component does end up having to be updated often, supporting the automation could be more hassle than it’s worth. “Don’t trade fixing problems for fixing automation,” said Laffey.

    4. Start Small

    Here’s some sage advice applicable to most new projects with a big scope—start small. Laffey recommended starting by automating the easy stuff that has proven immediate value. These tasks should be modular and do one specific thing, like rebooting a machine or draining a log file. “Automate things in chunks that you can put into a toolchain.”

    Too often, teams try to automate everything all at once. But by taking big bites, you can quickly get off track and potentially add to technical debt. If you keep things modular, you’ll find it easier to change and reuse this automation.

    5. Share and Be Open

    This last tidbit is more of a cultural change—don’t code so that you’re the only one who knows about it! Sometimes, engineers don’t necessarily want the world to know about the automation they create. They might worry about automating their job away. Yet, Laffey said, these worries are misplaced. There will always be more work to do, and automating toil away can free up time to help you focus on more exciting projects.

    So, Laffey encouraged DevOps and SREs to be open about the automation they create. Ensure they’re well documented and stored in a centralized repository to allow other departments to reuse your code. This can avoid the duplication of having five different versions of the same script written in different languages within the same organization!

    Next Step: Move From Scripts to Industry-Standard Tooling

    It can be challenging to argue for investing time and energy into infrastructure automation, especially when engineers are hard-pressed to keep the lights on. Yet, small gains in automation can make huge waves in the long term.

    Eventually, said Laffey, you’ll want to move away from using scripts within your toolchain, which he describes as a low-end DevOps maturity. But as you update your process, you’ll want to replace these scripts with more modern industry-standard tools like Terraform or Puppet’s DSL. He recommended slowly revamping scripts in a toolchain as you update them one by one. “Start where you are and avoid a monolithic approach.”

    In essence, platform engineering, Laffey explained, is wrapping DevOps principles at a higher level to make them more efficient. And we will likely see an increased emphasis on infrastructure automation as platform engineering gains more mindshare. “It’s going to become a necessity.”