staging-devopsy.kinsta.cloud

Tag: toolsets

  • So Many DevOps Tools, So Little Time

    So Many DevOps Tools, So Little Time

    I’ve received thousands of emails, ads and sales pitches for tools for every stage of the app life cycle, from automating infrastructure buildout to release planning to building and testing, to delivering applications. If you’re building or rearchitecting an application, you have a surfeit of choice of DevOps tools.

    Ironically, when an organization invests in various DevOps tools, it can lead to disconnected teams that are siloed by tooling. Gartner calls this “disconnected islands of automation.” In addition to a culture problem, it can lead to major incidents and downtime.

    You can’t tool yourself into DevOps. But here’s how to make sure you’re not hurting your team with the wrong tools.

    Step 1: Process Before Tools

    Find out what problems you need to solve, then find tools to fit that need. It sounds simple, but many companies do the opposite: they’ve already picked the tool, and then use it to attack every problem.

    “If all you have is a hammer, everything looks like a nail.” If all you have is one or two main tools, every problem gets fixed by those tools. Our team has been guilty of this, too. When you get really good at one tool, such as Puppet or Chef, the answer to every question is inevitably Puppet or Chef.

    For our team, the fix has been to create process and architecture diagrams without tool names. Instead, we describe what needs to happen at each stage. If I know I’m building a long-running virtual machine, I know I need a configuration management component. Once we agree on the process, then we discuss tools.

    Engineers are practical people who are trained to be constrained by the tools we use. To break out of this mindset, write out your desired code pipeline without tool names, even if you “know” the DevOps tools you’re going to use.

    Step 2: Give Equal Priority to Infrastructure and Application Tools

    A software company recently told me an unhappy story: The company spent years perfecting its continuous delivery pipeline and everything worked together like clockwork. Developers could test and run code in minutes.

    The development team knew how to push code, but nothing was reachable in their region. Some of its delivery tooling was running in the same cloud region, so even though other regions were up, the tools the team wanted to use to perform failover weren’t available. When the team went to fail to DR, the configuration for its DR infrastructure had not been kept up to date with production, so instance sizes were not sufficient to run the application.

    The moral of the story here is that your infrastructure and application need the same level of automation and agility. Even if you have a very mature CI/CD pipeline with gating, approvals, linting, etc., if you don’t have that same discipline and tooling around your infrastructure stack, then you’re equally down. You should have the ability to move your application from one region to another (and if you’re really ambitious, from one cloud to another). Also, if you’re not incorporating application development into infrastructure development, over time your infrastructure will grow out of date.

    You should be asking questions such as:

    • What steps can we take to automate infrastructure provisioning?
    • How can we continually test failure?
    • What steps can we take to survive a platform failure?

    Step 3. Integrate Infrastructure and Application Tools

    The best way to survive a platform failure is to continually “practice” building out entirely new infrastructure and application stacks—as one continuous process, not as two different activities.

    Here’s one extreme example. The team at Logicworks recently worked with a large enterprise software company that was launching an internal product with a custom deployment pipeline. The pipeline included dozens of parallel automated and manual tests, each in a separate development environment. When a new deployment to the development environment occurs, a new set of 45+ dev instances must be created that has the new version of code; when the test is finished, the instances must be terminated. This means that instances in their environment rarely last for longer than 24 hours, and over the course of a single week, hundreds of instances are terminated and rebuilt.

    As you can imagine, this company has never experienced an infrastructure outage it couldn’t recover from, since it already recover from “failure” hundreds of times a week. This is the true meaning of immutable infrastructure: virtual instances are disposable and once you instantiate the infrastructure and code, you never change the instance. This way the infrastructure never strays from its initial, known-good state, operations are simplified and the system is so good at replacing itself that failure is a non-event.

    Ideally, infrastructure automation and application automation are not different stacks with different teams, but the same stack with the same level of destructive testing, so that an advance in your application needs to be tested against the current state of your infrastructure.

    Summary

    The constant release of new DevOps tools and features can be overwhelming. But the biggest challenges for your DevOps team probably have nothing to do with CI/CD tools. I have found that it’s easy to overemphasize tools and deprioritize crucial (albeit less glamorous) automation components such as infrastructure templates and testing.

    Your developers are constantly looking for new libraries and experimenting with new languages and tools for your applications. You need the same level of attention and testing in your full infrastructure and application stack. This will make you choose better application-level tools and prepare you for failure.

    — Jason McKay

  • IBM: DevOps Begins and Ends with Culture

    IBM: DevOps Begins and Ends with Culture

    Making the transition to DevOps requires as much a shift in cultural mindset as it does new platforms for building, deploying and managing applications. Between the two, however, it’s often the extent of the cultural challenges needed to be overcome that gets most underestimated.

    Michael Elder, a IBM Distinguished Engineer, said one of the most important DevOps philosophies that organizations must embrace is that No Code is Sacred (NCIS). NCIS fosters an experiment-driven culture that encourages self-editing and promotes a “don’t be afraid to fail” mentality. This approach allows teams to form questions, quickly implement means to gather data, try alternatives and incorporate real user behavior and feedback, he said.

    Manage the Scrum (Meeting)

    Other cultural issues that need to be addressed include leadership and how daily scrum meetings are managed.

    An overzealous build champion can wind up adding technologies to the continuous integration/continuous deployment (CI/CD) platform that don’t get adopted simply because the team as a group did not agree to their inclusion, said Elder.

    Daily scrum meetings don’t necessarily need to take a full hour. Non-critical, time-consuming status updates should be eliminated from the agenda. Daily scrum is more about problem-solving and looping the wider team in on critical decisions than talking about updates most team members can already read about for themselves, Elder noted. The goal of the meeting is not necessarily to inform team members of events. They should be expected to know what’s occurred. The real goal of the meeting is to decide what to do next.

    Mindsets Matter

    Elder said DevOps teams be cognizant of Conway’s Law, which says that organizations that design systems are constrained to produce designs that are copies of the communication structures of these organizations. If the goal is to develop a more agile application environment by employing microservices, Elder said, then the organization should create smaller, more independent teams to construct and manage them. However, that doesn’t mean every monolithic application needs to turned into a set of discrete microservices.

    Elder also advised organizations to embrace Design Thinking principles that encourage DevOps teams to focus on what needs to be achieved instead of enabling processes. The focus ultimately needs to be on the experience of the users who are being asked to employ an application, he said.

    The Tools Issue

    The biggest cultural challenge, however, still may be to tooling. Organizations trying to embrace DevOps clearly need new tools. But acquiring new tools won’t drive meaningful change. Too many organizations buy tools in the hopes that processes will eventually be transformed. The decision to acquire new tools needs to be closely connected with the transformation of processes, said Elder. That said some tools, such as an enterprise instance of Github, are core to encouraging the sharing and reviewing of code within a DevOps process, he added.

    Of course, Elder noted the most critical cultural change as far as tooling is concerned is keeping an open mind. Developers and IT operations teams tend to get overly attached to specific programming languages and platforms. A true DevOps culture, above all else, encourages experimentation.

    — Mike Vizard

  • DevOps Gets More Exciting in 2018

    DevOps Gets More Exciting in 2018

    As 2017 winds down and the IT industry prepares for 2018, there are a number of DevOps trends that everyone should be keeping their eyes on. It’s not so much that something incredible and explosive is expected in the new year. Rather, 2018 will be a time when DevOps starts to mature alongside other trends important to developers.

    These trends don’t guarantee organizations will be immediately successful with DevOps. Most still will struggle with the management aspects of DevOps, especially the need to keep feuding silos within IT working together as one happy group. Assuming developers, QA, and Operations can all learn to get along and act as a team, the technology supporting DevOps will continue to improve and the ecosystem will grow.

    Here are some of the DevOps trends that Amalgam Insights is following for 2018:

    Everyone Loves Containers and the Ecosystem Grows

    Containers have become a common technology in many IT departments. Although the majority of applications remain housed on single servers and virtual machines, container deployments are growing. Containers are an important enabling technology for DevOps—by creating small, transportable bits of applications, containers make it easy for IT silos to work from the same page.

    In 2018, growth in container deployments is expected to accelerate as large companies outside the software industry, such as those in financial services and health care, adopt microservices infrastructure. Containers that are especially suited to microservices will continue to expand rapidly.

    Increased container growth is also due to an expanding ecosystem. More companies have entered the market with products necessary to large-scale production applications. Products now exist for securing, managing, deploying and enhancing container services that weren’t available a few years ago. What’s more, containers are now available in all the major cloud services including Oracle Cloud, Microsoft Azure, AWS, IBM Cloud and Google Cloud. Containers are becoming enterprise-ready and large enterprises are feeling more confident using them.

    Takeaway: Growth in the ecosystem contributes to growth in container deployments while expansion of containers entices companies to enter the ecosystem. Containers have reached mainstream normalization and we expect to see fast expansion throughout 2018.

    Kubernetes Eats the Container Orchestration World

    In the container orchestration space, Kubernetes has become the tool of choice. Orchestration is the deployment and management of container clusters and is necessary to deploy them at scale. Kubernetes, originally a Google open source project and now managed by the Cloud Native Computing Foundation, has become the most popular orchestration tool available. That became obvious when Docker decided to distribute and support Kubernetes, despite having invested in its own orchestration system called Swarm. CoreOS, Microsoft Azure, Red Hat OpenShift, Docker and others have their own enterprise distributions as well.

    Kubernetes brings to the container ecosystem a relatively easy way to manage containers at scale. In large container deployments, containers are constantly being spun up, shut down, moved and copied. Once container clusters are in place, they must be monitored for security and service failure.

    During development, there will be a number of clusters that must be managed for development, test and final release to production. Doing this manually is close to impossible when even a moderate number of containers are in the DevOps pipeline. Kubernetes provides the tools and workflow to simplify these tasks.

    Takeaway: As container production clusters grow and as they are integrated into the DevOps pipeline, the need for orchestration becomes acute. Kubernetes has won the game and is now the tool of choice for orchestration. 

    The Pipeline is Getting More Integrated

    DevOps is often described of as a “pipeline,” In this context, a pipeline is a linear workflow that moves from build to test to release. As DevOps becomes more common in organizations, the pipeline grows to include additional test phases, monitoring and continuous improvement to create more of a cycle than a linear flow.

    One of the great impediments to making DevOps a reality is a lack of tools that support the entire pipeline. There are plenty of tools that help build applications, such as IDEs and code repositories from companies such as IBM, Microsoft, GitHub and Oracle, as well as dozens of open source products. Testing is well-supported with test data management and test plan management tools from companies such Atlassian, Informatica and Oracle. Release management and deployment products are widely available from Atlassian, Microsoft, IBM and many other companies. The problem lies not with the support for the individual steps in the pipeline, but with the integration of these tools to provide a complete end-to-end workflow. This creates a disjointed workflow, prone to errors and supportive of silos instead of collaboration.

    This is changing, however, and announcements late in 2017 herald toolsets that can provide a continuous DevOps pipeline. A great example of this is the newest versions of Microsoft’s developer tools that were debuted at Microsoft Connect(); in November 2017. Tying together the Visual Studio debugger with Kubernetes allows developers to test and debug as close to the production environment as possible and then deploy cluster to the test environment. That, plus the upgrades to Team Server, shows the commitment Microsoft has to creating integrated DevOps systems.

    Takeaway: While DevOps tools are still more or less disjointed, we are seeing a movement to fully integrated toolsets that enable the end-to-end DevOps pipeline.

    2018 is the Year of DevOps—Unless IT Management Gets It Wrong

    This coming year could be the year that DevOps truly takes off and becomes mainstream. Those who believe it is already mainstream now are fooling themselves. Management may believe their organization is implementing DevOps, but too many barriers still exist—especially management itself. IT departments remain heavily siloed and independent of each other, as do development tools. DevOps at scale requires, above all else, a collaborative and cooperative approach. Sadly, that does not yet exist in many IT departments. As tools reach maturity and best practices become common, managers become the inhibitors of DevOps. New management structures, incentives and thought processes need to be in place or the money spent on tools is wasted.

    Takeaway: All the DevOps pieces are in place except for the breaking down of functional silos that exist in IT departments. 2018 may be the year of DevOps if IT managers can create a new environment of cooperation.

    About the Author / Tom Petrocelli

    Tom Petrocelli is a contributing analyst with Amalgam Insights. His area of interest is collaboration and new ways of work, developer tools, IT project efficiency, governance and methodologies and DevOps. Most recently, Tom worked for a large, global banking corporation. Previously, he was the research director for Enterprise Social, Mobile and Cloud Applications at Neuralytix. He is an experienced marketing, technology and business executive with 30+ years in the computer technology industry. Tom’s background spans software engineering, systems architecture, IT, product management and marketing and general management. Connect with him on LinkedIn and follow him on Twitter.