Tag: DevOps processes

  • Applying Automation to DevOps

    Applying Automation to DevOps

    In the book, “Life and the Art of Engineering,” author Haresh Sippy said, “Automation is cost-cutting by tightening the corners, not cutting them.” Today, businesses and organizations are constantly on the lookout for ways to improve productivity while reducing inefficiencies across their operations. Automation has emerged as the natural answer as it addresses these issues while enabling organizations to scale their operations.

    And this is a natural fit with DevOps, which aims to automate and streamline and accelerate the development process from code generation to application performance monitoring. Automation promises better results, greater efficiency and reduction or elimination of errors, but many organizations remain hesitant to deploy it in critical operational areas.

    But there are many benefits to DevOps automation that organizations shouldn’t ignore:

    • Consistency: The most crucial benefit of DevOps automation is consistency. The absence of human effort can remove errors because automation tools repeat processes in the same way, ensuring the development environment remains of consistently high quality.
    • Scalability: DevOps automation not only allows new code to be released quickly but in larger quantities, as well. Automation also allows teams to deliver multiple projects in multiple environments simultaneously, reducing manual processes.
    • Speed: One of the first visible benefits DevOps automation will bring to the table is the sheer speed of the product delivery process. DevOps automation streamlines the configuration process, the development environment and other aspects of deploying new software, making the process significantly faster. With DevOps automation, software delivery becomes less dependent on people. 
    • Flexibility: Flexibility can often be an overlooked aspect of automation. However, with DevOps automation, organizations can change and modify critical code based on new requirements and deploy these changes in real-time, rather than having to train the team to manage the operations based on the new changes. 

    Critical DevOps Automation Processes

    Once an organization is aware of the benefits, the obvious next step is to figure out which processes to automate. 

    While a core tenet of DevOps is to automate everything logically possible, that is a long-term goal. On a more immediate basis, organizations would be well-advised to focus on automating a few critical standard processes. 

    This will allow for a smoother transition as well as more efficient results. But the question of which exact processes to automate is a subjective decision that will depend on each organization’s unique needs and technological requirements. However, some fundamental areas that may benefit from DevOps automation in any organization include the following:

    • Software Testing: Any organization with a digital service or product to offer spends considerable time and human resources testing the new offering or updates before rolling them out. Naturally, automating the testing process would bring greater efficiency to the test results while significantly reducing any errors, especially human errors. Additional tests that may be automated include unit tests, smoke tests and UI tests. 
    • Monitoring: Most organizations understand how important it is to adopt a proactive approach to monitoring. A key aspect of DevOps is consistently reading, recording and maintaining crucial data from regular monitoring. By automating this process, organizations can further increase the extraction of valuable B2B data and insights while ensuring greater security against possible breaches and data theft. 
    • Infrastructure: Infrastructure management is a critical area that serves a vital role in ensuring things run smoothly. This includes networks or servers that are consistently in need of maintenance, configuration modifications and initial setups. By deploying an automated infrastructure, organizations can increase efficiency while freeing up valuable human resources to be deployed elsewhere. 
    • Continuous Integration/Continuous Delivery (CI/CD): CI/CD’s entire purpose is fast application development and delivery. By its very nature, this task is ripe for automation as it requires testing and monitoring every little change or modification. Similarly, automating the deployment phase of CI/CD would ensure the distribution of successful updates is carried out proactively the very second it is ready. As a result, organizations will see updates being rolled out more efficiently with minimal errors or bugs in the final updates.
  • 5 Ways to Reduce DevOps Toil

    5 Ways to Reduce DevOps Toil

    Over the last several years, DevOps has become a bit of a buzzword. It has become simultaneously a practice, a culture, a team, a job title and a vendor product. You can hire some DevOps, buy some DevOps, adopt DevOps and sprinkle a little bit of DevOps on top for good measure.

    But, at its core, DevOps is about service ownership: having a single team with end-to-end ownership of software running in production. You should not have one team building software and another team deploying and operating it. That’s slow. DevOps is literally combining development and operations into a single consolidated team. DevOps means embracing the mantra of “you build it, you own it.”

    The primary benefit of DevOps is increased speed and agility of engineering teams. With DevOps, exactly one team is responsible for a given piece of production software. There are no handoffs or knowledge silos. Practices like continuous delivery can emerge. Features can be put in front of your customers’ eyes with a shorter turnaround time. A secondary benefit is increased reliability and security. With “you build it, you own it,” it is no longer someone else’s job to ensure that software is correct or performant or reliable or secure. These responsibilities are “shifted left” onto exactly one team.

    So, where do things go off the rails?

    Service ownership and DevOps mean giving teams the agency to make whatever changes are necessary to their software. That can and will come at the expense of product functionality at times. So, the first place things go off the rails is when there is not a healthy balance between product needs and engineering needs. If teams are constantly working on new features but do not have the capacity to fix operational issues that are affecting them, product quality will eventually suffer. Ultimately, product-engineering imbalance generally comes from misaligned incentives or poorly defined goals from management.

    The second place where things go off the rails is insufficient training. Especially with organizations early in their journey to adopting DevOps, which may have separate development and operations teams, there is a lot of training on new technologies and processes. Developers need to learn about observability, responding to incidents and debugging production applications. Operators and sysadmins need to learn about coding and the principles of software design. These are new skills for both camps and investment in training and mentorship is required.

    What are some best practices for keeping DevOps on track?

    There are lots of these best practices. It really depends on your starting point as an engineering organization, but here are five important ones:

    1. Track Service Ownership

    If DevOps is about “you build it, you own it,” you need an authoritative list of a) what’s running in production and b) which team owns it. If a piece of software goes down, you do not want to find out that the last person to work on it was the intern who left three years ago. Many folks start tracking this kind of information in spreadsheets or wikis, but keeping the data up-to-date becomes challenging. SaaS service catalogs have emerged as the easiest way to automatically keep this data accurate, complete and consistent.

    2. Adopt Continuous Integration, Continuous Delivery and Feature Flags

    Committing small changes multiple times per day is superior to release trains where one large change is committed on an infrequent basis. CI/CD allows for new features and improvements to be tested faster and put in front of customers more quickly. Feature flag tools let you easily test new functionality and product hypotheses for different segments of customers.

    3. Embrace Incident Management

    Software breaks sometimes. Having a robust process for how to triage and respond to both alerts and incidents is important. Within the context of DevOps, the team that owns a piece of software should be responsible for being on-call to respond to associated incidents. If the software breaks or behaves in some unexpected way, the team is expected to fix it. The team is also responsible for identifying what happened and investing in ways to ensure the same error does not occur again.

    4. Treat Everything as Code, Then Automate all the Things

    Applications and microservices? That’s code. Infrastructure? That’s code, too. Security policies? Also code. Deployments? You guessed it–code.

    Treating everything as code allows for processes like “code review” to emerge beyond just your applications. You can have multiple eyes on every change, which catches bugs sooner and spreads knowledge to others. As well, version control systems (like Git) are a natural audit log, allowing anyone to see what was changed, when, and by whom.

    With everything defined as code, the next step is to invest in automation. How many steps does it take for a developer to deploy new code to production? Or to spin up new infrastructure? Or to hook up monitoring or error tracking tools?

    Your goal should be “one step:” run a command. It can be a shell script, a Slack bot or a button with a nice web UI. But it should always be one step. If there are more, you’re now in “manual steps” territory, and that’s brittle. You have no guarantees that successive individuals will run whatever sequence of steps in the same order, which can lead to divergence in how various parts of production are set up.

    Another example is maintaining a list of a) what’s running in production and b) who owns it. Manually keeping that list up-to-date is impossible as the size of your production infrastructure or engineering team scales. You’re better off investing in service catalog tooling that can automatically integrate and capture this information.

    5. Invest in Reducing Toil

    Service ownership and DevOps are two-way streets between management and their engineering teams. On one hand, consolidated ownership will lead to faster delivery and more reliable software. That’s good for management. But, on the other hand, teams require actual empowerment and agency to make decisions about their software. Part of that is having capacity on their roadmaps to make improvements to issues that are causing toil.

    For example, if a particular service is paging a lot or experiencing load issues, service ownership means giving the owning team the time to make any necessary changes. That will come at the short-term expense of new production functionality.

    There’s no free lunch on this last point. Engineering management can’t just combine dev and ops responsibilities and magically expect results. Service ownership is a two-way street. Otherwise, toil will kill your teams in the long term.

  • Report Identifies DevOps Skills in Highest Demand

    Report Identifies DevOps Skills in Highest Demand

    When it comes to DevOps there’s general agreement that a skills shortage is holding back broader adoption. The real issue, of course, is identifying what specific DevOps skills are required most.

    With that goal in mind, the DevOps Institute this week published a report that finds the DevOps skills most in demand fall into three broad categories: automation skills (57 percent), process skills (55 percent) and soft skills (53 percent).

    DevOps Institute CEO Jayne Groll said that, given the fact that DevOps is all about automating repeatable processes, the fact that automation skills are the most sought-after is not all that surprising. But when it comes to process skills, the top five areas where organizations need the most help are software development life cycle (47 percent), understanding of process flow and analysis (46 percent), agile development expertise (42 percent), experience with test-driven deployment concepts and methods (40 percent) and system thinking (39 percent).

    The survey also identifies areas where survey respondents need additional functional skills; most notably, IT operations knowledge (52 percent), security (47 percent), IT infrastructure (44 percent), application design and development (42 percent), quality assurance (34 percent) and testing (31 percent).

    While the survey makes it clear what skills are needed most, it’s less clear whether organizations want to train people with the right attitude toward DevOps skills or find individuals with proven skills. About 30 percent of the survey respondents said they look for soft skills first, with an eye toward training or educating new hires on the process and technical skills later. But 32 percent of respondents said they first look for technical skills.

    In that context, however, it’s also worth noting that 43 percent of the C-level executives and IT management executives that participated in the survey identified business skills—defined as communication, influencing, negotiation, strategic thinking and leadership skills—as being the most important attribute of a new hire.

    Other noteworthy attributes of the organizations participating in the DevOps Institute include:

    • Two-thirds (66 percent) have already adopted DevOps to some degree.
    • More than three-quarters (76 percent) employ agile development processes.
    • Nearly half (47 percent) have adopted the ITIL framework as a best practice for IT service management.
    • Only 10 percent rely on site reliability engineering (SRE) practices defined by Google.

    Groll said the survey makes it clear there is still much work to be done in terms of advancing best DevOps practices. One the biggest challenges are all the silos within IT organizations that have their own nomenclature and culture. Each of those silos creates technical and cultural inertia that must be overcome to integrate processes end to end, she said.

    On the plus side, Groll noted the next-generation of IT professionals are being taught best DevOps practices in academic settings, which means the pool of job candidates with familiarity with DevOps practices should increase steadily. The downside is that it might take a while before those individuals learn enough about the business of the company that hires them to effectively employ those DevOps practices.

    — Mike Vizard

  • A Definition of DevOps for the Masses

    A Definition of DevOps for the Masses

    If you asked the average person in the street, “What does DevOps mean to you?” you’d probably get a blank stare. And there are those who aren’t directly involved but are vaguely aware of DevOps as a concept. While a vague understanding is better than nothing, it leaves room for misconceptions and misinformation, which can be problematic in the long term. An accurate definition of DevOps is necessary.

    How can you provide an accurate definition of DevOps for a non-technical audience? Long-term DevOps practitioner Alex Honor echoed earlier agile software development principles when he coined the phrase, “People over process over tools.” This mantra provides a handy list of DevOps priorities, and as such offers a useful framework for an easily understandable definition of DevOps.

    Start with the People

    Everyone loves a good organizational chart, and discussing how DevOps teams organize themselves is a useful way to understand how DevOps should work in practice. DevOps is very much about bridging traditionally separate development and operations departments, but this involves more than simply increasing collaboration between teams.


    Related Content:

    5 Biggest Ways to Fail at DevOps

    DevOps Gets More Exciting in 2018


    In a traditional setup, developers build applications and fix them when necessary, while operations teams work to maintain, support and monitor those applications.

    In DevOps, the wall dividing these two worlds disappears. But it’s not just about getting dev and ops teams to work more closely—it’s about creating an environment where developers and engineers work alongside each other at every stage of a product’s life cycle. In other words, there should be no separation between dev and ops, just a unified DevOps team.

    By emphasizing the people factor and contrasting a DevOps team structure with more traditional arrangements, it’s easier to visualize DevOps playing out in the real world. This is a great place to start when it comes to demystifying DevOps for a less-technical audience.

    Move on to the Process

    When put into practice, DevOps is supposed to be faster, more efficient and more reliable than more traditional software engineering methodologies—once the organizational structure is in place, these benefits can be achieved via innovative DevOps processes.

    In traditional development and ops, manual tasks have a habit of slowing down processes, and getting in the way of high-frequency deployments. But in DevOps processes based on infrastructure as code (more on the tech later), teams can automate the provision and management of resources.

    Take the task of provisioning server resources, for example. In the past, the developer would email the infrastructure team and ask for a certain set of resources in a specific configuration. Then someone in the infrastructure team would look at a document and follow a checklist of tasks before finally deploying the server.

    In a DevOps process, not only are developers and infrastructure engineers working alongside each other in the same team, they’re also implementing processes through infrastructure as code (IAC).

    Within the context of IAC, the provisioning of IT resources is automated via code, so rather than provisioning servers manually, the developer simply clicks a button. The desired infrastructure then spins up automatically, accelerating the overall process and giving DevOps teams the means to deploy far more frequently and efficiently.

    By describing the automated processes made possible by IAC, the differences between DevOps and more traditional practices become much clearer. For less-tech-savvy employees, this explanation provides some much-needed context in terms of the day-to-day working of DevOps teams.

    Then Talk About the Tools

    Technology, and the benefits it brings in terms of automation and management, is a key driver of DevOps. The maturation of cloud computing and virtualization, and the emergence of infrastructure as a service (IaaS) in particular, provides the flexibility and speed demanded by DevOps.

    With IaaS, it’s possible to instantly provision computing resources for highly customized and constantly shifting IT requirements, without on-premises hardware or manual configuration. This in turn enables the infrastructure as code (IaC) processes detailed above, and the level of automation required for DevOps teams to function.

    So cloud computing, in the form of IaaS and by extension IAC, lays the technological foundations of DevOps. That’s the big-picture stuff, but what about those brand-name tools that non-technical people might have actually heard of? Docker, Kubernetes, GitHub, Jenkins, Travis CI … the list goes on, but how vital are these to the core concept of DevOps?

    Arguably, not very. While containerization, source control and continuous integration tools all can perform very important roles within a DevOps context, they’re hardly defining features of DevOps itself.

    Continuous configuration automation solutions such as Puppet and Chef come more directly under the IAC umbrella, enabling the automatic configuration and scaling of data center resources. But like the others, these tools would be equally at home in a more traditional setup.

    In this sense, there’s no such thing as a “DevOps tool,” just tools that can be used within a DevOps organization for DevOps processes. It’s how you use them that really counts.

    So when offering a definition of DevOps for a novice audience, don’t fall into the trap of simply reeling off a list of tools. DevOps is far more about people, organization and process. And while it’s very reliant on IaaS and IAC technologies, many of the tools utilized by DevOps teams are more or less incidental.

    And Finally, a Reminder that DevOps Works

    The ultimate goal of DevOps isn’t simply to unify development and operations, but to result in shorter dev cycles, more frequent deployments, higher-quality products and happier customers. More bluntly, the ultimate goal of DevOps isn’t just to say, “We do DevOps,” but to increase productivity and make the business more profitable.

    Just look at sprawling organizations such as Google and Amazon. The success of these companies hinges on their ability to deploy new software multiple times every day, or even every hour. Amazon and Google may not agree on a precise definition of the term, but what they do agree on is that DevOps, as a set of unified software engineering practices, is crucial to their profitability.

    With a consensus on the broad strokes of DevOps emerging over the last few years, lack of definition of DevOps will hopefully become less of an issue as its culture crystallizes. In the meantime, the good news is that the big ideas behind DevOps are also the most straightforward to explain to a non-technical audience.

    By breaking DevOps down into people first, process second and tools third, an accurate and relatively simple definition of DevOps is possible. DevOps is fundamentally about people, how developers and engineers organize themselves and work as a unified team to deliver better products faster. It’s on top of this structure that DevOps processes are built, with a heavy focus on automation and innovative tools—always within the context of the core DevOps principles and practices.

    — Neal Thoms