staging-devopsy.kinsta.cloud

Tag: ecosystem

  • Rugged DevOps: Good, Bad & Ugly

    Rugged DevOps: Good, Bad & Ugly

    The Good: DevOps has been a success for Dev.

    With higher expectations, better processes, more management focus and new tools like advanced PaaS and containers, developers are churning out more and more software faster than ever. Success! Or is it?

    The Bad: DevOps has been a success…for Dev.

    For Dev teams, this may result in pats on the back and high fives. But for Ops, this can look like one thankless tidal wave of work after another. The same software velocity that is praised in Dev is currently overwhelming their counterparts in Operations. And frankly, many DevOps tools released in recent years happen to benefit Dev much more than their counterparts in Ops. (My pet theory is that developers at IT automation firms are acting and thinking like…developers – focusing their creative energies on solving Dev problems.) A similar problem holds true for IT Security teams who want to build quality in by “shifting left” – see  https://staging-devopsy.kinsta.cloud/2015/12/10/getting-rugged-devops-right/ . From the vantage point of IT Operations and Security teams, they are doing more work than ever but without any more human or IT automation resources. To add insult to injury amid their increased workloads, many IT Ops teams have to invest countless hours (away from family and personal time) in training courses and migrations to reorient their processes to fit into shiny, new Dev-centric “DevOps” tools and paradigms. 

    The Ugly.  

    Work. Overwork. Glitch. Search. Find. Rework. Repeat.

    A quick glance at IsItDownRightNow.com will show you which IT Operations teams are in a living hell at the moment. The other thing you will notice is that these IT Ops teams are from some of the most reputable companies on the planet. They are talented teams but stuck in rework cycles of web performance problems, security vulnerabilities and outages.

    These problems get the attention of everyone. Operations…and Dev…and Security…and customers…and Finance…and auditors and the press.

    No one likes rework. And no one likes the panicked fire drills that cause panicked rework. So what causes those rework cycles?

    Drama-free IT. Visibility Cures Chaos.  

    Much of this chaos can be tied to miscommunication. Boring? Maybe. Simple? No. Part of miscommunication is cultural. (Hint: OrcaConfig’s “How to Screw Up DevOps” white paper covers the cultural aspects of miscommunication). But miscommunication is also cured by automation solutions that promote transparency and visibility.

    If we want drama-free IT we need IT automation solutions that eliminate miscommunication and missed assumptions between Dev, Ops and Security.

    What better way to do this than with full transparency and visibility?

    Bird’s Eye Views:

    Many IT automation tools cater to SysAdmins who are interested in managing configurations of individual servers. That’s great if your focus is on individual severs. But IT Operations, eCommerce teams, Application Owners, Compliance & Security teams and IT Managers often care more about the health and performance of their application ecosystem (including related applications, middleware, databases and configurations that tie them together). So there is often a mismatch between the tools that SysAdmins and Developers have versus what IT Ops needs.

    Operating at Enterprise Scale Requires at-a-glance Visibility:

    IT Ops teams need bird’s-eye views of applications, databases, middleware and ecosystems to promote ease of ongoing operation. They need to be able to see, at a glance that a configuration compliance problem exists and they need to know right away the location and nature of the error. Large scale IT Ops teams do not have the luxury of toggling back and forth between multiple reports and multiple screens searching for that non-compliant, broken needle in an ecosystem haystack. If they are faced with a glitch or an outage, they need to find and fix it right away. One size does not fit all. Dev, Ops and Security each have their own interests and their own need to view their application through a lens that is most meaningful for them. Multiple ecosystem views eliminate guesswork between Dev, Ops and Security and save precious time in the find-fix cycle.

    Forrester Research spells it out well in their 7 Habits of Rugged DevOps white paper, Habit 1: Increase Trust And Transparency Between Dev, Sec, And Ops:

    “I&O pros pride themselves on understanding the interconnectedness of the hardware and software environment, and they are unnerved by complexities that poorly governed delivery practices create because the firm rates I&O pros on their ability to improve the performance and stability of the production environment. To maintain the environment, they must reduce the chances of an outage or interruption of service, and they must reduce the MTTR when an issue does occur.“

    Understanding that interconnectedness depends greatly on how much intuitive ecosystem visibility we provide to our colleagues in IT Operations. But it’s not just IT Ops that benefits from visibility, Dev and Security do too.

    Developers are able to write more secure code when they get early, in-context feedback. As David Mortman, distinguished engineer and Dell chief security architect, said, “Security issues are product quality issues, and no one wants to write buggy code.”

    Even better than efficiently finding and fixing errors is of course to prevent them in the first place and to proactively eliminate the damage they cause. Here too, the answer is better visibility. With true application ecosystem visibility, Dev and Ops teams would be able to model relationships between applications, middleware and their configurations so users will know that “changing this, affects that”.

    My teenage son has better visibility via his smartphone as to his driving route and weather conditions than I have for my multibillion dollar IT operation.

    Visibility into the Future:

    Ecosystem visibility is a key to communicating between Dev, Ops and Security and running complex application environments in the present. But what about the future?

    Release automation tools and custom scripts are excellent at deploying changes across large environments. Unfortunately they can be just as efficient at deploying typos and other mistakes across large environments, causing the chaos that we discussed earlier. Here again Operations teams would benefit from improved visibility – the ability to preview and pre-validate proposed changes across the entire ecosystem well before those changes are deployed. This is the real promise and savings of ecosystem visibility. Better visibility, with more use-case lenses cuts out the miscommunication that causes too many performance glitches, security vulnerabilities and outages. This chaos takes a toll on our IT Operations teams, our employers and our customers.

    What Next?

    We are soon entering a new season with new budgets. The temptation to “buy something” will be great. In the spirit of Rugged DevOps my recommendation is to take a full, bird’s eye view of your SDLC and to shift Ops and Security leftwards, building their critical considerations into your processes and into your IT automation tool selection.

  • The 3 practical change elements in enterprise DevOps

    The 3 practical change elements in enterprise DevOps

    Culture, Technology & Ecosystem

    Most enterprises approach DevOps adoption in a singular fashion: they use a bottom-up approach that is typically technology-driven and groups all considerations like, culture, metrics, processes and communication into a single stream of work that is often led and implemented solely by technology-oriented champions.

    However, this singular approach presents a few issues:

    (1) First, it leads to poor planning on the change aspects of DevOps that are not (and should not be) technology driven – although we (DevOps Evangelists) are not change experts, we believe we can properly assess impact & lead organizational change just because we are the subject matter experts on the technical topic (DevOps). Accordingly, we miss the wider context and relationships with non-IT functions that would be impacted by DevOps.

    (2) Second, it naturally focuses the efforts on the areas inside our comfort zone (technical aspects) and potentially disconnects the DevOps program from the Enterprise priorities. If it didn’t, it limits DevOps to a technical change of tools and automation.

    (3) Third and most important, it contributes to the absence of realization that the changes in areas outside of IT is critical to the success of DevOps adoption and that specialist (internal or external) help in these areas may be required and better yet, often acts as a catalyst for change.

    The consequence of the 3 points above is that post the initial phases, enterprises reach an incomplete point of “DevOps Equilibrium”; this is the state that enterprise IT reaches after installing some automation tools, setting up a basic practice and changing some behaviors. It takes a lot of effort, dedication and pressure on the organization’s DevOps evangelists to make this initial launch and bring the beginnings of DevOps into their Enterprise. By the time they reach that point, energies are depleted, the bigger challenges of enterprise politics and ecosystem become apparent and the thought of another push to bring in more teams, departments, tools and stakeholders on board becomes a burden.

    The question then becomes, why is it a struggle and why is it difficult to adopt DevOps in an Enterprise? The reason is that the singular, technology-driven approach that “considers” the soft aspects of DevOps fails to leverage the value it delivers by transforming it into executive sponsorship and political influence; and the reason this happens is because executives consider a bigger enterprise view that stretches outside of their departments and outside of IT but the technology-driven, singular approach has a narrower view and fails to address the ecosystem of the enterprise that surrounds the technology and the culture, and accordingly all the executives’ concerns.

    In my experience, one of the main reasons DevOps evangelists in Enterprises fail to move beyond their point of equilibrium is because they approach DevOps as a singular, technology change not as an enterprise change.

    What I have seen from multiple enterprise DevOps journeys is that Enterprise DevOps adoption should be broken down to 3 main “Change Elements” and DevOps Evangelists should plan for each “change element” separately under a single program umbrella:

    1st Change Element is “Technology” – Enterprise IT landscapes are typically diverse with bespoke and packaged products. Having the technical expertise to understand what DevOps tools fit the enterprise landscape and how to integrate them into bespoke and packaged products is key. Also, understanding what does “implementing DevOps” mean to different IT systems is crucial; for example, do you need to automate the release of revenue-assured systems where speed is not as important as quality or is it enough to implement quality inspections only? Also, which systems do you start your journey with and based on what criteria did you choose these systems/projects?

    2nd Change Element is “Culture” – This is looking into how the IT teams would change the way they work, for example, developers need to check-in more frequently, business analysts need to learn to write behavior-driven requirements and operations should start understanding more about what developers do. Another example is how the developers’ (or testers … etc.) objectives will change to become more DevOps-Friendly? (For example, objectives should focus on the handover points between teams rather than team-specific activities). This is more focused on internal IT rather than the rest of the enterprise. Challenges arise here when adoption is phased without change planning – when developers potentially are required to change their approach when testers and operations teams are not. In this scenario, frictions arise.

    3rd Change Element is “Ecosystem” – This is the core of the message and it is about how to plan for organizational change outside of IT. This element is about how DevOps impacts the enterprise outside of the IT space and how to position DevOps to the rest of the enterprise. Here are a few examples of what should be done in this space:

    (1) Finding a Context: DevOps evangelists need a context for their initiative and it needs to complement other efforts in the organization. For example, evangelists should choose an organizational pain point like “poor customer experience driven by poor product quality”, and position DevOps as the IT solution among a wider enterprise program to tackle the issue – this helps evangelists take a benefits-driven approach to DevOps adoption, which goes along way in navigating the politics of the enterprise.

    Another example is the “lean initiative” – in most organizations I’ve seen, there is usually a process improvement or lean initiative going on in non-IT functions; evangelists need to hook into these relevant initiatives, present DevOps as aligned with the organization’s direction of becoming lean and position DevOps as the IT part of that lean journey.

    (2) Second, evangelists need to understand and assess the impacts of DevOps outside of IT. For example, as they change the culture by changing developers’ objectives (see above), how does that impact the overall HR strategy of the enterprise that places roles in bands and have a clear definition of a “Star” and how to promote this “Star”? Similar to HR is financial governance: how does a DevOps lifecycle work with the budgeting and financial governance process of large enterprises that releases budgets in gates/phases that are typically not iterative?

    Another example is how do evangelists link their DevOps metrics with the wider organization performance KPIs and business objectives? How do evangelists deal with the enterprise’s managed service providers and system integrators who may very well be located oversees and get them to work in a DevOps way without breaking existing contracts?

    (3) Finally, evangelists need to understand the political landscape and how to gain IT and non-IT support to their DevOps journey. Activities in this space should address the following questions: how do evangelists gain IT and non-IT Executive sponsorship to get non-IT teams to support their journey?

    Answering all of these questions above requires partnerships and help from other non-IT enterprise departments – but how do evangelists build these bridges?

    Because of all these Ecosystem challenges, I believe DevOps adoption in the enterprise should be approached as an enterprise change not a technology change. DevOps evangelists should dedicate separate plans for the 3 change elements above under a single DevOps program and they should not be shy of reaching out for non-IT help (external and internal).

    In the next few blogs, I will discuss each change element separately. Then I will answer the question that probably most of you are asking by now (continuing with the analogy): So what happens if the enterprise passes this “DevOps Equilibrium” and reaches a point of “DevOps Escape Velocity”? The point when DevOps is an established and acknowledged enterprise standard. What benefits will the enterprise reap? And why isn’t the point of equilibrium enough?

    About the author/ Tarek Negm

    tarekTarek is a DevOps and Lean IT enthusiast with a specific interest in the change aspects of the movement. He is a founding partner at Do Consultants where he helps enterprises adopt and realise the business value of DevOps and Lean IT. Before that, Tarek spent his career in global organisations in the UK and Europe working in IT leadership and Technology roles in a wide variety of industries including Telecoms, Financial Services, Hospitality and F&B. Tarek’s experience spans all the technology lifecycle starting with Linux kernel development to Enterprise Architecture and IT Strategy, in addition to IT change where he previously led a global IT transformation program across 5 European countries. Tarek holds an MBA from London Business School focused on change management in technology.  Reach him on LinkedIn: