staging-devopsy.kinsta.cloud

Tag: devops

  • How Good Developers Become Good Engineering Managers

    How Good Developers Become Good Engineering Managers

    Being a good engineering manager means that, to some degree, you have a natural affinity for the work involved. And if you’re a good developer, you likely already have some of those innate skills and attributes.

    But simply having the skills doesn’t mean you’ll enjoy being an engineering manager. For example, the nature of the role is that you’re no longer the one writing the code. I’ve known great architects who became great managers, but they ended up going back to being individual contributors (ICs) because they missed working with nitty-gritty coding details and the joy that brought them.

    It helps to take a realistic inventory of your skillset and preferred working style before making the transition. There are certain competencies you should have completely dialed in, and other tips to know that will help your transition.

    Basic Skills of a Good Engineering Manager

    If you’re a good engineer, you likely already have the following foundational skills that also make a good engineering manager. With these in your toolbox, you’ll have a better chance at success.

    1. Attention to detail. The buck stops with the engineering manager. You need to know when to push and ask questions and how to set outcomes for your team’s accountability. If you work loosely as an IC and need reminders about your own work, you probably won’t do well as a manager.
    2. Amazing personal planning skills. As an engineering manager, you’ll be planning an entire team’s work. If you’re terrible at planning your own work, maybe the engineering manager role isn’t for you.
    3. Speak intelligently about broad technical topics. If you’re a good developer, this shouldn’t be a concern. You’ll need this knowledge to advocate for your team and justify their work to upper management.
    4. Team player attitude. Even as an IC, you should have a collaborative, team-oriented mindset. It goes without saying that you’ll need this when leading others.
    5. Interest in managing others. This might seem obvious, but if you don’t have a natural interest in managing others and in the people side of engineering, you’ll be unhappy as an engineering manager.

    Making the Transition From Developer to Engineering Manager

    If you can do all of the above and are ready to make the transition from developer to engineering manager, here’s what to be aware of as you navigate that process. These tips aren’t meant to deter or discourage you but rather provide a helpful dose of reality.

    Satisfaction in your work takes new forms. If you’re an excellent developer, you know the value you bring and likely get personal satisfaction out of the contributions you make. As a manager, it’s harder to see the direct value you provide. Shift your mindset and realize that now, you’re the one enabling developers to do that rewarding work, and that’s the value you provide. You’ll learn to take enjoyment and fulfillment from your team’s accomplishments.

    People take up a lot of your time. What most frequently catches new engineering managers off guard are the people-related aspects of the role. They’re not prepared for how much time managing people takes. I’ve seen it over and over again where a new manager thinks they can still set aside time to code, but you really have to prioritize learning the craft of management.

    Remember to communicate upward. In the absence of information, it’s human nature to assume the worst. When I was at BitBucket, I assumed that because I had always done a good job, the leaders would think my team was crushing it. The opposite was true; leadership thought everything was on fire.

    I started sending updates every two weeks to various levels of management, and that’s when I learned the power of upward communication. I realized I needed to sell my team’s work, contributions and challenges. Learn what higher-level leaders are looking for and then give them that information regularly.

    Context-switching is part of the job. As an IC, constant context switches kill your day. But being an engineering manager means dealing with endless context-switching. Going to multiple meetings is baked into the job, and you’ll still have to find ways to block off time to get other work done.

    Shield your team. You also need to be prepared to help protect your team from context-switching. You might learn information that’s important to your team during your many meetings, but not all of that needs to be communicated with them. Filter out the junk so it doesn’t rain down on them and they can maintain their flow.

    Being an engineering manager can be a rewarding role that has a considerable impact on an organization. The more prepared you are for the realities of the role and its differences from being a developer, the more you’ll enjoy it and the more successful you’ll be.

  • Platform Reality Has a Surprising Amount of Detail

    Platform Reality Has a Surprising Amount of Detail

    Developer platforms and the platform teams that create them have been the center of attention for quite some time now. There have been many (wildly different) opinions about the subject, ranging from how bad the notion of a platform team is to how essential platforms are to make an organization’s DevOps journey to cloud native successful.

    Since early this year, the CNCF TAG App Delivery team has been working on a whitepaper about platforms. It explains what platforms are, why they are needed and how platform teams might work to create them. Furthermore, there’s an abundance of end-user talks at conferences that show how platform teams and their platforms were essential in enabling developers at scale. Even on the KubeCon + CloudNativeCon keynote stage, we’ve seen success stories of platform teams like the one at Mercedes Benz.

    Developer Platforms Should Enable Developers

    There is no silver bullet to successful platform teams and platforms. However, it is important to understand that the main goal of a developer platform is to enable developers. It is essential for platform teams to focus on delivering that value, which is why we are seeing a trend to introduce actual product ownership and product management roles to platform teams.

    The CNCF whitepaper stated that the three core jobs of a platform team as follows:

    1. Research platform user requirements and plan a feature roadmap.
    2. Market, evangelize and advocate for the platform’s proposed values.
    3. Manage and develop interfaces for using and observing capabilities and services including portals, APIs, documentation and templates and CLI tools.

    I couldn’t agree more. But the reality of day-to-day business for many platform teams means they rarely get to focus on those three jobs.

    Reality Has Detail

    The situation reminded me of a blog post I read a few years ago, titled “Reality has a surprising amount of detail.” The author explained how it is easy for people to get stuck even when faced with a task that might look simple from the outside. They explain how a task as simple as building wooden stairs can, in its many subtasks, become quite complex. Furthermore, when faced with reality, even the tasks that seemed plannable get more complex and need real-life adjustments.

    Most engineers have had analog experiences in their day-to-day work. We can translate this notion into platform engineering quite well. Building a developer platform sounds straightforward at the onset. However, once we get into it, it becomes a rabbit hole of figuring out the CNCF landscape, adding company-specific glue, compliance, security, fixing bugs, getting deep into the open source projects and sometimes, faced with the reality of open source, even becoming contributors and maintainers of such projects.

    Now, I’m not saying that it’s all bad. All these things can be great and rewarding in their own way. Especially with a community like the CNCF, participating and contributing to open source can be a very rewarding experience. You’ll make lots of friends, learn DevOps and development skills and might even further your career significantly.

    While it’s a lot of work, there is a risk that every platform team basically reinvents the wheel. This is because an organization’s requirements seem too large to rely on a single product (or even a suite of products). We end up with bespoke platforms that need a lot of attention. These platforms also have platform teams that don’t have enough time to focus on the higher-level jobs that they were tasked with.

    Platform Evolution

    Over the years talking to and working with platform teams we have built a model of prototypical platform evolution and its risks at each stage.

    Usually, a platform team starts with the task of building a developer platform to enable the product teams to build great software and deliver value to the end users.

    As they build the platform from the first Kubernetes clusters to observability tooling to security, the backlog for features constantly grows. Even with a growing platform team and sub-teams for certain technology areas, there’s a risk that some of these backlog items never get attention. Especially, delayable, non-critical, daunting tasks (like upgrading to the next Kubernetes release) often fall victim to this problem.

    Next, when it comes to operations, the platform team needs to limit the possibilities and breadth of what they offer. Operations responsibility for functionality that is outside of those limits, (for example, a specialized data store) goes back to the product teams themselves, which results in developer distraction.

    Even with a wide coverage of the platform, Day 2 and operations, the next big task for the platform team is advocating for adoption. This is not just marketing for their platform, but also means the platform team has to build user-friendly interfaces and documentation. The risks and alternatives we often see here are that product teams build their own mini-platforms and systems that result in shadow IT. Another common risk is ending up with several competing platforms within big enterprises that are hard to integrate and centralize, which in turn leads to inefficiencies.

    Once we get over the adoption hurdles, it is the platform team’s job to deliver value and high-level capabilities to the product teams’ developers. However, at this stage, often the platform teams have grown and might encompass several sub-teams for technology areas. Additionally, some areas like security might already be under central IT control. The risk we see here is that the capabilities offered are reduced to technology silos. They might cover all areas like observability, connectivity, security and release engineering, but to achieve real maturity, a platform team needs to offer capabilities that span several of those areas.

    Many of the advanced capabilities that we see with more mature end users cover several or all technology areas. Think about a simple high-level capability, like progressive rollouts. It starts with release engineering capabilities to build and deploy the software. From there, we need connectivity capabilities to dynamically route traffic to the new service. Then we need observability capabilities to observe our new version and from there have automated feedback loops to roll back or go forward with the rollout to production. And if that wasn’t enough, as we’re a security-aware organization, we want the whole software supply chain that facilitates this to be secure and trusted.

    Question for the curious cloud native enthusiasts: How many tools do you need to integrate to achieve the above “simple” high-level capability in an automated way? And this is only one of many capabilities.

    PaaS-Like Platforms Without the Downsides

    So, while platforms and platform teams are definitely the way to go, we need to be aware of the amount of work and effort required.

    We need to be aware of the risks and strive towards building highly integrated platforms that deliver high-level capabilities to the product teams.

    However, cloud-native developer platforms, while having similar goals to the PaaS systems of the past, should not follow the same approach of full abstraction. If we just abstract away into bespoke systems that hide everything from the user, we go back to the world of PaaS where at some point the end-user developers hit limits and build workarounds around the platforms.

    Additionally, if we build these bespoke internal platforms, we’ll be stuck with them and need to maintain and evolve everything by ourselves.

    Or, to go back to the blog post I referenced earlier, “At every step and every level there’s an abundance of detail with material consequences.” And we have to remember what we care about. The way to avoid both of these downsides is to bet on open source and stand on the shoulder of giants.

    By building our platforms on open source and using as many of the established or soon-to–be-established standards of our community we can avoid our platforms becoming bespoke. Recent developments like Cluster API, Gateway API and sigstore are some good examples.

    Furthermore, we need to make our abstractions poke-able. While our platforms should come out of the box with user-friendly interfaces and sensible production defaults that enable new product teams to onboard quickly and get their jobs done without having to bother with deep understandings of the underlying technology, we need to keep them open to mature users that know their way around and want to tweak it to their needs. We achieve this by giving mature users the option to access said open standards and APIs safely.

    This is the strength that we gain from a community like the CNCF, so we should use it to build great platform products that stay true to the core of open source and strive to make our product teams as happy and successful as possible. And remember to be happy with ourselves as successful and appreciated platform teams.

  • You’re Already a Distributed Developer

    You’re Already a Distributed Developer

    For many coders, the idea of distributed systems development might seem like a dark art. They think of high-performance computing wizards crafting mysterious spells with job schedulers to work magic with hundreds of connected nodes in massively-parallel systems. Or large, distributed content delivery networks that balance resources to deliver content across the world no matter how high the demand. The folks doing those jobs have often worked in specialist distributed computing environments for years. But the complex problems they face at a massive scale aren’t so different from those that you deal with every day. In short, you’re a distributed developer too. It’s just that no one bothered to tell you.

    Everything is Distributed

    The world is a lot more distributed today than it used to be. Back in the day, code would talk directly to one mainframe or microcomputer, generally on the same LAN. Your PDP-11 and your batch order processing software had an intimate, exclusive kind of relationship.

    Computing environments began to change with the advent of open computing in the late eighties. It ushered in the era of multiple computers working together other over a network. The concept continued to evolve beyond PCs. Now phones, servers and smart kettles all expect an easy way to talk to each other.

    Today, distributed operation is so ubiquitous that people barely see it happening. When you type anything into a browser, it fires off dozens of queries to different backend systems owned by different organizations. These systems generally run on different infrastructure.

    Even a small application that makes a database query is a distributed system. The application often talks to a system—or even multiple systems—halfway around the world.

    Many apps these days are compositions of multiple API calls to different online services. When a developer gathers weather data from a service provider and feeds it into a cloud-based online geographical information system that decides how much to irrigate crops in Iowa tomorrow, what is that if not a distributed query?

    Software is becoming more distributed over time. As software became more complex, developers have long wanted to carve it up into smaller pieces. This began in earnest with the software component movement around CORBA in the mid-to-late nineties, gravitating to XML web services in the early noughts and then to RESTful APIs. All of these involved forms of remote procedure call to increasingly granular services.

    In the last decade, this atomization of monolithic software has become the norm rather than the exception. Container-based architectures ushered in the use of microservices for better development agility and maintenance. Most developers that started their careers in the last five years or so will have been forced to think about how these services communicate from the outset. Distributed design patterns are a fact of everyday life.

    And it’s Still Difficult

    While your distributed applications might not be as complex as those facing HPC and hyperscale services, they still come with some challenges that increase as systems become more granular. What happens if a remote API takes longer than expected to deliver a result because it is overloaded, meaning that API queries to different systems arrive in a different sequence than expected? How should you react if a network or system outage or an expired access key prevents an API from responding at all?

    Problems like these arise unexpectedly, meaning that you have to get more creative with code from the outset. Good developers do more than master object-relational mapping and send queries. They program defensively to ensure that their programs retry steps when a failure somewhere else delays that job or stops it altogether.

    There are different design patterns to handle most common distributed programming scenarios today, even though you might not think of them as such. For example, developers have been building load-balancing code into their applications for decades, distributing requests between different servers to maintain performance.

    Anyone who has tried to maintain transaction integrity when writing to multiple databases will also have dabbled in two-phase commit design, which ensures that all of the databases get updated (or that none of them do).

    The more experienced you get, the more complex and esoteric these patterns become. At some point, many developers move beyond basic CRUD models in their applications to handle more complexity when reading and writing data. For example, they might read data from a website in one format but collapse multiple records into one when writing data to the website’s back-end store. That means writing code that creates different types of read and write queries. That pattern, called command and query responsibility segregation (CQRS), is also a form of distributed programming.

    Then, there’s the Saga pattern, which people use to maintain application state across long-running processes comprising multiple transactions. This pattern relies on an event bus along with messaging and queuing code to handle services with unpredictable delays in response times. It ensures that the overall process transaction finishes in an acceptable state or rolls back.

    As you venture into more sophisticated patterns that hold transactions together while maintaining performance, you’ll find your code becoming more complex and, therefore, application development becoming more difficult. Such is the life of a distributed developer, even if you didn’t realize that you were one until now.

    Gaining this knowledge and experience as a developer is important, but once learned the hard way, you don’t have to keep doing it manually. There are technology solutions that manage some of these complex distributed patterns for you. As the volume of plumbing code to support distributed design patterns increases, so does the need to offload it. In an industry increasingly built on abstraction, it’s time to turn distributed systems management into a separate service and unburden your code.

    And—if you’re looking for an example platform specifically designed to address this issue—check out Temporal, a completely open-source (MIT license) microservice orchestration platform which enables developers to build scalable applications without sacrificing productivity or reliability.

     

  • Preparing for Machine Learning Readiness to Unlock its Full Potential

    Preparing for Machine Learning Readiness to Unlock its Full Potential

    With the rapid advancement of artificial intelligence (AI) and machine learning (ML), companies need to understand and evaluate their readiness to adopt these technologies and drive material business outcomes.

    To assess a company’s machine learning readiness, we must consider several key factors, including:

    ● Vision/strategy
    ● Data availability
    ● Expertise, infrastructure and resources
    ● MLOps and governance

    In this article, I will review these three key factors in assessing ML readiness and how they can help your organization prepare.

    1. Vision/Strategy

    A company’s vision for its AI/ML strategy is essential to consider when assessing its machine learning readiness. This includes ensuring that the expected AI transformation aligns with the company’s strategic goals and that it will ultimately be meaningful for the business. To evaluate this, organizations should consider their overall strategy for AI adoption and determine how it fits into their broader business objectives. Additionally, companies should identify specific use cases and applications of AI that support their goals and how those can be implemented to drive business outcomes. A company that has already woven an AI/ML strategy into its business planning is in a far better position to assess its ML readiness.

    2. Data Availability

    I believe data availability is another crucial factor in assessing ML readiness. Machine learning algorithms require vast amounts of data for training and validation to offer real value. It would be challenging for a company to successfully implement machine learning without access to sufficient data, or they’d risk a model that cannot be trained effectively and that would, in turn, perform poorly. Companies must consider the quality and quantity of data at their disposal, as well as any potential future data sources when assessing their readiness for this important element of machine learning. I also think organizations should consider the feasibility of collecting and integrating data from various sources, whether internal or external, to have the most impactful machine learning initiative.

    3. Expertise, Infrastructure and Resources

    It’s no secret machine learning requires a diverse set of skills, including data science, programming and statistics, and every organization must be sure they have all the necessary talent and resources when building that function. These resources include data preprocessing, model development, model validation, deployment, monitoring, maintenance and the ability to train and onboard new employees to oversee AI/ML programs. Companies should also enable collaboration across these different units and roles. If a company chooses to outsource machine learning projects to third parties, the skills needed internally may need to shift toward project management and collaboration with external teams, as well as a nuanced and sophisticated understanding of the business problem, industry and technical capabilities of the vendors, to ensure proper execution.

    A company’s systems and technology stack are key to any operation. This includes selecting the right technologies and tools to enable the end-to-end creation and use of AI/ML-powered analytical applications. Before adopting ML, companies should evaluate their current stack to determine if it’s equipped to handle AI, including the availability of hardware, software and programming languages. Organizations should also consider the costs associated with selecting and implementing the right technology stack for AI/ML, including the costs of any necessary upgrades or new investments. Decision makers should also research and evaluate different technology vendors, their products and services, and find the one that best suits their needs in regards to cost, value and alignment with long-term goals.

    4. MLOps and Governance

    To be machine learning-ready, I find it important for companies to have machine learning ops (MLOps) in place. MLOps is a set of practices that allows companies to manage the end-to-end machine learning life cycle. It covers the entire machine learning process, from data preparation to model deployment and monitoring. Implementing MLOps includes continuous integration and delivery, model versioning and management and monitoring and alerting. Additionally, MLOps practices such as automated testing, model evaluation and deployment can help organizations improve the overall efficiency and effectiveness of their machine-learning efforts over time.

    A company’s governance structure for AI/ML adoption is another factor to consider when assessing machine learning readiness. Companies should evaluate their existing governance structures and determine if they are equipped to handle the implementation of AI, including data governance, compliance and risk management. We must also consider how we will manage the deployment and scaling of AI, including the development of policies and procedures to govern the use of AI and the identification of internal stakeholders and decision-makers who will be responsible for its implementation.

    Ultimately, I believe the key to machine learning readiness is to assess current capabilities, identify areas for improvement and develop strategies to address them. By taking the time to assess their machine learning readiness, companies can ensure they are in the best position to take full advantage of the benefits that machine learning can provide.

  • The Fallacy of Continuous Integration, Delivery and Testing

    The Fallacy of Continuous Integration, Delivery and Testing

    We know that continuous integration and continuous delivery (CI/CD) have become a DevOps best practice. And many have learned that by adding continuous testing (CT), they can create a virtuous loop, ensuring perpetual code quality and security. They’re not wrong.  Yes, testing continuously is good practice, but some incorrectly equate the concept of continuous with monitoring, thinking they’re able to see everything that could eventually affect a customer’s experience (CX) with their software application. That’s a fallacy. Many of the most glaring blind spots that wreak havoc on CX exist outside the environment that CI/CD/CT is able to “see.” CI/CD/CT, even when supported by application performance monitoring (APM), doesn’t and cannot monitor for and alert to blind spots across the internet stack. And there are many. Only internet performance monitoring (IPM) can take CI/CD/CT as input streams and shift to continuous resilience as a measurable outcome.

    How CI/CD/CT Typically Works

    As customer requirements evolve, software development teams write code that triggers a new software build. In the continuous model, each new build moves to a runtime environment for integration and quality assurance and, ultimately, is deployed to end users across public or hybrid clouds. This is an application-specific view of the world.

    Since quality assurance is part of the typical continuous process, monitoring is an assumed benefit or outcome. This assumption might be because many now complement their CI/CD/CT processes with APM. What is achieved with APM in this approach is testing, not monitoring. While testing provides a better picture of performance at the application level, it doesn’t provide full visibility into the blind spots that exist between the application layer and the end user.

    APM Isn’t Enough

    While APM tools focus on code—including everything that negatively affects an application such as database wait times and inefficient code—IPM focuses on the internet, which has become your network. IPM monitors what impacts your customers, workforce or application (API) experiences across the internet.

    While APM and IPM may seem similar, they are nothing alike. They may share common tools, such as using synthetic agents, but the difference is in what they monitor and how. In this age of highly distributed applications, that difference means everything.

    Where APM can “simulate” what a user may see from cloud location to cloud location, which can confirm application performance, this simulation isn’t true or comprehensive monitoring because it is blind to network-related problems with end-user experience, which are vast. Customers and employees each have a unique internet performance fingerprint and access applications from myriad devices across multiple locations and network infrastructures, not through the individual cloud locations that APM synthetics simulate.

    IPM approaches synthetic testing more holistically, replicating multiple user journeys across a much more complex Internet infrastructure to proactively flag performance issues before they impact end users. This true monitoring takes place 24/7, not as point-in-time tests.

    Forensics vs. Monitoring

    With APM and similar observability solutions that embrace real user monitoring (RUM), it’s important to focus on what monitoring really means. APM passively collects data that is a proxy for user experience, enabling postmortem assessment of causes and possible fixes, but after the fact. Observability looks at logs, metrics and traces and provides an assessment of whether users are having a good experience or why they did not. Neither solution by itself will prevent poor experience, and many fixes won’t take place until a postmortem analysis feeds the next iteration. The poor experience has already done its damage, and the costs are already sunk.

    With thousands of potential blind spots across the globe that can slow or disrupt applications, from ISPs and wireless carriers to CDNs, it’s not hard to grasp the increasing and differentiated importance of IPM. Where the combination of CI/CD/CT with APM can proactively prevent deployment of poor code that would affect user experience, only IPM is continuously replicating – even more valuable than monitoring–millions of experiences across a dynamic multi-node network where the unexpected is now expected.

    Internet Resilience Vs. Retrospection

    Businesses today understand the importance of resilience: It’s a board-level concept. Any business that has lost market share and market cap during sustained downtime or poor user experience understands what’s at stake: Millions if not billions of dollars. IPM is squarely focused on “internet resilience,” and its pillars are availability, reachability, performance and reliability. For businesses reliant on the cloud and employers operating with highly distributed workforces, retrospection alone is a risky proposition. Now that the internet is the new corporate network, internet resilience is a new organizational mantra.

    The Takeaway

    Whether your organization embraces CI/CD/CT already or is rethinking its approach to DevOps, this article should give you pause. Your job–perhaps as part of a larger team–is to catch performance issues and potential disruptions with your application before client impact is realized. Without IPM, only part of that job is being done.

    Why is this so important? Even conservative estimates put the cost of an outage at $6,700 per minute (Gartner) and the “cost of slow,” where something causes an application to perform poorly and it goes unnoticed, is likely significantly higher because it happens all the time. The fallacy isn’t that there’s value in CI/CD/CT–there is. The fallacy is that you’re in control of the entire CX of your internet-deployed products without monitoring the internet stack and ensuring internet resilience. Only by adding IPM can you have this assurance.

  • The Rise of NetDevOps and CI/CD Pipeline Solutions

    The Rise of NetDevOps and CI/CD Pipeline Solutions

    Over time, as organizations, networks and technologies have changed, the way enterprises think about managing networks has shifted as well. Enterprises are continuing to evolve their networking strategies, specifically with regard to how they expand their efforts to adopt infrastructure-as-code (IaC) practices. Looking ahead, the industry will continue to see the proliferation of solutions that help teams adopt a modern network operating model that delivers an agile, data-driven change management process. In most organizations, the software development team members and the networking team members have operated independently, with separate tools and processes. Developers would leverage pipelines to test and deploy code and then open a ticket for network changes such as VPNs or firewall changes. The network team would implement the changes, and end users could then use the application. The handoff between dev and network could be immediate or, more likely, it could take a few days–delaying the ability of the end users to access the new functionality. NetDevOps is focused on addressing this situation, which has become one of the market’s more critical infrastructure questions: How do you build an organization that can function smoothly across different skill sets, different technologies and different processes? Those same enterprises will be looking to leverage continuous integration/continuous delivery (CI/CD) practices from their software development best practices to help them overcome some of the challenges involved with implementing automation and orchestration into their networks.

    CI/CD Accelerates the Automation Process

    Let’s start with CI/CD. Adoption of network automation and orchestration will accelerate as network operators change the way they think about network updates and configuration management. According to Cisco’s recent white paper on how “Automation is driving network engineer skills transformation,” the reasons we’re seeing an increase in automation adoption is due to organizations relying on continuous connectivity, a greater need for the network to become more dynamic and the increased rate of changes both in the service supply side (applications) as well as in the service demand side (application consumers).

    However, some enterprises trying to build their own automation infrastructure struggle with resourcing, skill sets and maintenance effort struggles. To solve this, many of them are looking to incorporate the traditional DevOps practice of building and updating code with traditional continuous integration/continuous delivery (CI/CD) pipelines into their network automation and orchestration initiatives.

    Establishing a CI/CD pipeline is a workflow best practice for DevOps teams to follow. Today, organizations rely on CI/CD pipelines to help minimize bottlenecks in the software development and delivery process, while continuous integration optimizes building, integrating and testing code within a development environment.

    By integrating both CI/CD with network automation, teams will gain greater velocity and improve delivery. And by integrating network automation with an application CI/CD pipeline, they can reduce the time to perform network changes and enable secure access between the application and an in-house sales database.

    Why NetDevOps?

    NetDevOps brings the culture, technical methods, strategies and best practices of DevOps to networking.

    When DevOps principles are applied to network operations, organizations minimize operator interactions and optimize programmable network processes to configure, deploy and manage the environment, increasing efficiency and ensuring network availability, quality and reliability.

    NetDevOps takes those principles and applies them in a networking environment, offering tools to empower teams to collaborate and build an organization that can function across different skill sets and different processes — enabling everyone to participate in network orchestration.

    And it’s a movement that’s taking hold. According to Cisco’s Hybrid Cloud Trends report, 55% of those surveyed have created a cross-functional team with technical and business representation and 50% have a centralized CloudOps and NetOps function to help ensure their organization’s hybrid cloud strategy meets business objectives.

    Network Management Tools Will Continue to Evolve

    In today’s networking environment, where the expectation is to have an “always on” approach, teams are expected to build and release new applications at a faster pace than ever before. And every team has its own processes and methods for doing that which can create delays in the time it takes to execute the application. Incorporating network automation into the pipeline can empower teams to make application changes and deploy them to the network in seconds instead of the multiple days/weeks required for manual network changes. By integrating CI/CD pipelines and NetDevOps principles with network automation and orchestration, organizations can reduce the network change bottleneck even more.

    As we face the new year, we expect the way in which teams consume network automation and orchestration services as part of their software pipelines to become more sophisticated, with NetDevOps and CI/CD practices becoming a standard. And as more services become available, the adoption of IAC becomes easier. By abstracting and tweaking to abstract the traditional CI/CD pipeline, organizations of all sizes can accelerate the use of network automation and orchestration.

  • AI Chatbots: Game Changers for DevOps Transformations

    AI Chatbots: Game Changers for DevOps Transformations

    In today’s fast-paced digital world, organizations are constantly looking for ways to accelerate their software development processes and ensure the seamless integration of their development and operations teams. DevOps has emerged as a popular methodology to achieve this goal, but even with its adoption, many companies still face challenges in their transformation journey. AI chatbots can revolutionize DevOps transformations, making them more efficient and effective than ever before.

    How AI ChatBots Improve DevOps Transformations

    Enhance collaboration by automating routine tasks and facilitating communication between teams: AI chatbots can handle repetitive tasks, freeing up team members to focus on more critical issues while also providing a centralized communication hub for both development and operations teams.
    Expedite incident management by providing real-time alerts and smart recommendations: AI chatbots can analyze logs and system metrics to quickly identify issues and suggest the most appropriate resolution steps, reducing downtime and improving overall system performance.
    Streamline deployment and monitoring processes using AI-driven automation: AI chatbots can intelligently manage deployment pipelines, ensuring that the right code gets deployed to the right environments while also monitoring system health and performance in real-time.
    Offer real-time analytics and insights into the development pipeline for informed decision-making: AI chatbots can gather and analyze data from various sources, providing valuable insights that help teams make better decisions related to code quality, release schedules and resource allocation.
    Identify skill gaps and recommend targeted training resources for continuous improvement: AI chatbots can analyze team performance and individual skill sets, helping organizations identify areas for improvement and providing personalized learning resources to close those gaps.

    Roadmap to Implement AI Chatbots in DevOps Transformations

    Evaluate your organization’s current DevOps maturity and identify areas for improvement: By assessing your existing processes and infrastructure, you can better understand the gaps where AI chatbots can make the most significant impact.
    Select an AI chatbot platform that aligns with your organization’s technology stack and goals: Choose a platform that complements your existing tools and supports your specific DevOps objectives to ensure seamless integration and adoption.
    Define and prioritize use cases for AI chatbot implementation in your DevOps processes: Identify the key areas where AI chatbots can provide the most value, such as incident management or deployment automation and prioritize their implementation accordingly.
    Develop a phased implementation plan that includes integration, testing and deployment of AI chatbots: Create a step-by-step plan to integrate AI chatbots into your existing DevOps processes, ensuring proper testing and validation at each stage to minimize potential disruptions.
    Train your teams on how to leverage AI chatbots effectively and continually refine their usage based on feedback and performance metrics: Provide education and training resources to help your team members understand how to use AI chatbots to their full potential and foster a culture of continuous improvement.

    Benefits of AI Chatbots in DevOps Transformations

    AI chatbots offer several benefits that can significantly improve DevOps transformations, including:

    Increased efficiency by automating manual tasks and streamlining processes: This leads to faster development cycles, quicker releases and a more agile response to changing business requirements.
    Enhanced collaboration and communication among development and operations teams: AI chatbots can break down silos between teams, fostering a more collaborative and unified approach to software development and delivery.
    Faster incident resolution and reduced downtime: With AI chatbots’ help, issues can be identified and resolved more quickly, minimizing the impact on end users and maintaining a high level of service quality.
    Real-time insights into the development pipeline for more informed decision-making: AI chatbots enable teams to make data-driven decisions by providing real-time analytics and insights into various aspects of the development process, such as code quality, resource allocation and release timelines.
    Continuous learning and improvement through AI-driven recommendations: By identifying skill gaps and suggesting targeted training resources, AI chatbots can help teams continuously improve their knowledge and expertise, ultimately leading to better performance and more successful DevOps transformations.

    Summary

    In conclusion, AI chatbots are poised to revolutionize DevOps transformations by addressing common challenges, enhancing efficiency and fostering a culture of continuous improvement. By automating routine tasks, facilitating communication and providing valuable insights into the development pipeline, they can significantly improve the effectiveness of DevOps processes. As organizations continue to adopt and refine their DevOps strategies, incorporating AI chatbots into their workflows will become increasingly essential to remain competitive in the ever-evolving digital landscape.

  • Five Great DevOps Job Opportunities

    Five Great DevOps Job Opportunities

    staging-devopsy.kinsta.cloud is now providing a weekly DevOps jobs report through which opportunities for DevOps professionals will be highlighted to better serve our audience.

    Our goal in these challenging economic times is to make it easier for DevOps professionals to advance their careers.

    Of course, the pool of available DevOps talent is still relatively constrained, so when one DevOps professional takes on a new role it tends to create an opportunity for others.

    The five job postings shared this week are selected based on the company looking to hire, the vertical industry segment and, naturally, the pay scale being offered.

    We’re also committed to providing additional insights into the state of the DevOps job market. In the meantime, for your consideration:

    Monster.com

    Ace American Insurance Company (Chubb)
    Philadelphia, Pennsylvania
    DevOps Engineer
    $157,477 to $172,000

    LinkedIn

    DRISHTICON
    Mountain View, California
    Senior DevOps Engineer
    $120,000 to $150,000

    Dice.com

    Srivango Technologies
    Cary, North Carolina
    Java Developer with Azure and DevOps Expertise
    $120,000 to $140,000

    Indeed

    The Coca-Cola Company
    Atlanta, Georgia
    Cloud DevOps Engineer
    $102,00o to $109,000

    SimplyHired

    Moody’s
    Remote
    DevOps Engineer
    $90,000 to $159,300

  • Turning Software Testers Into 10xers

    Turning Software Testers Into 10xers

    How do you make technical processes faster and more scalable? If your answer is simply to automate them, you’re stuck in the 2010s. Today, automation is widespread, and automating processes in a basic sense is no longer enough on its own to maximize efficiency and velocity. Instead, teams who want to excel–who want to become 10xers, so to speak–need to look beyond automation in its simplest form. They must take automation to the next level in a way that makes processes not just a little faster but orders of magnitude faster. Allow me to explain by discussing how software testers can put this principle into practice.

    The Limitations of Basic Test Automation

    When I talk about “automation in its simplest form,” I’m referring to automation that helps to execute core processes.

    In the context of software testing, basic automation means using frameworks like Selenium or Appium, which provide a way for engineers to execute tests automatically. That saves a lot of time and effort compared to executing each test manually.

    To be sure, there’s nothing wrong with simple automation. But the fact is that by now, most teams have caught on to the value that test automation provides. Automatically executing most of your tests no longer makes you much faster than average. It might have been a big deal a decade ago, back when test automation was less widespread, but it’s hardly enough to make you a 10xer today.

    Going Beyond Simple Automated Testing

    What does set you apart? What can testers do to supercharge their efficiency and velocity?

    The answer, again, is to take automation to the next level. Rather than simply using test automation frameworks to execute tests, they can also use automated tools to generate the tests in the first place.

    For example, modern automated testing solutions can automatically create test scripts based on actions that testers perform manually, making it possible to generate tests without having to write any code. That means you can create many more tests in much less time.

    As another example, next-generation automation can supercharge the interpretation of test results. If engineers want to know, for instance, when a screen finishes rendering in an application or when a crash occurred, they don’t have to parse test results manually, or write custom scripts, to find that information. They can leverage modern automation tools that collect that data on their own.

    In a similar vein, modern test automation tools can accelerate the process of troubleshooting tests that fail to run. For example, imagine that an automated test fails because it is supposed to test an element that developers relabeled in the newest version of an application. Traditionally, fixing that issue would have required engineers to review the test, sort through application code and then update the test with the new element ID. But this process could instead be automated using tools that can identify the application elements that a test should assess and update test scripts with the appropriate element IDs, all without requiring manual intervention.

    A Copilot-Inspired Approach to Software Testing

    You could draw analogies between the types of opportunities that modern test automation software makes available to testers and those that AI-powered tools like GitHub Copilot have opened up for software developers.

    If you’re a software developer today, you don’t have to settle for basic automation tools, like continuous integration servers (which automatically integrate code into larger codebases) or source code management platforms (which automatically track changes to code over time). You can also benefit from a new breed of automation tools, like Copilot, which can automatically generate code, unit tests and much more.

    For developers, Copilot is a prime example of a next-generation tool that can dramatically accelerate software development. Developers who learn to use Copilot effectively can work potentially ten times faster than their counterparts.

    The types of next-generation test automation solutions I’ve described above do the same thing for software testers. Instead of relying just on basic automation tools to run tests, modern testers have a host of additional, more powerful automation at their disposal to help create, manage, interpret and troubleshoot tests. Those who learn to use those tools will become the 10xers of today’s generation of software testers.

  • The Now and Next of Automation

    The Now and Next of Automation

    Today’s never-ending flood of data has many of us drowning in noise. And while humans are still essential for thoughtful analysis and driving action, it’s no longer possible for a human to parse gigabytes of data alone.

    How do you begin to filter out the noise and then understand what to do with it? How can you get to the root cause of an issue so you can act on and resolve it? Most of all, how can you do all of this while being more productive and driving more innovation instead of constantly putting out fires?

    Automation is the obvious answer. For many organizations, automation isn’t just about efficiency – it’s about survival. Automation can make organizations more agile, resilient, and capable.

    Where you sit within an organization or team, however, often dictates how automation can help with your specific goals or alleviate your immediate pain points. Mandi Walls, a DevOps advocate with PagerDuty, offers some advice on how even less experienced teams can use automation to perform simple tasks, solve problems and improve the work environment.

    We hear a lot these days about A.I. and machine learning, or tools like Chat GPT which to some is a harbinger of robots finally replacing humans. Putting the hyperbole aside, where do you see automation today? Is it still a “nice to have” for engineering teams or a “must-have” to do their jobs effectively?

    Automation is definitely a “must-have.” Manual processes are risky; they can inadvertently introduce too many errors, especially in complex environments. The scope and complexity of modern technical ecosystems is just too vast for teams to effectively cope with using only manual methods.

    This is especially true for teams that develop and manage customer-facing services. Having a successful product leads to more customers, which requires more capacity and creates more complexity. To deliver the kind of performance customers expect, teams need to be fast and efficient. That’s where automation comes in.

    What are some specific types of automation that engineers and IT pros should be aware of?

    Some automation has become more or less ubiquitous, such as tools that enable configuring and building large numbers of systems with little human intervention. Other processes might feel less like “big-A” automation–for example, continuous integration/continuous delivery (CI/CD) pipelines are automation.

    Humans also aren’t manually building many software packages anymore. The use of automation in the software testing process has greatly expanded in the last several years, changing the jobs of QA engineers significantly.

    Is self-service automation – aka true autonomy that doesn’t require human intervention – the ideal end state for a digitally mature organization? When is the best time to jump in?

    Organizations that wait for a perfect opportunity to automate their processes risk wasting a lot of time and resources in the interim. Teams can look at any of the tasks that they regularly perform and assess them as candidates for automation.

    Tasks can be assessed in a number of different ways, but for teams just starting out with automation, seeking tasks that are low complexity and low impact is a strong starting point. Even if we look just within the realm of tasks that might be utilized during incident response, teams might have any number of tasks related to information gathering, log preservation, system diagnostics, and other telemetry that responders run for every incident. There’s no reason to require humans to run those well-understood tasks!

    As teams get more comfortable with creating and using automation, they can seek out more complex tasks to automate. And it’s entirely possible that there will be tasks that teams aren’t able to automate fully! Those tasks might be too complex, require a lot of human input or interaction or have too many edge cases for the team to create automation for.

    Are the principles of automation the same in simple versus more complex environments like Kubernetes?

    The principles are essentially the same, but the constraints and tools may be different. In some environments, a simple shell script might be enough to accomplish our goals. In other environments, we might need to make use of APIs or vendor SDKs to do what we want.

    On that topic, is self-service automation in cloud environments the preferred path?

    Actually, self-service is sort of how the cloud really got going! Development teams armed with nothing more than a credit card got the environments they needed without going through lengthy procurement processes.

    Now that cloud is a critical part of many workloads, organizations are putting more rigorous guidelines in place. The cloud platforms are also much more complex, with more service offerings than in the early days, which makes automation more important than ever. Automation helps organizations ensure that their cloud environments meet their security and compliance requirements, helps manage spend and still provides enough flexibility for all teams to get work done. And with self-service automation, developers can request an environment in the cloud to develop their service and have that service meet all the organization’s standards without having to keep up with all the details themselves.

    What about toil? Can that be fully eliminated through automation, or is some toil still necessary?

    I’m not sure I’d say toil is necessary, as much as it tends to be inevitable. There will likely always be some boring, repetitive tasks that are too unpredictable or unstable to automate well.

    That said, teams certainly don’t need more toil; having too much repetitive work tends to lead to less job satisfaction and higher attrition rates, so teams should be looking to minimize as much toil as possible where they can.

    Is there one myth or misconception, either about automation overall or self-service automation in particular, that you’d like to clear up? What’s the reality?

    First of all, don’t worry. You won’t be able to replace your team with a set of shell scripts! Seriously though, automation isn’t about “doing more with less.” It’s about doing more with the same. Automation is an approach to help teams cope with the day-to-day running of complex environments and will need to be managed as part of those environments. Managers need to make sure they are allotting time for the creation and maintenance of automation in the same way they would for other tools and applications.

    Interested in learning more about PagerDuty? Stop by Booth S49 at KubeCon to speak with an expert or see a demo!