staging-devopsy.kinsta.cloud

Tag: devops

  • Zero-Trust Network Access Platforms Slash DevOps Bottlenecks

    Zero-Trust Network Access Platforms Slash DevOps Bottlenecks

    In a prior article, I indicated why a new remote zero-trust platform is needed to support DevOps teams.

    Traditional remote access configurations are not well-suited for DevOps given a shifting user population comprised of employees, contractors and other third parties often using their own devices; the large number of tools and systems to which access must be provided and the hybrid nature of the infrastructure itself (on-premises, hybrid and multi-cloud).

    DevOps Pain Points

    Three pain points are associated with traditional approaches when used with DevOps.

    1. DevOps user experience is poor in traditional remote access configurations. Each DevOps service must be accessed as separate secured sessions, which is confusing and tedious.
    2. System management requires IP whitelisting rules in VPN, static SSH keys in Bastion, firewall rules controlled by different teams and application-specific authentication. The coordination and execution of so many touchpoints for common actions—including the onboarding of new team members, changing roles or adding a new service—invites mistakes that cause productivity issues and introduces unforeseen security holes.
    3. Security is complex and error-prone. DevOps users must be granted restricted access to the individual services they need to be productive. Administering restricted access for individuals and groups is time-consuming. Security policies typically require monitoring detailed audit logs of user access.

    How a Zero-Trust Remote Access Platform Works with DevOps

    In my whitepaper, Zero-Trust-Remote Access-For-DevOps, I explain how a zero-trust remote access platform can resolve the requirements for improved DevOps user experience, consistent and manageable administration and simplified, less error-prone security.

    Improved DevOps User Experience

    By replacing the need for the user to tediously set up separate sessions for each DevOps service they need to use, a client-side DevOps service catalog enables all DevOps services to be accessed with a single click. This service catalog approach is more convenient for DevOps users and provides a form of DevOps governance. It does this by encouraging DevOps teams to select choices that are available in the catalog instead of creating new choices that result in expensive DevOps tool sprawl.

    Simplified DevOps Management

    Management of remote DevOps users is simplified by consolidating access for all DevOps users and services under a single unified service manager and connector. The need to manage each user and service with different teams, each with their own processes, is unified and streamlined when the management requirements for DevOps users and services are consolidated under a common zero-trust remote access platform.

    In addition, this approach makes it much easier to implement more adaptive policies across the breadth of users and services.

    With a zero-trust remote access platform, onboarding new DevOps users and groups is accomplished within a single session—not separate sessions for each onboarding requirement.

    Simplified and Less Error-Prone Security

    Consolidating access to DevOps services under a common zero-trust remote access platform with unified services and services connectors enables simplified and less error-prone security controls, improved security monitoring and policy management. Audit logs of access are reported consistently for all services. Monitoring and changing access rules can be supported for individuals or groups when adjusting a policy. Trust-based access controls can be implemented consistently. Granular policies ensure access is predicated on appropriate device trust score thresholds being met. Rather than a single check prior to granting access, these production services policies continuously assess the security of the users and devices accessing the services.

    Transforming to a Zero-Trust Remote Access platform for DevOps

    Transformations of all types, and especially those for DevOps, are made more certain by following a
    seven-step transformation process, described in my book Engineering DevOps. This process serves as a framework for implementing the zero-trust remote access platform.

    The first two steps, vision and team alignment, involve getting the leaders, key influencers and implementers of an organization to recognize and establish actionable goals for the implementation of the zero-trust remote access platform.

    The third step, assessment, requires taking inventory of the current state of DevOps users, DevOps services and policies that need to be supported by the new platform.

    The fourth step, road map, requires determining the plan for implementation. Typically, it makes sense to start with one application. Start with a set of DevOps users and DevOps services needed for a single application or pipeline which can serve as a model to drive adoption to other applications as the solution is proved and accepted.

    The fifth step, realize, is the step in which the zero-trust remote access platform is installed and set up for the initial model application or pipeline. Banyan Security explains how an example tool vendor helps to simplify this step.

    Capabilities to look in for in zero-trust remote access tools include:

    • Deploys in minutes, with no integration requirements on any existing DevOps services or infrastructure.
    • No need to set up VPNs, open inbound firewall ports or manage DNS.
    • Use existing DevOps user devices without diminishing security.
    • DevOps service catalog provides users visibility and one-click access to DevOps services.
    • Support for bundled services, favorites, and autorun capabilities.

    In step six, operationalize, the procedures for commissioning and using the zero-trust remote access platform are proven, benefits are measured and playbooks are hardened for the organization.

    In step seven, expansion, the zero-trust remote access platform deployment is expanded to other DevOps applications pipelines, DevOps users and DevOps services across the enterprise.

    What This Means

    Traditional remote access configurations are not well suited for DevOps administrators or users given the dynamic nature of services, team members, and environments. A zero-trust remote access platform addresses the need for an improved DevOps admin and user experience, simplified access management to DevOps services and simpler, less error-prone security practices and monitoring.

    I want to express my appreciation for the time spent with the Banyan Security team, who has pioneered many of the concepts outlined in this blog and contributed valuable insight.

  • Is AI Taking Over DevOps Functions?

    Is AI Taking Over DevOps Functions?

    Artificial intelligence has taken the tech world to a new level of automation. Today, almost every specialization demands the intervention of machine learning to develop AI technologies that help businesses do more with less time and resources. Still, some organizations question whether leading with AI is a good investment. 

    For DevOps, the answer is a resounding yes. AI can enhance DevOps practices to accelerate the pace of software releases, helping businesses achieve continuous delivery. This enables programmers to release software about 10 times faster and allows programs to be reviewed before they are released. 

    AI also has improved DevOps culture, from enabling more efficient decision-making processes to code quality enhancements and automated operations. So, let’s take a deeper dive into how AI impacts DevOps.

    Recent Changes in DevOps

    Monitoring and managing all the data generated in DevOps environments involves a high degree of complexity, making it difficult for teams to collect and use data effectively. In addition, the volume of data any given team may receive can reach well into exabytes, so AI tools provide much-needed assistance. After all, analyzing massive data sets manually would take too much time for a human and won’t meet the demands of modern businesses.  

    What’s more, surveys show that 57% of developers in the United States have less than five years of experience. For this reason, software testing must be an ongoing and in-depth process to check and double-check that no vulnerabilities impact the security of the code. AI can supplement manual review to increase speed and accuracy in finding bugs. 

    That being said, integrating AI into DevOps functions doesn’t mean developers aren’t necessary, even though their roles have evolved considerably. Backend developers are involved in the development and operations sides as they take on ops tasks and tests, especially around cloud-based infrastructure and cybersecurity. And frontend developers will always be needed for the technical support of creative design—which no one can do better than humans. 

    In today’s dynamic and fast-paced world, developers need to release code faster than they used to while the operations teams ensure that the existing systems function with minimal disruption. This partnership is easier when integrating AI, as it makes the collaboration between development and operations teams more efficient. AI-powered systems provide a unified view into systems and glitches in the complex chain of DevOps. 

    Is AI Taking Over DevOps?

    Short answer: No. AI is supplementing human activity in DevOps to make things more efficient and secure. 

    Due to the complexity of DevOps requirements, people have been looking for a more automated solution that could assist humans in performance and accelerate the processes. What’s more, you can expect to pay around $60 an hour for an experienced backend developer in the U.S. This means using AI to automate the grunt work can be a big cost saver as well.

    Here are a few ways AI transforms DevOps functions:

    Enhanced Security

    Distributed denial of service (DDoS) and related attacks have become a widespread threat to the security of websites and online services these days. AI-powered tools have the ability to identify and mitigate these threats. In addition, AI can help in the pivot to DevSecOps in teams still using traditional DevOps.

    AI security tools detect anomalies and threats based on real-time data and by analyzing past behavior. In this way, AI has played a vital role in protecting many organizations, including schools and colleges, from cyberattacks by supplementing human security experts.

    Improved Data Access

    The massive amount of data that needs to be collected, organized and analyzed in DevOps is beyond human capabilities. However, AI has resolved this issue by compiling data from multiple sources and organizing it for analysis. This drastically improves and accelerates the process of DevOps.

    Rapid Software Testing

    Software development and software testing has accelerated with the integration of AI. As a result, various testing types used by DevOps, such as user acceptance testing, regression testing, and functional testing, are more efficient and accurate than ever.

    These tests produce a large amount of data that would take humans ages to identify and collect manually. With the help of AI, identifying patterns and coding practices that lead to errors and vulnerabilities has become convenient and fast. This data can then be used later by the DevOps team to increase their efficiency and improve development practices. 

    Timely Alerts

    DevOps teams sometimes struggle to react and respond to the alerts during incident response. With no priority tags attached, it becomes challenging for the team to prioritize incidents effectively. A more efficient, comprehensive alert system should be capable of spotting flaws instantly and marking the more severe ones as higher priority. That way, the team can tackle the issues more systematically and resolve them without failure.

    AI can assist in prioritizing the responses to alerts based on certain factors, including the intensity of the alert, the source of the alerts and past behavior. This leads to efficient management in situations where systems get flooded with massive amounts of data and have to respond immediately. AI-powered alerts are particularly important in DevOps security, so this is a critical benefit of AI in the discipline. 

    Superior Implementation Efficiency

    DevOps teams are often burdened by the managerial tasks of a rule-based environment which decreases their time spent in more innovative and creative areas. When AI takes over, these tasks become self-governed, increasing efficiency and reducing human intervention. AI systems can work on their own, sparing humans from these tasks to focus more on innovation and creativity.

    This way, AI has turned the DevOps into self-governed systems that no longer require the rule-based, human management of analysis. This should resolve the complexity of research that DevOps teams were struggling to achieve and enable faster adaptation.

    Feedback Loops

    The primary function of DevOps is to use monitoring tools to gather feedback from every stage of operations and then use that to improve software development and delivery processes. Performance monitoring tools use machine learning to gather information such as performance matrices, log files, datasheets, etc. The feedback collected is then used to track potential issues in advance and make suggestions to resolve them accordingly. The application is then altered by applying these suggestions to make it more efficient.

    Artificial intelligence is driving increased efficiency in DevOps functions. However, it would be a stretch to say that it is taking over, as there are still many areas that require human intervention. The idea behind AI integration is not to change the players but to change the game.

    Even though AI is assisting humans in accelerating software development, developers still lead many aspects of the craft. But it also seems impossible to perform DevOps functions at the required pace without the integration of AI. So, if you wish to release software at a pace that keeps up with modern tech trends, you need to incorporate AI in your DevOps functions.

  • 3 Ways to Prevent Developer Burnout

    3 Ways to Prevent Developer Burnout

    Without developers, software companies wouldn’t exist—and yet, the experience of these experts can be far from ideal. A recent report from Haystack found that nearly 80% of developers report being burned out and the pandemic isn’t even one of the primary factors. Despite similar job requirements, the experience of developers significantly varies across the industry, adding complexity to an already challenging job. Conflicting coding styles, communication styles and requirements from different teams conspire to make the development process inefficient and difficult and leave developers feeling overwhelmed. Between different organizations and even teams within the same organization, the differences in team culture and educational opportunities can lead to radically different job experiences for the same job title. To retain talented developers in the workforce and maximize their potential, the collective developer experience needs to be improved through a developer-first and inclusive culture and better access to professional development for developers at every level.

    Build a Strong Culture of Documentation, Communication and Knowledge Sharing

    One of the biggest challenges team leaders encounter is how to communicate key information between team members. Many developers are incredibly skilled at their trade but aren’t properly trained or given the time to work on sharing that information with either colleagues or end users. Having skilled technical writers means that the team can provide consistent, clear information about the software—and this is crucial for company success. Having technical writers on the team improves communication and efficiency and enables developers to more quickly and easily locate the resources and support that they need to perform their tasks

    The developer’s goal is to innovate and build successful and maintainable systems. Achieving this requires good team processes and a solid system of documentation. Without user-facing documentation, adoption will suffer as users struggle to get things done with the system. Similarly, without documentation of the individual components, knowledge cannot be shared between teams and so the teams working on the application cannot collaborate effectively. A positive culture around documentation and investment in best practices by all contributors, including developers, makes a huge difference in how successful an organization can be.

    Welcome Employees with Comprehensive and Constructive Onboarding

    Strong communication is especially vital during the onboarding process; it’s one of the most important periods for developers early in their careers. Getting developers up to speed in a new team quickly is crucial—and streamlining education beginning during onboarding can make a big difference. Onboarding is a golden opportunity to bring new developers into the cultural fold and enable them to contribute to the team quickly. Doing so also reduces the risk of losing developers from the team.

    Prioritize an Inclusive and Supportive Culture at all Levels

    The experience of software developers is greatly influenced by the culture of the company they work for. The values of a company and the way that departments interact make a big impact on developers—and it’s not always a positive one. Opportunities for career development, promotions, mentorship programs and training are critical for all employees, but especially those in fields with the potential for high turnover or burnout like developers. Providing developers with resources to further their own understanding and mastery of the job—such as access to a budget for attending conferences and further study—is a critical investment for their success. Organizations both large and small must take initiative to ensure developers see a path forward for themselves at their current company and within the industry—or risk losing their talented developers entirely. 

    While developers’ experiences vary across the industry, between departments and even smaller internal teams, there is inherent value in improving communication, investment and culture on a micro and macro scale. To attract and retain highly skilled and innovative developers and ensure success for teams and companies as a whole, executive leadership must work with their developers to find a cohesive system and create an experience that works for everyone. Like any other specialized professional in a complex trade, developers are invaluable to their teams and community; advocating for the improvement of their work experience throughout their careers is the best way for tech companies to keep developers creating software that improves our world.

  • Solving the DevOps Accountability Problem

    DevOps accountability is the idea that both the development and operations teams have equal responsibility for success. But putting this approach into practice has been a thorn in the side of DevOps teams for the past decade, largely due to outdated organizational structures that place responsibilities on a single employee or team.

    These outmoded structures restrict the lean flow of operations needed for DevOps and contribute to costly downtime when issues arise. But redistributing functions across the organization can help solve this problem. 

    For cross-functional teams that rely on organization-wide collaboration, redefining the org chart can clarify ambiguities about accountability and help organizations overcome internal obstacles.

    Legacy Organizational Structures

    Many organizations have not instituted an organizational structure that reflects a shared accountability model. This is partially due to leadership’s assumption that large teams work faster and more dynamically. In reality, after a team exceeds nine individuals, the rate of return diminishes and people are more likely to assume someone else on the team will take responsibility for a task. As a result, accountability falls through the cracks.

    Additionally, there is frequently an imbalance in assignments and responsibilities between junior and senior employees. When senior employees take on the majority of important or interesting tasks, junior employees can feel overlooked and less responsible for their team’s success. This is both an individual and a company-wide issue; younger workers tend to bring fresh perspectives and insights—they can also adapt more quickly to unprecedented changes and issues.

    Picture a team consisting of a line manager and team members, with a structure that places all major responsibilities on the manager. If the manager happens to be out of the office when an issue arises, the team would have to wait for their boss’s return to resume operations.

    But the consequences of single-point accountability go far beyond lost time and downtime costs. When employees feel like they don’t have a significant stake in a project, they feel less aligned with the organization’s overall purpose and are less likely to be engaged.

    DevOps is most effective when teams are built around a common goal rather than around the performance of individuals. If an employee can’t see the impact their work is having on the overall organization or project, it can create blind spots and disconnect between the business and its people. Shifting business frameworks is no easy feat, but with the right strategy, it can improve how tasks and responsibilities are allocated in your organization.

    Three Ways to Switch up Your Org Chart for Greater Accountability

    The days of allowing responsibilities to fall on a single employee are over. But to successfully embrace a shared accountability approach, you need to reimagine your entire organizational chart. Beyond de-prioritizing seniority and minimizing team sizes, consider these three initiatives to help settle DevOps accountability issues:

    1. Blueprint your organizational structure around products. Organize your work structure around products—not projects. Traditional project management models rely on passing responsibilities between specialized teams, which can slow down innovation and degrade the quality of work. With an organizational structure built around products, you can build a cross-functional team with all necessary skills for each system. For example, if you have a team that’s accountable for the payroll system, all necessary expertise you need to keep that system running smoothly is based on a single contact point.
    1. Redistribute talent. The product-oriented approach requires changing up talent distribution across teams. Let’s say you have a 24-person team for a single SaaS product and seven of those team members have similar skill sets and backgrounds. In this instance, you could move several of those workers to a different team where their skills are in demand.
    1. Establish hybrid workplace environment norms. Face-to-face communication has always been a critical part of successful teams. But in light of the COVID-19 pandemic, workplace norms are evolving and organizations need to adapt to new ways of working. Many teams are no longer able to participate in informal knowledge sharing and other typical in-office interactions, so it’s critical to identify new norms for your team. Initiate conversations about applying DevOps concepts in a hybrid world, along with creating rules of engagement and accountability that are realistic for remote work.

    The Great Redistribution

    As technology advances, so should your organizational structure. To optimize efficiencies and ensure products reach the market on time, consider rethinking your org chart. In practice, this will enable you to more seamlessly adopt a culture that celebrates shared responsibility and employees who feel they have a stake in your product’s future.

  • How Does That Stack Up?

    How Does That Stack Up?

    There’s an entire class of successful innovations made up of things that people didn’t realize they needed. Things that were an obstacle they found a way around and didn’t think about until the innovation showed them an easier way. Think of things like virtualization (hardware was the bottleneck, but “That’s just the way it is,” until it wasn’t) or web monster shopping sites (“They don’t have that in the stores; I guess I’ll live without,” until you could order the weirdest or smallest things from the likes of eBay and Amazon).

    My rule here is that I only talk about products and services if I use them and find value in them. And even then, I try to do it sparingly. But in the spirit of the innovation story you just read, I’m going to point you at one of the coolest new websites on the net. I don’t know the company; have not done business with them, couldn’t even name a principal. But I have spent a surprising amount of time on their site, both in the Feed section and Browsing Stacks.

    The site is Stackshare.io, and the reason I think it is so innovative is what and who it is aimed at: Listing companies’ DevOps stack and offering advice on different parts of the DevOps stack.

    The Browse Stacks section is for presenting what other organizations are doing, so you can get good ideas for your own stack. But the developers of the site appear to have recognized the other useful aspect of this information: Matching organizations to prospective employees based on technologies. You can go look and see that Company X is using GitLab, for example, and that can then influence your job search.

    The Feed section is just a running discussion that uses categories to ask questions or offer insight and advice about decisions a given organization came to—so, “Chose X over Y” is actually a standard subheading for posts, as is “Need advice on Z” or “Recommends A”. It makes for great reading, and no doubt while reading through, you’ll have some advice to offer one of the “Looking for advice” posts.

    The Stackshare.io participants seem to range in size from startup to massive, and that is one of the things I hope they change as they grow—the options for looking at DevOps stacks are “Popular” and “Trending”. We all want more filters than that—organization size, number of apps, stuff like that—to help us zero in on environments that are similar to our own. My little company does not have the needs that Facebook does. And while it’s true that some startups use more tools in their toolchain than Facebook, you get the idea.

    Another issue I had with Stackshare.io’s site (that I fully understand, but still wish could be different) is forced registration/login to see some data. It’s not even particularly sensitive data—I know what ELK and Azure Data Services are, so putting the definition/description behind a registration page/login is kind of silly. But every free site needs data to pay the bills, and I suppose I’m glad it is info that’s also available elsewhere that is behind a reg wall.

    The company offers APIs for integration to your site/tool and private company instances of StackShare to help you manage one of my other gripes—the five (or fifty!) different DevOps stacks many organizations have allowed to develop in-house.

    So if you are job hunting, Stackshare.io can help you find companies doing what you would like to do or know you’re good at. If you are an organization trying to improve or set up a new toolchain (which is how I found the site), it has a lot of information to digest, though not all of it is easily accessible at this point. Either way, keep kicking it and keep an eye out for tools that do the job, no matter where you are looking.

  • Site Reliability Engineers to the DevOps Fore

    Site Reliability Engineers to the DevOps Fore

    [pdf-embedder url=”https://staging-devopsy.kinsta.cloud/wp-content/uploads/2022/02/site-reliability-engineers-to-the-devops-fore.pdf”]

  • MediaOps and DevOps Institute Earn Inc. 5000 Honors

    MediaOps and DevOps Institute Earn Inc. 5000 Honors

    I am really proud to announce today that both MediaOps and DevOps Institute have been honored by Inc. and earned inclusion in the Inc. 5000 list of the fastest-growing companies in America.

    For MediaOps, this is the second year in a row we have earned this honor. For DevOps Institute, it is the first year they have been honored. First of all, congratulations to all of the great people who work at and make both MediaOps and DevOps Institute the great companies they are. None of the success of these companies would be possible without their talent, skills, hard work and dedication. Thanks to all of you.

    Secondly, as the founder/co-founder of both of these organizations, I can’t tell you how proud I am that both of these companies have not only had the success they have (as evidenced by this honor) but that they have done so in the middle of a global pandemic. It really makes all of the hard work and the long days and nights worthwhile.

    For anyone who has ever started a business, you know that you put a certain amount of your blood, sweat and tears into any venture you are involved in. It is no different here! Since 2014 I have put everything I could into growing MediaOps into the company it is today. For DevOps Institute, I was lucky to have my partner Jayne Groll, DevOps Institute CEO, take the lead in making DevOps Institute the leading member organization/certification authority in the DevOps space today.

    Yes, you hope the monetary rewards of founding a company make it all worthwhile. But this kind of recognition is also very gratifying. It gives you the extra spring in your step and makes the load lighter.

    These awards also highlight something else. The ascendancy of technology. It is no coincidence that both of these companies play in the digital transformation space and have seen such success. Digital transformation is driving the world right now. Software is driving digital transformation, and DevOps, cybersecurity and cloud are driving software.

    At the end of the day, it is a great honor to have not one but two companies you started win this award at the same time. I think it speaks volumes about the future of both of these organizations. More news on that front is coming, so stay tuned for more great things coming from MediaOps and DevOps Institute!

  • 5 Ways to Reduce DevOps Toil

    5 Ways to Reduce DevOps Toil

    Over the last several years, DevOps has become a bit of a buzzword. It has become simultaneously a practice, a culture, a team, a job title and a vendor product. You can hire some DevOps, buy some DevOps, adopt DevOps and sprinkle a little bit of DevOps on top for good measure.

    But, at its core, DevOps is about service ownership: having a single team with end-to-end ownership of software running in production. You should not have one team building software and another team deploying and operating it. That’s slow. DevOps is literally combining development and operations into a single consolidated team. DevOps means embracing the mantra of “you build it, you own it.”

    The primary benefit of DevOps is increased speed and agility of engineering teams. With DevOps, exactly one team is responsible for a given piece of production software. There are no handoffs or knowledge silos. Practices like continuous delivery can emerge. Features can be put in front of your customers’ eyes with a shorter turnaround time. A secondary benefit is increased reliability and security. With “you build it, you own it,” it is no longer someone else’s job to ensure that software is correct or performant or reliable or secure. These responsibilities are “shifted left” onto exactly one team.

    So, where do things go off the rails?

    Service ownership and DevOps mean giving teams the agency to make whatever changes are necessary to their software. That can and will come at the expense of product functionality at times. So, the first place things go off the rails is when there is not a healthy balance between product needs and engineering needs. If teams are constantly working on new features but do not have the capacity to fix operational issues that are affecting them, product quality will eventually suffer. Ultimately, product-engineering imbalance generally comes from misaligned incentives or poorly defined goals from management.

    The second place where things go off the rails is insufficient training. Especially with organizations early in their journey to adopting DevOps, which may have separate development and operations teams, there is a lot of training on new technologies and processes. Developers need to learn about observability, responding to incidents and debugging production applications. Operators and sysadmins need to learn about coding and the principles of software design. These are new skills for both camps and investment in training and mentorship is required.

    What are some best practices for keeping DevOps on track?

    There are lots of these best practices. It really depends on your starting point as an engineering organization, but here are five important ones:

    1. Track Service Ownership

    If DevOps is about “you build it, you own it,” you need an authoritative list of a) what’s running in production and b) which team owns it. If a piece of software goes down, you do not want to find out that the last person to work on it was the intern who left three years ago. Many folks start tracking this kind of information in spreadsheets or wikis, but keeping the data up-to-date becomes challenging. SaaS service catalogs have emerged as the easiest way to automatically keep this data accurate, complete and consistent.

    2. Adopt Continuous Integration, Continuous Delivery and Feature Flags

    Committing small changes multiple times per day is superior to release trains where one large change is committed on an infrequent basis. CI/CD allows for new features and improvements to be tested faster and put in front of customers more quickly. Feature flag tools let you easily test new functionality and product hypotheses for different segments of customers.

    3. Embrace Incident Management

    Software breaks sometimes. Having a robust process for how to triage and respond to both alerts and incidents is important. Within the context of DevOps, the team that owns a piece of software should be responsible for being on-call to respond to associated incidents. If the software breaks or behaves in some unexpected way, the team is expected to fix it. The team is also responsible for identifying what happened and investing in ways to ensure the same error does not occur again.

    4. Treat Everything as Code, Then Automate all the Things

    Applications and microservices? That’s code. Infrastructure? That’s code, too. Security policies? Also code. Deployments? You guessed it–code.

    Treating everything as code allows for processes like “code review” to emerge beyond just your applications. You can have multiple eyes on every change, which catches bugs sooner and spreads knowledge to others. As well, version control systems (like Git) are a natural audit log, allowing anyone to see what was changed, when, and by whom.

    With everything defined as code, the next step is to invest in automation. How many steps does it take for a developer to deploy new code to production? Or to spin up new infrastructure? Or to hook up monitoring or error tracking tools?

    Your goal should be “one step:” run a command. It can be a shell script, a Slack bot or a button with a nice web UI. But it should always be one step. If there are more, you’re now in “manual steps” territory, and that’s brittle. You have no guarantees that successive individuals will run whatever sequence of steps in the same order, which can lead to divergence in how various parts of production are set up.

    Another example is maintaining a list of a) what’s running in production and b) who owns it. Manually keeping that list up-to-date is impossible as the size of your production infrastructure or engineering team scales. You’re better off investing in service catalog tooling that can automatically integrate and capture this information.

    5. Invest in Reducing Toil

    Service ownership and DevOps are two-way streets between management and their engineering teams. On one hand, consolidated ownership will lead to faster delivery and more reliable software. That’s good for management. But, on the other hand, teams require actual empowerment and agency to make decisions about their software. Part of that is having capacity on their roadmaps to make improvements to issues that are causing toil.

    For example, if a particular service is paging a lot or experiencing load issues, service ownership means giving the owning team the time to make any necessary changes. That will come at the short-term expense of new production functionality.

    There’s no free lunch on this last point. Engineering management can’t just combine dev and ops responsibilities and magically expect results. Service ownership is a two-way street. Otherwise, toil will kill your teams in the long term.

  • DevOps Database Platforms Slash DX Bottlenecks

    DevOps Database Platforms Slash DX Bottlenecks

    An enterprise with many databases and various data requirements for DevOps continuous delivery presents a problem. For example, test data provisioning is a challenge today, particularly in pre-release testing of database updates. Standard practices involve shared database environments that need to be maintained from one test to the next, slowing test cycles and generally not fitting into an automated CI pipeline. As I explained in A New DevOps Database Platform is Needed, at best, it makes things chaotic; at worst, problems with database integrations with DevOps is a gross impediment to overall organization agility—which undermines the very purpose that DevOps is meant to serve.

    A recent whitepaper, A New Kind of DevOps Database Platform, indicates that there are multiple reasons why a new kind of DevOps database platform is needed, which are:

    • Compatibility requirements
    • Integration with current CI/CD pipelines
    • Integration with portfolios of value stream management
    • Infrastructure cost efficiency
    • Productivity for developers, testers and operations
    • Security and governance
    • DevOps performance

    The paper explained that a DevOps database platform requires the following capabilities to satisfy these requirements.

    • Automated database provisioning: the ability to deploy a database using automated procedures.
    • Database cloning: writable database environments are delivered on demand, based on a built versioned image.
    • Distributed repository: database images can reside on Linux and Windows servers, as well as storage systems.
    • Simple declarative builds: a simple, predefined method to make the creation of versioned data images simple, while enabling user/group access controls, data masking and other data engineering functions.
    • Scalable data cloning: database clones are served on demand, with many simultaneous clones served from a single image.
    • Comprehensive database support: the repo supports all common relational, NoSQL, machine learning and file-based data stores.
    • Versioned databases: each image becomes a versioned, immutable DevOps artifact.
    • Deploys equally to all cloud environments: DevOps databases can be deployed and operated on any on-premises infrastructure or cloud.
    • Cross-platform:  DevOps databases can be deployed and operated on common operating system environments such as Windows and Linux.
    • Security features including access management, data masking and encryption: database images easily incorporate access control, data masking and synthetic test data via SQL scripts, batch files or command line tools.
    • Compatible with CI/CD platforms:  DevOps databases can be deployed and operated equally well in multiple CI/CD platform tools such as Azure DevOps, Jenkins and CloudBees.
    • Compatible with containers and container orchestrators: DevOps databases can be deployed and operated equally well in multiple container environments such as Docker, Kubernetes and Red Hat OpenShift.
    • Deliver writeable data at the speed of DevOps: Deliver writable databases in seconds.
    • API for easy integration with DevOps tools: An API is available to enable integration of the DevOps database control plane with other tools.
    • Audit logging: The DevOps database includes a complete audit log of operations.

    Transforming to the New Database DevOps Platform

    A DevOps database platform in an enterprise requires more than simply buying and installing software and telling everyone to use it. With multiple entrenched database platforms already in place, there are practical and political considerations that must be overcome. Introducing any change to a large group of applications in an enterprise can be chaotic, and will often be more costly and take too much time if the changes are not compelling.

    To be sufficiently compelling, the database DevOps platform needs to be affordable, easy to understand, easy to set up and use with a wide variety of CI/CD pipelines. The following will be sufficiently compelling benefits to drive adoption:

    • Versioned, secure database environments or images managed as a DevOps artifact.
    • Images can be built and managed by the DevOps team, without requiring new infrastructure.
    • Multi-DB and multi-TB data environments in seconds.
    • Database environments can be served on demand for a CI/CD pipeline.
    • Lower-level database environment infrastructure is streamlined, with a single VM capable of supporting many simultaneous database environments which drives an immediate cost savings and positive ROI.
    • Data security and governance benefits with versioned database environments, with built-in and auditable security policies.

    What This Means

    Digital transformation is imperative for modern enterprises and organizations to remain competitive. However, the transformations of many enterprises and organizations are seriously impeded by the challenges of integrating databases into their DevOps application pipelines and value streams. A new DevOps database platform can allow the databases to be treated like any other DevOps artifacts that are orchestrated and automated within the DevOps toolchains.

    Suggested resources for automating database environment delivery include Windocks, Cohesity and other storage appliances that provide virtualized (cloned) database environments that can be adapted for DevOps purposes. I want to particularly acknowledge input from Windocks, who has introduced a DevOps data repo that features database virtualization on standard Linux and Windows servers, for versioned data environments and CI pipelines and cloud-native support for data delivery for containers, Kubernetes and conventional instances.

  • Puppet Survey Suggests DevOps Adoption is Stagnant

    Puppet Survey Suggests DevOps Adoption is Stagnant

    Puppet today published an annual State of DevOps report that suggests overall adoption of DevOps practices has stagnated.

    The survey of 2,650 IT, development and information security professionals finds 83% are implementing DevOps practices, to a degree. However, only 18% of respondents are working for organizations that have highly evolved, automated DevOps processes in place. More than three-quarters of respondents (78%) are somewhere along the path toward achieving that goal.

    A full 90% of highly evolved DevOps teams have automated most repetitive tasks compared to only 67% of mid-level organizations, the survey found. In addition, 91% of highly evolved teams report a clear understanding of their responsibilities.

    Not surprisingly, the biggest hurdles identified are more cultural than technical. The most common culture blockers at mid-level organizations include a culture that discourages risk (21%), unclear responsibilities (20%), deprioritizing fast flow optimization (18%) and insufficient feedback loops (17%). In contrast, less than 2% of highly evolved organizations report resistance to DevOps from the executive level, the Puppet report found.

    Nigel Kersten, a field CTO at Puppet, said during an online panel that, after more than a decade of DevOps advocacy, people continue to be the biggest limiting factor when it comes to adoption. Specifically, from a compensation perspective, many IT operations personnel tend to feel undervalued compared to developers, noted Kersten.

    Another challenge enterprises encounter is trying to balance the need to create a standardized IT environment that can be managed and secured consistently and the desire of DevOps teams to embrace new tools as they see fit. Senior IT executives often tend to make platform decisions based on cost criteria that don’t always account for the need to enable agile development methodologies. Most of those platforms are designed to be managed by an IT administrator using a graphical tool rather than via a command line interface (CLI) and application programming interfaces (APIs) that appeal to DevOps teams.

    As a result, individual pockets of DevOps teams that select and maintain a platform of their choosing are emerging within enterprise IT organizations. Those teams often view their chosen platform as a better alternative for advancing developer productivity. Finding and retaining site reliability engineers (SREs), for example, to manage those DevOps platforms is a major challenge for most enterprise IT organizations. Most IT administrators lack the programming skills needed to become an SRE, and it’s not clear how many IT administers are inclined to acquire those skills.

    Conversely, it’s not clear how many programmers want to programmatically manage IT infrastructure at the expense of time they would otherwise devote to writing code. Organizations that can only hire a limited number of developers may not want to shift responsibility for managing IT operations to the proverbial left simply because they need more code to be written faster. Finding and retaining developers, in many cases, is just as much a challenge as hiring and retaining SREs.

    Advances in automation and artificial intelligence (AI) should make it easier for organizations to embrace DevOps best practices in the months and years ahead. However, it should be apparent to all that adoption of those best practices across an entire enterprise still requires a significant amount of time and effort, and will for many years to come.