staging-devopsy.kinsta.cloud

Tag: devops

  • Managing Endpoint Compliance-as-Code

    Managing Endpoint Compliance-as-Code

    The rapid pace of modern software delivery and increasing fleet sizes have transformed how organizations see and handle security strategies. Compliance-as-code is required for today’s organizations that need security as a fundamental part of business processes. It is no longer possible to manually manage compliance with dedicated security teams.

    Balancing security with growing infrastructure needs means IT security and compliance are non-negotiable. There is no room for more uncomfortable trade-offs between risk and an organization’s ability to deliver market-ready solutions quickly and efficiently.

    Complying with growing security and compliance regulations in today’s world of rapid innovation is a constant challenge that affects all organizations, large and small. Organizations are implementing automated solutions to eliminate reliance on traditional, slow, error-prone manual processes because of tighter regulations in the industry and greater risks associated with security attacks and compliance violations.

    What is Managing Endpoint Compliance-as-Code?

    Compliance-as-code is the codification of compliance controls to automate their adherence, application and remediation. It includes the tools and practices that enable DevOps and developer teams to incorporate the three key compliance activities:

    1. Detect: Discovering non-compliance through automated estate scanning and notifying stakeholders when offending infrastructure is discovered.
    2. Remediate: Correcting non-compliance by implementing immediate infrastructure changes to ensure the highest level of compliance at scale.
    3. Automate: Avoid non-compliance by automatically verifying that planned changes comply.

    There are many important use cases for managing the compliance of endpoint state as-code including confidential data protection, detecting shadow IT resources, network security, data exposed to public access and code licensing compliance.

    The Importance of Compliance-As-Code

    Most organizations still struggle to stay secure and comply with regulatory standards. Many organizations lack visibility across heterogeneous infrastructure and applications. They also possess inconsistent language for communicating requirements between development, security and operations (dev, sec and ops) teams and are unable to remediate findings.

    Compliance-as-code enforces a comprehensive compliance strategy that involves a robust set of controls; for example, managing data storage locations and access control management.

    Ensuring that these are followed at scale is critical to the success of a digital business as more organizations shift to the cloud, there are more possibilities of non-compliance. Reports suggest that a regulatory compliance violation can cost businesses $15 million on average.

    This violation is avoidable if DevOps, development and DevSecOps teams can automate compliance by adopting a different mindset and writing understandable code. The adoption of the “as-code” strategy led to infrastructure-as-code (IaC) which is one of the success factors in DevOps teams. Audits are streamlined and allow individuals to focus on higher-value activities.

    The Challenges of Managing a Fleet Through Compliance-as-Code

    Managing a fleet through compliance-as-code is not without its challenges. First, teams must bridge the talent gap as endpoint compliance is typically managed by IT admins, who are not inherently comfortable with an as-code approach.

    Typically, compliance requirements are dense and difficult to comprehend by other stakeholders. Understanding the compliance requirements of the organization and collaborating with other stakeholders to convert them to as-code is the biggest hurdle to getting started.

    Developer teams must find the right set of tools and integrate them into DevOps and DevSecOps workflows. This is not just a process change but a culture change. A boil the ocean approach will not just be painful but success will be close to impossible. Taking small steps and continuously improving is the mindset one has to adopt.

    Visibility Throughout the Ecosystem

    Whether you are talking about end-user devices like desktops, laptops or mobile devices or non-server edge devices like point of sale (POS) or IoT devices, endpoint devices are extremely vulnerable to exploitation from bad actors. Thus, it is necessary for organizations to bring them under the organization’s security policy and governance requirements just like server-side devices.

    There are a number of advantages to adopting an endpoint compliance-as-code approach:

    • Users can easily understand what is going on across the organization in terms of compliance status at any time.
    • The organization can keep an audit log of everything that occurred throughout the development life cycle and calculate an appropriate risk acknowledgment rating.
    • Organizations can evaluate any deviations from fundamental compliance requirements.

    The Continued Prioritization of Compliance-as-Code

    Managing diverse IT infrastructure fleets through compliance-as-code ensures that endpoint devices meet a variety of security standards. The IT team is empowered to create rules that can enforce compliance and establish security baseline standards within the organization. Compliance checks are validated and more visible at each stage of the software development life cycle.

    This guarantees detailed visibility and thorough compliance standards throughout the IT ecosystem.

  • What Does AIOps Mean for SREs?

    What Does AIOps Mean for SREs?

    It seems SREs are of two minds when it comes to AIOps. On one hand, AIOps’ potential is pretty exciting. By automating complex workflows and troubleshooting processes, AIOps could make your life as an SRE much easier.

    But on the other hand, some SREs may choose to view AIOps with disdain and distrust. They might think of AIOps as just another trendy buzzword that doesn’t live up to the hype and that can become a distraction from the SRE tools that really matter.

    Which perspective is right? Should SREs embrace AIOps with open arms, or should they resist marketers’ efforts to position AIOps as the latest, greatest tooling innovation in the IT industry?

    Those are subjective questions that we can’t answer definitively, but let’s at least gain some perspective by examining what AIOps means for SREs.

    What is AIOps?

    If you keep up with IT buzzwords, you’ve probably at least heard the term AIOps—which is short for artificial intelligence for IT operations—and that it refers to the use of AI and machine learning (ML) to help automate IT operations workflows.

    The big idea behind AIOps is that, by using AI and ML to perform advanced analysis of large volumes of data from IT systems, IT and SRE teams can solve complex problems more efficiently than they could using a traditional, manual approach.

    AIOps can, for example, help to surface the root cause of a performance issue in a complex, multilayered environment like Kubernetes. Or it could make recommendations about how best to resolve an incident.

    AIOps entered the IT lexicon in 2016, when Gartner coined the term. Now, six years later, it’s a relatively well-established tool domain.

    How SREs View AIOps

    Despite the fact that AIOps has been around for some time, it doesn’t yet appear that many SREs have bought into the AIOps revolution. Catchpoint found in a 2021 survey that just 7.5% of SREs reported that AIOps tools delivered high value to their organizations.

    It’s unclear exactly why there are such low rates of excitement about AIOps, but we’d speculate that there are a few key factors at play:

    • AIOps is a new term for an old idea: Many monitoring and observability tools have included at least basic AI and ML analytics features for a long time, even before tool vendors slapped the AIOps label on their products. SREs probably realize this and, to some extent, view AIOps as an effort by marketers to rebrand functionality that is not actually new.
    • AIOps is hard to implement: Setting up an AIOps tool requires integrating it with diverse data sources and customizing it to fit your workflows and environment. It’s possible some SREs view this setup work as requiring more effort than it’s worth.
    • AIOps can’t replace human insight: While AIOps tools may be useful to a point, it would be unwise to place blind trust in AIOps-based analyses or recommendations. For this reason, some SREs may believe that AIOps encourages organizations to rely too heavily on automated tools at the cost of the expert analysis and perspective that only SREs can provide. (This is similar to the fact that it sometimes makes sense to prioritize human intuition and expertise over playbooks.)

    From an SRE’s perspective, then, AIOps may appear over-hyped, overly complicated and underperforming compared to traditional approaches to SRE.

    What SREs can Gain From AIOps

    SREs’ wariness toward AIOps is valid—but only to a point. It’s important not to let suspicions about the limitations of AIOps turn into excuses not to use AIOps at all. AIOps has some value to offer to SREs, even if it’s not perfect.

    For example, AIOps can play a role in reducing toil. To the extent that AIOps tools can recognize complex patterns or discover interrelated data sets more quickly than human engineers, AIOps reduces the time SREs have to spend manually troubleshooting problems or poring over complicated information.

    AIOps also helps to enable a more proactive approach to monitoring and incident management. If AIOps tools can alert SREs to emerging issues before they would otherwise recognize them, AIOps can help the SREs to get in front of the problems before they turn into true incidents. That’s better for SREs and end users alike.

    There is also an argument to be made that AIOps can help SREs do more with fewer engineering resources. If you can use AI to automate some aspects of monitoring and incident response, you can maintain the same levels of availability and performance with fewer human engineers on hand.

    Conclusion: AI Won’t Replace SREs—But it can Help

    None of the above is to say that AIOps can replace SREs, or that it magically solves every problem SREs face. Anyone who believes AIOps is a silver bullet has bought into the marketing hype to an unhealthy degree.

    Nonetheless, AIOps tools do offer value to SREs by making their jobs easier in some respects and improving reliability outcomes.

    So, while it’s wise to maintain a healthy perspective about the limitations of AIOps, SREs shouldn’t rule out AIOps tools as one way to improve reliability engineering.

  • What is MLOps? DataOps? And Why do They Matter?

    What is MLOps? DataOps? And Why do They Matter?

    Let’s look at three distinct disciplines—DevOps, MLOps and DataOps. In 2011, Marc Andreessen famously proclaimed that software was “eating the world.” A little more than a decade later, it’s all but impossible to argue with his premonition as software has embedded itself into virtually every industry. 

    But now another shift is underway and AI is gobbling up software as we know it. Our emails can finish our sentences for us, connected cars are helping us stay safer on the road and predictive technology is helping minimize supply chain disruptions caused by COVID-19. In the near future, almost every piece of software we interact with will have intelligence built-in, and as AI gains acceptance and new use cases are rolled out, applications will only continue to become more intelligent. The widespread move toward ML-enabled software has the potential to be as transformative and far-reaching as what Andreessen predicted. 

    But innovation is rarely easy or simple—and intelligent applications are no exception. While conventional software involves one thing—code—and not much else, intelligent software relies on a complex relationship between three interconnected variables or legs of a three-legged stool.

    A Three-Legged Stool: DevOps, MLOps and DataOps

    Model:  The one or more AI/ML models (e.g., linear regressions, rules, deep neural networks, etc.) trained to recognize patterns in data and make decisions are what makes an application “intelligent.” 

    Data: Most often, a model is trained on historical data and programmed to emulate it. As a result, an application’s behavior depends heavily on the data—the raw input data, labels and features, as well as the new data to which the model is applied and ground-truth for this data.

    Code: Code is the language an application uses to function. When used in the context of intelligent applications, the code pillar might refer to business logic, calls to models, receiving outputs, decision making and calls to other data systems. 

    A Real-World Example

    To see how these pillars play out in real life, consider the example of a perception system for a self-driving car. Such systems use data (provided by offline sources and real-time cameras and sensors), models and code to determine myriad ever-changing variables (for example, if there are obstacles on the road, what lane markings look like, how fast a car can safely travel, etc.) While the following illustration is greatly simplified, it outlines the basic steps needed to build and deploy such a system and indicates which pillar each step belongs to. 

    Regardless of application type, the legs that intelligent applications stand on are always the same: Model, data and code. And if you try to remove any one of them, the application will topple over.

    With the three distinct pillars that make up intelligent applications, three distinct disciplines have emerged to keep each individual pillar functioning as efficiently as possible: DataOps, MLOps and DevOps. The rest of this post examines the details that make each discipline distinct and necessary to the model life cycle.

    DevOps

    Once upon a time, companies used waterfall processes to develop software. The process often moved so slowly that products didn’t realize their potential before they died or were replaced. In 2000, Agile was introduced, allowing developers to work quickly and iteratively in tight sprints where they were constantly delivering. By 2009, Flickr, for example, was the envy of the industry, routinely performing 10+ deploys per day. Companies seeking to increase their own productivity combined software development with established operations principles—and the term “DevOps” was coined. 

    As the familiar loop graphic below illustrates, the DevOps cycle goes from the planning stages to the creation, testing/verification, packaging, release, configuration, monitoring and then starts over again with planning. Over the past two decades, a host of unique tools have emerged to support individual steps in the DevOps life cycle—helping developers ensure quality by building in continuous integration and continuous testing from the start and helping speed up time-to-market with workflow automation. The DevOps ecosystem is robust, well-thought-out and always iteratively improving.

    MLOps

    MLOps helps teams move more quickly and efficiently through the model life cycle—a loop that involves six distinct steps, each of which contains unique considerations. 

    Since MLOps is related to DevOps, businesses sometimes try to adapt the processes used to write conventional software to execute the unfamiliar task of machine learning operationalization. But the approach doesn’t work, as the two disciplines are actually quite different.

    MLOps primarily deals with models, not code—and models are quite different from code. For example, model training and testing look very different from code building and testing. (What does “test case” even mean for a model?) And monitoring model performance is completely different than monitoring traditional software. (DevOps looks at things like CPU utilization, latency and throughput, while MLOps looks at model quality, drift and data quality.)

    Furthermore, the people that MLOps engineers are building for are quite different than their DevOps counterparts. As one might expect, the primary users of models are data scientists while DevOps generally serves software developers. Most often, MLOps engineers come from a background in ML and data science or software engineers who picked up ML. 

    As with DevOps, no single all-inclusive MLOps platform exists. Rather, different tools support different individual steps in the model life cycle and provide some amount of workflow automation to speed up processes. Different tools support specific pieces of the ML life cycle—but the ecosystem is still somewhat fragmented. Ideally, MLOps will evolve to offer more DevOps-like tools that work well together and offer more automation. For companies looking to build an MLOps platform, build a suite of supporting tools with speed, safety and automation in mind.

    DataOps

    We’ve seen very well-defined life cycles for the other two pillars discussed above; however, when it comes to DataOps, the life cycle is still being defined. But the overall objective of DataOps should sound familiar: DataOps involves processes and tools to ship high-quality data frequently—which requires a combination of data engineering, data quality, data security and data integration. 

    As with DevOps and MLOps, what the DataOps platform or DataOps engineer is doing is supporting the tools that help perform one of these activities and help build workflows. For example, a workflow may take data from a database, apply some transformation to it and then make it available for a business intelligence (BI) tool. That’s the kind of automation that is involved in DataOps. 

    As you might imagine, the skillset and the tools required for DataOps are quite different from the ones needed in MLOps and DevOps—so expecting a DevOps engineer to also be an expert in DataOps seems a bit unrealistic. While MLOps requires a much tighter integration with data than DevOps historically has required, MLOps is not the same thing as DataOps. More so than the other pillars that support intelligent applications, DataOps tends to be quite separate and distinct. 

    Conclusion

    From perception systems to enterprise-scale fintech software, every intelligent application depends on three distinct disciplines: DevOps, MLOps and DataOps. And therein lies the complexity of developing intelligent products. Each of these fields is distinct, dealing with a different set of questions and objectives within the ML life cycle and requiring different kinds of people and tools. However, they are all fundamentally united by a common goal: Optimizing quality and speed of iteration of the ML life cycle. The tools, practices and organizations that will rise to the top are the ones that will enable more seamless collaboration between different ML teams, helping them move through the ML life cycle quickly and efficiently. 

  • Service Level Objectives – Techstrong TV

    Service Level Objectives – Techstrong TV

    Service Level Objectives (SLOs) are being more widely discussed as they provide a clear path to delivering exceptional results by defining clear reliability goals. Kit provides updates on the community, use cases and best practices. The video and a transcript of the conversation are below.

    Recording:              This is Digital Anarchist.

     

    Alan Shimel:            Hey everyone. Welcome to another Techstrong TV segment. I’m really happy to have my friend Kit Merker all done up in his Seattle Kraken hat and his winter beard for the pacific northwest up there joining us today. Ken of course is the COO of Nobl9, a recent winner of the devops dozen awards as best new company on the devops space. Hey Kit. Welcome to Techstrong TV.

     

    Kit Merker:             Hey Alan. Great to see you.

     

    Shimel:            It’s good to see you again my friend. I’m thinking Kit we may actually see each other in person soon. I’m hoping.

     

    Merker:             Wouldn’t that be great? Hopefully somewhere sunny and I can get rid of the beard.

     

    Shimel:            Well I’ll tell you. I was just looking at airfare for Valencia, Spain for Cube Con in May. I don’t know if you guys are planning on being there for that one, if that’s like a CNCF thing you go to or what or – actually we have a road show coming up in Seattle at some point. I’m not sure when. We’ll talk about that.

     

    Merker:             Well in May I’m not sure about Cube Con but we will be having SLO Con and we’re doing it online again this year in May so that’s what I’m looking forward to.

     

    Shimel:            Yes. I think I saw that. What are the dates?

     

    Merker:             It’s May 9th through 12th. And that will be completely online and asynchronous and attend while you work just like last year. So we’ll have tons of speakers and all the fun from last year that we had and hopefully even bigger. The CFP is open right now so if you go to slowconf.com you can submit a talk. And it’s going to be exciting. And you guys covered the, were a media sponsor last year so hopefully we’ll have you back.

     

    Shimel:            Yes, we were.

     

    Merker:             Not to put you on the spot.

     

    Shimel:            I remember. We’ll talk about it.

     

    Merker:             All right.

     

    Shimel:            So that works out well because I think Cube Con if I’m not mistaken is like May 16th.

     

    Merker:             Yeah. We intentionally didn’t put it the same dates so.

     

    Shimel:            Yeah. So that will work. Cool man. Let’s talk a little Nobl9 though.  I guess we really shouldn’t. I’m assuming everyone knows who Nobl9 is. But I forget you’ve still only been around two, what, two, three years and at most. And not everyone may be familiar. Why don’t we back up and start there?

     

    Merker:             Sure.

     

    Shimel:            Tell people about Nobl9.

     

    Merker:             Of course. Yeah. So Nobl9 we make a service level objective platform, SLOs. And the whole idea of SLOs is about measuring your observability system and bringing new context, shared context so that you can make better decisions and better automation within your organization. So if you think about how you’re pulled in all these different directions, right? Some people on your team say we’ve got to ship features. Some people say if we don’t make it reliable and perfect then we’re going to hurt our reputation. If we don’t fix our tech deck we’re going to burn out all of our employees and we’ll have the great resignation to deal with. They’re all getting pulled in all these different directions.

     

    And what’s interesting about this is that we have actually more data about our software than we’ve ever had before. We have source control. We’ve got our AWS Cloud. We’ve got our Kubernetes cluster. We’ve got all these observability tools. We have so much data and yet we come to these different conclusions about what we should work on. And this is a big problem for so many organizations. So what Nobl9 is doing is we’re bringing some additional context. This is what’s really missing is people need to understand in an organization what are the risks to the system. What are the expectations that customers have and what are the tradeoffs we have to make in order to deliver the right kind of service to our customers?

     

    So what we’re doing with service level objectives in Nobl9 d is we collect all this observability data, we add this context and we drive automation and decisions so that whether you’re on call, you’re planning your future roadmap or trying to justify your investments you can see exactly what the impact is on the business and everybody can start pulling in the same direction. And that’s really what we’re doing. And like you said we’ve been around for a couple years. We now have a product in market. We both have kind of an entry level thing called Hydrogen, an enterprise class product that’s being used by some of the world’s largest enterprises and it’s very exciting and we’re having a great time.

     

    Shimel:            Very cool. Kit when we look over the evolution of the last couple years, right, ‘cause I remember with Nobl9, right, we first thought of it around well this is an SRE kind of thing. Right?

     

    Merker:             Yeah.

     

    Shimel:            And SREs early on were inherited a world of SLAs. Right? Where it was perform or a quick bullet to the back of the head will work.

     

    Merker:             Metaphorically speaking.

     

    Shimel:            But that wasn’t a very forgiving model or a model that would really kind of foster cooperation and team building and a lot of the principles behind devops, right, about how we get stuff done. It’s very hard to have a blameless postmortem when if you blew an SLA someone’s job is on the line. Right?

     

    Merker:             Well it’s really interesting because I think definitely in the sort of earlier I would say IT service model, right, where you have SLAs, especially if you have maybe outsource contracts. Right? SLAs became really a form of punishment. Right? One of the interesting things that I learned from sort of the Google SLA model is that there’s an assumption of reliability at Google, an assumption of infrastructure investment that’s not actually there in most enterprises. The SLOs that are prescribed by the SREs in the Google model are really trying to hold back over investment in infrastructure and this sort of gold plate infrastructure problem. Well most companies in the real world.

     

    Shimel:            It’s the other way.

     

    Merker:             It’s the other way around. Their issue is they can’t keep up with the reliability needs and the feature pressure is really the issue that they have. And so with this SLO model and being able to define them in fine grained ways across multiple data sources, being able to use it with automation. What changes is that you can start to put guardrails around these different conditions that people care about. You start to model the business. And this is really what we’ve got to get to. It’s about building the technology for digital business because all the businesses are digital now. Right?

     

    Shimel:            Right.

     

    Merker:             So you’re point about kind of like how you adopted sort of like devops method and bring this to the table, I mean really devops is trying to get better at building software, become more kind of diligent and efficient and make the right tradeoffs. It’s about ownership, right, a sense of responsibility but not punishment. And I think once people realize – actually I just had this customer from Flexera. We just did this testimonial video together. And he said it’s like once people realize that they weren’t building SLOs to point blame at them or punish them, that we were trying to do it to help them so that the people in the organization could see when they had problems, right, when they needed help.

     

    Then people really got open to this idea. This to me this is the game changer. If we can help organizations become more transparent, to be able to do more through APIs instead of meetings if you know what I mean. Right? Like using the data and the systems to help make these better decisions. This is what changes culture. A lot of people want to talk about reliability culture and devops culture and agile culture but that’s such a scary idea. The way I like to think about this and what we’ve talked about with our customers and community members is it’s about how you make decisions together. Right?

     

    And if you can make lots of decisions in a way that is supportive of the strategy, supportive of the business and also the realities of the pressures that we’re on as humans. And let’s not forget that everybody is going through a hard time and maybe not totally happy being on call and all these other things. Taking all that into account and making a decision together this is what becomes culture, not the other way around. Right? You don’t change culture first to get the decisions. You start making good decisions and then the culture emerges from that. And that’s I think a really important idea.

     

    Shimel:            I agree with you 100 percent. No doubt about it. And it is. But when we focus on devops being about delivering software and clearly that’s the goal. Right? Delivering software better, faster, higher quality. In order to do that you need just that whole culture thing. And some people say oh culture is BS. It’s not. Having a team that works together, having goals that are clearly enunciated, right, we’re all trying to do – you understand what the guy or gal next to you is doing. Right? And why they’re doing it and how you fit into it. That’s all part of that. All of those things help you get software better, faster, higher quality. And that’s where it is.

     

     What’s interesting about Nobl9 Kit from where I sit, right, because I sit – I mean beyond being on different sides of the continent I sit in a different chair than you. I get to look at the whole market without a horse in the race so to speak. You guys have done a really great job of defining and owning SLOs. Right? I think for a lot of people in the marketplace, right, SLOs aren’t necessarily synonymous with Nobl9. But you’ve probably done more to promote SLOs and their use and why they’re beneficial than anybody else in market.

     

    Merker:             Well first of all I appreciate that a lot. Personally I think it’s a reflection of the community. I think it’s a reflection of a nerve that we hit at some level. Because to your point, right, goals that are well enunciated and it’s funny to me because I think oftentimes we think of goals as things that are kind of squishy. Right? We set stretch goals – oh sorry. I thought I muted everything.

     

    Shimel:            It’s never all muted.

     

    Merker:             We’re all muted. We want to set goals. Sometimes they’re a little squishy is what I was saying. So it’s like maybe you have OKRs. Maybe you have stretch goals. Maybe you have some different perception of what the goal is for your service. And what SLOs are doing is turning it into pretty much the most precise type of goal you could imagine which is code, right? Like actually describing it in code and something that’s checked in where it’s the truth. There’s no arguing about what it is. Now we can change it, right? We can always change our minds about what the goal is. But at any given point in time there is a goal that’s described and there’s a set of actions associated with that goal. And to me this is exactly what is so attractive to people in the market about SLOs because they’re no longer guessing.

     

    They’re no longer sending each other Slacks and emails and writing documents and putting it in dashboards. They have a very precise way of describing what they want and expect from their service that takes in account the risks and tradeoffs and customer expectations that they have for the service and then allowing the team to engineer to that goal, allowing the business to understand the tradeoffs. Right? Hey. If you want the service to be faster we’re going to have to spend more to buy more computing power. Right? Hey. If you want these features we’re going to have to put these other reliability concerns on hold and vice versa. This is something that brings visibility to all the little engineering decisions that are made on a daily basis and then aggregates them up to the size of an entire enterprise ‘cause it’s birds eye view of how the enterprise functions and doesn’t limit them to a prescriptive way. ‘Cause I think this is one of the other big challenges.

     

    When I look at the observability space or anybody’s infrastructure space is trying to bring a prescriptive guidance. I think today’s world is about the right tool for the job. Right? You’ve got to fit inside of an ecosystem whether that’s what language you use, what CICD system you use, what package manager you use, what source control you use. All these things are choices that engineering teams have to make for themselves. And you can’t necessarily dictate that Team A and Team B are going to agree on that stack if you will. Right? You give them some flexibility. But what you can get them to agree on is the concept of the service level objective. Like what should my service provide to customers in terms of availability, latency, performance, throughput, whatever the characteristics are that deliver that end to end service.

     

    This is what becomes almost the contract because if I know that I’m using your service and you’re going to give me x number of latency over y number of nines then I can build my system to account for it. I can decide what my retry strategy is, what my caching strategy is. Right? I can design and engineer a system that takes into account what your service says it’s going to do. Well that level of clarity allows engineering teams to operate in a completely fundamentally different way because now they aren’t guessing and they’re not hoping and they’re not wishing. Right? They’re working toward a clearly articulated goal. And I think this is what’s so important. So now the whole idea of us promoting this concept. I mean we’re just like we’re in love with SLOs. I mean I don’t know how else to explain that. Like we’re just like losing our mind about how amazing it is.

     

    But it is really about the community. We now have literally thousands of people that are involved in the SLO, the SLO Conf community. We have the monthly meetups. The conference as I mentioned will be in May. And we’re expecting hopefully about 5,000 people to show up this year. Last year we had 2,200 registrations. So we’re pushing hard for this but based on what we’ve seen in the market and the companies that are literally deciding if they’re going to build or buy an SLO platform right now.

     

    There is a huge groundswell of interest in this concept to take their observability systems, their operation systems, reliability systems to the next level. And all the practices and tools and techniques and tricks everybody wants to share it and learn together which is I think the most exciting thing. And it fits in with so many other areas, incident response and chaos engineering and how we do monitoring and logs. All this stuff kind of fits into this whole story about SLOs. Even finops, right, even like cost management, all this stuff fits into this same story. So yeah. It’s kind of overwhelming to be honest how much the market seems to have appeared and materialized over the last couple of years with SLOs.

     

    Shimel:            Cool man. Hey Kit, we’re up on the clock here. So you mentioned SLO Con and SLOconf with an F dot com. Right?

     

    Merker:             That’s right.

     

    Shimel:            Right.

     

    Merker:             Yep.

     

    Shimel:            For people who want to maybe submit a talk or just find out more about attending this year’s virtual conference in early May. And then for people wanting to get more information on Nobl9 is it Nobl9 dot com or IO? I forget.

     

    Merker:             Nobl9.com. Yeah. N-O-B-L number 9, dot com.

     

    Shimel:            That’s it. Hey Kit, man. It’s great seeing you. I hope to see you in person sooner than later here. But keep doing what you’re doing man and congratulations on the devops dozen award as well.

     

    Merker:             Thank you so much Alan and hopefully we’ll catch a hockey game together.

     

    Shimel:            Come on down. Actually I don’t have Seattle Sounder tickets when they’re playing the Panthers but we’ll look into it. I’ll talk to you later man.

     

    Merker:             All right..

     

    Shimel:            Great seeing you.

     

    Merker:             All right. Take care Alan.

     

    Shimel:            All right. Bye bye.

     

    Merker:             Bye.

     

    [End of Audio]

  • Analyzing SRE Job Postings

    Analyzing SRE Job Postings

    You can find plenty of high-level definitions out on the internet about what site reliability engineering means and what site reliability engineers do.

    But if you want to understand what it’s actually like to work as a site reliability engineer, there is perhaps no better source than job descriptions. SRE job ads explain what real companies are looking for and what they expect on a day-to-day basis.

    To provide insight on what SRE looks like in practice, we’ll look at some job ads from several leading companies. As we’ll see, each company has a slightly different take on what the site reliability engineer role entails, but it’s possible to identify some core themes that underline what businesses expect.

    To understand the content of SRE job descriptions, let’s examine examples from four leading companies.

    GitLab’s Job Ad

    GitLab’s SRE job description contains most of what you would expect from a typical site reliability engineer job ad. It highlights the importance of having a mix of technical skills and collaboration skills. It also emphasizes expertise in areas like Unix/Linux, infrastructure-as-code and Kubernetes.

    What may be slightly surprising, however, is the emphasis that GitLab’s job description places on programming skills. GitLab expects SREs to know how to code in specific languages, like Ruby and Go. The takeaway is that SREs need to know more than just how to use code to solve IT operations problems. They must also be able to develop applications or contribute code to application platforms (the job description mentions “contributing to code in GitLab” as a required SRE technical skill).

    Also interesting is the fact that GitLab expects site reliability engineers to be able to write blog posts. Blogging may not be one of the first skills that come to mind when you think of a site reliability engineer’s responsibilities, but at GitLab, at least, it’s part of the job.

    LinkedIn’s Job Ad Template

    LinkedIn provides an SRE job ad template that businesses can use when posting their own SRE jobs on the site.

    LinkedIn’s ad is pretty high-level (which is unsurprising, given that it’s a template rather than an ad for a particular job). For the most part, it doesn’t mention specific tools—although the main exception is programming languages, where it says SREs should know how to code in languages like C, Python, JavaScript and Ruby. Like GitLab, then, LinkedIn clearly sees development as a core SRE job responsibility.

    The LinkedIn job description also places a heavy emphasis on the ability to “gather and analyze metrics.” It doesn’t use the word “observability,” but that seems to be what LinkedIn is getting at.

    An SRE Job at Microsoft

    Continuing the theme of development-centric SRE work, a recent job ad for a specific SRE opening at Microsoft reads a lot like a DevOps engineer job description. The ability to “design, write and deliver software” is the top responsibility listed in the ad, which also mentions familiarity with CI/CD tools.

    The Microsoft job description also mentions the importance of written and verbal communication skills. Unlike GitLab’s ad, this one doesn’t say SREs will need to write blog posts; still, it’s clear that companies expect people in this role to be creative communicators in addition to technical experts.

    It’s worth noting, too, that Microsoft wants site reliability engineers who know both Windows and Unix. If you need further proof that Unix skills are absolutely central to SRE work, look no further than this job ad from the company that created Windows.

    Amazon’s Site Reliability Engineer Ad

    Even Amazon—a company that is more of a platform provider than a software vendor—places major emphasis on the ability of SREs to code in one of its recent SRE job ads.

    Interestingly, the Amazon job also highlights networking skills as a major qualification. That’s understandable given the complexity of networking in the AWS cloud. But it’s a bit surprising because most other site reliability engineer job ads don’t call out networking expertise as a key skill—despite the fact that network engineering is critical to SRE work.

    The Amazon job description mentions the importance of an applicant’s willingness to be on-call, too. We suspect that’s critical in virtually any SRE job, but at least Amazon is upfront about the fact that, at time, you’ll be woken up at 4 a.m. to manage incidents.

    Conclusion: Takeaways

    There are a few key takeaways from these various SRE job descriptions.

    • Development skills are absolutely critical in order to be an SRE. Scripting for administration or IT Ops purposes is not enough on its own.
    • The ability to communicate clearly—including both verbally and in writing—is more important to SRE work than you may realize.
    • Network engineering skills, too, are a crucial part of SRE jobs, at least at some companies.
    • In practice, SRE job descriptions sound more similar to DevOps job ads than you might expect.

    This is not to say that all SRE roles are identical or that all companies expect the same things from their SREs, of course. But the jobs ads or templates we’ve surveyed do highlight how the high-level understanding of SRE translates to on-the-ground reality at actual companies.

  • DevOps for Customer Experience

    DevOps for Customer Experience

    Charlene O’Hanlon and Amy Hudson of Cyara discuss the concept of DevOps for customer experience, which takes DevOps methodology and focuses it on CX software. But there are pitfalls DevOps teams face when dealing with customer experience software. The video is below followed by a complete transcript of the conversation.

    Announcer: This is Digital Anarchist.

    O’Hanlon: Hey, everybody, welcome back to Tech Strong TV. I’m Charlene O’Hanlon, and I’m here now with Amy Hudson, who is the VP of Global Center of Excellence at Cyara. Amy, thank you so much for taking a couple minutes and talking with me today—I really appreciate it.

    Hudson: I’m happy to be here.

    O’Hanlon: Great, great. So, Amy, tell me a little bit about Cyara to start off.

    Hudson: Oh, absolutely. Cyara has been around for about 10 years at this point, but we really focus on what we call customer experience assurance. Companies spend a lot of money in designing the right customer experience and all the technology that supports it, and Cyara is in the business of making sure that you are getting the experience you’ve invested in. And if you are, how do we help you continue to innovate and double down? And if you’re not, how do we help you fix it very quickly so you can stay ahead?

    O’Hanlon: Alright. That sounds great, and I imagine your business has been booming over the last year or so, because customer really came to the forefront for a lot of organizations as they kinda had to double down on their digital transformation efforts just to stay alive over the pandemic, over the last year and a half or so. And some of them have done it really well and others have not done as well.

    So, what are you guys seeing in the market right now as far as customer is experience is going?

    Hudson: Yeah, no, that’s a great question, because it has changed a lot in the last year, with the pandemic. I mean, listen, customers have historically always been demanding, right? That’s in our nature as customers. But now, with their hands on the technology, customers have very high expectations right now. And switching costs are low. So, if a customer doesn’t get what they’re looking for out of an experience, you may lose them.

    And so, the importance of having a good customer experience and staying ahead of it is at an all-time high from our perspective.

    O’Hanlon: So, we’ve been, obviously, you know, through staging-devopsy.kinsta.cloud, we focus on the benefits of going with a, kind of a DevOps methodology and mindset and culture shift and all that good stuff. And we’ve seen kind of that whole premise of DevOps kind of shifted to other parts of an organization as they recognize the benefits of kind of blending two different maybe mindsets and areas of an organization together to kind of streamline operations and actually build a better product in the long run.

    I imagine customer service is not—has not been left out of the conversation when we’re talking about organizations adopting those DevOps principles for customer experience.

    Hudson: Yeah, that’s right. It’s interesting. So, about four or five years ago at Cyara, we started seeing DevOps fold into the contact center/customer experience space, but it was a little bit more of a rarity then. DevOps hadn’t necessarily intersected with customer experience yet at that point, but over the last three couple years or so, we’ve really seen DevOps become much more commonplace in the customer experience space. Which I think is a good thing, because, exactly what I mentioned before—you need to stay ahead. You need to be able to innovate quickly for customer experience, and I think these companies are starting to see that.

    So, we’re starting to see DevOps adopted almost universally. Either somebody has started it or is halfway through or a little more mature, but it’s in the conversation with almost everybody we talk to these days in terms of customer experience.

    O’Hanlon: That’s great. That’s really great to hear, because there are certainly a lot of benefits to DevOps in the customer experience space, you know, not the least of which is being able to service the customer much more effectively and quickly.

    But it’s not kind of a one to one adjustment, though. You know, it seems like, especially since, when we’re talking about DevOps, we recognize that each DevOps implementation and each DevOps experience, if you will, is different. You know, everyone is kind of a snowflake.

    So, how does that equate to the customer experience and how might that actually kinda trip up organizations if they’re looking to kind of add DevOps to their customer experience?

    Hudson: Yeah, absolutely. So, when I first started at Cyara, I led a very small strategic consulting group, and we worked with customers on how Cyara can help them through any major transformation. And because of the shift of the industry, it ended up being a lot of DevOps transformations. And so, we’ve seen customers at multiple different points in the journey, and the struggles that they’ve bumped into. And I think there are a number of different kind of pitfalls if you will, or lessons learned from that.

    And, you know, one of the first things—and we can go into as many of these that are of interest, but one of the first things and most important that pops out at us is that leadership level cultural shift. I think, so frequently—and I’m leading with this because if you don’t get this part right, it’s really gonna hold you up. But so frequently, DevOps is viewed as a technical or process change and so, it happens at the day to day level. But really, it is a mind shift, cultural shift, has to be top down, has to be embedded in the culture, and it has to be leadership driven.

    O’Hanlon: Mm-hmm, yeah. I agree, agree, and that goes across an entire organization, no matter where they’re implementing DevOps, whether it’s in IT or if it’s in the back office or marketing. Because we’re seeing this blending of so many different disciplines in an organization these days as ways to advance and accelerate just their ways of doing business.

    But to your point, if we don’t have, or if an organization doesn’t have that buy-in from the top, no matter where they’re implementing DevOps in an organization, it’s doomed to failure.

    Hudson: That’s right, that’s right. And it’s, you know, and it’s not just the kickoff, it’s not just somebody at the C level saying “rah rah” at the beginning.

    O’Hanlon: Right.

    Hudson: This is, like, through the process, embedding into how you motivate, incentivize, how you reward people, how you—you know, in your regular cadences. It’s everywhere. It needs to be treated as a cultural shift.

    O’Hanlon: Yeah, yeah. I totally agree. Now, what about, you know, beyond kind of that initial, as organizations start to kind of adopt this mindset and these principles, what are some of the things that maybe we call it kinda DevOps Phase Two or DevOps 2.0, the Second Day DevOps—what are some of the things that you guys are seeing that organizations need to be very mindful of to make sure that they are successful?

    Hudson: Yeah, another thing that we’re kinda bumping into at more of the kind of operational level is, as you know, the importance of flow in DevOps, right?

    O’Hanlon: Right.

    Hudson: You don’t wanna disrupt the flow. And to do that well, you have to think about automation, right? It’s just—it’s a no-brainer. It’s a pillar, right? And to me, when I think about automation, there’s a lot of different pockets you can automate. But for me, I think primarily around deployment automation and testing automation.

    We did an interview—or I did an interview with a consulting, one of the big four consultants with the DevOps expertise and a customer. And both of them agreed that automating the testing, while it’s not always completely simple or straightforward, it is, they would almost consider it the low hanging fruit, right? Get the testing automated so you can just kind of plug and play that part of it, don’t disrupt the flow.

    From a deployment perspective, there’s a number of different environments, right, to get something into production. So, the more you can automate from that perspective, the more you can continue to push things out and learn fast and fail fast, if you will.

    And so, finding those pockets of automation and having a strategy on how you can automate as much as possible as to not disrupt the flow is something that we’re really recommending to folks to realize the total benefit of the transformation.

    O’Hanlon: What about from a people perspective? You know, DevOps is as much about culture, as we know, as it is about the processes and the technologies. So, are there ways that organizations—maybe they’re not thinking about particular things when it comes to culture and people in their adoption of DevOps specifically for customer experience?

    Hudson: Yeah, sure. So, I would answer that a couple different ways. The first one I would talk about is essentially around—this is, again, it’s a culture thing. One of the pillars of DevOps is to create an environment that encourages learning and a safe place to fail. And so, you want your teams to be encouraged to fail if they have to and learn from it and get better the next time and feel safe doing that. So, that’s one from a person perspective.

    The other thing I would mention is just around measuring, key performance indicators, right? You can’t know if you’re successful without measuring, right?

    O’Hanlon: Mm-hmm.

    Hudson: And then publishing those KPIs so they’re transparent. Transparency is really important, so everybody is charging towards the same goal, and everybody feels like they are part of achieving that goal. So, let’s get the performance indicators out there, let’s be clear what we’re trying to achieve, let’s make people feel safe in that process. And, you know, that’s kinda the culture you wanna build around it.

    O’Hanlon: So, when we’re talking about customer service applications and software versus, say, an internal process or internal DevOps process or an internal application that is being advanced by these DevOps processes, what is the difference when we’re looking at what may be what those KPIs are or what the service level objectives are, things like that? Are there certain things that an organization needs to be more mindful of when they’re working on a customer experience type software rather than maybe more internal focused or maybe other customer that’s not maybe customer facing, end user customer facing?

    Hudson: Yeah, I think that’s a great question. And what we’re trying to do within our industry is create this mind shift where you are thinking about the end customer all the time—not just in the production environment, you are always thinking about the end customer experience through every step of the journey.

    And I’ll give you an example of what I mean by that. Traditionally, the contact center space, how they would monitor things in testing or production is kind of server loads or capacities, right? If it’s reached capacity, it’ll set off an alarm and that’s a problem, right? You don’t want it to be at capacity.

    Great, but you might not know what the customer experience is if that server is at capacity, right?

    O’Hanlon: Mm-hmm.

    Hudson: And so, at Cyara, what we’re trying to do is say—absolutely, that server might be at capacity and that’s a problem, but what is the customer experiencing in the moment right now in production? But we also wanna test in Dev and QA and UAT and performance testing, what the customer experience would be. So, if there’s an issue that comes up, you know it could be a pretty serious one, it could be calls being disconnected. If, in your revenue generating phone lines or chat windows, customers can’t get through—you wanna know that before it gets into production.

    O’Hanlon: Mm-hmm.

    Hudson: You don’t wanna know necessarily that—or it may be less helpful to know if a certain piece of technology is at capacity. What’s most valuable is to know what the customer would experience in that moment, even before it gets into production, if you can do that.

    O’Hanlon: Interesting. You know, it’s really fascinating to have the conversation about DevOps in different vertical markets, because everybody’s got something different that they’re really, really focusing their efforts on, but at the end of the day, it really is all about just making sure that the end user experience is the best it can be, and that the application is working as advertised and certainly that there’s a level of security as well within that application.

    So, I do imagine that security is also a big part of the equation, especially with the customer experience and the amount of sensitive data that a lot of contact centers, to your point, that they’re dealing with on a regular basis. So, heavy emphasis on Sec on DevSecOps?

    Hudson: Oh, absolutely. And that’s really from two ends. One is that customer data, right? There’s—you have to protect that. There are sensitivities, and those need to be protected, and we need to invest in those, and you need to be able to test that in production or prior to that if possible, right?

    But the other side of it, too, is around the regulatory requirements, right? Customers will—our customers, large companies—will get fined if certain requirements aren’t met, right? And so, from a pure business perspective, you’re motivated to make sure the customer is experiencing that requirement. But then from their own protective perspective, you’re also motivated.

    And so, DevSecOps, we really encourage people to fold security in very early in the discussions. It’s often one of the last things to come up—“Oh, wait, did we talk about security, did we bring in that team?” Fold them in early, complete transparency, get the conversations in and weave them into the process, because it’s critical.

    O’Hanlon: Excellent, excellent. Amy, this has been a great conversation. I’ve really, really enjoyed it, and would definitely love to have future conversations with you about other topics related to DevOps and the customer experience, because I really think that this is something that is—it’s not gonna be a one and done conversation. It’s like DevOps. Customer is an evolving conversation.

    But for now, thank you so much for taking the time and having the conversation with me. I do appreciate it.

    Hudson: Thanks for having me. It’s a privilege to be here.

    O’Hanlon: Alright, great. Alright, everybody, please stick around. We’ve got lots more Tech Strong TV coming up, so stay tuned.

    [End of Audio]

  • Irreconcilable Differences: DevOps Grapples With Issues

    Irreconcilable Differences: DevOps Grapples With Issues

    DevOps issues can be a thorny subject. The coming together in harmony of Dev (developer) and Ops (operations) represents an arranged marriage of sorts that, for some, will never have a ‘happily ever after’ ending as both parties ultimately grapple with irreconcilable differences.

    But despite the naysayers, the very existence of DevOps as a portmanteau (and perhaps sister labels like DevSecOps, to form a security sandwich) is, in many ways, a validation of the need to form a bond of tighter unity and harmony between two factions that have often been at odds with each other over the years.

    Even with the popularization, extrapolation and democratization that DevOps has enjoyed through the last decade, not everyone will be convinced of its worth and/or its effectiveness—some even doubt its very existence. Some say that DevOps already existed without a formalized label if you look at the software development practices IBM championed a quarter-century ago.

    One thing we can be certain of is that DevOps almost always throws up a few meaty bones of contention. Let’s take a closer look at these issues in the hope that we can harness some of the angst and turn it into positive energy for a DevOps-enriched future. If we are to analyze the most contentious and sensitive DevOps issues in this space, what should we be aware of and how do the big issues break down?

    Is it a Bird? Is it a Plane?

    Arguably, the biggest problem DevOps faces is categorizing it. Some call it a methodology for software application development. Proponents of ‘classic’ software methodologies disagree with this idea as, for the purists, a software methodology can only be Agile, waterfall, extreme, rapid, joint or Scrum.

    These semantics mean that many people would prefer DevOps to be classified as a culture, a workplace ethic, a technique, a culture and/or a mindset or, perhaps in the vaguest terms, an approach. To be truly successful with DevOps you must take a holistic approach and look at the entire development cycle from beginning to end. Closely evaluate the processes that go into development to find bottlenecks and roadblocks to productivity. By finding the areas where your current processes come up short, you will be able to identify the areas to emphasize when implementing a strong DevOps strategy.

    Cloud-Native DevOps

    Anyone who hasn’t been living in a cave since the turn of the millennium will know that cloud computing has grown, faltered occasionally and, ultimately, blossomed to become a new standard barometer for all organizations on their journey toward so-called digital transformation.

    That’s the good news. The not-quite-so-good news is that cloud (and the contemporary drive to go cloud-native) is still complex from initial deployment to ongoing maintenance, management and reporting.

    Taking DevOps into the world of cloud-native will ultimately pay off, but first, there will be challenges as teams get used to the new shape, form and function of the altogether more abstracted and virtualized software toolsets.

    The key to getting cloud-native DevOps right will be a prudent approach to monitoring and observability. As cloud-centric DevOps teams ingest the logs, metrics and traces they need to perform solid and dependable system observability, they will need a consistent set of tools to monitor and observe systems in the long term. Within this practice, it will also be important to remember that monitoring shows you lower-level granular information related to system performance. Observability includes all the information gleaned from monitoring and also includes higher-level tools, some of which will be data visualizations for DevOps practitioners to use. When the two are combined for cloud-native DevOps, system health can be optimized, preserved and maintained while any debugging or root cause analysis is executed at the lower level.

    Machines in the Application Loop

    With machine-to-machine events now happening more prevalently inside the internet of things (IoT), the need to capture, integrate and analyze the event streams that go on outside of the human loop and bring those log metrics back closer to the sphere of ‘human’ software engineering is not a plug-and-play quick fix.

    Getting machines to play nicely and be a part of expansive and progressive DevOps teams should be a given. After all, they’re just machines. In reality, there will be specific skills needed and human DevOps personnel will be required to get used to updates and other alerts that they have not considered before.

    AIOps and Automation Limits

    Related to our last point on machine events, we need to think about AIOps. When we build AI-driven DevOps and create AIOps, we will need to stop and question where we set limits on process automation and where the wider elements of automation hit the limiter/restrictors. To know just how far we should build out our AIOps practice, exhaustive auditing and analysis must to be carried out; the whole technology proposition here is no simple point-and-click affair. Even when we do establish workable AIOps, know that there will be perimeters.

    Add all that to the fact that some AIOps implementations will require what we like to call ‘human handoff’ support when the AI intelligence engine doesn’t know what to do next and needs additional data and training—only then can we start to appreciate that AIOps itself is no simple plug-and-play affair.

    Lift and Shift Doesn’t Scale

    One of the unfortunate truths in too many real-world IT shops today is that, where the cloud is used, it is used poorly. Too many technologies are unceremoniously lifted and shifted to cloud instances that an organization procured. This makes cloud-centric DevOps harder to achieve. But why?

    Because what happens back down in terrestrial (i.e., non-cloud) programming is the development of individual cross-functional software teams. This silo-based approach happens due to corporate legacy structures, company management systems and, sometimes, for personal, parochial or cultural reasons. When different teams are all reinventing wheels (running diagnostics tests, setting up CI/CD pipelines, delivering upgrades and so on) that should be shared and accessible to all, a natural level of inefficiency surfaces.

    DevOps Issues: Conclusion

    These are the five most contentious DevOps issues in the near term, and it would, of course, not be too hard to find and categorize even more. In the future, we might even need to argue over which developers should be categorized as core devs and which core members of the operations department to include.

    For software developers, software architects—really, all the software engineering professionals who work in systems administration, database admin, testing, security and all other forms of operations—there’s quite a lot at stake and quite a lot to fight over. And if you ask me, that’s what makes DevOps great.

  • Why Developer-First is the Future of AppSec

    Why Developer-First is the Future of AppSec

    DevOps culture and rapid cloud adoption mean developers are shipping code faster than ever and, in many cases, security teams struggle to keep up. To avoid relegating security to afterthought status, organizations must shift left and adopt a developer-first approach to application security (AppSec).

    Organizations that depend on software development need a solution that accomplishes two essential things to adapt and survive: Risk-appropriate security measures and an even distribution of functions across the stack. First, risk-appropriate security measures must be applied to all software before it is delivered or deployed. Second, they must adopt processes that enable security functions to be distributed across the development stack in a way that will not slow down the pace of development.

    Just as the entire development process starts with developers, integrating security should also start with a developer-first approach. Developer-first AppSec is the future; here’s how organizations can evaluate tools that will help them adopt a developer-first approach.

    Why Developer-First is the Future

    Developers outnumber application security engineers by as many as 100 to one, and AppSec functions cannot scale if security practitioners are the only ones responsible for security. This imbalance suggests that organizational leaders must better distribute security ownership across the developer teams that own the software asset.

    As it stands today, many companies expect developers to build and deploy software faster than ever before. Security teams often can not keep pace with software development, and they become a roadblock to software delivery. To meet project development deadlines and their key performance indicators (KPIs), developers tend to leave security teams behind. They do not have the time or the incentive to slow down development because application security tools or processes are slow and cannot keep up.

    Security and developers are at cross-purposes in today’s software development model. They are pseudo-adversaries because AppSec teams know what it takes to make code secure, or at least they know how to find vulnerabilities. And, on the other hand, developers need to write code that works well and meets the sprint’s deadline.

    Friction Between Teams

    This growing problem creates friction between these two teams. It’s not that security teams don’t care about the organization’s need to produce quality software quickly or that developers don’t care about security. It’s just that each team is measured and incentivized to achieve opposing objectives. AppSec programs must create a developer-first approach to building software quickly and securely to combat this growing problem.

    It is not feasible for organizational leaders to provide security engineers with the entire business or environmental context behind software applications. In this sense, AppSec teams are working with limited vision. They may not see how the software fits into the organization’s big picture or understand its priorities.

    With limited contextual understanding, security teams necessarily rely on developers to make decisions about acceptable risks. If left to their own devices to view the world through a security-centric lens, AppSec teams could get bogged down—potentially spending too much time enforcing security measures that do not apply to the task at hand.

    Since developers are building the software and know what the services are designed to do, security engineers must defer to a developer-first approach so that decisions about security issues will align with the business context and acceptable levels of risk.

    Finding a Developer-First AppSec Platform

    Below is a condensed guide to help organization leaders evaluate potential AppSec platforms. These questions will help identify a solution that will enable the organization to build a developer-first security program.

    • Will the solution flood the developers and AppSec team with more and more results, or will this help them fix the essential gaps?
    • Will the solution provide context to help developers focus their limited time on security work that matters, or will it end up wasting time on things that may not even present a risk?
    • Will the solution help build security measures as guardrails to prevent security issues in the first place?
    • Is the solution flexible enough to enable building custom application security checks and policies in the SDLC?
    • Does the solution provide flexible ways of communicating actionable security information directly to developers in real-time within dev tools like source control systems, CI/CD platforms or engineering task management systems?

    There are an overwhelming number of application security solutions on the market today, most of which are designed to create security alerts about every conceivable threat regardless of the business context of the software being developed. More tools are then needed to manage the identified vulnerabilities.

    What This Future Will Look Like

    AppSec teams must recast themselves as security facilitators providing expertise for solving complex challenges and maintaining oversight of the developer teams’ security efforts. Developers should own tactical security tasks, but the AppSec team should continue to be the experts in making risk-based decisions and driving security accountability across the business.

    Inside many organizations, application security is only beginning to become integrated into developer workflows. This integration creates both opportunities and challenges. As organizations strive to make AppSec an enabler that aids developers in creating secure software quickly, developer-first AppSec programs let businesses make their digital transformation journey safer and faster.

  • 4 Ways to Uplevel Your Software Engineering Team

    4 Ways to Uplevel Your Software Engineering Team

    In my 20 years leading software development teams, I’ve witnessed the industry’s continuous growth alongside technology’s evolution. The only thing we know for certain is that the industry will continue to change and winning businesses will be those that build with change in mind, backed by effective, fast-moving software teams.

    This is the year we must focus on fine-tuning our DevOps processes and uplevel our teams, but where do we start?

    Below are four ways you can begin to transform your teams into engineering powerhouses, no matter how big or small or where they are in the world.

    1. Embrace New Delivery Models

    Remember when deployments were a big deal; a highly-anticipated event that often meant late nights? These days, many teams have managed to make deployments a non-event and are deploying any time of day. Those that haven’t adapted risk being left behind. The rate of deployments will only increase, so how teams handle this progression will be a deciding factor for success in an already competitive field.

    In 2022, teams should continuously evaluate their procedures as they push the boundaries of their tech stack. Navigating the complexity of the digital era—from massive datasets and machine learning to networks of third-party services—will force both developers and company leaders to embrace flexible tooling and processes to ultimately refine the risk/innovation tradeoff.

    2. Prioritize People and Culture

    While 2021’s Great Resignation saw many software professionals jumping ship in search of a fresh work environment, in 2022 we expect to see a Great Reshuffle. Realizing their new or current gig still may not be quite the right fit, engineers and developers will start looking for roles that are more in line with their long-term career and personal goals.

    Don’t fear this reshuffle. Instead, recognize it as an opportunity to build a strong and stable team. Be honest about what, exactly, the job entails and the hard and soft skills you’re looking for. Honesty throughout the recruitment process leads to a better match between employer and employee and also contributes to resilient company culture.

    3. Deploy Low-Code Solutions Strategically

    Pressure to get the highest productivity out of software development teams has led to the adoption of no-code and low-code solutions. But that means these tools are being used predominantly by people who already know how to code. The folks with big ideas who don’t actually want to be programmers remain unserved and untapped.

    Expect this disparity to soon lead to a split in the category of no-code and low-code solutions—those built specifically for optimizing developer teams, and those that can serve everyday creators. To really get the most out of your team, keep an eye on the low-code space and ensure the solution you use is the one that’s actually best suited for your goals.

    4. Build With Change Validation in Mind

    The growing adoption of cloud storage, service-oriented architectures, third-party API-based services and open source code (to name a few) makes modern software development monumentally complex.

    A way to manage this complexity is by pulling our focus away from the application itself and onto myriad sources of change that affect the application—a concept we can think of as change validation.

    Ultimately, change validation builds upon tried-and-true software best practices such as continuous integration, continuous delivery and release orchestration. Cloud-native architectures can be effective and scalable, but you need to have the right tools and systems in place to validate changes to your code at every step.

    In the past, the general consensus was that you shouldn’t test in production; you should just release and wait for the bug reports to tell you what’s wrong. But in the world of extensive software supply chains and cloud-native architectures, testing in production is the only way to truly know whether changes to your code are working as intended before releasing software to customers.

    In the new year, software developers will be of critical importance as organizations continue to expand on digital services and projects. Embracing new methods of software delivery, prioritizing people and culture and keeping an eye on how to best leverage automation tools like no-code/low-code and change validation will all help to transform your team for the better in 2022. This forward-looking approach and a willingness to reassess and pivot guarantees that your software engineering team will remain agile amidst a rapidly changing tech landscape.

  • Front-End Frameworks Enable Better Dev Collaboration

    Front-End Frameworks Enable Better Dev Collaboration

    The general public often thinks about software development in terms of a lone software developer hacking away tirelessly at their keyboard. Of course, that is an anachronism, at best. What is true is that collaboration among software engineers, especially with their colleagues in IT operations which evolved into DevOps, has become the norm for rolling out better code faster.

    We are on the cusp of the next big evolution as software development becomes more of a ‘team sport’ than ever before. Now, collaboration is extending beyond IT (developers and operations) to cross-functional areas across various business teams. Product development, marketing and others also are providing input and feedback into the software development process and, most significantly, becoming part of the process.

    Front-End Frameworks are Driving Collaboration

    Historically, software programming, including front-end programming, happened in local IDEs. However, the rise of modern front-end web frameworks has led to an entire class of developer tools that speed up the build process and allow applications to be deployed more quickly.

    There are a number of front-end frameworks that do this: Netlify, Vercel and Appsmith, an open source internal app builder, are three examples. These tools are getting widespread adoption, because, in a world where front ends are becoming more complex, they make things like hosting, deployment, authentication and serverless functions much simpler. The other reason for adoption is also because many of these tools are built on top of popular open source frameworks such as React, which already has a vast user base.

    This is enabling a new kind of collaboration that is unlocking a lot of value for all the stakeholders involved. Inspired by how Figma reshaped the design process by allowing real-time commenting back in 2016 or how Google’s suite allowed real-time editing, there is a similar phenomenon taking place over the last year in software engineering.

    Visual, Real-Time and On the App

    The most obviously striking feature is that collaboration is happening on the app (or the app prototype) and is becoming increasingly visual. Writing or editing code in the command line is great, but it doesn’t necessarily help the developer to visualize what is happening. Vercel’s Next.js Live, for example, allows teams to chat, draw and edit.

    collaboration

    This also means that collaboration is happening on the app instead of through tools like email, Slack and Jira, leading to targeted feedback without having to toggle between multiple tools.

    Collaboration and Feedback Loops Become More Inclusive

    While developers often collaborate at the code level, non-developers’ feedback is often given at the design level. This is more often the case for internal tools (or apps), which typically account for anywhere from 20% to 40% of engineering bandwidth in organizations.

    For example, Appsmith today has thousands of developers self-hosting and deploying internal applications for users who are often business or operational teams like support, sales and/or finance, for example. Often, these business users don’t even know who has built the application. Real-time commenting via front-end frameworks facilitates collaboration between software engineering teams and their business counterparts who can now communicate directly, provide feedback and collaborate with their engineering peers. This close collaboration empowers all parties, increases efficiency and saves significant time while improving the overall quality of the software and ensuring that it meets expectations.

    AppSmith

    In addition, non-engineers don’t need to familiarize themselves with a new tool like GitHub or go through a painful replication issue on Jira involving a variety of screenshots or screen recordings. Sharing feedback can be done much more easily.

    Better Version Control and Feedback

    For developers, having the ability to visualize a particular branch of an application and give detailed feedback on it and then track it on GitHub or GitLab speeds up the development cycle. Netlify, for example, pioneered this when it launched Deploy Previews earlier this year. Netlify takes this feedback and integrates it into project management tools like Jira, Trello and many others in addition to GitHub and GitLab.

    collaboration version control

    We are seeing other companies, like Sturdy, building on this much-needed trend of real-time collaboration in app development.

    Collaboration is the Trend

    Great software is the story of teams expanding collaboration beyond developers and there is a lot of good tooling available to help achieve that. This trend should continue to gain traction in the years to come because it taps into a number of other underlying trends in the software development space.

    First, the explosion of alternatives in the backend-as-a-service space, in which Heroku and Firebase were early pioneers, means BaaS is embraced by many more organizations for a variety of use cases. This trend speeds up development because they present a much more tightly integrated ecosystem of services and organizations don’t have to worry about deployment, hosting, content delivery network (CDN) and/or authentication themselves, for instance.

    Second, the proliferation of APIs is driving the emergence of more complex applications built on a variety of data sources. Everything is connected to everything. Of course, this also means that a lot more data is available to support decision-making, and this leads to many different kinds of applications being created both for external as well as internal purposes.

    And finally, the larger low-code/no-code movement is creating millions of new citizen developers and blurring the boundaries between who is and isn’t a developer.

    Summary

    DevOps marked the start of the evolution of software development into a team sport, and collaboration is now accelerating beyond the boundaries of IT. Like DevOps, the result of this collaboration in the software development process will be better software that propels businesses forward created faster than ever before.