staging-devopsy.kinsta.cloud

Tag: devsecops

  • Supply Chain Security: Has the Next SolarWinds Already Happened?

    Supply Chain Security: Has the Next SolarWinds Already Happened?

    More than two years after the now-infamous hack of the firm SolarWinds, the incident is very much in the foreground of conversations about cybersecurity defense, threat detection and incident response. As evidence of that: The SolarWinds incident was the subject of five separate talks and panels at the recent RSA conference in San Francisco. Dozens more talks addressed issues related to supply chain security that–directly or indirectly–invoked the SolarWinds incident. 

    That’s because the SolarWinds attack raises important—if not existential—questions for both software development and DevOps teams and the companies running the software. First: If sophisticated cybercriminals can insert malicious backdoors directly into application code from reputable developers, what organization can afford to blindly trust software? Second: If it is now the responsibility of software teams  to spot sophisticated, methodical compromises of their development pipelines, what is the best and most reliable means of doing that? Finally, if it is also now the job of software consumers to monitor for—and detect—compromised binaries from otherwise reputable vendors (like SolarWinds), how should they go about doing that given ongoing staffing and technology constraints? 

    The answers to these questions are matters of debate within DevOps, software development and information security communities. What is beyond debate is that the SolarWinds incident was not a fluke. In the months since that attack, attackers—both criminal and nation-state-backed—have leveraged software supply chain security blindspots to target development organizations as means of gaining access to sensitive IT environments. While we haven’t yet heard of another incident of the scale or importance of SolarWinds, it is entirely reasonable to assume that the next SolarWinds-caliber compromise could already be unfolding, but has yet to be discovered and/or publicly acknowledged. 

    Software Supply Chain Security Weaknesses

    SunBurst, the malicious implant in SolarWinds software, endures as a topic of interest and conversation for information security professionals and for DevOps and development organizations precisely because of the ways it exposed glaring weaknesses in the security of development organizations and software supply chains. As noted in my analysis of the hack in December 2020 and FireEye’s analysis of the same compromise, the SolarWinds hackers gained access to the company’s development environment months before they actually launched their attack. During that time, they closely studied the underlying source code and mimicked SolarWinds style and nomenclature as they added malicious code and integrated it into the SolarWinds Orion application via a compromise of the company’s build system.  

    Though carefully planned and conducted, the SolarWinds attack wasn’t foolproof. The attackers left clues, such as their additions of native Windows functions, use of cryptography, compression and more. The attackers obfuscated these using compression and Base64 encoding designed to fool simple threat hunting rules that looked for their plaintext variants and other suspicious strings. Still, the obfuscation repeated throughout the implanted code was enough to send up a red flag to anyone who bothered to look closely. That left the attack open to discovery. 

    The problem is this: Very few software publishers have the staff, expertise or tooling to comb through massive codebases to spot a handful of sketchy obfuscated strings or analyze large binaries for deviant behaviors before deploying them. As evidence of this, ReversingLabs sponsored a survey of more than 300 IT professionals at software-driven enterprises. It found that 87% were aware that software tampering, like what occurred in the SolarWinds attack, could lead to enterprise security breaches. About 40% of those surveyed said CI/CD toolchain exposures posed a risk to their organization. Still, barely a third of respondents (37%) said they had the means to detect such software-tampering attacks. 

    Threats to Software Development Organizations Grow

    Attackers are aware of this weakness within the software development pipeline, which is why threats to development organizations are metastasizing. Just in the last year alone, we have seen a series of attacks on development infrastructure and software supply chains. In one April 2021 incident, attackers were successful in pushing malicious, unreviewed code directly to the PHP main branch, ultimately resulting in a compromised, formal version of PHP being pushed to all PHP websites. That same month, there was a breach at the firm CodeCov, which involved unauthorized access to and modifications of the company’s Bash Uploader script. An investigation of that incident ultimately concluded that the alteration of a single line of code in that script by an unauthorized third party enabled the attackers to (potentially) export information stored in the users’ continuous integration (CI) environments. 

    More recently, ReversingLabs observed proof-of-concept and “white hat” activities testing so-called “dependency confusion” attacks, in which attackers set up malicious packages in public repositories like GitHub that squat on the names of private, non-public repositories and sport high version numbers. Misconfigurations in package management systems can result in these malicious components being imported in lieu of the legitimate, privately managed namesakes. As ReversingLabs found, a number of malicious modules discovered on the npm platform were used to target German technology, manufacturing and entertainment firms. It was later revealed to be the work of a white hat firm conducting red team exercises on behalf of those firms. Red team exercises aside, RversingLabs research also detected other, malicious packages in npm, as well as similar attacks targeting Ruby Gems and Python Package Index (PyPi). 

    And that doesn’t even count issues like the Travis CI flaw that exposed (potentially) millions of credentials to developers’ accounts on GitHub, Amazon Web Services and Docker Hub. Lapses like that can fuel supply chain attacks, giving malign actors the ability to insert malicious code into otherwise legitimate open source or proprietary code repositories. After reading such headlines, and without the means to check code intent, how can developers trust their software to update? And how can they do so while at the same time being forced to frequently update their dependencies to reduce or eliminate critical software vulnerabilities?

    Dev Teams Need New Tools

    For software development organizations struggling to meet delivery deadlines and satisfy customer expectations, reorienting to address amorphous software supply chain threats and attacks is a bridge too far. The same is true for enterprises struggling to staff security teams and simply keep on top of targeted phishing attacks or threats to public-facing IT assets. Too often, these entities only find religion about software supply chain attacks after they have occurred. 

    What software development organizations need is a way to secure the CI/CD pipelines and release packages, ensuring that their software builds are free of code tampering and other supply chain risks introduced through build systems. That should include the ability to detect compromises by monitoring for malicious software changes and enforcing security controls and failure conditions on software build processes to ensure that you can release your code with confidence.

    On the customer end, organizations need a way to detect malicious behaviors and unauthorized software changes in compiled binaries before they are released into production, while also monitoring for risks like exposed secrets or exploitable software vulnerabilities that might lurk in compiled code. 

  • The Secret to Secrets Management: Reduce Source Code Vulnerabilities

    The Secret to Secrets Management: Reduce Source Code Vulnerabilities

    We are in the cyberattack age. From Microsoft to Crypto.com, no company is completely safe from malicious attackers, and it seems as if a new major cyberattack is announced weekly, if not more frequently. As more companies not only go digital but expand their digital offerings and initiatives, there are more opportunities for cyberattackers. Security teams within IT organizations are hard at work to ensure that their company isn’t next in the headlines. However, as the tech world moves to everything-as-a-service (IaaS, PaaS, SaaS, etc.), the move toward everything-as-code is happening in tandem. As a result, many developers are unknowingly keeping sensitive data (secrets, passwords, certs, keys, etc.) stored in Git source code repositories. This information is often pushed to production which puts sensitive information at greater risk of exposure.

    The good news is there is a way to ensure that secrets are locked tight within source code. Let’s explore the complexity and realities of securing source code today and how to take the guesswork out of finding at-risk code that could land your company on the front page of the news tomorrow.

    Secrets Management is Challenging and Costly

    Sophisticated cyberattackers will stop at nothing to breach valuable data assets. Secrets leaked from source code make it easy for threat actors to access applications using the same credentials intended for legitimate purposes.

    At the pace of continuous software development, developers must move quickly to build applications. However, security teams that are expected to prevent breaches are not often well-versed in all facets of the software development life cycle (SDLC). This becomes an issue as the number of lines of code (and the secrets kept within) rise alongside the number of apps developed. According to a recent report from 1Password, “two in three (65%) IT and DevOps employees estimated their company has more than 500 secrets.” And, according to the same report, “the average cost of a secret’s leak is $1.2 million. Poor secrets management can result in organizations losing $8.5 billion annually.” Secrets are vital to building any connected application or process. If hard-coded or left unprotected in source code, the proverbial keys to the kingdom are at risk of exposure—as are your loyal customers.

    It’s clear that source code security should be a priority for security teams and DevSecOps professionals. Let’s examine how they can achieve this without sacrificing developer productivity.

    How to Reduce Threats Through Secrets Management Best Practices

    Manually searching source code repositories for vulnerable secrets can be almost impossible. However, there are some best practices to follow to ensure vulnerabilities are caught early and often:

    ● Identify and remove secrets in code: You can’t protect what you can’t see. It’s important to have visibility into source code to be able to see where vulnerabilities may exist. There are a few ways of potentially going about shining a light on vulnerabilities. One is through manual scanning of source code, but this takes away valuable time developers could be spending on more innovative tasks. There are also some open source solutions that can help with scanning source code repositories, but as with everything open source, this means developers will have to maintain it (keep it up to date, integrate it, etc.). Instead, security teams should consider looking for a unified toolchain platform to help with source code vulnerability identification and remediation.
    ● Alert and remediate in code repositories: Provide a fast feedback loop to a developer or DevOps engineer before releasing code into production. Remediation should happen at the time of commit. The idea here is to block secrets from being committed to your source code repositories in the first place.
    ● Adopt a secrets management tool: Don’t store secrets or binaries within source code—nothing but code should be in your source code repositories. Remove secrets to a secrets manager (i.e. HashiCorp Vault), keys and certificates manager (i.e. Azure Key Vault), etc. Modularize and isolate these managers, but make sure they still communicate with each other so that they don’t expose secrets.

    As cyberattacks continue to increase, it is imperative that security teams take a hard look at their source code to ensure that it is locked up tight. Keeping secrets out of source code entirely is recommended in theory but difficult to ensure in practice. Following the above best practices will help reduce the risk of exposure of sensitive data and reduce the risk of losing loyal customers and revenue dollars.

  • GitLab Research Shows Security Concerns in DevOps Teams

    GitLab Research Shows Security Concerns in DevOps Teams

    The threat of breaches is top-of-mind for many DevOps teams, and while security continues to shift left it appears to be a driving force for many decision-makers when it comes to choosing a DevOps platform or other technologies.

    These were among the results of a GitLabs survey that also found getting security right is the number one challenge for DevOps teams. 

    Getting it Right

    Among the top reasons respondents said they chose DevOps were better code quality, developer productivity and operational efficiency—cited by 37% of survey takers, followed very closely by better security/more secure applications.

    The survey also revealed that 53% of developers said they are fully responsible for security in their organizations, a 14% jump from 2021. 

    Michelle McLean, vice president at Salt Security, said it is risky for organizations to depend on development teams alone for security.

    “Security and developer teams need to collaborate and work together–at every point in the application life cycle–not just development,” she explained.

    For example, security teams must bring runtime insights back to development so they can be applied for education and training.

    “It’s fundamentally important to choose a DevOps platform that either has security capabilities built-in or that can easily integrate with security platforms to facilitate collaboration by security and DevOps teams,” she said. “Otherwise, organizations run the risk of pushing out unsecured software or introducing other risks into the software supply chain.”

    Tim Mackey, principal security strategist at the Synopsys Cybersecurity Research Center, added that an obsolete mantra in the cloud-native community held that security wasn’t a high priority as proper DevOps workflows meant that incremental changes were small and easily deployed.

    He said the Log4j and Log4Shell experiences called that philosophy into question with a recognition that security issues can, and do, occur out-of-band from a release or feature and that continuous monitoring for external threats is part of a properly functioning security process.

    “By implementing monitoring for new threats, regardless of source or nature, the ability for cloud-native development to quickly issue tested patches becomes itself a security feature,” he explained. 

    Understanding the Pipeline

    McLean pointed out that while there is a growing understanding of security in the pipeline with DevSecOps, it’s not enough.

    “This is particularly true, for example, in the case of APIs. Because APIs aren’t just straight code, you must see them being exercised to spot logic flaws,” she said. “You need the ability to monitor them in runtime in order to spot anomalies and find areas where APIs might be exposing critical data. Although developers write APIs, their testing often falls short for security.”

    To accurately see what’s going on with APIs, developers need to exercise the API–even in pre-production.

    McLean said this is an area where security and development collaboration and integration is essential–APIs underpin a significant portion of the modern economy and the disconnect between security and development teams can have a direct economic impact.

    John Bambenek, principal threat hunter at Netenrich, a security and operations analytics SaaS company, added there has always been a delay between the release of new platforms and the awareness that security capabilities for those platforms need to be implemented.

    “With cloud-native and serverless technologies, that delay is as short as I remember in my career,” he said. “I greatly prefer the world where everyone at least understands the need for security instead of having to spend time getting buy-in that it matters in the first place.”

    He said if you can manage and implement security in a seamless and efficient way early in the development process, it is far easier and cheaper to address issues than addressing them after code has already shipped—and that’s without adding in breach or liability costs.

    “You can either fix it in dev or in prod, but you’re going to have to fix it sooner or later,” he said. 

    McLean noted many organizations are still maturing in their DevOps practices, not to mention their DevSecOps initiatives.

    From her perspective, educating developers as to what is expected of them from a security perspective before a line of code is written is key.

    “To help increase compliance, we’ve begun to see organizations taking steps to introduce security tooling into their pipeline process and add testing into the build process,” she said.

    Then they’ll add a step that triggers an alert for non-compliant builds but won’t prevent the build. Then they’ll add a step that mandates sign-off by the developer that they’ve seen the security alert for a non-compliant build but still won’t interrupt the build.

    “In rare organizations with highly mature DevSecOps processes, they’ll fail the build for non-compliance,” she added.

    McLean said organizations need to understand what’s realistic given their maturity and risk tolerance and implement controls that match.

    They also need to reevaluate those controls to look for opportunities to increase adherence to security policies and other compliance regulations.

    Overall, continuing to educate through compliance automation in the software development life cycle (in development, testing and deployment phases) further helps reinforce best practices while ensuring issues are uncovered before they become a risk or technical debt.

    Mackey said because DevOps platforms touch the software powering your business, when choosing any DevOps platform, the security of the platform itself and the competencies it enables should always be must-haves.

    “In effect, any decision about new software should be based on how it improves the current security capabilities of the business,” he said. 

  • Everything is DevOps

    Everything is DevOps

    It’s funny how we in IT tend to be absolutists, shoving everything into the current bucket. XML was going to eliminate programming—even though it wasn’t even particularly good at data representation. NoCode was going to eliminate programming (yeah, that’s a theme, I’ll skip the other 50 or so ‘going to end programming’ cycles we’ve been through), AI will eliminate IT, you must have a mobile app, even if no one will ever use it, my OS/browser/language is better than yours and will win—you get the idea.

    We’re currently coming to the end of the “Everything is DevOps” phase. DevOps is a great methodology that helped solve several intractable IT problems but, like everything, marketing and a ton of smart people that want to be “right” have taken it too far. Or have they?

    The thing with DevOps is that contrary to what those who try to make a separate market out of DevSecOps or SRE try to tell you, it was designed to literally encompass all of human IT activity. We can excuse the DevSecOps crowd, since DevOps’ roots had a definite “Security is an unimportant bottleneck” feel to it. The SRE crowd, on the other hand, appears to be another case of “my silo is special”. Or, better yet, a stompy marketer saying “We’re late to the DevOps game! Let’s invent something just like it, claim we do it more broadly and give it our own name!”

    Let’s look at what each does, for those of you who are skeptical. 1) break down silos. Check. 2) Monitor everything. Check. 3) Use monitoring results to improve product performance and stability. Check. 4) Design and implement products and product features. Check. There’s more, but you get the idea.

    Do you know what all of this ignores? You.

    One of the issues that our absolutism brings about is that marketers get to dictate what is important. Sometimes that is what enterprise IT needs. Far too often it is what the marketer is paid to dictate.

    I don’t care if you call it DevOps or SRE, I care that your organization feels like it is getting its money’s worth out of IT—something, I might add, that most organizations did not feel prior to DevOps. At least most of the ones I dealt with. I even had a job where they created a whole new department and hired me to run it to “get around those people in IT.” I spent as much time educating business leaders about not duplicating entire groups like storage services and InfoSec as I did building and deploying new products. So avoiding that kind of issue is near and dear to my heart. And we’re more ‘there’ than we have ever been.

    I do care about whether you call it DevSecOps or not. It should not be different. Shifting security left is the appropriate solution to the double-edged sword of security staffing shortages and increased adversary capabilities. So if your developers are getting secure coding feedback in their IDE or each time a build is kicked off, great! But call it DevOps. Say it is part of the DevOps process. Same with configuration reviews like cloud configuration management—it’s just part of the DevOps process, not a security step to be thought of differently.

    I could go on, but hopefully I’ve spurred your imagination a bit and don’t need to.

    You rock. Streamline your thinking to streamline your processes, ignore absolutism and call it macaroni for all it matters. Just keep doing what needs doing—at the most convenient place to do it. And keep developing, deploying and monitoring the systems that keep the company running.

     

  • Postman Survey: Increased Usage of APIs Results in More Security Incidents

    Postman Survey: Increased Usage of APIs Results in More Security Incidents

    A survey of more than 37,000 developers and application programming interface (API) professionals published this week by Postman, a provider of a platform for developing and managing APIs, suggests that as more APIs are deployed, more organizations are seeing a rise in cybersecurity incidents.

    The survey finds more than half of respondents (51%) reporting that more than half of their organizations’ development effort is spent on APIs. About 20% also noted they are responding to API security incidents or breaches at least once a month at their organization. Overall, more than half (52%) said API security incidents occur less than once a year.

    Kin Lane, chief evangelist for Postman, said that while a lot of the concerns over API security could be classified as “theater” being put forth by providers of API security platforms, it’s clear there is a need for a zero-trust approach to building, deploying and maintaining APIs.

    The bulk of APIs are used primarily to interconnect internal-facing applications and systems, with a subset then used to integrate partners. Only a comparatively small percentage of APIs are externally facing. However, Lane noted many organizations fail to appreciate how quickly an internally facing API can become externally facing over the course of its lifecycle. As such, IT teams need the tools and platforms required to manage and secure APIs as they evolve, he noted.

    In fact, nearly three quarters of respondents (72%) said being able to build APIs remotely is very important to them. That would suggest best DevSecOp practices will need to be extended to API development teams that are often working from home.

    Despite those concerns, however, a full 89% said Investments in APIs will increase or stay the same over the next 12 months to drive, for example, digital business transformation initiatives. However, the survey also suggests most organizations are still coming to terms with API development and deployment. Only 8% of respondents identified themselves as API-first leaders, which Postman identifies as an elite group that produces APIs faster, deploys more often, has fewer failures and recovers sooner when failures occur.

    Lane said the survey makes it clear that while APIs are playing a crucial role in advancing business goals, there is a need for increased visibility as more APIs are deployed across the enterprise. That’s especially critical as more types of APIs employed also continue to expand. Most APIs in use today are based on the REST format, but there are also now APIs based on GraphQL, event-driven architecture and various messaging platforms. None of these newer APIs are going to supplant REST APIs any time soon. Rather, they are being applied to drive additional processes alongside REST APIs, noted Lane.

    It’s not clear how many organizations are adopting a design-first approach to building APIs before they write application code, but as more APIs are deployed, the way IT is managed is rapidly changing. Application services are essentially becoming more disposable so long as the API used to expose them to other applications remains stable. In fact, it’s now only a matter of time before most organizations fully appreciate the degree to which their entire IT strategy now revolves around their APIs.

  • Survey Shows Steady DevSecOps Progress

    Survey Shows Steady DevSecOps Progress

    A survey of 200 DevOps and IT/information security professionals published this week by Mezmo, a provider of an observability platform, conducted in collaboration with the market research firm Enterprise Strategy Group (ESG), finds only 22% report their organization has a formal DevSecOps strategy to integrates security into their software development lifecycle (SDLC) processes.

    Among those that have adopted best DevSecOps practices, however, 60% saw an acceleration in incident detection, while 53% saw a reduction of incidents in production environments.

    Overall, the survey finds 62% of respondents report their organization is actively evaluating use cases for, or has plans to implement DevSecOps.

    Mezmo CTO Rob Fry said the primary challenge organizations face when it comes to DevSecOps remains cultural. In fact, 99% of survey respondents said establishing a culture of collaboration between development and cybersecurity teams is either critical (51%) or important (48%). Building cross-functional DevOps teams that have the required level of security expertise, however, is a major hurdle when there is a general shortage of cybersecurity expertise, noted Fry.

    The survey finds that on average it takes 17.5 person hours to triage and understand security incidents, an amount the survey finds that 82% of respondents desire to reduce.

    However, organizations are also struggling with the sheer volume of data collected by DevOps teams. More than half the respondents (52%) said the variety of data to be captured, analyzed and be acted on throughout the software development lifecycle (SDLC) hinders security efficacy/efficiency. A total of 43% said the scale of data to be captured, analyzed and acted on throughout the SDLC also hinders security efficacy/efficiency.

    Organizations surveyed capture several (54%) or even hundreds (32%) of terabytes per month, with 6% capturing a petabyte or more per month. Well over two-thirds (69%) do not capture certain data sources because of the high cost of storage/retention, which Fry noted is going to be problematic if there is an incident. Log data (35%), followed by metrics (29%) and traces (22%) account for most of that data, the survey finds.

    In all, 84% of respondents said getting the right data and tools to developers is key to achieving success. A full 91% of respondents are using multiple tools to get the most value out of their data. Two-thirds (66%) have dedicated logging analysis tools, while 59% have a security information event management (SIEM) platform. Cloud to two-thirds 64% have a data lake hosted in the cloud, while 55% have one in an on-premises IT environment.

    A full 87% are using open source tools as part or all of their observability stack because they are more customizable. However, 84% noted that over time it will become challenging to manage, adapt and scale with these solutions. Nearly all survey respondents (98%) said they will likely investigate a managed observability solution over the next 12 months.

    The application security issues that plague organizations today have been around for decades, so it’s not likely they will be resolved in a few short weeks. On the plus side, at least, there’s now a lot more awareness of the root cause of the problem than ever.

  • Why DevOps Teams Need Security Engineers

    Why DevOps Teams Need Security Engineers

    In the episode of View with Vizard, Mike Vizard sits down with Om Vyas, chief product officer for oak9, as he explains why security engineers need to become part of every DevOps team. The video is below followed by a transcript of the conversation.

    Vizard: Hey, guys. Thanks for the throw. We’re here with Om Vyas, who’s Chief Product Officer for oak9. We’re gonna be talking about all things related to DevSecOps and cloud security. Om, welcome to the show.

    Om Vyas: Yeah. Thanks for having me, Mike. 

    Vizard: We, of course, have been talking about DevOps for the better part of a decade. And then we decided that the security people couldn’t be left out, but they wanted to have the word “sec” thrown in there to make things a little more confusing, so then we got DevSecOps. And some of them even argue that I guess it should be SecDevOps. But who knows? My question to you is it seems like we’re still struggling with getting this idea across to people. It’s become more important in the cloud era, obviously, ’cause more developers are programming stuff. But what, from your perspective, is the fundamental challenges that we’re seeing with getting people to really implement DevSecOps practices?

    Vyas: Yeah. I think if we take a quick step back, I think you hit it on the head. There’s DevOps, there’s DevSecOps, there’s SecOps. There’s a lotta buzzwords and a lotta different ways that these adoptions are being thrown around. I think fundamentally, my take is DevSecOps or DevOps, it’s first and foremost a cultural transformation. The technology and the people process really come after. So really, one of the biggest challenges is that embracing this notion of being able to work together, being able to have shared responsibility and a collaborative environment.

    If you think about kinda where we’ve come from, and in fear of dating myself, I’ll say that when I used to develop long ago with my DevOps, my development teams, our focus was only towards application development. And then it was like the packaging, the art effects, once they were created, it was like, “Hey, kinda ” the notion around, “Hey, throw it over the wall to IT ops people,” or having formalized security review processes, right? And really, this cultural transformation is, I think a key area where I think a lotta the folks in the organizations, they’re struggling. They’re struggling to embrace the culture of DevOps, culture of DevSecOps.

    And then when you factor in, as you said it yourself in the beginning, that the security people are getting left behind. And what that really means is they don’t have the visibility and they don’t have the automation to be able to keep up with the amount of churn that’s happening with the development, the DevOps. Because the dev teams are using the automation, which is the technology part of this transformation, to really create this velocity in which they are churning over new features, doing rapid deployment into production. Five, seven years ago, we used to do maybe one release a month, one release a quarter. And now, I’m seeing organizations, ours included, where we’re doing tens of releases a day, hundreds of releases a day. So this is really where the security or the “sec” part of the DevSecOps is really struggling to keep up with that velocity and that change.

    Vizard: Does that mean we need to slow down? Or can we build and deploy secure applications just as fast? What’s the tradeoff?

    Vyas: Yeah. I mean, I think maybe slow down is not the right answer. Because I think velocity is really being utilized as a differentiation for a lotta the – lotta the organizations. I think the right answer is – I think, not necessary just slow down to a halt. The right answer in my opinion is really, how do we integration security into the processes of DevOps to truly give them the visibility at the point of time when these changes are happening and then really arm them with the automation that they really need to give them full visibility, give them ability to be able to really quickly identify how that application is changing? How are those new capabilities that are being added, and how is that impacting security of that application? And the sooner we give visibility to security engineering of all of those changes and where security matters in those changes, the sooner we can continue to operate with that velocity.

    Vizard: How do we go about doing that? ‘Cause a lotta the developers, they didn’t really set out to build something that’s insecure. It’s just more often than not, they have a tool that lets them manage a cloud as infrastructure, as code. But then they misconfigure something ’cause they’re not security experts, and maybe a port gets left open or something like that occurs. So how do we lock all that down and how do we kinda tighten that up without necessarily requiring the developer to become a security expert, ’cause that seems unlikely?

    Vyas: Yeah, you’re absolutely right. I think that a developer is not a security expert. And I think if you look at kind of where dev engineering is today, especially with the Cloud Native architecture, especially with DevOps, it’s dev engineering or developers are taking on lot more responsibility, right? But I do think as a developer myself, what I’m interested in is not that I don’t care about security. It’s that what I wanna know is, “Hey, help me understand what I need to do to move fast. What do I need to do? What are the five things or ten things or whatever number of things that are related to security that I can do to not only help my security team, but really quickly move this feature capability from development and implementation to testing and to production?”

    So from that perspective, I think what really also helps is if I have the ability to, within my own workflow, within my own tools – if somebody educated me and gave me kinda the prescriptive guidance of, “Hey, here is all you need to do to ensure that all of these different security aspects of security-related things in this infrastructure’s code that you change. Here’s what you need to be aware of. Here’s what you need to change, and to really help you move forward.” I think that’s what, I think as a developer, I’m looking for. Because again, I have many, many different things that I’m managing today beyond just application code. 

    Vizard: Mm-hmm. How automated can all this get? Seems to me the key to this whole thing is if the developers still need to program the cloud environment, we need to find a way for them to make fewer mistakes, put more guardrails in place, and kinda just gently lock down the environment, I guess, is the phrase I’d say. So what is the challenge, what is the state of the art at the moment?

    Vyas: Yeah. I think it’s interesting. You said “gently lock down the environment.” I think absolutely, right? I mean, I think that developers, what we’re seeing today with a lot of our customers is they want lotta freedom because that freedom will enable them the agility and the velocity that they are looking for. But at the same time, we do need guardrails because they’re important.

    And I think how do we gently lock down the environments? I think that’s kinda the key. I think one way we can talk about this is that when we think about that infrastructure’s code, we’re really talking about codifying the entire environment under which your application is running in the cloud. And what that actually enables us to do is be able to then say, how do we actually remediate? How do we actually focus on automation that doesn’t always just point out all the challenges or all the problems in that code around security? But then taking it that last mile, as I like to think about it, which is actually fixing it for that developer.

    So I think that’s really where the trends are. And I think that’s really where things need to be, is really say, “Hey, I analyzed this infrastructure’s code. I have identified where all of the security challenges are. Here’s how to risk-appropriately secure this code. Because all of this infrastructure’s code is declarative. So it’s predictable in how it can be changed and what the impacts would be.”

    So I think that’s really where it needs to be, is when I look at some of our customers and infrastructure’s code, it’s getting pretty complex, right? So we have customers that have 40,000, 50,000, 60,000 lines of Terraform code. So even cloud infrastructure code or infrastructure code itself is getting pretty complex. So no security engineers or no number of security engineers or even DevOps or developers can actually keep up with understanding, what are all the security implications in this code? So the real answer in my mind is, how do we automate this thing, take it to the last mile, and say, “Hey, here are all the recommended changes in this code as far as the security aspects of it is concerned?” So as a developer, now I can focus on the functional aspects, and I have an automated tool, an automated process, that will really enable to me to then not have to worry about the security aspects of that infrastructure’s code.

    Vizard: And maybe no one ever needs to know how much I don’t know about security. But do you think that we’re gonna see a lot more of so-called artificial intelligence thrown at this problem? And is that part of this automation curve? Or are we making too much of AI and security these days and it’s just gonna require something a little more fundamental?

    Vyas: Yeah. I think the short answer is yes. But I think my opinion is that AI kinda gets thrown around a lot. And I think when we’re talking about some of the deeper learning concepts in AI, we’re talking about some neural network things, I think those things are coming. But I think we still have to recognize that we’re in the infancy stages of this. So I think really to me, it starts with the fundamentals first. Can we actually provide these recommendations, this automation, to really help educate the user, that developer, that practitioner, that is today, lot more often than not is kind of utilizing lotta the modularized code that’s available in the web without really understanding, what are the implications as it relates to their own world or their own applications or their own context?

    And lemme sort of dive in deeper. So not all applications or workloads in the cloud are created equal. So if you are a health care company that’s securing health care applications that’s utilized by your patients, that’s using PII data, their requirements around security may be completely different than if I’m building a marketing website where most of the data that’s on that website is publicly available. Both of those applications are not created equal. So how do we secure those two different applications is actually different.

    So there’s an element of this dynamic nature. There’s element of understanding, what are the requirements of that application? What are maybe the compliance adherence to those applications? So really then, when we talk about securing those two applications, that’s also different. So it’s really what I think about the context behind that application and how you secure that application really becomes important or a focal point of, then what are the recommendations that you’re providing to that end user, that developer, within that infrastructure’s code? So I think that has to take that into consideration.

    And then I think, to your point, it’s not simply just saying, “Hey, here are the ten things you need to do, developer. Go and here’s a tool that will actually remediate it for you.” That’s great. But I think part of it can be used really as an educational tool to really better understand, “Hey, developer, help us a little bit. Help us understand the context of this application. What are, what is it that you’re trying to do?” And then I think once you kind of cover those fundamentals, then I think as we go deeper beyond the infancy stages of this cloud infrastructure journey, then I think what we can do is we can learn behaviors. We can learn from the industry that this developer or this organization is in. And we can say, “Yeah, this is a health care industry vertical or this is a particular type of architecture or workload that’s being deployed in the cloud,” and secured.

    And then I think we can start to get into intelligence around deep learning and other heuristics to say, “Hey, how do we react to then risk-appropriately secure this application,” right? Because otherwise, if we start treating all applications, all workloads that are run in the cloud, in the same way, there’s a lot more that we’re asking of the developer. There might be hundreds of things that we need to do to risk-appropriately secure a health care application that’s using PII data versus we might only need to do a handful of things to ensure that a marketing website that has publicly-available data is secure.

    Vizard: What will be the role of the security team in all of this? Because one of the things that comes to mind is if we are shifting more responsibility or at least the enablement left, then what exactly does the security team need to do to kinda work alongside, then? Or will they just kinda set the policies and the rest of it will be automated?

    Vyas: Yeah. I think, first and foremost, about kinda the role of security engineering today in the DevOps world or DevSecOps world, is really understanding and gaining visibility into what’s happening. Because if you have engineering teams that are deploying tens of times a day into production, they’re fundamentally changing, potentially, the architecture of that entire application. So I’ll give you an example. Today, with lotta the sort of building blocks or the way I like to think about it is cloud service providers are really offering these Lego pieces. So they’re outta-the-box capabilities.

    Now, I can plug in an analytics service into my architecture and seamlessly provide analytics dashboard into my application or into my workload that I didn’t have a week ago. But that fundamentally changes, or it fundamentally has the ability to change, the architecture, and therefore the security implications. So first and foremost from the role of security engineering is really getting the visibility and getting the visibility as early as possible so that they can understand how, fundamentally, this application is changing and what are the security implications of that. So it, and I’ll say for that aspect of it, this is the people process side of embracing DevOps, is really engaging that security engineer as early as having that requirement or having that capability from business that says, “Hey, we need this analytics dashboard,” really getting the security engineering involved as early as possible.

    And from that point forward, again, then it goes back to technology and automation. How can security engineering really understand, what are the implications of adding this analytics service from AWS or some analytics service from GCP or Azure? What does that mean for the security of my architecture? So there might be hundreds of different configurations. There might be thousands of different implications around putting in that one analytics service that’s available outta the box as a feature or service from the cloud service provide. And the reality is no security engineering can actually keep up with that.

    So having a tool that really understands down to the configuration level, what are the impacts or the configurations of that analytic service to securely deploy that service and then what security engineering can do is security engineering can say, “Hey, let me understand the implications and the architecture when my development team is adding this analytics service.” And if we start early, if we get the security engineering involved early, if we enable, then, automation to take away the implications of down to the hundreds of configurations of securely deploying that analytics service away from security engineering, but then give them the visibility, then I think we have a really great starting point to enable all of the teams to work together with the same operational velocity that the dev engineering really desires.

    Vizard: Mm-hmm. What is your best advice to folks? I mean, should I just take all these developers and security people and lock ’em in a room till somebody comes to an agreement? Or is there some more intelligent way to go about this?

    Vyas: Yeah. I think locking everybody up in a room is not the right answer, right? I mean, I think what I’ve seen work really well is there’s lotta different ways to sort of come to a solution. I think there’s no one silver bullet here. But I think depending on the size of the organization, depending on the size of the teams, I think one of the things that should be done, and I’m starting to see some trending around this, is really add a security engineer to the engineering team as a whole so you really have a dedicated security engineer that’s spending part of their time with these engineering teams.

    So typically, we’ve seen, in more of the modern development shops, you see lotta the dev engineering, it started out with just developers in a single team. Then we kinda focused on scrum teams. So there was development teams, there was a product engineering or QA involve.We’ve seen a DevOps engineer or SREs get involved in these teams. And I think, to me, the answer is, again, being able to put that security engineer in these teams, as well, to work closely with the developers, work closely with the SREs. Because then, what will start to happen is that engagement, that early visibility and understanding. So I think that’s one.

    And then I think the other area for lotta the larger organizations where this may not work is I think the notion around excellence centers , so you have visibility across these little center of excellence teams. So there’s cloud ops teams, there’s security teams. It’s really good to incorporate lot of your security engineering and security architects into these cloud center of excellence teams. Because I think then again, it goes back to that visibility, goes back to engagement, and it goes back to helping security teams really understand, what are the different architectures? What are the different capabilities that lotta the dev engineering groups are using today? Is it the serverless architectures? Is it the micro services that are being built with containerized architectures?

    And then what security engineering can start to do is they can start to build these reference architectures or guardrails around 80 percent of the, “Hey – ” recognizing that 80 percent of the workloads across my organizations are built with serverless architectures, as an example, in the cloud, or it might be that containerized architectures in the cloud. And that will really help them start to build these guardrails in an effective way. And then I think, you talked about sort of, early on, gently locking down environments. So I think if we then start to think about, in the DevOps pipeline, how do we incorporate these security automation tools that enable us, both the development teams, the SRE teams, and the security teams, to identify, what are all of the security underpinnings, vulnerabilities, design gaps that we are identifying, be it with the application code, be it with runtime, be it with cloud infrastructure security?

    So it’s really, where possible, building these automated tools within the pipelines without actually blocking the pipeline so that you’re actually what you’re doing is you’re really enabling the development engineering and the DevOps engineering to really start to build in the process around, how do we ensure that all of these security design and the best practices are actually incorporated within the pipeline? And then as the teams learn and grow, then these tools can become those guardrails. So maybe you’re not blocking or completely blocking the pipeline or failing the pipeline at or in the development or sandbox stages, but then you start to guardrail those as that feature capability’s moving through user acceptance testing or staging or before it goes into production. And I think those are some of the key things that can be done to really enable more of this collaborative, but shared responsibility culture that I think is really going to drive lotta the benefits that are needed to build security in.

    Vizard: Hey, Om, thanks for being on the show. It’s all about making sure that if you find a pattern, build a guardrail, and rinse and repeat.

    Vyas: Yeah, absolutely. Thanks for having me, Mike.

    Vizard: All right. Back to you guys in the studio.

  • Contrast Security Adds API Support to Security Platform

    Contrast Security Adds API Support to Security Platform

    Contrast Security has fully extended its vulnerability scanning and runtime application self-protection (RASP) platform to now include application programming interfaces (APIs).

    Company CTO Jeff Williams said the Contrast Secure Code Platform embeds agents directly into code to detect vulnerabilities using a combination of integrated static and dynamic application security testing tools. That approach makes it simple to embed vulnerability scanning and exploit prevention capabilities within a DevOps workflow as an application is built and deployed, he noted.

    In contrast, traditional approaches to application security depend on legacy platforms such as web application firewalls (WAFs) that are deployed by cybersecurity teams to protect applications in a way that doesn’t provide the same level of granular control at the API level, said Williams.

    Contrast Security claims the Contrast Secure Code Platform improves scan times by a factor of ten while fixing defects 45 times faster. It also reduces false positive alerts by being 200% more accurate, the company claims.

    As more organizations start to embrace best DevSecOps practices to better secure software supply chains, many of them are trying to achieve that goal without slowing down the rate at which applications are built and deployed. That approach, however, requires giving developers and DevOps teams the tools needed to build and deploy secure applications without requiring a lot of direct intervention on the part of a security operations team.

    In general, there is also now a lot more focus on API security as more organizations build and deploy microservices-based applications that rely on APIs to interconnect discreet components. Those APIs should be built, deployed and secured by DevOps teams before they are deployed just like any other software component, noted Williams. The current process of building applications is at its core, from a security perspective, fundamentally broken, he added.

    Application security, in general, has always been somewhat problematic. Cybersecurity teams tend to fund the tools and platforms they regularly employ. The assumption is that application development teams will invest in the tools they need to build and deploy applications. Alas, application development teams that have limited cybersecurity expertise assume that cybersecurity teams are securing both the underlying infrastructure and the applications deployed on top of it. As a result, application security is frequently underfunded.

    In the wake of a series of high-profile breaches of software supply chains there are now at least more reviews of application security being conducted. Contrast Security is making a case for securing the entire application, including the APIs, it invokes using agent software that is injected into the application at the time it is constructed.

    It’s not clear to what degree application security issues will be addressed using agent software rather than traditional cybersecurity platforms managed by security operations teams. Responsibility for application security is shifting left toward developers and the DevOps teams that support them, but as long as humans write code, mistakes will be made, said Williams

    The issue, of course, is many of the members of those teams are still focused a lot more on writing code and deploying applications than they are on cybersecurity.

  • Avoiding Security Review Delays

    Avoiding Security Review Delays

    In the summer of 2021, I had lunch with a senior security developer at one of Seattle’s leading tech firms. Even though we were relaxed in the sunny and cool afternoon of the Pacific Northwest, there was no doubt my friend was frustrated. Part of his job was running code through a popular scanning product and then evaluating the results. It was a tough role because he sat outside the team that was developing the code and he was often under pressure to run his analysis within the constraints of their deadlines. 

    All too often, he had to deliver the bad news that the scan had found security issues. But that alone wasn’t why he was frustrated. His issue was that the tool that he was using was taking several hours to complete scans. He’d taken to starting the scan at the end of the week and hoping that it would be complete by Monday morning. The very tool that was supposed to make his job easier was actually a headache.

    This is one of the fundamental challenges of analyzing software systems for security vulnerabilities. All too often, developers and stakeholders focus their project timelines on the functional specifications. Teams work feverishly to rapidly iterate from prototype to finished product. 

    Security concerns are never disregarded, but the culture of “Let’s see if we can get it working the way we want first and button it up later” ends up costing more in the long run because the evaluating and testing cycles take longer. Not to mention that if substantial security problems are found, it can be difficult to remediate them without breaking things that were supposed to already have been completed. This problem is becoming acute in the SaaS world where solutions are being built faster than ever before with little to no security oversight.

    The Right Tools and Software Are Critical

    First, managers and executives need to mandate that the entire software development lifecycle and every environment should have security. Particularly for web-based SaaS tools, any vulnerability that exists is a potential attack vector. Attackers know that going after a dev version might not yield a jackpot, but it might give them access that exposes credentials that work in later versions or identifies a vulnerability that flies under the radar into production.

    Second, automation and tooling are simply the name of the game. Using tools with code repositories, pipeline management and testing frameworks help ensure the elimination of as many potential vectors of error as possible. Security checks should be the same. We can’t always rely on the skills of a developer or an administrator to follow security best practices. 

    We need tools that provide reliable and rigorous checks. Moreover, these tools need to run frequently. SAST scanning should be fast enough that a developer can run it with every save, or at least every commit. Coupling SAST and IAST scanning together is the gold standard for secure development. Every time code is compiled into a dev or test environment, it should be tested to make sure that vulnerabilities haven’t crept in and that the permissions and architecture of the system don’t work at cross purposes to expose vulnerabilities.

    Finally, there is a fundamental issue that must be addressed in every manager’s office. Code quality is not the same as security. Sure, there might be some overlap, but it doesn’t make sense to have 200 rules for code quality and 50 for security. The two things are not equal, and it’s a fallacy to expect that having high-quality code without frequent security checks will ultimately lead to less time spent in security review. Don’t miss the forest for the trees. Make sure security is the team’s top priority.

    Advocating for Security in SaaS

    My friend didn’t have a problem holding teams accountable for the code they’d written. In fact, he relished being able to insist on high standards and was proud that his organization empowered him to do so. That’s exactly what we need in the software industry: Individuals who are unafraid to demand the best from their peers. But it’s imperative that we provide effective tools they can rely on to succeed.

    The problem of security testing delays are not new to software development. It has been well-documented that it can take 640x the money to fix bugs in production, but due to a lack of effective tools this problem has been amplified in SaaS development. A SaaS development environment presents unique challenges that cannot be met by just employing code analysis or configuration testing for SSPM vendors. It requires a holistic view of the environment and the ability to use customization to be effective in the identification and remediation of vulnerabilities. 

    With a comprehensive view, those responsible for security and those implementing it will see security as a necessity to innovate quickly rather than a necessary evil that is put off until last in hopes it can be avoided. Maybe then my friend won’t be as frustrated next time we meet.

  • Putting the Security Into DevSecOps

    Putting the Security Into DevSecOps

    The non-Newtonian fluid that’s composed of cornstarch and water has been around a long time, but Dr. Seuss’ 1949 book was the impetus for what it’s often called today – Oobleck, from “Bartholomew and the Oobleck.” When not under pressure, Oobleck is a thin liquid; when pressure is applied, its resistance increases to the point of being solid. A couple of other examples are quicksand and Silly Putty ™ (for fun, this video shows how resistant Oobleck is against Thor’s Hammer). Company application security is much like Oobleck. When not put under stress, security appears to be a fluid mess of policies, procedures, guidelines and a host of other controls that seem to have no real connection. But when there’s an event of interest, incident or even a breach, those policies and procedures (P&Ps) come together to make their worth known.

    DevOps—What’s in a Name?

    DevOps has been around for a while now, and its definition is akin to a Newtonian fluid: Not well-defined except within each company or organization. This lack of a standard definition has its merits, though; giving each company flexibility in determining what it means for them, and the benefit of maintaining the best of development, security and operations no matter what it is called. Attempts at bringing all aspects of these disciplines together, regardless of the nomenclature used, are both old and recent. It’s a reframe of Shakespeare’s line, “What’s in a name?”

    Security has to be part of the DevOps culture, whether it’s called DevSecOps, SecDevOps, SecOpsDev or just simply DevOps. Creators of software need to go beyond the desire to have a good process and move to understand that customers need the software to be secure in addition to being useful, delivered on time and with a good UX and that the business needs to develop and maintain a positive reputation and avoid breaches to the best of its ability.

    How does one put security into the DevOps process? One approach is to proceed as if the goal is to attain SOC 2. SOC 2 reports hold great value. While not universally recognized (SOC 2 is a US-focused security controls examination), they give useful details about the controls used. While SOC 2 is not on the level of ISO 27001, for international (to the USA, that is) prospects, it still divulges security control details that aren’t present in other standards.

    What if Attestations Are too Expensive?

    AppSec transparency is not only good for improving sales but for mitigating the problems caused by third-party breaches.

    For some vendors, especially newer or smaller ones, something like SOC 2 is on their radar but not yet feasible. Why? Cost–those and other attestations are often expensive. Is it worth it? Yes, but not if the financial expense will put the company at risk. Whether one has achieved SOC 2 or not, potential customers want to see transparency in a company’s security controls.

    In the process of going for SOC 2, there are two advantages: 

    • You’ll improve your security posture.
    • You’ll be ready for when those opportunities roll around.

    Let’s get started.

    Pieces of the AppSec Puzzle

    Actually, let’s back up a bit first. There are some policies that cover multiple areas of the company or product: Infosec policy, risk assessment, vendor management and acceptable use (AUP). These will require full company interaction and arrangement. I want to focus on what you can do, starting today, to improve your product security.

    Here are the policies to work on to put security into DevSecOps.

    • Software development life cycle (SDLC)
    • Incident response
    • Access control
    • Passwords
    • Change management
    • Logging and monitoring
    • System backup and recovery

    Each of these will require, for full effect, more people and departments than just you and your own. But these P&Ps can be directed internally toward your development and production teams; perhaps even IT and security.

    Here’s a list of each policy and its purpose:

    Software Development Life Cycle (SDLC)

    This is one of the first P&Ps to develop. You’re already doing this – just because it’s unwritten doesn’t mean you don’t do it. In the world of customer assurance, if it isn’t written, it doesn’t exist. Make it real and purposeful by codifying what you already do and improving as you go. This is a primary tool for AppSec (which includes web applications and APIs) and is almost solely within the purview of devs, engineers and QA. The product or service provided needs to have this foundational level of control to prevent common forms of attack.

    Incident Response

    This is also high on the list of P&Ps on the list. This is separate from the corporate IR plan; DevOps should have its own plan for dealing with technical emergencies, incidents and breaches. And this plan can be part of the overall company plan.

    Access Control

    Access control includes how users are authenticated and authorized, and includes least-privilege access, multifactor authentication (MFA) and service accounts. Prospects need to know that access to your production environment is well controlled.

    Company Passwords

    Company passwords as a whole are managed by IT and security, and often have a minimum of eight characters, require a certain number of special characters and other industry-standard requirements. But your software environment can go beyond that to enforce more stringent controls.

    Change Management

    Customers need to be aware that the software they bought is changed in an orderly and authorized manner, not through an ad hoc and ad lib process.

    Logging and Monitoring

    This is important for ensuring that activities such as access control and change management in your company have been performed properly, in addition to providing information for forensics in case there’s a breach.

    System Backup and Recovery

    Document and ensure your capabilities to restore data in a timely manner. Things happen; not just attacks—user errors, natural disasters, provider outages and the like.

    The process of each P&P looks something like this:

    It’s a straightforward and natural process, but the simplicity of the ingredients (like cornstarch and water) makes it very easy to overlook its effectiveness.

    Additionally, make these P&Ps available—whether in full or by using excerpts or summaries—to prospects and customers. A major part of driving security is opening oneself up to public scrutiny. This could be via a secure portal, a public-facing page or a ticket/request system.

    BONUS Assessment for Transparency

    Take a look at the Consensus Assessment Initiative Questionnaire (CAIQ) by Cloud Security Alliance (CSA). Level one, which is a free self-attestation, is an often-requested assessment. It’s fairly rigorous and may require you to level up security in your company, but it’s a well-respected and no-cost attestation and is available free for prospects and customers to view.

    The Next Step

    Inserting or improving security in the DevOps process is part of the true DevOps process. It’s a mindset and culture more than a title or official designation–and it’s possible.