staging-devopsy.kinsta.cloud

Tag: devsecops

  • Why Your DevSecOps Initiative Will Fail

    Why Your DevSecOps Initiative Will Fail

    When you’ve been in and around the security industry for long enough, you get used to the industry hype machine turning a cool innovation into, uh, meh. This hype cycle starts at the RSA conference each year, and folks like me look for new hot stuff on the show floor. For perhaps only the second time in 25 years, I wasn’t at the show in person, but it seems that DevSecOps was the preferred bandwagon on which every vendor jumped. Sigh.

    I’m not sure whether it’s my increasing age or this business that has made me cynical, but here we are.

    To be clear, from a customer’s perspective, there is great value in the concept of DevSecOps, as evidenced by the upcoming DevOps Connect: DevSecOps @ RSAC 2022 virtual event. Having these (sometimes) mortal enemies working together toward a common goal is great, in theory. But alas, in practice (and similar to what happened when DevOps hit the scene), it’s easier said (or displayed in a vendor booth) than done.

    Thus, I’ve come to a sobering conclusion: Your DevSecOps initiative will fail.

    Most organizations will talk a good game for a while and they may even try some stuff, buy some kit, send folks to training and the like. But in the end, despite all these efforts, you guessed it—they will still fail.

    Is this a foregone conclusion for every organization? Well, no. But this isn’t my first rodeo, and I’m pretty sure this one will end like the others—in failure.

    But let’s say you want it to be different this time. Really, you do. And you are willing to take some actions to give yourself a chance at success. If that’s the case, here are a couple of tips to increase your chances, however small.

    Culture

    First and foremost, DevSecOps is not a thing. It’s not a set of tools (no matter what the vendors tell you); you don’t buy it on a VAR line card. DevSecOps (like DevOps) is a cultural shift. Your teams need to work together. They need to collaborate. They need to have shared incentives. And they need to experience the consequences of not playing nicely in the sandbox.

    Cultural shifts are hard. People don’t like to change. They resist. They undermine. They sabotage. It all sounds nefarious, but they do. So unless you can change your culture, you’ve got no shot.

    Shift Left

    Another one of these security-marketing-terms-run-wild is shift left. It means you want to integrate security testing directly into the development process as early as possible. You should be running security checks in the developer’s IDE to ensure they don’t violate security policy (like hardcoding API keys in their source, for example) before they even commit the code.

    You must also integrate security testing into the CI/CD pipeline(s). Thus, you need tooling that works with your preferred pipeline.

    Security Champions

    Security testing will create angst for your developers. They will have false positives. They will have to fix things that they probably don’t understand. It will cause them to miss delivery dates. Of course, they should build it right the first time, but in their mind, they’ll see it as the security folks causing delays.

    The developers will be annoyed, and you’ve got to manage that annoyance—or have senior management quash any rebellions, but that’s not a long-term solution.

    On the other hand, you could establish a security champions program, which anoints (and trains) developers on each team to “represent” security within the development team. They help educate the developers on the tools and secure coding. They diffuse irritation when something goes wrong.

    A strong security champions program is one of the leading indicators of DevSecOps success.

    Break the Build

    Finally, the security team must have the juice to break the build if a critical issue is found during the deployment. We talk about this with many organizations and they nod their heads. But they are lying to me and to themselves. When it comes down to it, someone on the dev team calls someone in senior management and, all of a sudden, that critical issue is now an alert, not a failed deployment.

    Not being able to stop a build will kill your DevSecOps program. It’s like when a quality control issue forces someone to stop the assembly line. If you don’t stop the line, faulty products can result. In a security context, that means some critical vulnerabilities will be much more expensive to fix later.

    So, there you have it. If you can evolve your culture, integrate security testing into your toolchain, empower security champions and empower your team to stop a deployment, you’ve got a chance at a successful DevSecOps program.

    Unfortunately, you probably can’t do these things, and you’ll likely fail. In that case, I have some lovely property by a waterfall I’d like to sell you.

  • The Two Types of Code Vulnerabilities

    The Two Types of Code Vulnerabilities

    Spell checkers, grammar checkers and predictive keyboards all help reduce errors in written communication, but in a creative medium where writers need to innovate new expressions, nothing can eliminate all errors. The same is true in the code we write.

    Of course, your code is probably perfect but, on average, all code has bugs. Some of those bugs are benign. Some will cause unexpected early termination. A few will cause runaway resource consumption. And some bugs lead to security vulnerabilities that let users do things that are unexpected and sometimes undesirable.

    Those security bugs are behavioral vulnerabilities because they depend on how the code behaves when executed. Another type is an intelligence vulnerability, something that’s part of the correct behavior of the code, but which gives away details about how to exploit a system. These are the two main types of code vulnerabilities; both come with risks.

    Behavioral Vulnerabilities at Runtime

    Tl;dr: How the code works and how attackers can leverage what it does to their advantage.

    This covers a broad range of problems, but all of them are a result of the behavior of the code leading to undesirable outcomes in specific circumstances. Most of these are unintentional. An unescaped user input sent to a database can lead to a SQL injection vulnerability or an unsanitized string sent to a browser could create a cross-site scripting (XSS) vulnerability. Some behaviors are intended, but have unexpected consequences; the Log4j string substitution behavior that allowed it to output environment variables or read data from external resources was intentional, but the malicious usage we now call Log4Shell was not. In another recent example, a combination of behaviors could be exploited to gain root privileges on Linux desktop computers.

    Intelligence Vulnerabilities in the Content

    Tl;dr: What the content of the code tells an attacker about the system and how to access it. 

    In the 1950s, the phone company—there was pretty much just one at the time—began implementing completely automated dialing that didn’t require a human operator at the switching station to connect the call. The details were first published in a technical journal in November 1954 (see pages 1209 and 1309 here), and the information gleaned from that led to the creation of “blue boxes” that used insider knowledge to allow outsiders to control telephone equipment and make free long-distance calls.

    That’s hardly the first example of insider knowledge being leveraged for outside advantage, but its role at the heart of computer hacking and the origins of the PC (the ‘hack’ inspired Wozniak and Jobs) is so legendary that it demands recognition here.

    The phone system’s intelligence vulnerabilities were published in the company’s technical manuals—manuals that many thought were only for internal use, and nobody realized were available to untrusted readers. Today, though, intelligence vulnerabilities are in our code.

    Secrets abound in code, along with the details about which systems those secrets connect to. These secrets were once difficult to manage and putting them in code was the only practical choice. Now we have tools like Hashicorp Vault, as one example, and secure keystores in almost every cloud, but the legacy and security debt of all those secrets in legacy code are a goldmine for attackers like Lapsus$ and others who specifically target code.

    Security Is a Process, not a Product

    There’s a lot hidden under the surface in the two categories outlined here. Each of us can name many types of behavioral vulnerabilities—many more than the different types of intelligence vulnerabilities—but the two classes of vulnerabilities represent relatively equal risk. Recent events have demonstrated how an attacker with inside knowledge gleaned from code—and with passwords and keys taken from it—can move quickly despite firewalls and other access controls.

    Identifying secrets and other intelligence risks in code is relatively easy, but few teams do it because detection is the easy part. Integrating detection in developer workflows so they can take action in-context while working on and reviewing code is the most effective way to eliminate creating new security debt. And tools that give developers the information they need to understand the overall picture of intelligence risks in their code, the areas of highest risk—and that point them to actionable issues they can fix immediately to improve their code health—are necessary to help teams mitigate historical risks, ideally by rotating any secrets found.

    Tools exist that can automate that process and help teams to eliminate more than half the risk in their code—but the first step is recognizing where that half resides.

  • Common RDS Misconfigurations DevSecOps Teams Should Know

    Common RDS Misconfigurations DevSecOps Teams Should Know

    Relational databases have become the option of choice for organizations wishing to streamline and scale the use, storage and retrieval of data. Many organizations choose AWS Relational Database Service (RDS) to forego the resource-intensive tasks related to database administration including management and continuous oversight. RDS is a fully managed service that simplifies and automates these processes, making it easy to set up, operate and manage relational databases in the cloud—including MySQL, PostgreSQL, MariaDB or Aurora—and their various time-consuming activities. Reducing costs and automating processes makes significant business sense and allows organizations to focus on performance, scalability and cost-effectiveness as data grows.

    That said, AWS is not the owner or the party responsible for securing your data—this duty remains in the hands of the organization. As the cloud provider, AWS does offer an impressive suite of security measures for protecting the infrastructure that runs its cloud services, including access controls, database snapshots and encryption. Their policies, however, clearly state that safeguarding data is within the responsibility of the organization and, as the moderator of your environment, you will be accountable for what occurs within it.

    The following are the six most common RDS misconfigurations related to security and compliance that DevSecOps teams should be aware of to secure their data stores and organizational assets.

    Broken Geo-Trust (aka Data Sovereignty): RDS provides the convenient option to distribute data across various AWS regions. There are various benefits to this data migration including minimizing latency, disaster recovery capabilities and scaling operations. However, the security and governance risks of cross-geo replication may significantly outweigh these benefits. A common misconfiguration includes migrating replicas or storing backups and snapshots in another location. When sensitive data travels and is stored in another geographic location, the organization may be at risk of violating international compliance requirements such as Schrems II (for EU-to-U.S. data transfer). Additionally, data traveling to various locations may not be visible to security teams or managed by DevOps teams—without a continuously updated inventory, data will spread unobstructed and ungoverned. 

    The screenshot below displays a copy-snapshot operation of a U.S.-based RDS datastore to an EU region.

     

    Inadequate Retention Periods: AWS allows organizations to retain database instances for a specific period of time as configured by the user. A common misconfiguration of this feature is the existence of unmonitored standalone snapshots, short-lived backups and a failure to permanently delete records. Organizations that retain records for a long period of time may be in breach of their GDPR obligations, whereas short-lived backups may jeopardize the organization’s ability to recover critical data in case of data loss or a ransomware attack. Clear guidelines and open communication between DevOps teams and compliance leads is the first step toward a healthy environment.

    The screenshot below presents a recommended and comprehensive backup retention period configuration from an RDS datastore creation process.

    Exposed Resources (Network Access): Publicly accessible database ports and RDS instances are a lucrative configuration loophole for malicious actors. Common misconfigurations may cause RDS database snapshots to be publicly accessible, thus providing unauthenticated users with privileges to access and abuse it through brute-force attacks, known CVEs and DDoS attacks—thereby exposing sensitive data. This risk will also impact the organization’s ability to comply with international regulatory requirements and should be a high-priority configuration when creating and maintaining such resources.

    The screenshot below presents recommended network (public) access and default port configuration from an RDS datastore creation process.

    Weak Encryption: Encryption of sensitive data—both at rest and in transit—is a necessary security baseline to ensure the integrity and confidentiality of organizational data and make it inaccessible to unauthorized users. Common encryption misconfigurations may include implementing inadequate encryption for communication channels such as TLSv1.2 (and higher), weak cipher suites or untrusted CA which may lead to a false sense of security. For data at rest, another common misstep is a lack of key rotation or exposure of the encryption key to unauthorized entities.

    The screenshot below presents the recommended basic encryption-at-rest configuration to be enforced when creating an RDS datastore.

    The screenshot below presents recommended basic encryption-in-transit configuration to be enforced when creating an RDS datastore.

    Improper Authentication & Authorization (RBAC): A cornerstone of adequate data security hygiene is ensuring that access is only associated with a unique account based on the principle of least-privilege, using strong identification methods with appropriate justification. With RDS, users may not use IAM accounts, provide over-privileged access and permissions to either the main instance or to backups and refrain from implementing a user-access quarterly review.

    The screenshot below presents the recommended authentication configuration to be enforced when creating an RDS datastore.

    Incomplete Audits: Visibility cannot be considered a point-in-time activity. Monitoring of access and use of sensitive data is an essential component of tracking organizational assets and will provide crucial information in case of an incident or potential data breach. Incomplete or misplaced audit records including login/logout events, short-lived audits or leakage of sensitive data from RDS to audit logs increases the risk. 

    The screenshot below presents the recommended audit and log configurations when creating an RDS datastore.

    Misconfigurations and single points of failure in cloud environments may be unavoidable for all organizations, especially those operating at scale, due to knowledge gaps, high velocity of data and changes and a lack of visibility as the amount of data increases. While the security baseline may be covered by RDS configurations, these are only the minimum required to protect your environment. DevSecOps teams are encouraged to work closely with their compliance leads to ensure that customer data is secured; leveraging RDS for its benefits is a complex and entirely more substantial responsibility. Organizations must view their data as an inherent part of their internal assets. As its owner, they should create appropriate ecosystems that allow continuous oversight and monitoring of the data, gaps that may arise and, most importantly, the controls in place to secure it. 

  • Quick! Define DevSecOps: Let’s Call it Development Security

    Quick! Define DevSecOps: Let’s Call it Development Security

    For a good long while, DevSecOps referred specifically to vendors like Veracode that did static application security scanning, dynamic application security scanning, software composition analysis and some form of runtime monitoring (usually interactive scanning). Then we realized that DevSecOps was potentially a lot more than that and, like DevOps, we drove the word to encompass all things that shift security left.

    That’s where we are now. But that leaves the DevSecOps market confusing to a lot of people. I propose we rename the tools that specifically focus on code quality to Development Security—even though that name is not perfect and all-encompassing. It’s a better fit than terms in other spaces, like Password Management, which includes two-factor authentication (2FA), multifactor authentication (MFA) and, increasingly, passwordless and secrets management, none of which actually use passwords for the focus of their functionality.

    This way, we can all understand that DevSecOps is the mindset and methodology and Development Security is one piece of implementing DevSecOps.

    There are probably people smarter than I am who have come to this (or alternate) conclusions, and I’m happy to run with their solutions, too—this is just an attempt to make it less confusing for those who don’t live and breathe the security and DevOps markets. If a new person says, “I think we need DevSecOps,” they are in for a journey to actually figure out what they meant and if what they need is actually DevSecOps. We should strive to clarify that so, at the beginning of their research, it is clear that DevSecOps is like DevOps—an overarching term for policies, procedures, mindset and products while the tools that can help with DevSecOps, like those that make up the overarching DevOps toolchain, belong to a market segment named X. In my case, I proposed Development Security because it’s a close fit. What we actually call it is irrelevant, as long as new people understand it is a subset of DevSecOps.

    While you continue to keep the systems spinning and the organization online, when you research these products, use phrases like, “I’m looking for Development Security tools that fit with a DevOps environment.” Markets are largely named by vendors and pundits but, in the end, all these people want to talk in terms their customer base recognizes. So, if enough of you ask for Development Security, that’s what they’ll all start calling it. Effective immediately, that’s what I’m calling it. It is a distinctive set of functionality that all orgs who do internal development need, so let’s give it a distinctive name. And yeah, drop me a line if you have a better name. I’m tied to making it easier, not to a specific term.

  • Chip-to-Cloud IoT: A Step Toward Web3

    Chip-to-Cloud IoT: A Step Toward Web3

    During the first six months of 2021, IoT devices were breached 1.51 billion times, a significant increase from only 639 million breaches observed for the entirety of 2020. This problem can be attributed to the widespread adoption of the internet-of-things (IoT) and the Windows Server Message Block (SMB), and neither can be avoided in the age of remote work. 

    But while these technologies offer unprecedented levels of data mobility and oversight for many interconnected digital and physical systems in industrial settings, they also make it easier for malicious actors to penetrate the very fabric of our lives if left unsecured. 

    This makes it all the more important for DevOps teams and developers to prioritize cybersecurity while creating systems for the IoT. It is important to address the inherent flaws in security like easily-breakable usernames and passwords or saving data without protection on remote access servers. 

    There is a way developers and DevOps professionals can help curb the increasing instances of cybercrime, and it’s called chip-to-cloud IoT. Let’s explore how it is emerging as a viable way to construct a safer, more valuable and more decentralized system for everyone.

    IoT: Understanding the Inherent Security Challenges

    It’s best to investigate potential cybersecurity issues that may befall your organization before finding ways to secure your IoT platforms with solutions such as a shift left security approach (meaning security is embedded in the earliest phases of the application development process) or an IoT decentralization approach, as we’ll expand on below. 

    Some common security concerns include:

    Vulnerabilities in the Software and the Firmware

    It can be challenging to guarantee the safety of IoT systems because of the following security flaws:

    • Lack of computational capacity to provide adequate built-in security
    • Inefficient access control in IoT systems
    • Insufficient funds for adequately testing and further developing the firmware’s security
    • Lack of regular patching and upgrades as a result of IoT devices’ technical and budgetary restrictions
    • Unpatchable vulnerabilities because users don’t regularly update their gadgets
    • Older devices may not be compatible with newer software updates
    • Easy accessibility means attackers could insert corrupted chips or hack via radio waves

    Possibility of Cloud Breaches

    Hackers can gain access to the data processed by your IoT system by intercepting unencrypted messages. 

    Your device might contain sensitive information such as your location, bank account information and medical records. Attackers can access critical information in many other ways, including by exploiting weakly protected communications.

    Even if every piece of data is transported to cloud storage, intruders can potentially target cloud-hosted applications. Therefore, the risk of data leaks is not just from the devices themselves but also from the cloud environments to which they are connected. That is why DevOps pros, developers and software engineers must find ways to protect the technology from the start and prevent hackers from breaching the cloud.

    Risks of Malware

    A recent survey by Zscaler revealed that smart TVs, set-top boxes and smartwatches are most susceptible to malware attacks. 

    If attackers successfully infiltrate an IoT system or insert malware, they may be able to change its functionality, acquire personal data and carry out other malicious activities that can endanger device owners and users. 

    The trouble is that manufacturers struggle to maintain proper software security to keep their devices safe. Security often takes a backseat when these gadgets are built, leaving buyers to deal with the aftermath if they get infected with malware. 

    How Chip-to-Cloud IoT Minimizes Security Risk

    There are two ways for organizations to enhance their IoT security. 

    The first step is ensuring both device manufacturers and software developers adhere to sound security principles. If the value chain is exposed, the solution must be chip-to-cloud. 

    Second, they must take an in-depth approach to protecting key technology by layering security controls to create an iron-clad IoT infrastructure.

    The chip-to-cloud architecture enables the creation of secure, low-energy networks of IoT devices that can communicate directly with cloud hosting platforms. The collective effort to build a cloud-first technology stack has been costly, inefficient and unreliable.

    Traditionally, IT professionals have used non-machine-hosted security tools like firewalls to secure IoT devices. Chip-to-cloud architecture aims to fix this underlying flaw.

    Features of Chip-to-Cloud IoT

    Onboard security and computational power are two of the most significant shortcomings of IoT devices. In addition to being more powerful and safe than previous models, IoT devices built with chip-to-cloud security are also more energy-efficient.

    Here are the features that make the chip-to-cloud technology so effective:

    • An onboard cryptography engine
    • Generator of random numbers
    • Sufficient random-access memory (RAM)

    How It Works

    Reliable software design is essential for IoT devices and other internet-connected devices. It keeps hackers from stealing your identification or duplicating your device for their ulterior motives. Chip-to-cloud delivers on all fronts. 

    These chipset characteristics confer an extra security advantage. Each IoT node is cryptographically unique, making it nearly impossible for a hacker to impersonate it and access the more extensive corporate network to which it is connected. Chip-to-cloud speeds things up by eliminating the need for traffic delays between the logic program and the edge nodes that are ready to take action on the information.

    The chip-to-cloud architecture of the internet-of-things is secure by design. New tools are being developed to provide bespoke and older equipment with data mobility capabilities, just like the current IoT. 

    Nevertheless, chip-to-cloud chipsets are always connected to the cloud. As a result, the availability of assets and the speed of digital communication across nodes, departments and facilities will be significantly improved. Chip-to-cloud IoT is a significant step forward in the evolution of the IoT toward Web3. Experts predict IoT device customers and value producers will benefit more from these design principles than centralized providers.

    Chip-to-Cloud Enables IoT Decentralization

    The traditional approach to IoT is to cover an already-existing cloud of linked devices with third-party firewalls and other security measures. Its equivalent imagery would be a single umbrella covering a family of 12—or at least attempting to do so. The protocols that govern security must be central to decentralization.

    Chip-to-cloud IoT is a powerful, uniquely secured chipset that can be installed inside each device, giving it a far greater connection in the IoT chain than the average IoT gadget.

    The cloud manager or analytics application receives data from every node in real-time using the chip-to-cloud technology. The process does not add another layer of protection; instead, it reduces the latency and loss of data packets as they travel from one destination to another.

    The Future of IoT

    In the wake of recent global events, general cloud technology and chip-to-cloud architecture are receiving huge investments. IoT can supply the data that companies and organizations need to fuel their client engagement portals, machine maintenance platforms and enterprise planning tools.

    That said, the internet-of-things will not be entirely safe even with chip-to-cloud technology. Businesses must have a security-focused culture, standards for managing new IT investments and the expertise to pick the best technology partners.

    The Web3 transition will likely be coupled with future Web3 innovations such as blockchain to improve its utility and security. Seeing how the world’s genuine value-creators use these new solid tools will be intriguing.

    Conclusion

    Considering security at the very onset of any IoT project is critical. Maintaining comprehensive cybersecurity in IoT environments is challenging due to the rising frequency of cyberattacks and increasingly complicated system vulnerabilities. 

    The safety of IoT devices is essential to the operation of many businesses—and even entire economies. Engineers are under increasing pressure to ensure that the rapidly expanding number of interconnected devices is safe. This is why chip-to-cloud technology is poised to become the next big thing in this field. Are you ready to be the change?

  • DevOps Connect: DevSecOps — Building a Modern Cybersecurity Practice

    DevOps Connect: DevSecOps — Building a Modern Cybersecurity Practice

    Malicious actors are constantly looking for new ways to gain access to sensitive data and corrupt systems. As software supply chain attacks are on the rise, security has become a top priority and a growing area in the regulatory and standards landscape. DevOps teams need to approach security as an ongoing part of the software development cycle by implementing DevSecOps practices at scale to improve the security of their software supply chain and ensure the components of the application code are trustworthy.

    At DevOps Connect: DevSecOps Virtual Summit, on July 12, we’ll share exclusive content from our RSAC 2022 event to help DevOps teams integrate security in their process from the start. Internationally recognized DevOps and security leaders will examine the supply chain risk landscape, market trends and practical solutions, and provide strategic insights on emerging regulatory topics.

    Attendees will gain valuable insider knowledge, tips and tricks they can share with their team and immediately put into practice to build a modern cybersecurity practice within their company.

    Things You’ll Learn

    The agenda features a mix of DevOps and cybersecurity topics, including:

    • AppSec and DevSecOps best practices
    • SBOMs and security regulations
    • API, open source and supply chain security
    • Cloud and Kubernetes security risks
    • Pentesting at scale
    • The three ways of DevOps
    • Shift-left security
    • Open Zero Trust
    • And more

    Meet the Speakers

    The speaker lineup features some of the foremost innovators in the DevSecOps community, including:

    • Jim Zemlin, executive director at the Linux Foundation
    • Chenxi Wang, founder and general partner at Rain Capital
    • Jennifer Czaplewski, senior director of cybersecurity at Target
    • John Willis, senior director of the global transformation office at Red Hat
    • Caroline Wong, chief strategy officer at Cobalt
    • Priyanka Sharma, executive director at CNCF
    • Dr. Amit Elazari, director of global cybersecurity policy at Intel Corp
    • Cherilyn Pascoe, senior technology policy advisor at National Institute of Standards and Technology (NIST)
    • Larry Maccherone, DevSecOps transformation architect at Contrast Security
    • Tracy Bannon, senior principal at The MITRE Corp
    • And many others

    Visit the Exhibit Hall

    In addition to the sessions, attendees can visit the virtual exhibit hall to interact with partner sponsors including Sysdig, Sonatype, Check Point Software Technologies, Prisma Cloud, SUSE, Qualys, Traceable AI and more. The expo hall will be open throughout the event, giving attendees the opportunity to chat with the sponsors and download information at any time.

    Registered attendees will also receive a link to access the virtual environment on-demand. To see the full agenda and to register for free, please visit the DevOps Connect: DevSecOps Virtual Summit website.

  • The Age of Software Supply Chain Disruption

    The Age of Software Supply Chain Disruption

    The software supply chain is swiftly becoming a widespread attack vector, and securing it is now in the spotlight. Software supply chain attacks have become a given in 2022, reports Darktrace.

    SolarWinds, Kaseya and GitLab are just a few examples of organizations that have been vulnerable to attack in recent years. We’ve also witnessed an increasing number of exploits, like Log4j and Log4Shell that have the potential to affect billions of devices. What these instances have in common is they leverage vulnerabilities in open source software and third-party dependencies commonly used in the software delivery process — meaning one corrupt package could have broad encompassing impacts on the tech industry at large.

    I recently chatted with Doug Dooley, COO, Data Theorem, about the consequences of today’s intricate software supply chains. “The software supply chain problem is bad, but it’s only going to get worse in the next five years,” he said. According to Dooley, application security requires a continuous context of how the app functions at runtime in a software ecosystem—otherwise, it will continue to miss gaping holes in the big picture.

    Understanding Software Supply Chain Disruption

    Most people understand what a traditional product supply chain is—it encompasses all the components involved to bring a product to the end consumer, from raw materials to manufacturing, distribution and retail locations. But, amid the pandemic and recent international conflict, we’ve witnessed how brittle this chain is. If you shut down one of these components, all consumers feel the effect, resulting in everything from gas hikes to baby food shortages. A full 61% of organizations were hit by supply chain attacks in 2021.

    In software, the supply chain can be thought of similarly—there are many components involved between a software product and the end consumer. The developer writes code, integrates third-party libraries and frameworks, puts it in source code systems, pushes it to test environments and eventually deploys it to production. And often, DevOps has the view of this whole picture.

    “DevOps has become some of the most powerful people in the business,” said Dooley. “They can take a business person’s idea and turn it into reality in the form of an application.”

    Third-party exposures can create huge disruptions in this above workflow. This could take the form of insecure libraries, underlying cloud vulnerabilities or API vulnerabilities, says Dooley. “Customers will embed APIs in the software stack—to them, it’s a black box.” There’s also the potential for misconfigured S3 buckets, which continually leak millions of records. More serious application security threats include remote code execution, which can invoke a detrimental loss of control.

    Why Now?

    So, why the sudden peak in software supply chain-related security incidents? After all, open source software, APIs and third-party vendors have been around for decades. Well, according to Dooley, there are several factors at play here.

    For years, the Silicon Valley mantra has been to ‘move fast and break things.’ Take Netflix engineering, for example, which is constantly evolving services and releasing dozens of changes per day. According to Dooley, such rapid application development often means that security is following development rather than preceding it or becoming integrated into the process, à la DevSecOps. This could lead to what he calls an “ambulance chaser moment.”

    The attack surface is heightened as the number of third-party suppliers and dependencies increases. The average company has 254 SaaS apps, Productiv data found. Also, on the attacker side, we’re seeing increasing levels of automation and sophistication. “You only have to be right one time when attacking,” said Dooley. “The deck is stacked in an attacker’s favor.”

    Strengthening The Chain

    The software supply chain issue is a complex problem that won’t be solved by any single action. However, Dooley offered several tips for companies to consider as they confront this broad problem:

    • Source code analysis and vendor management. It’s important to vet partners and have a process to audit new software by scanning for common vulnerabilities and exposures (CVEs). Yet, source code analysis falls short in many areas, admitted Dooley, as it is unlikely you will be able to view the source code behind cloud API calls and production behavior doesn’t always match documentation.
    • Continuous runtime security monitoring. New exploits are being found constantly. Thus, organizations require a continuous defensive posture at runtime, even if they were found to be secure in development. Continuous full-stack discovery must consider many third-party components, like SDKs, open source code, mobile apps, web services, cloud services, APIs and identity and access management, explains Dooley.
    • Increased transparency with SBOMs. More widespread use of software bills of materials (SBOMs) will be critical to increasing transparency across the industry. Just as nutrition facts describe the food we eat, SBOMs spell out the internal makeup of software components. “We’re trying to get to a place where we get more transparency into the software we depend on,” Dooley said.
    • Rapid vulnerability detection. Have a good handle on the attack surface. Businesses will want to collect an accurate inventory of what can be attacked.
    • Build protection around the “crown jewels.” Sometimes, you must prioritize which steps to take first. Dooley reiterated that securing applications that interface with customer data is critical, as these are the “crown jewels”—leaks here could have serious consequences around regulatory penalties and brand reputation.

    Caution: Heavy Lifting Required

    “The moment you ship a piece of software, you’ve taken on a risk,” said Dooley. But at the same time, if you’re not creating digital interfaces, you’re missing out on massive opportunities.

    Reusing code and open source packages have become necessary to maintain agility to help continuously release digital products. “Even if you build everything from scratch, you’re still reusing a lot,” said Dooley. “Without it, you’d never get to market.” While these tools enable unprecedented speed, we can’t overlook the potential vulnerabilities that may lie within the software supply chain.

    With its unique window into the software deployment and automation process, the DevOps sphere will undoubtedly have a role to play in protecting this paradigm. “Let’s push for more transparency and disclosure,” said Dooley. “But don’t hang your hat on that to make your program great—if you really care about this problem, you’ll have to do some heavy lifting in the real world.”

  • At Some Point, We’ve Shifted Too Far Left

    At Some Point, We’ve Shifted Too Far Left

    Those of us involved in DevOps have a tendency to see the world with blinders on. It is rather easy to fall into the “If all you have is a hammer, everything looks like a nail” trap. We have used the phrase “shift left” with the attitude that this is the solution to every problem in IT.

    Just for clarity, it is not. That is as blunt and unambiguous a statement as I can make. DevOps is great and has helped in many areas, but it is not the only solution, and not every problem is a DevOps toolchain problem.

    In the end, much critical work in IT is not DevOps-related, nor can it be impacted by DevOps. I’ll mention two areas that are obvious and talk in-depth about one of them.

    First off, no matter how far down the Agile/DevOps trail you travel, the wagon you are riding in is separate and distinct from the DevOps process. If you have a data center, all that hardware is still, in the end, hardware that must be configured and managed. For the “But we’re 100% in the cloud!” crowd (who say that loudly and forcefully enough that I suspect they’re trying to convince themselves that this is a good thing)—in the end, you are still using a network and computers to connect. The massive work-from-home migration due to lockdowns made it abundantly clear that IT is needed to set up and support both. Some of the horror stories coming across ops and DevOps social media were … ugly. Employees don’t want to be—and should not need to be—IT specialists to get their job done.

    The other easy one is security. Yes, insert security into development and DevOps workflows. Yes, make operations staff more aware of security. Yes, automate security as much as possible because we’re chronically short of security staff. But, in the end, lots of security is external to DevOps. While Kubernetes security is absolutely DevOps, password management, for example, absolutely is not. And that’s just one of many, many examples.

    So I am going to ask (but not answer) the question: Where is the line? How far left is too far? The answer is yours to decide with regard to your organization and the tools/environment that exist in your organization. In a primarily cloud environment, the answer to this question is far different than in a primarily data center environment. It is rare that any organization be purely one or the other at this point in time, so there are similarities, but few exact solutions we can name.

    Security is akin to a cost center. It is actually insurance, but few orgs view it that way. And even if they did, it would not make much difference. Much as my father once declared that he and Mom were “insurance poor” and canceled half their policies, most orgs want to contain the costs of security whether they admit it or not. When ‘security merged into DevOps’ hits the ears of people trying to stretch budgets just as far as they can, bad things tend to happen.

    DevSecOps implements security where it should have been all along—at the development stage—but it does not replace other security, it just makes our apps more protected. There are cases where security testing hours are seriously reduced, but since testing in all forms is the first thing to go when staffing is tight, again, it is not replacing something that was happening—it is making it possible for us to do what we should have been doing all along.

    It doesn’t help that our toolchains have taken on lives of their own—at many organizations, staff are required to maintain them—because they, too, are apps that DevOps struggles to help. Once you’ve invested time and money in building that edifice, you want to use it as much as possible. So, many orgs want to shift left everything in IT, right into the toolchain octopus.

    You know your org better than anyone. You are the ones making it work day in and day out, in all of your environment’s glorious complexity. Start talking about what is too much; when adding a step in an already complex toolchain needs to be justified. Push back against everything needing to shift left (where appropriate, I’ll say again for clarity, DevOps has done amazing things for us, but too much of a good thing is our living nightmare and, historically, is tomorrow’s technical debt). Keep rocking it, and make sure you are rocking the right things in the right places.

     

  • Survey Uncovers Depth of Open Source Software Insecurity

    Survey Uncovers Depth of Open Source Software Insecurity

    A survey from Snyk and the Linux Foundation published today found that less than half of respondents (49%) work for organizations that have security policies in place for the use or development of open source software.

    The survey, which polled 550 software development professionals, was conducted by Snyk, a provider of tools for securing software, and the Linux Foundation. Just under a third (30%) of respondents without any policies in place also admitted that no one on their team is currently directly addressing open source security. A total of 41% said they didn’t have high confidence in their open source software security. Nevertheless, 59% of respondents said the open source software they deployed is somewhat or highly secure.

    Matt Jarvis, director of developer relations for Snyk, said many organizations are not aware of the degree to which open source software is now being targeted by cybercriminals looking to surreptitiously insert malware into widely used software projects. Those same organizations also lacked clarity about how open source software is constructed, he added.

    In many ways, the open source software community has now become a victim of its own success, noted Jarvis.

    Overall, the survey found the average application development project has 49 vulnerabilities and 80 direct dependencies on open source software. Only 18% of respondents said they are confident of the controls they have in place for their transitive dependencies. A full 40% of all vulnerabilities discovered were found in transitive dependencies.

    Only a third of respondents (33%) are using static application security testing (SAST) tools within a continuous integration process or software composition analysis (SCA) tools to discover those vulnerabilities, according to the survey. Overall, 44% reported they use some type of tool to analyze source code.

    In terms of additional capabilities, 59% said they would like to see more intelligence added to these tools, while 52% cited a need for more clearly defined cybersecurity best practices. Nearly half (49%) also wanted increased automation and tools for conducting security audits.

    The survey also found that fixing vulnerabilities in open source projects takes 19% longer than in proprietary projects.
    Among organizations that do have security policies in place, 80% vested responsibility with the security team. Only 40% of organizations without such policies did the same, according to the survey.

    The Open Source Security Foundation (OpenSSF), an arm of the Linux Foundation, is focusing on 10 streams of investment that, in total, would require more than $150 million in funding to drive greater adoption of DevSecOps best practices among maintainers of open source software projects. Developers today routinely reuse open source software. The issue is that many of those projects are maintained by a small number of programmers that voluntarily contribute their time and effort to build components that others are free to use. Like any other developer, the amount of security expertise those individuals have is limited. The onus for making sure the projects and software are secure is on the organizations that decide to deploy that software.

    Unfortunately, many IT vendors and large enterprise IT organizations reuse that code without contributing anything meaningful back to the project—whether that be in terms of financing or helping open source maintainers find and remediate vulnerabilities. Hopefully, following an executive order issued by the Biden administration, the level of open source security will steadily improve in the months ahead.

  • TechStrong Con: Downturn Brings Additional Sense of DevOps Urgency

    TechStrong Con: Downturn Brings Additional Sense of DevOps Urgency

    Regardless of whether the overall economy is experiencing a correction or is on the cusp of a recession, organizations are going to prioritize some projects over others as resources become more constrained.

    During the “DevOps: Has the Bubble Burst?” panel made up of DevOps executives and investors from venture capital firms that kicked off the virtual TechStrong Con event today, panelists said organizations will need to continue to invest in DevOps automation and cybersecurity to ensure their long-term survival.

    However, the number of IT projects that organizations will be able to simultaneously invest in is likely to decline, said Chenxi Wang, general partner for Rain Capital. “We are starting to see a slowdown of projects, for sure,” she said.

    Fundamentally, organizations will need to decide not so much whether they will modernize legacy monolithic applications but rather at what rate. More than 70% of organizations’ codebases are still based on legacy monolithic architectures, noted Wang.

    No one knows for sure how long the current economic contraction might last, but, as in any downturn, organizations will seek to conserve cash, said JFrog CEO Shlomi Ben Haim. DevOps initiatives will continue to be prioritized because otherwise, organizations risk falling far enough behind that they may not be able to ever recover, he added. “DevOps is in a good place,” said Ben Haim.

    In fact, thus far many DevOps companies have yet to see a decline in demand because organizations are investing in automation to reduce their technical debt costs, noted Thomas Krane, managing director for Insight Partners. “It’s a matter of life or death,” he said.

    At the same time, Krane said organizations might want to take a hard look at the profitability of the vendors they are counting on to drive that innovation. Vendors that have been counting on venture capital funding to maintain operations may soon find that money much harder to come by as capital becomes scarce, he noted.

    The core issue will become how quickly returns on investment (ROI) are generated, added Incredibuild CEO Tami Mazel Shachar. “There is going to be a lot more focus on ROI,” she said.

    In the meantime, the debate over what constitutes DevOps continues. Some narrowly define DevOps as a set of development activities that revolve around continuous integration/continuous delivery (CI/CD) platforms. Others definitions of DevOps include everything from application testing to observability and application security. No matter how big the proverbial DevOps tent is defined to be, however, the number of organizations that are embracing DevOps—and now DevSecOps—workflows more deeply only continues to increase.

    The challenge is focusing those efforts on the projects that deliver the most value for the business in the face of executive uncertainty about the future. Of course, nothing quite focuses the mind like an impending crisis; for better or worse, it will require more organizations to honestly assess their current capabilities.