staging-devopsy.kinsta.cloud

Category: DevOps Toolbox

  • Diffblue Integrates Generative AI-Based Testing Platform With GitLab

    Diffblue Integrates Generative AI-Based Testing Platform With GitLab

    Diffblue this week generally made available an integration between its automated unit testing platform for Java and the DevSecOps platform from GitLab.

    The Diffblue Cover platform makes use of generative artificial intelligence (AI) to automatically create unit tests based on reinforcement learning technologies that don’t rely on LLMs—avoiding their drawbacks of sometimes introducing hallucinations and also requiring human review.

    Diffblue CEO Mathew Lodge said the integration with the continuous integration capabilities found in the premium and ultimate editions of the GitLab platform would, for example, streamline regression testing in a way that should ultimately improve both application quality and security. Diffblue Cover, for example, can update tests as much as 250 times faster than a human developer can write them manually without developer review.

    That approach also serves to reduce the level of friction many DevSecOps teams encounter when bottlenecks involving testing processes emerge, noted Lodge. The overall goal is to make it simpler for developers to test as much of their own code as possible before integrating it into a build, he added. Otherwise, developers will get fed up because testing is continuously breaking the build, noted Lodge.

    Ultimately, instead of having to write unit tests, developers become supervisors of a platform that automatically generates them on their behalf, said Lodge. The job of a developer doesn’t go away, but it does fundamentally change, he added.

    To achieve that goal, developers need to be able to access a platform that writes the tests and can then also execute them automatically. If it takes too long to create the test, chances are high that most developers won’t run it. On average, writing and evaluating a single unit test can take a developer 10 minutes. Over the course of any project, thousands of tests need to be written, so the amount of time testing takes away from coding is often much greater than most IT leaders fully appreciate.

    Automating those tests should improve developer productivity as more time is available to focus on writing code rather than testing. That doesn’t necessarily eliminate the need for a dedicated testing team, but it does mean that more tests will be run without slowing down the overall pace of application development.

    Developers naturally want to be able to test code at the very instant they create it. AI platforms can make that happen by, for example, employing reinforcement learning to write unit regression tests.

    Most developers are not going to resist assuming more responsibility for testing if the tools to automate that task are more accessible. Instead of merely shifting responsibility for testing left toward developers, DevOps teams need to find ways to streamline the process. Otherwise, testing just becomes one in a series of tasks that are being shifted to developers in ways that many of them are coming to resent.

    It may take some time before AI is fully integrated into software engineering but it’s clear with each passing day more previously manual tasks are being automated. Among the lowest hanging fruit for applying AI to software engineering are clearly testing processes that, if truth be told, few indeed enjoy conducting.

  • Survey: Widespread Inability to Remediate App Vulnerabilities

    Survey: Widespread Inability to Remediate App Vulnerabilities

    A survey of 621 IT and security professionals published this week found only 20% are confident in their ability to detect a vulnerability before an application is released.

    Conducted by the Ponemon Institute on behalf of Security Journey, a provider of cybersecurity training services for developers, the survey finds 61% struggling to remediate vulnerabilities effectively, with 55% blaming misalignment between development, security and compliance teams for delays.

    A third of respondents (33%) said teams lacked a common view of applications and assets, with 38% acknowledging they lack the ability to hold departments accountable for patching applications. Only 11% of organizations believed they patch vulnerabilities effectively in a timely manner, the survey finds.

    In the 12 months prior to the study, 54% of respondents suffered a security incident due to an unpatched vulnerability, with 51% experiencing more than eight incidents.

    Overall, half (50%) admitted they fail to test the security of their applications after they have been released. Only 36% specifically teach developers to write secure code. Just one-fifth (21%) educated developers on vulnerability remediation. Less than half (43%) have invested in training provided by a third party.

    When training is provided, 68% of respondents also admitted immediate feedback is not provided and less than half (47%) said secure coding training is customized to meet developer needs. Half (50%) of those who provided training have no formal assessment to measure knowledge gain, the survey found.

    Amy Baker, chief marketing officer and security education evangelist for Security Journey, said the survey makes it clear much work still needs to be done in terms of teaching developers how to write secure code.

    In addition, too many software development teams are overly optimistic about their ability to remediate known vulnerabilities the next time they plan to update an application, said Baker. Many developers move on to the next application simply because either the pressure to meet delivery deadlines is too intense or there is some other emerging technology, such as large language models (LLMs), that needs to be learned, she added. In fact, too many organizations are far too focused on the rate at which applications are built and deployed at the expense of cybersecurity, Baker noted.

    That’s especially problematic because cybercriminals have become much more adept at monitoring application deployments and updates, she added.

    The amount of time between when code is deployed and when cybercriminals begin scanning it for vulnerabilities is now measured in hours rather than days. Developers, unfortunately, are deploying more insecure code than ever. In fact, much of that code is insecure by design, noted Baker.

    Overall, nearly half of respondents (48%) only train developers annually, bi-annually or when an incident occurs. More than two-thirds (68%) only train developers on how to implement secure coding practices to meet compliance requirements. In effect, because developers are not taught security in school, what training they do receive is on the job, said Baker.

    Clearly, there is still much work to be done when it comes to implementing DevSecOps best practices to improve application security. In the meantime, the amount of code that needs to be patched has already reached proportions that most application development teams are never going to be able to effectively address.

  • Mobb Extends DevSecOps Reach of Tool to Generate Patches

    Mobb Extends DevSecOps Reach of Tool to Generate Patches

    Mobb today added an ability to instantly surface patches for vulnerabilities at the time when code is being committed during a DevSecOps workflow.

    Previously, the tool provided by Mobb to create validated patches based on the scan results of third-party SAST tools would only surface issues to individual developers.

    Mobb CEO Eitan Worcel said organizations can now address these issues within the context of DevSecOps workflow.

    The Mobb platform combines proprietary security research and traditional semantic analysis with generative artificial intelligence (AI) to automatically create patches for vulnerabilities. The overall goal is to make it simpler for DevSecOps teams to consistently ensure applications with known vulnerabilities are remediated before they are deployed, said Worcel.

    The Mobb approach eliminates the need for IT teams to sort through static application security testing (SAST) scans themselves. Each vulnerability typically requires at least 30 minutes to fix, so the Mobb platform improves application security while simultaneously making developers more productive.

    Historically, one of the reasons for the wide divide between application development teams and cybersecurity professionals is that many of the vulnerabilities discovered don’t actually impact applications running in production environments. Development teams then find themselves wasting time investigating vulnerabilities and, when they do determine a vulnerability is an issue, spending time developing a patch. The Mobb tool automatically creates the necessary patch, so there is less of a need to debate which vulnerabilities are worthy of the time required to create a patch.

    For decades, application development teams have been attempting to find a way to remediate vulnerabilities as quickly as possible. With increased focus on adopting DevSecOps best practices to address those vulnerabilities, more organizations are now reviewing software supply chains as part of a larger effort to improve application security.

    There will, of course, be vulnerabilities that the Mobb tool will not be able to address simply because there may not be enough affected applications to warrant pre-building a patch. Even with the most robust set of rules, it is improbable that an automated tool can, with 100% certainty, safely fix every issue. However, given the number of routine vulnerabilities present in applications, such as SQL injections, there’s plenty of opportunity to improve developer productivity by relying on patches that address common vulnerabilities.

    The process by which patches are prioritized, developed and then applied has, of course, long been a source of DevOps frustration. Anything that streamlines that process will go a long way toward improving the overall state of application security at a time when cybercriminals are becoming more adept at exploiting vulnerabilities. The challenge is finding a way to remediate vulnerabilities that developers will embrace rather than resist. The Mobb approach still provides them with control over what patches are applied but the toil involved with creating a patch in the first place has been substantially reduced.

    It’s not clear how long it might be before automated approaches to building patches might improve the overall state of application security but, as always, every journey needs to begin with a first step in the right general direction.

  • Legit Security Applies AI to Detect Vulnerable Application Secrets

    Legit Security Applies AI to Detect Vulnerable Application Secrets

    Legit Security today announced it has expanded the scope of its application security posture management (ASPM) platform to make use of artificial intelligence (AI) to discover secrets more accurately in applications that cybercriminals can actually exploit.

    Scanners typically generate high false positive rates to the point where developers often start to ignore them. As a result, it’s not uncommon for secrets to be discovered in an application even though an alert was generated.

    Legit Security CTO Liav Caspi said the company is now applying a large language model (LLM) specifically trained for source code along with prompt engineering techniques and heuristics to reduce the level of alert noise that developers would otherwise see when scanning for secrets. The overall goal is to correctly identify access keys, passwords, application programming interface (API) keys and other personally identifiable information (PII) that should not be exposed once an application is deployed in a production environment, he noted.

    That capability can be applied to detect secrets across all development assets, including code repositories, source code management (SCM) tools, build tools and logs, artifacts and private and public documentation. In addition, this approach also uncovers toxic combinations of secrets or ones that might lie buried within assets such as source code history or modified Confluence pages that developers often forget exist, noted Caspi.

    Legit Security also enables DevSecOps teams to apply preventive guardrails on developer endpoints using the Legit command line interface (CLI) to stop secrets from being exposed before code is pushed. In addition, automated workflows can reach developers as part of a pull request or create Slack messages or Jira tickets to streamline remediation. DevSecOps teams can also continuously triage results to create a baseline that includes exceptions.

    As part of a larger effort to secure software supply chains, Legit Security has developed a software-as-a-service (SaaS) platform designed to apply security policies across an entire software development life cycle. The company is now extending the capabilities of that platform using AI technologies to detect secrets in both source code and all the components that make up a development pipeline as they are added.

    In the wake of an executive order requiring federal agencies to better secure software supply chains, there is now more focus than ever on application security. Unfortunately, many developers are a little too cavalier when it comes to managing secrets during the application development process. They often leave secrets in plain text to facilitate workflows, but more often than many care to admit, those secrets are not removed before an application is deployed. Knowing this, cybercriminals have learned to scan applications for those exposed secrets. Once gained, they then appear to be just another legitimate user of an application that might not be discovered until malware is activated.

    It may be a while before the cultural and technical issues that conspire to make the software supply chain insecure are fully addressed. In fact, the problem may get worse before it gets better—simply because the pace at which distributed applications are being built and deployed only continues to accelerate. At the same, the cost of making a secrets mistake is also only going to increase as regulations become more stringent. In fact, it’s now only a matter of time before stiff fines intended to make an example of one or more unfortunate organizations are eventually going to be levied.

  • Chronosphere Acquires Calyptia to Process Log Data More Efficiently

    Chronosphere Acquires Calyptia to Process Log Data More Efficiently

    Chronosphere today announced it has acquired Calyptia to add additional pipeline management capabilities to its observability platform.

    Calyptia created Fluent Bit and Fluentd, a set of open source log processors that are now being advanced under the auspices of the Cloud Native Computing Foundation (CNCF).

    Chronosphere CEO Martin Mao said these log processors will be embedded into the company’s namesake observability platform to enable the routing, transformation and optimization of log data at scale. That issue has become problematic because cloud-native applications based on microservices create exponentially more telemetry data than legacy monolithic applications. If it becomes too costly to analyze that data, then DevOps teams will begin to limit the number of applications they will observe, noted Mao.

    The Calyptia platform achieves that goal using filters at the data source to reduce the volume of data that might otherwise be shared by 30% or more.

    The shared data can then be analyzed in real time rather than waiting for it to be stored and indexed. In addition, DevOps teams can enrich that data in flight to include additional context.

    Calyptia cofounder Anurag Gupta added that the primary mission for the company has always been to make log data more accessible to developers as they build and deploy applications.

    The acquisition of Calyptia comes on the heels of a log management alliance between Chronosphere and CrowdStrike through which a log storage and virtualization capability, dubbed Logs powered by CrowdStrike, has been added to the observability platform. The overall goal is to simplify the management and analysis of logs via a single click of a graphical tool or within the context of a larger DevOps workflow based on a command line interface (CLI) without requiring data engineering expertise, said Mao.

    It’s not clear how fully DevOps teams have embraced observability, but it’s apparent the legacy monitoring tools that collect a set of predefined metrics are not able to keep pace with the complexity of cloud-native application environments. An observability platform makes it simpler to query log data and discover the root cause of an issue—hopefully before a catastrophic event occurs. The challenge, of course, is providing that capability without incurring massive increased storage costs for log data.

    Despite those concerns, the need for observability is becoming a more pressing issue with each passing day. Organizations have never been more dependent on applications to drive revenue. A degradation in performance is usually an early indicator of a disruption that often has a direct impact on revenue and profitability.

    There is, of course, already no shortage of observability platforms. The issue most DevOps teams encounter is finding the funding needed to employ one at a time when organizations have become much more sensitive to IT costs. There may be an opportunity to rationalize some monitoring tools to help pay for the cost of a more modern observability platform, but regardless of approach, an ounce of prevention is always going to be worth a pound of proverbial cure.

  • Swimm Adds Generative AI Chat Tool for Documentation

    Swimm today announced it has added a chat tool that enables developers to use natural language to surface insights into code.

    Company CEO Oren Toledano said the /ask Swimm tool makes it simpler to launch queries that enable developers to better understand how code was constructed.

    The /ask Swimm tool aggregates documentation along with other related data to surface factors that are not evident in the code itself, such as documentation of business decisions, product design considerations and decisions concerning why specific architectural choices were made. It automatically captures and updates code-related knowledge to provide a continuous feedback loop as code, documentation, files and repositories are created and updated to provide deeper insights into code that goes beyond what might have been actually documented.

    The overall goal is to provide a mechanism that enables developers to understand how and why code was constructed within the context of an integrated development environment (IDE), said Toledano.

    Swimm has previously made available a platform that uses generative AI platform to create a static analysis of documentation. That capability should make it simpler for organizations to track documentation at a time when generative AI platforms such as ChatGPT are exponentially increasing the amount of code being written. In theory, those platforms should be able to also create documentation for that code, but there will still be a need for tools to track and analyze it.

    In the longer term, Swimm plans to continue to extend its usage of generative AI to provide deeper levels of insights across entire applications and software ecosystems to address that challenge, noted Toledano.

    In the short term, however, it’s clear that AI is making developers more productive than ever. In fact, the volume of code being generated might soon overwhelm existing DevOps pipelines and workflows. Most DevOps teams will need to revamp those workflows as the pace at which more applications than ever are being developed and deployed faster than ever.

    It’s still early days as far as generative AI adoption is concerned, but it’s clear many developers are already using it to write code. How much of that code makes it into a production environment is difficult to determine, but a lot of that code is going to be of varying quality. A general-purpose AI platform such as ChatGPT was trained using code collected from all across the web. The code generated by these platforms is only as good as the examples used to train it, all of which were created by human developers who may have made mistakes, such as including code that has known vulnerabilities.

    Regardless of whether code was generated by a machine or a human, the probability that documentation will need to be analyzed as part of a code review process is always high. The challenge and the opportunity is to determine the best way to apply AI to a longstanding challenge that is now being exacerbated by the existence of AI tools that, with each passing day, are only becoming more accessible to developers.

  • Broadcom Survey Surfaces Raft of IT Automation Challenges

    Broadcom Survey Surfaces Raft of IT Automation Challenges

    A survey of 501 IT leaders published today by Broadcom suggested that islands of automation that operate independently of each other are major contributing factors to organizations failing to meet service level agreements (SLAs).

    The survey found that 98% of respondents encountered at least one automation issue, with 61% noting that SLA issues are a monthly occurrence. More than a third (35%) said these issues occur weekly. A full 88% indicated SLA breaches directly impacted customers in a significant way.

    The survey also found nearly three-quarters (74%) lacked visibility across their data center and cloud environments, with an equal number lacking end-to-end visibility of their automated business processes. Only 28% possessed tools that could predict automation issues, and more than two-thirds (68%) were unable to model or predict whether a new or updated automation job would actually meet SLAs. A full 69% said they are unable to properly size automation workload resources, with 61% having no way to simulate automation workloads before implementation, the survey found.

    Only slightly more than half (55%) of respondents, however, could provide SLAs and key metrics in a format that is appropriate for business stakeholders.

    Aline Gerew, head of automation for the Agile Operations Division at Broadcom, said that while it’s routine for IT teams to leverage a range of automation tools and platforms to manage IT, it’s clear they are challenged to manage those platforms in a comprehensive fashion. As a result, whenever there is an issue with one tool or platform, it creates a bottleneck that impacts customer experience in a way that ultimately adversely impacts revenue, she noted.

    In fact, many IT teams are now starting to identify automation architects within their IT teams to enable DevOps teams in a more holistic manner, added Gerew.

    Overall, the survey found that 80% of respondents are using three or more automation platforms, with just over a quarter (27%) using six or more.

    Nearly three quarters (74%) are also relying on three or more observability solutions, with just under a quarter (24%) using six or more. Just under three-quarters of respondents noted they are consequently subjected to alert storms that ultimately serve to make it more challenging to resolve issues. More than half (56%) of respondents are blind to the downstream effects of an initial alert and, as a result, do not inform response teams of which systems, jobs, applications and business processes are affected.

    In some ways, IT automation has become too much of a good thing. Every provider of an IT platform makes automation tools available, but as they are invoked, each one creates yet another island that IT teams are expected to manage in isolation. However, all it takes for an SLA not to be met is for an issue to arise that IT teams can’t easily determine the root cause of because one or more automation platforms obscure the underlying issue in the absence of any standardization, noted Gerew.

    Given the complexity of modern IT environments, automation is now a requirement. The issue, as always, is finding a way to apply it all in the most transparent way possible.

  • 5 Time-Saving and Problem-Solving Technologies for DevOps Teams

    5 Time-Saving and Problem-Solving Technologies for DevOps Teams

    DevOps has transformed almost every aspect of software development, taking us from the dark ages of big-bang deployments every 12 months to the reality of releasing software multiple times per day. However, DevOps professionals have to overcome several hurdles to successfully implement and sustain the DevOps culture within the organization. Let’s review some of these challenges—and five technologies that can help.

    Cultural and Organizational Differences

    One persistent challenge for DevOps teams in a mature setting is navigating cultural and organizational differences. These differences often manifest in varied working styles, communication methods and priority management across teams. For instance, development teams might prioritize rapid innovation, while operations teams focus on stability and security, leading to conflicts. Managing these diverging priorities requires constant negotiation and alignment, which can be time-consuming and complex. 

    Furthermore, as organizations grow and incorporate diverse teams, including remote and international members, ensuring cohesive communication and understanding becomes even more critical. This diversity, while beneficial, brings challenges in terms of aligning on common goals and working practices, often leading to delays and misunderstandings that impede efficient workflow.

    Complexity of Automation

    In mature DevOps environments, the complexity of automation can become a significant day-to-day challenge. As the infrastructure and application landscape grows, the automation scripts and tools that were once efficient can become unwieldy and difficult to manage. Keeping these automation processes up-to-date with the constantly evolving technology stack is a never-ending task, requiring significant time and resources. 

    Additionally, the integration of new tools or updates to existing ones often introduces complexity, leading to potential errors and downtime. This complexity not only slows down the deployment process but also requires DevOps teams to invest time continuously in learning and adapting to new tools and methodologies. Ensuring that automation is both robust and flexible enough to handle a dynamic environment is a delicate balance that DevOps teams must constantly maintain.

    Cross-Team Collaboration

    Effective cross-team collaboration is vital for DevOps teams but can be challenging in day-to-day operations. The interdependence between various teams–such as development, operations, QA and security–requires a high level of coordination, often hindered by differing goals, tools and terminologies. Communication barriers can lead to misalignment on project objectives, causing delays and inefficiencies. 

    Moreover, in a fast-paced DevOps environment, keeping everyone on the same page regarding updates, changes and issues requires meticulous attention. This is compounded when teams are distributed geographically, as time zone differences and cultural nuances add another layer of complexity. Regular synchronization meetings, clear communication channels and shared tools are necessary, but managing these effectively without causing meeting fatigue or information overload is a continuous challenge.

    Consistency and Standardization

    Maintaining consistency and standardization across various projects and teams is a significant challenge for mature DevOps teams. As the number of services and microservices increases, ensuring that each adheres to the same set of standards for coding, security and deployment becomes increasingly difficult. Inconsistencies can lead to significant problems in integration, testing, and deployment, slowing down the entire delivery process. 

    Furthermore, as different teams may prefer different tools and practices, creating a standardized environment that accommodates everyone’s needs without compromising on efficiency or quality is a complex task. Regular audits, effective governance and a balance between flexibility and standardization are essential to manage this challenge, but they require constant effort and vigilance.

    5 Lesser-Known Technologies That Can Help

    While there is no one technology that can completely alleviate the above challenges, there are several tools that can help. Let’s review five technologies that can make the life of a DevOps team just a little bit easier.

    Chaos Engineering Platforms

    If you’re part of a DevOps team, you’re probably all too familiar with the panic that ensues when a system fails unexpectedly. Chaos engineering platforms aim to save you from such unforeseen surprises by embracing the philosophy of ‘breaking things on purpose.’

    Chaos engineering involves intentionally introducing failures into systems to understand how they respond and to identify potential weaknesses. By doing so, DevOps teams can proactively find and fix issues before they manifest in production, thereby saving significant time and preventing potential downtime.

    Platforms like Gremlin and Chaos Monkey offer controlled environments for running these chaos experiments, providing teams with valuable insights and recommendations. In the long term, this can not only save time but also instill a sense of confidence and resilience in your systems.

    Application Dependency Mapping

    Application dependency mapping (ADM) is a process that helps DevOps teams visualize and understand their applications and their dependencies. It is a crucial technology that aids in quickly diagnosing and rectifying application performance issues, thereby saving time.

    ADM provides a holistic view of the application architecture, showing how different components interact with each other. This comprehensive visibility is invaluable during troubleshooting as teams can easily pinpoint problematic areas and rectify them without having to sift through mountains of code.

    Furthermore, ADM is crucial when planning for application changes or upgrades. Understanding how changes in one component may affect others helps in preventing potential disruptions, leading to smoother and faster software delivery.

    Policy-as-Code Solutions

    Policy-as-code is a practice where policy definitions are codified and managed as code files within version control systems. This approach offers a host of benefits that can save time for DevOps teams.

    Policy-as-code solutions enable automatic enforcement of policies, eliminating the need for manual checks. This significantly reduces the time taken to ensure compliance and allows teams to focus more on developing and delivering quality software.

    Moreover, since policies are managed just like any other code, they can be versioned, tested and reviewed before being deployed. This allows for a more streamlined and error-free policy management process, saving valuable time and effort. Tools like Open Policy Agent and HashiCorp Sentinel are commonly used to implement policy-as-code in DevOps environments.

    Service Mesh Technologies

    Next on the list is service mesh, a dedicated infrastructure layer for managing service-to-service communication in a microservices architecture. Service mesh technologies offer a range of features that can save time for DevOps teams.

    One of the primary benefits of a service mesh is its ability to offload common communication concerns from individual services to the mesh itself. This includes functionalities like load balancing, service discovery, and traffic management. By doing so, it saves DevOps teams the time and effort required to implement these functionalities within each service.

    Additionally, service mesh technologies provide detailed observability of the service communication, making it easier for teams to monitor, troubleshoot and optimize their applications. Istio and Linkerd are two common open source options for implementing service mesh.

    Codeless Test Automation Tools

    Codeless test automation tools represent a significant shift in the way DevOps teams approach software testing, offering a more efficient and less time-consuming method. These tools allow the creation of automated tests without the need to write extensive code, making test automation accessible to team members who may not have deep coding expertise.

    Using a graphical user interface, codeless test automation tools let users design tests by simply dragging and dropping elements into a workflow. This approach not only accelerates the test creation process but also simplifies the maintenance and updating of tests as applications evolve. Additionally, these tools often integrate seamlessly with various CI/CD pipelines, ensuring that automated tests can be easily incorporated into existing DevOps workflows.

    With the speed at which technology is advancing, it’s crucial for DevOps teams to stay updated on the latest tools and technologies that can help them optimize their processes. The technologies listed in this article can significantly save time for DevOps team and make complex DevOps workflows a little smoother.