staging-devopsy.kinsta.cloud

Category: DevOps Toolbox

  • Out of the Silo: Big Data and DevOps Meet in the Middle

    Out of the Silo: Big Data and DevOps Meet in the Middle

    Given the myriad benefits they provide, it seems crazy that big data and DevOps have been implemented independent of each other

    If you have been working from home during the recent lockdowns, you probably don’t feel like you are making many new connections, but when we come to look back on this time in a few years, it might prove to have been one of the most effective networking sessions of the past decade.

    Nowhere is this more true than in the twin worlds of big data and DevOps. It seems that being forced to work from home, far from locking engineers in both disciplines deeper within their silos, has brought them out of their shells. In recent months, we’ve seen an outpouring of speculation on how DevOps and big data can work together, and even a number of impassioned manifestos on why it is time to bring DevOps ideas to big data.

    In this article, we’ll look at why this crossover is suddenly so popular, how these two disciplines can engage in a symbiotic relationship with each other, and how they are already doing so.

    The Pandemic and the Silo

    For consumers, the experience of lockdown has accelerated internet usage trends. While some of the changes that have recently swept the online world were easy to guess, the speed at which they have taken hold has been astounding. It’s therefore no exaggeration to say, as the New York Times put it recently, that the virus has changed the way we use the internet.

    These consumer trends are driving changes throughout the software development industry, forcing firms to adapt to rapid release, testing and update cycles.

    One of the biggest changes in the way that developers collect data over the past decade has been the source of the data. This year, and largely due to the COVID-19 pandemic, we passed a notable watershed: Mobile devices and smartphones now account for more than half of all internet traffic, resulting in an increased data flow that boggles the mind and creates issues of its own. This increased data load from consumer mobile devices, rather than from corporate data acquisition systems, is forcing teams to work more closely together.

    In the words of Bill Detwiler, editor in chief of TechRepublic, in a recent CBS interview: “The COVID-19 pandemic has accelerated the blending of data analytics and DevOps, meaning developers, data scientists, and product managers will need to work more closely together than ever before.” For most firms, the way in which these consumer trends are fed into the development life cycle is via analytics teams. This has meant that, during the pandemic, analytics teams were generating actionable insights at an increased rate, and this has been the driving force behind the newly intimate relationship between big data and DevOps staff.

    Waterfall vs. DevOps

    To understand the way in which big data and DevOps teams can best work together, it’s instructive to visualize the way in which developers, operational staff and data analytics agents work.

    A decade ago, most software developers were using some form of waterfall development methodology. As the name suggests, this approach was essentially linear—the specifications for software were set in advance, and after this point multiple teams worked in tandem to deliver a continuous “stream” of software updates.

    The DevOps framework changed that by encouraging communication and collaboration between operations and development staff. The idea, in short, was to ensure that software underwent continual testing—not just in terms of usability and security, but also against the requirements of the operations staff who were charged with maintaining it.

    Now, we might be facing a shift of similar scale. Many firms have now adopted big data frameworks, but these systems generally have been seen as a way of harvesting and collating market insights. This was always a waste of potential for big data because as long as it was regarded as part of marketing, it’s utility in other parts of an organization were difficult to discern.

    Not any longer, it seems. The changes caused by the pandemic—both in consumer behavior and in the way that organizations communicate internally—have given developers an appreciation of the value of the data that analytics teams hold.

    Pioneers Show the Way Forward

    It’s also worth noting that big data and DevOps teams working in tandem is not a utopian vision; rather, this way of working was already being explored within leading companies even before the pandemic.

    In retrospect, in fact, it seems crazy that these two teams ever worked independently. If you’re a transportation company, the ability to track your loads on the road and the health and safety of the cargo that they’re carrying has become mission-critical. If you’re in the armed forces and using drones on the battlefield to conduct and report reconnaissance in real-time flyovers, the ability to adapt software parameters on-the-fly can be the difference between success and failure.

    The military is, in fact, the place to look for examples of how big data and DevOps will shortly become one process. Big data in the military has long been more advanced than that in the civilian sector and is beginning to inform everything from IT predictive analytics to real-time usage analytics.

    The Future of Big Data and DevOps

    There remain, of course, significant challenges to be overcome before the promise of DevOps working with big data teams can be realized. Perhaps the most thorny of these is the fact that computing performance is crucial for big data in a way it hasn’t been for development: The latter teams may need to build capabilities to make use of the insights generated by big data teams.

    Still, communication can only be a good thing, and if the pandemic has forced some of us to emerge from our bunkers and start to work together, then at least one good thing will have come from it.

  • Broadcom Extends AIOps Platform Reach

    Broadcom Extends AIOps Platform Reach

    Broadcom today extended the reach of its artificial intelligence for IT operations (AIOps) platform to provide full-stack observability via integrations with 15 third-party data sources including Dynatrace, Datadog, AppDynamics, PagerDuty, Splunk, ServiceNow, SolarWinds and Microsoft Systems Center Operations Manager.

    In addition, Version 2.0 of AIOps from Broadcom now captures data at the chip level to enable IT teams to triage network congestion and packet loss in real-time.

    The latest release also adds visualizations and dashboards for different IT personas along with a recommendation engine that enables IT teams to automate tasks once the root cause of an issue is surfaced by the platform.

    Sreenivasan Rajagopal, director of products and strategy for Broadcom Enterprise Software, said the new approach allows IT teams to decide what level of automation to apply based on the complexity of the problem and potential risk to the business.

    Collectively, these capabilities are intended to make it easier to align people and processes with business outcomes by creating a continuous feedback loop, he said, noting it takes about 90 days for the machine learning algorithm embedded within AIOps from Broadcom to start delivering on that promise.

    Other capabilities included as part of the latest release are additional machine learning topology and correlation capabilities, causal graphs and intelligent situational analysis for automatic root cause analysis, noise mitigation, alarm clustering and anomaly detection. Broadcom has also added a library of machine learning algorithms, natural language processing and an ontological graphical topology to enable situational correlation.

    Finally, Broadcom has added capacity analytics tools that can be applied to specific groups of services that IT teams are collectively managing and maintaining.

    Rajagopal said it’s now only a matter of time before AI is applied more broadly to enable IT teams to become more agile. AIOps won’t replace IT staff as much as it will simply augment the capabilities of the IT staff, he noted.

    AIOps also will prove vital as older members of the IT team retire, Rajagopal added. Newer members of the IT staff will be able to become productive much faster if they can rely on insights and recommendations surfaced by an AIOps platform.

    It’s not clear to what degree AIOps platforms will exist as a separate entity apart from traditional IT services management (ITSM) platforms that are also being infused with machine learning platforms. Broadcom is making a case for a distinct new IT operations platform that relies on data to align people, processes and platforms. The single biggest obstacle in terms of achieving IT agility is not the tools as much as it is people in different roles who can’t empathize with one another because there is no common source of data that every stakeholder can access, said Rajagopal.

    Regardless of the AIOps path chosen, it’s clear that many of the existing tools IT teams rely on to manage IT environments are slowly becoming obsolete. In fact, there may come a time soon when IT teams won’t even consider an IT management platform that doesn’t have some level of AI capability.

  • InfluxDB to Revamp Storage Engine for Time-Series Database

    InfluxDB to Revamp Storage Engine for Time-Series Database

    InfluxData has launched an open source InfluxDB IOx project that it said will become the foundation for the next generation of the company’s time-series databases.

    At the same time, the company has made InfluxDB Open Source 2.0 generally available. This version of the company’s database comes in the form of a single binary that makes it easier to secure.

    InfluxDB Open Source 2.0 also provides access to Data Explorer visualization tools as well as a lighter-weight Flux query language purpose-built for time series data.

    InfluxData CTO Paul Dix said while improvements are still being made to the existing database offering, the time has come to start working toward addressing a number of issues via a new project anchored around a storage engine that will make it possible to query petabytes of data across thousands of servers. The InfluxDB IOx project will address that issue by eliminating current restrictions on cardinality, data size and cluster size that are inherent in the current platform, he said.

    The InfluxDB IOx project is written in the Rust programming language and incorporates Apache Arrow, an open source in-memory analytics engine that incorporates a columnar data store that is being advanced under the auspices of the Apache Software Foundation (ASF). That approach will also make it possible for InfluxDB to natively support SQL along with InfluxQL and Flux query languages.

    The current plan is to make InfluxDB IOx an optional storage backend in a future point release. By the latter half of next year, InfluxData plans that InfluxDB Enterprise customers will have a commercially supported version of InfluxDB IOx and InfluxDB Enterprise. InfluxDB IOx will have its own builds and can be run separately. In effect, Dix noted, InfluxDB IOx will make it easier to separate compute and storage requirements.

    Dix said that while InfluxDB has served as the foundation that enables a wide range of monitoring tools to collect data in real-time, current cardinality limits that impact the number of tables that can be joined requires a new approach to add support for distributed tracing capabilities required to observe microservices-based applications. The IOx version will also employ object storage platforms to reduce the total cost of ownership while making it possible to address the requirements of distributed computing environments that now include a wide range of edge computing platforms, he noted.

    The decision to embrace Apache Arrow also opens the platform to end users who prefer to work with SQL and other scripting languages, he added.

    It may be a while before observability tools that leverage InfluxDB IOx find their way into DevOps tools. However, it’s clear that time-series databases will evolve to meet the requirements of cloud-native applications that will be based on large numbers of microservices spanning distributed computing environments. The only issue is to what degree the current rate of deployment of these applications might outpace the current ability of InfluxDB to keep pace.

  • The Future of Application Development

    The Future of Application Development

    What lies ahead for application development? Can we look forward to a less hierarchical system?

    The current state of application development has an often overlooked problem: It’s inaccessible.

    Top tier companies such as Google, Microsoft, Apple and Facebook are at a major advantage. With their large capital, they can invest in innovative development tools and round up extensive workforces of leading talent to use them—which matters, especially amid an ongoing developer drought. Meanwhile, companies lacking these privileges must make do with a leaner toolset and smaller, less cherry-picked workforce—which is not to say that they don’t make smart, creative use of what’s available to them, but that they deserve to play ball on a more equal playing field.

    Unfortunately, the pandemic compounded this issue, at times bringing even greater advantages to the tech giants. To overcome what some consider a trend of software elitism, efforts have been made to accelerate the pace of application architecture for all. Employing no-code and low-code tools, for example, could help the smaller guys compete. However, as Robson Grieve, CMO of enterprise software company OutSystems sees it, “these latest approaches are mostly point solutions solving a limited set of problems.”

    I recently invited Grieve to share more of his take on this issue and how he envisions a future of application development that works inclusively for a wider set of companies.

    Application Development: The Other 99%

    “It’s frustrating that the best, most advanced development tools have traditionally only been accessible to the largest, richest companies,” said Grieve. “Everyone else suffers with ballooning backlogs and struggles to take advantage of the latest, greatest technologies.”

    While it may sound like a platform out of a Bernie Sanders speech, recognizing the advantages of the top 1% in the application development world brings up a valid line of questioning as to whether maintaining this status quo is really how we want to go forward. After all, wouldn’t it be better if more companies had access to the best technologies? Wouldn’t we all benefit from that?

    Creating a More Equal Playing Field in Application Development

    Luckily, we are already feeling the winds change in this department, as top tier companies are beginning to lose their stronghold on the crème de la crème of development tools. What’s beginning to change, Grieve believes, is “access to the most sophisticated tools and advanced technologies are being democratized … sophisticated, feature-rich, beautiful, and mission-critical applications are no longer the sole domain of the elites.”

    How is this being realized? In many ways, it’s coming from the makers of the development tools themselves, who are bringing careful thought and creativity to tool design. Some of the strongest approaches in Grieve’s eyes include the following:

    • Holistic approaches that streamline the most critical innovations into single, integrated platforms.
    • Low-code approaches that help accelerate development while retaining the power and expressiveness of traditional code.
    • Platform approaches that offer design, development, deployment, monitoring and management capabilities in one place.
    • Multi-functional approaches that serve a diverse team, support the full DevOps life cycle and, in effect, improve productivity and collaboration. For example, platforms crafted to serve the unique needs of UI/UX experts, business analysts, professional developers and architects.
    • Automated approaches that allow applications to be continuously updated and seamlessly modified once they’ve been deployed, using an AI-powered automation layer that underpins the tool, managing application dependencies and providing self-healing capabilities to ensure that regardless of what changes are made, the tool never breaks.

    One thing that stands out here, according to Grieve, is the idea that broad, multi-functional development platforms reign supreme. As Grieve noted, “the most valuable tools are the ones that solve the broadest set of problems.” In general, approaches that keep in mind security, scalability, reliability and availability will be robust solutions for diverse teams, both large and small. A bonus is if the tool is visually appealing and model-driven for the best ease of usability and user experience.

    All for One, and One for All

    The issue of software elitism is one problem that can be seen as a clear opportunity, and that opportunity is not being missed among the smartest makers of development tools. As Grieve said, “Modern application platforms have the potential to help the vast majority of the economy succeed by offering a better approach—one that takes state-of-the-art technology and makes it more approachable, accessible and adaptable.”

    Again, we see a version of AI democracy at play. This is an exciting shift for many companies and teams that exist outside the elite tier. As an example, Grieve said that instead of “procedural code or minimalistic point solutions,” modern applications platforms “harness the power of advanced technologies and AI to create efficiencies for companies outside of the tech bubble.”

    This could be one way to level the playing field and allow companies of all sizes to complete the work needed, shrink backlogs, overcome developer drought with tools that are friendlier to citizen developers, support team collaboration, accelerate efficiency, and—for the underdogs—compete with the big dogs.

    Looking to the Horizon

    So, what does Grieve see for the future of application development? To him, it is bright if we continue on the current trajectory. “More and more,” he said, “successful development projects will require business and IT to work together and collaborate closely.”

    This means investing in sophisticated technologies, with powerful built-in AI, that can help companies ensure that even though applications are built quickly and efficiently, they are built securely and reliably—and in ways that can be easily scaled and evolved.

    If this is all to be realized, Grieve forecasts that we can look forward to “a bright future that is welcoming of a much more diverse and representative community of people who build applications together to solve all the most pressing problems.” It’s an optimistic view, but one that many will welcome as a fresh breath of air in what have been some particularly trying times.

  • Buildkite Tightens CI/CD Integration With GitHub

    Buildkite Tightens CI/CD Integration With GitHub

    Buildkite has tightened the integration between its namesake continuous integration/continuous delivery platform and the GitHub source code repository.

    Fred Showell, a senior product manager for Buildkite, said the integration will accelerate application development efforts using the Buildkite application in GitHub. Any connected GitHub account can be selected as a source when creating Buildkite pipelines, he noted.

    Repository webhooks can also be created automatically, making it faster to kick off builds on a Buildkite pipeline from pull requests and then feed build information back into GitHub using the Status application programming interface (API).

    The bulk of organizations that employ Buildkite are software vendors that are continuously updating software-as-a-service (SaaS) applications. Most of those organizations routinely employ software artifacts that are stored in the GitHub repository.

    While GitHub continues to expand its DevOps capabilities in lockstep with its parent company Microsoft, Showell said there’s plenty of opportunity to extend capabilities of frameworks such as GitHub Actions, a set of APIs that DevOps tools can invoke to automate workflows.

    In general, Showell noted the number of SaaS-based platforms continues to expand as organizations embrace cloud applications to drive digital business transformation initiatives. Some of those efforts are now also expanding to include use cases involving edge computing initiatives, which is expanding the reach and scope of the Buildkite platform, he said.

    Of course, many enterprise IT organizations are now applying many of the same best DevOps practices employed by SaaS platform providers to their own application development efforts. Many of those organizations are realizing they are essentially software companies that also manufacture something or provide a service within the context of a specific vertical industry. As such, the number of organizations embracing best DevOps practices should substantially increase in the months ahead.

    Most of those organizations are already trying to manage a large portfolio of applications. Digital business transformation initiatives may allow them to in time retire some of those applications, but chances are good legacy applications will need to be maintained for some time until the digital process that has been put in place proves effective. It’s likely then that sound software development lifecycle management practices will be more relevant than ever.

    Competition among providers of CI/CD platforms to provide those capabilities is already fierce. Fueled by massive amounts of capital, providers of these platforms are expanding the scope of their offering to address a market that is expected to be valued in the tens of billions. Buildkite itself raised $28 million in additional funding last summer.

    It’s too early to say how the next phase of the DevOps platform wars might play out. However, it’s apparent that when it comes to DevOps platforms the level of investment required just to keep pace is substantial.

  • Gravitational Becomes Teleport to Streamline Access Management

    Gravitational Becomes Teleport to Streamline Access Management

    Gravitational today announced it has changed its name to Teleport and launched the Teleport Unified Access Plane, an open source security gateway that comes in the form of a single binary that acts as a proxy for remotely accessing Linux servers and Kubernetes clusters.

    Teleport Unified Access Plane (UAP) for managing those gateways can be either deployed by an IT team or accessed via a cloud service managed by Teleport.

    Offerings included in Teleport UAP are Teleport Server Access, Teleport Kubernetes Access and Teleport Application Access, which combined enable IT organizations to lock down access to an IT environment.

    Company CEO Ev Kontsevoy said Teleport UAP gives engineers via a single login access to just the IT resources they have permission to access. That approach reduces the myriad credentials that IT organizations currently need to manage across multiple networks.

    Teleport automatically discovers all the resources in an IT environment along with the protocols needed to access them. It provides a unified audit log in a JSON format that enables IT teams to track what resources are being accessed when and by whom.

    Access control is becoming a bigger issue because the overall IT environment is becoming more complex as the infrastructure stack required to run applications grows. Teleport provides a way to streamline access management across servers, clusters and databases.

    That issue is also becoming a bigger challenge with IT personnel now working from home to help combat the spread of the COVID-19 pandemic, added Kontsevoy.

    That fine-grained approach to access will ultimately eliminate the need to rely on virtual private networks (VPNs) at a time when the network perimeter continues to dissolve. In effect, Teleport enables IT teams to move toward a more identity-centric approach to unifying the management of access, he said.

    In general, he noted, Teleport enables organizations to manage security as code rather than having to rely on separate overlays. As such, they can shift responsibility further left toward DevOps teams.

    It may take some time for IT teams to aggregate all the access control protocols they currently employ under a single proxy. However, Kontevoy said, the massive amount of time and effort spent managing access control today will drive organizations that have embraced best DevOps practices to look for ways to automate what today is a very manual process.

    The challenge, of course, is finding the time to achieve that goal. Teleport is betting a single binary in combination with a control plane accessed primarily as a managed service will make it too simple for DevOps teams to ignore. Regardless of approach, however, it’s clear there needs to be a better way to manage access within the sea of servers, databases and applications that now make up the typical IT environment.

  • OpsRamp Makes AIOps Platform Simpler to Configure

    OpsRamp Makes AIOps Platform Simpler to Configure

    OpsRamp today took the wraps off an update to its namesake artificial intelligence platform for IT operations (AIOps) that makes it easier for IT organizations to declaratively configure the software-as-a-service (SaaS) platform.

    Ciaran Byrne, vice president of product management for OpsRamp, said the goal is to enable IT teams to embrace AIOps without having to hire a small army of consultants.

    The OpsRamp Fall 2020 Release adds a revamped user interface that provides access to a set of wizards that discover the assets that make up a specific IT environment and subsequently automate the configuration process.

    OpsRamp has also added curated dashboards that enable IT teams to launch queries using the Prometheus Query Language (PromQL), developed for the open source monitoring tool being advanced under the auspices of the Cloud Native Computing Foundation (CNCF).

    The latest edition of the AIOps platform also adds monitoring capabilities for containerized workloads and Kubernetes clusters. In addition, IT teams can monitor other cloud services including AWS ECS, Azure Functions, Azure Hyperscale (PostgreSQL) and Azure SQL Managed Instance.

    OpsRamp has also added IT process automation capabilities that can be triggered by alerts, updates to resources or on a recurring schedule. An OpsQ bot now also recommends automation workflows. Alternatively, IT teams can require manager approval before a process is launched.

    In general, Byrne said the goal is to improve the mean time to value for an AIOps platform managed by a team of data scientists who have extensive experience applying machine learning algorithms to IT processes.

    Efforts to automate IT have increased sharply now that most IT staff are working from home to help combat the COVID-19 pandemic. However, few organizations are able to set up and configure an AIOps platform in an on-premises IT environment on their own, especially now that outside consultants are no longer being invited to spend weeks working on projects, Byrne said, noting a SaaS-based approach makes it possible for IT organizations to take advantage of an AIOps platform while continuing to work from home.

    In general, the downturn in the economy has increased interest in AIOps because IT staff ranks are either remaining the same or shrinking even as IT environments continue to become more complex. Longer-term, Byrne said it’s not clear that hiring Level 1 and 2 IT administrators will be sustainable as AIOps platforms continue to automate processes.

    DevOps teams, of course, are especially keen to automate as many IT processes as possible, which is why there is a high correlation between organizations that have embraced DevOps and early adopters of AIOps.

    It may still be a while before AIOps is widely employed, but it’s now only a matter of time before machine learning algorithms are pervasive across IT. The challenge now is figuring out where the handoff between man and machine for managing IT ultimately will lie.

  • Puppet Makes IT Automation More Accessible

    Puppet Makes IT Automation More Accessible

    Puppet today launched two efforts to make its IT automation platforms accessible to a broader range of IT teams.

    The Puppet Enterprise Platform integrates the company’s flagship Puppet Enterprise offering with Relay by Puppet, the company’s existing event-driven automation platform.

    At the same time, Puppet announced Puppet Connect, a self-service platform for executing tasks on-premises or in the cloud without requiring any agent software to be installed, managed or updated. Based on the open source Bolt orchestration engine, Puppet Connect is a commercial platform scheduled to be available in early 2021.

    Puppet CTO Deepak Giridharagopal said Puppet Enterprise Platform is unique in that it allows IT teams with varying levels of skill and expertise to employ the same platform to automate IT operations. It is the first to support task-driven orchestration, model-driven infrastructure as code and application programming interface (API) event-driven workflows in a single platform, he said, noting as such, Puppet Enterprise Platform is designed to enable organizations to better approach IT management as a team sport.

    The company is also providing access to thousands of pre-built automation modules and hundreds of API integrations for Puppet Enterprise Platform that can be downloaded from the Puppet Forge repository.

    Puppet Connect, meanwhile, is designed to make it easier for IT teams to build self-service portals more consistently. IT teams regularly build self-service portals to automate tasks to help reduce the number of help desk requests generated by end users. According to the “2020 State of DevOps Report” published by Puppet, 63% of respondents have at least one self-service internal platform. Puppet Connect streamlines the process of creating a self-service framework using the orchestration capabilities of Bolt, said Giridharagopal.

    Interest in automating IT tasks has risen sharply as IT teams cope with more end users working from home to limit the spread of COVID-19. In some cases, the size of the IT staff has been reduced because of the economic downturn brought on by the pandemic. Regardless of motivation, IT staffs that have varying levels of skills all need to find an easier way to automate tasks. The Puppet platforms provide a more accessible layer of abstraction to enable IT teams to automate tasks spanning multiple platforms, noted Giridharagopal.

    It may be a while before IT teams of all sizes automate every IT task; however, the more those tasks become automated the further down the path they move toward embracing best DevOps practices. Those organizations may not deliberately start out to embrace DevOps, but once an IT team automates its first set of tasks it’s not long before other manual tasks become automated.

    Of course, there’s no shortage of IT automation frameworks to standardize on. At this point, the more important thing for many IT teams is to simply get started at a time when no one knows for sure how many years the impact of the COVID-19 pandemic will be felt.

  • DevSecOps Implementation: Static Analysis

    DevSecOps Implementation: Static Analysis

    One of the things I’ve done for Accelerated Strategies Group recently is looking into DevSecOps toolsets. This is a fun area for me, as development and security fit together well in my mind. Having a separate security group is useful, even necessary in some scenarios, but letting developers crank out code and looking for vulnerabilities only after has always seemed terribly inefficient. Whenever I’ve managed developers, security training is just part of the package I bring. It’s far easier to write it well the first time (the equivalent of shift-left) than rewrite it.

    I won’t name names here—this is work I did for ASG, it is not mine to give away—but I will talk about DevSecOps today, and moving into the future. I’m planning three to four installments for this series, discussing the different aspects of DevSecOps tools.

    For clarity, we’re defining DevSecOps tools as “developer-enabled security tools.” This means that in some way, they help developers improve overall security posture while leaving them to be developers. We all know security has to shift left in every way if it’s to be useful in a DevOps world, but most organizations haven’t seriously started the move to shift security. Hopefully, this series helps.

    Depending upon which financial analyst firm you prefer, projections are that the DevSecOps market will grow 28% to 33% a year for the next few years. While this is good news for DevSecOps vendors, it implies very few organizations have systemically adopted DevSecOps, or there would not be room for that level of growth.

    Whether a traditional team or a DevOps team, the static analysis portion of DevSecOps is (generally) easy to set up and configure and most products offer feedback at both coding time and build time. IDE plugins have come a long way, some even offering advice as-you-code and others allowing developers to run on-demand static scans of their work. The industry as a whole has worked hard to lower the false-positive rates that made most organizations reluctant to deploy them broadly, and prioritization of issues is possible in the best products, making identification and remediation pretty easy. Try a pilot out: Pick a vendor and project, grab the IDE plug-in and check out what they can do. Most do not yet directly support code smells, but I suspect that is coming rapidly, as the systems are already designed to look at source and determine issues. Adding to the collection of possible issues will be a differentiator that quickly turns into a market requirement.

    Things to check for:

    • Language support: Do the languages you use have a scanner? While most of the tools on the market offer support for a breadth of languages, there are differences, so check the corner cases. This truly is a make or break decision—the best product on the market is useless to your org if it doesn’t support the languages you use.
    • IDE integration: This is pretty straightforward, are the IDEs your developers use day in and day out supported? Most of these tools offer command-line variants also, so it is possible that even without support directly in the IDE, the security functionality of a given product dictates that command-line is better. In that scenario, a process to ensure developers are even running scans is a good idea.
    • Vulnerabilities detected: At a minimum, support for whatever is currently in the National Vulnerability Database (NVD) should be required, along with OWASP compliance. Most vendors offer support beyond the NVD, so a bit of research to see what types of vulnerabilities are detected will be in order.
    • Click-through support: With IDE plug-ins, one of the great strengths of scanning is the ability of the developer to simply click on the vulnerability in the results list and go to the offending code. Make certain this works—and works as advertised.
    • False-positive rate: This has historically been a problem, but the industry has brought it down overall. Ask your prospective vendors to share their numbers. Our experience shows low single-digits for a percent, which is far better than the past, but still leaves room for improvement.

    The other part of static analysis is plugging static analysis into the build process. Most of these products have integrations into the most popular build management tools. Scanning the entirety of the application allows the static analysis to see a broader vision of the app and note issues with interactions between portions written by different developers. Reporting for build-time scans is generally at the application/team level, and the best tools will show results of development-time scans side by side with build-time scans if you want them to.

    Things to check for:

    • Language support: Again with the languages. It’s pretty simple here: The core of the application may be built using languages a given vendor supports but critical parts might not. Integration is where that becomes a real issue. Make sure what needs scanning can get scanned.
    • Integrations: Most of these tools offer automated scans through integration with build tools or manual scans that the build manager can kick off. Automation is preferred by a long shot, so check if your tool of choice is integrated into your organization’s build tooling.
    • Depth: We’re not going to get into source composition analysis (SCA) here—that’s for another blog—but we are going to observe that vulnerabilities can come from your source’s sources. How far does the scanning tool trace in a static analysis situation? Is it deep enough to cover a given application’s integrations and inter-relationships? While SCA will detect a lot that static scanning won’t, things such as “unsafe values sent to [insert external API]” should be available in static code analysis.
    • Reporting: This is where you’ll find the most variety in popular tools. Look at their reporting and click-through support to find what best suits your needs.
    • False-positive rate: Same as IDE integrations above—make certain it is low enough for your organization’s comfort.
    • Scan run location: This is becoming a differentiator in the space—do you want your scans run in the cloud? In the data center? Some mix of both? Check your vendors; there are significant differences in the level of support for these options.

    This is just the tip of the iceberg, of course. It is an entire market segment, and static analysis tools have been around a long time. Many organizations have not taken dev-time security seriously yet, even though we all know we need to; I’m just offering a place to get started.

    DevSecOps just makes sense. Static scanning is mature and generally well-integrated. This is a great place to start shifting security left. We get it: Developers don’t want to be security analysts or they’d have filled one of those many security openings. But IDE integration makes these scanning tools more interactive and less intrusive, showing developers how to do it securely as they code. This is great to me because good coding practices will become second nature with real-time, on-the-spot reminders and pointers. It also means that if the developer acts upon the tool’s recommendations, the code is developed more securely the first time and there will be no need to revisit it for security proofing.

    You’re all keeping the lights on in a terribly tough time. DevSecOps attempts to make it more likely that your astounding work isn’t interrupted. It just makes sense to get these tools to protect your organization’s portfolio and your hard work.

  • LogDNA Extends Log Management Reach

    LogDNA Extends Log Management Reach

    LogDNA has extended the reach of its log management platform to add support for a Node.js platform from Vercel and a feature flagging tool from LaunchDarkly.

    In addition, LogDNA has expanded an existing alliance to IBM to add support for instance of Red Hat OpenShift running on VMware platforms hosted on the IBM Cloud, which enables IT teams to centralize log management using LogDNA and IBM Log Analysis.

    Chris Nguyen, chief strategy officer for LogDNA, said the goal is to continue to establish partnerships that will enable DevOps teams to manage larger swaths of the IT environment via a single pane of glass.

    Vercel has been gaining  traction as a provider of an open source reactive programming framework that is being employed to build more interactive front ends for web and mobile applications.

    LaunchDarkly is a feature management platform that serves more than 6 trillion feature flags a day spanning 14 million mobile devices and 3 million servers. The integration between LogDNA and LaunchDarkly enables developers to rapidly iterate on feature flags using logging data surfaced by LogDNA.

    Nguyen said that as IT environments continue to become more complex, more organizations are relying on log analytics to reduce the noise level being created by all the tools and platforms that make up a modern IT environment. The challenge IT organizations face is most of them can’t find or retain a site reliability engineer (SRE), so they need tools that enable developers to make better-informed decisions.

    LogDNA is making the case for a log management platform deployable in the cloud or on-premises, designed for DevOps teams as an alternative to existing log management platforms that were designed for IT administrators who rely on graphical tools. LogDNA instead provides developers access to log analytics that are accessed via command-line interface (CLI) or a user interface.

    It’s not clear whether or how much IT organizations may be willing to replace legacy log management platforms. However, as more organizations embrace best DevOps practices that include observability as a core tenet, LogDNA is betting those organizations will prefer an approach to log management aimed at developers rather than IT operations staff.

    As DevOps teams assume more responsibility for the management of IT on an end-to-end basis, a seismic shift in IT tooling is underway. Providers of legacy IT tools are quickly shifting toward making APIs and CLIs available in response to a range of startups that are making available tools designed from the ground up to appeal more to DevOps teams. It’s too early to say how this battle for control over observability will play out, but the number of providers of tools for DevOps teams has never been greater.

    In the meantime, many DevOps teams are still sorting out when best to employ log analytics, metrics and distributed tracing to optimize IT environments. The challenge these days isn’t so much being able to discover what’s occurring in an IT environment as much as it is making sense of it all.