staging-devopsy.kinsta.cloud

Category: DevOps Toolbox

  • Lightrun Tool Now Debugs On-Premises Apps

    Lightrun Tool Now Debugs On-Premises Apps

    Lightrun today announced it has made its namesake debugging tools available for applications deployed in on-premises IT environments that, for one reason or another, can’t be deployed on a cloud platform.

    lan Peleg, CEO of Lightrun, said Lightrun is unique in that software teams can capture live production data from software installed on their customers’ sites via a set of command line interfaces (CLIs). That approach eliminates the need for the end customer to set up a separate environment to troubleshot an application, said Peleg.

    Instead, software teams can troubleshoot their software without requiring on-site visits by support engineers or even a phone call, Peleg said.

    Lightrun makes it possible to add logs, virtual “breakpoints” and metrics while an application is running. Those capabilities are critical because many applications are not fully instrumented. Lightrun makes it simpler to triage issues by adding a missing log line, instrumenting a new metric or taking a snapshot of an application as it runs in a production environment. Those capabilities make it possible to find bottlenecks and errors without requiring the application to stop running.

    The performance impacts of Lightrun range from tens to hundreds of microseconds per invocation to minimize overhead, Peleg said.

    In general, Peleg said, debugging applications is becoming more challenging. Monolithic applications continue to grow in size while emerging microservices-based applications have dependencies that make them challenging to debug. Many organizations are employing application performance management (APM) platforms and other observability tools to try and address these issues. However, those platforms require IT teams to continuously collect data, and even then there is no guarantee the insights needed to troubleshoot an application will be surfaced. Lightrun, conversely, makes it possible for developers to troubleshoot an application and capture the metrics needed in real-time. That data can then be shared with an APM platform to make it easier to address similar issues if required, Peleg said.

    As developers become more accountable for application performance, many of them are not going to want to employ the same tools and processes that IT operations teams have historically relied on. Lightrun provides a lighter-weight approach to troubleshooting applications, within the context of a DevOps workflow, that doesn’t require as much overhead, said Peleg.

    Although observability is a core DevOps principle, many organizations have been unable to achieve it in a meaningful way. There is no shortage of tools for monitoring IT environments, but correlating all the data those tools generate is problematic. It’s not uncommon for organizations to spend weeks trying to identify the source of an issue that only takes a few minutes to fix. Now that developers are assuming more responsibility for managing the entire lifecycle of an application it is likely different approaches to achieving observability will be considered.

    In the meantime, as the number of applications deployed in production environments continues to increase, different approaches to debugging software will be required. Otherwise, at the current rate of deployment, it won’t be long before developers find themselves spending most of their time debugging existing applications, versus actually writing new code.

  • Datadog to Meld Observability and DevSecOps

    Datadog to Meld Observability and DevSecOps

    Datadog is planning to expand the scope of the security and observability services it provides following the acquisitions of Sqreen, an application security platform provider, and Timber Technologies, a provider of a tool, called Vector, for collecting and normalizing log data.

    Ilan Rabinovitch, vice president of product and community for Datadog, said Vector makes it possible to collect, enrich and transform logs and other observability data from both on-premises and in-cloud environments in a way that makes it easier to route that data to wherever it is needed via a data pipeline.

    Sqreen is a provider of a software-as-a-service (SaaS) platform that detects, blocks and responds to application-level attacks using runtime application self-protection (RASP) capabilities enabled by a web application firewall (WAF) embedded within an application.

    Rabinovitch said as IT continues to evolve, it’s clear organizations are looking to embrace DevSecOps best practices in a way that will require security tools that can be embedded within an application, coupled with observability tools that provide more context about the overall IT environment. That context is especially critical, given the amount of data that needs to be analyzed to determine the potential scope of a security threat to an IT environment, Rabinovitch said.

    The Datadog platform is already widely employed to monitor cloud computing environments. There is an industry wide effort to take monitoring to the next level by enabling observability, which applies analytics to provide more context. Surfacing that context, however, requires the ability to normalize data across a pipeline using Vector, Rabinovitch said.

    Of course, the Vector tools are equally applicable to DevOps workflows that have yet to incorporate security technologies. The goal, now, is to make it easier for IT teams, made up of developers, IT operations teams and security specialists, to collaborate by breaking down the various data silos created by IT and security management tools.

    It’s too early to tell what impact the rise of DevSecOps will have on roles within IT organizations. Most organizations will continue to employ a range of specialists that will be able to work more closely with IT generalists, Rabinovitch said. Managing IT is becoming the equivalent of a barn raising, in that it requires members of a community with different skills to come together for a certain amount of time to accomplish a specific task, Rabinovitch added.

    It’s also not clear whether IT organizations view observability as a distinct capability separate from monitoring, or whether they will simply view observability as the next logical extension of existing tools. At the core of that issue is debate over whether observability platforms need to be acquired separately, as part of an effort to replace existing monitoring tools, or whether those tools will evolve over time to provide observability capabilities.

    Regardless of the path forward, it’s apparent the increased complexity of modern IT environments – made up of microservices, containers and serverless computing frameworks – is about to force that issue.

  • SolarWinds Unveils Revamped APM Platform

    SolarWinds Unveils Revamped APM Platform

    SolarWinds is unifying its various application performance management (APM) tools within a single suite accessed via a software-as-a-service (SaaS) platform.

    Denny LeCompte, senior vice president for products at SolarWinds, said the APM Integrated Experience offering combines the company’s AppOptics, Loggly and Pingdom tools to collect metrics, traces and logs along with real user monitoring capabilities and tools to simulate application traffic.

    APM Integrated Experience also provides access to a landing page that identifies what has changed in an IT environment over the previous week and highlights which of those changes might become an issue. As most IT professionals well understand, most IT issues can be traced directly back to a recent change made to an IT environment that was previously running as intended.

    Other capabilities include advanced search tools and the ability to inspect lists and maps for hosts, containers and plugins for web servers and databases. In addition, APM Integrated Experience is integrated with both Amazon Web Services (AWS) and the Microsoft Azure cloud.

    LeCompte said the goal is to make it less expensive for the average IT organization to broadly instrument applications using an agent developed by SolarWinds that can automatically be installed.

    Ultimately, the goal is to enable organizations to better manage their software supply chains, LeCompte added. Of course, SolarWinds is at the heart of the software supply chain breach that involved its Orion network management software. That software runs on-premises. In comparison, APM Integrated Experience is a SaaS platform that is more secure. In fact, LeCompte said, IT professionals that manage software supply chains are generally sympathetic to the challenge the company currently faces. Most of those organizations have dealt with similar software supply chain issues, albeit on a smaller scale, LeCompte added.

    In general, LeCompte said it’s apparent most organizations need access to an APM platform that costs substantially less than rival platforms and still delivers 90% of the capabilities they require. In many cases, IT organizations are paying for extra capabilities that only very high-end enterprises require, LeCompte noted. In fact, one of the primary reasons APM platforms are not as widely employed is most organizations have found rival APM platforms simply too expensive to deploy, said LeCompte.

    LeCompte declined to speculate whether there is a meaningful difference between what has been historically known as monitoring and so-called observability platforms that have emerged in recent months. Regardless of approach, however, it’s clear as distributed IT environments become more diverse and complex, the number of organizations that need more visibility into their application environment is steadily increasing, said LeCompte.

    Of course, with the rise of microservices-based applications, that level of IT complexity is only going to increase. As applications grow and expand, the number of dependencies between microservices starts to increase. Understanding how an issue with one microservice is impacting a distributed computing environment will undoubtedly require more instrumentation. The challenge, going forward, will be determining how best to achieve that goal at a reasonable cost.

  • Applause Automates App Testing via SaaS Platform

    Applause Automates App Testing via SaaS Platform

    Applause today unfurled a software-as-a-service (SaaS) platform that makes it easier to automatically create test automation scripts without having to write code.

    At the same time, the company is adding support for an application programming interface (API) through which DevOps teams can send new features to be tested by the Applause platform.

    Both offerings are now being included with a rebranded Product Excellence Platform (PEP) offering, a suite of tools and platform through which Applause is making its portfolio of offerings more accessible.

    Rob Mason, Applause CTO, said the Applause Codeless Automation (ACA) platform makes it possible for testers to record a series of test case scenarios using a SaaS platform that translates those sessions into an automation script without anyone having to write any code. The current offering supports Android and Apple iOS applications with support for web applications planned, Mason said.

    An Applause In-Sprint Testing capability, meanwhile, makes the testing process more iterative using a bi-directional API that can be plugged into any software development life cycle (SDLC) process, Mason said. That bi-directional capability makes it possible to embed the feedback from the testing process into any bug tracking system implemented by an application development team, said Mason.

    Mason said that capability is critical, because it enables DevOps teams to identify issues earlier, thereby avoiding late-stage bug fixes that are more costly to fix. It also provides the added benefit of making it easier for developers to address issues in the moment applications are being written, versus days or weeks later when the context around that issue has long since been lost, Mason added. The SaaS platform provide developers with a just-in-time testing capability, Mason said.

    Ultimately, Mason said the goal is to make it simpler for a wider range of individuals to participate in application testing by making it easier to reuse tests. In addition, he said, it will allow subject matter experts to record tests that can be turned into test scripts on their behalf. That approach will enable organizations to push testing further left, beyond the developers that originally created an application, to include the owners of that application, Mason added.

    The rate at which applications are being built to drive digital business transformation initiatives is increasing, and makes testing more problematic. Automation is the only way to close the gap between the number of application testers an organization can dedicate and all the applications being developed. The automated approach not only reduces the time and effort required to create a test, it also makes it easier to reuse tests across different application use cases.

    Ultimately, it’s only a matter of time before machine learning (ML) algorithms and other AI capabilities play a larger role in application testing. The challenge will be finding exactly where the interface between humans and machines should lie in the context of application testing.

    In the meantime, the more automated testing becomes, the more widely it will be conducted. The results of those tests may not always be welcome news, but the issues they raise are always better addressed sooner than later.

  • Mabl Adds Beta of Desktop Continuous Testing Tool

    Mabl Adds Beta of Desktop Continuous Testing Tool

    Mabl today announced availability of a beta version of a desktop tool for creating tests that can be applied to browsers and application programming interfaces (APIs) using a low-code tool.

    Dan Belcher, co-founder of Mabl, said the goal is to make it simpler to streamline testing processes as responsibility for application quality assurance (QA) continues to shift left within the context of a DevOps workflow.

    Mabl provides a low-code tool for testing desktop and mobile browser applications, alongside the APIs they invoke, using an Electron tool that generates tests in a way that eliminates the need to write scripts. Previously available only via a software-as-a-service (SaaS) platform, Belcher said the desktop platform will make it simpler for developers to test code as they develop it on a local machine.

    Developers using the desktop tool can now also manage and run tests instantly via command line interface, versus having to completely exit from a project to invoke a separate testing tool, Belcher said.

    The Mabl testing tools are already integrated with multiple continuous integration/continuous delivery (CI/CD) platforms, such as Bitbucket and Bamboo, as well as the GitHub repository.

    In the longer term, Belcher said it is clear organizations will need to involve more developers in QA testing, as the rate at which applications are being built and deployed continues to increase. Testing teams, on their own, can no longer keep pace with the rate of change, Belcher noted.

    The challenge is achieving that goal in a way that creates the least amount of friction possible, Belcher added. The Mabl platform and desktop application make it feasible for developers to test both web and mobile applications without taking too much time away from developing application code, said Belcher.

    Testing has become a major DevOps issue because, as developers assume more responsibility for applications after they have been deployed in a production environment, they naturally want to vet applications more thoroughly. Issues that arise after an application is deployed simply result in less time available to write new code. There will always be issues that need to be addressed, but the more of them that are resolved as part of a QA process integrated within a DevOps workflow, the more productive an application development team will be.

    Unfortunately, testing is often given short shrift; it primarily occurs at the back end of the application development life cycle. Developers are typically behind schedule, so there is a natural tendency to reduce the amount of time allocated to testing in the hope of making an application delivery deadline. The vicious cycle that ensues is that, inevitably, there are more bugs to fix after an application has been deployed, simply because not enough testing was done during the original development cycle.

    Of course, some development teams have become conditioned to fixing bugs as part of an agile development process that, too often, encourages them to push known issues off to the next sprint. The trouble is, in the absence of continuous testing, it’s not long before all those known bugs start to pile up. It’s not uncommon for applications to be riddled with bugs that development teams – with mixed success – attempt to address all at once. The earlier those issues are addressed, the less stress there will be later for all concerned.

  • AppDynamics Survey Reveals High IT Stress

    AppDynamics Survey Reveals High IT Stress

    A global survey published today by AppDynamics, a unit of Cisco, suggests the last year took a substantial toll on IT teams at a time when many organizations either launched or accelerated digital business transformation initiatives.

    The survey, of 1,050 IT professionals in organizations with revenue of more than $500 million, finds organizations, on average, implemented digital business transformation initiatives three times faster in 2020 than they did in 2019 in response to the economic downturn brought on by the COVID-19 pandemic. That increased level of activity, however, appears to have come at a significant cost. The survey found 89% of respondents felt they are under immense pressure at work, with 84% admitting to having difficulty switching off after work.

    The survey also found stress levels are, not surprisingly, rising as well, with 81% of respondents experiencing increased frustration and 63% conflicting more with colleagues.

    The root cause of these issues cited most often include new sets of priorities and challenges (80%), technology sprawl and a patchwork of legacy and cloud technologies (78%), acceleration to cloud computing (77%) and multiple, disconnected monitoring solutions (74%).

    A full 85% also identified cutting through noise generated by increasing volumes of data to identify root causes of performance issues will represent a significant challenge in the year ahead. A total of 92% also noted the ability to link technology performance to business outcomes will be what’s really important to delivering on innovation goals over the next year.

    Three-quarters (75%) of respondents said they are already struggling to manage overwhelming “data noise.” Nearly three-quarters (73%) also noted they fear the inability to link IT performance with business performance will be detrimental to their business in 2021. Three-quarters (75%) of respondents also noted their organization needs to connect full-stack observability to business outcomes within 12 months in order to remain competitive. Two-thirds of respondents (66%), however, said right now they do not have the resources and support they need.

    AppDynamics regional CTO Gregg Ostrowski said the survey makes it clear that existing approaches to merely monitoring IT platforms are not going to be sufficient, at a time when observability across the full stack of IT platforms being employed is now absolutely essential. Rather than convening war rooms where IT teams responsible for different platforms compare reports in the hopes of identifying the root cause of an issue, Ostrowski said modern observability platforms provide a holistic view of IT environments that IT teams need to proactively resolve issues before they impact the business.

    That level of visibility will also play a crucial role in reducing the stress levels of IT teams that, in many cases, are now in danger of suffering from burnout, Ostrowski added. Of course, the more fatigued an IT team is, the more likely it is mistakes will be made.

    It’s not clear to what degree IT teams are embracing observability as a core IT management concept. The AppDynamics survey noted that 71% of respondents are wary of misleading claims from vendors who are re-labelling monitoring tools as observability platforms. Regardless of how observability is achieved, the general consensus remains: whatever observability means within the context of IT, most organizations lack it.

  • CircleCI Adds Private Orbs to DevOps Toolchain

    CircleCI Adds Private Orbs to DevOps Toolchain

    CircleCI announced today it is enabling IT reams to configure their own private integrations around its namesake continuous integration/continuous delivery (CI/CD) platform.

    The company enables integrations with third-party services via reusable config packages, based on YAML files, known as orbs. Now, internal IT teams can create private orbs that they can securely reuse to invoke, for example, a frequently accessed microservice via the command line interface (CLI) and invoked via the CircleCI platform.

    CircleCI CEO Jim Rose said the goal is to make DevOps teams more efficient at a time when the number of microservices organizations are reusing is increasing steadily. Rather than having to manually integrate each of those services every time they are needed, a private orb will consistently automate the process using the same template, said Rose. That approach, over time, will enable DevOps teams to more easily build and maintain an ecosystem of microservices, Rose added.

    In general, Rose said, with more DevOps teams now working from home to help limit the spread of the COVID-19 pandemic, there is a greater need to automate as many tasks as possible to enhance overall DevOps productivity. Constructing an orb gives DevOps teams a relatively simple way of achieving that goal, Rose said. Today, there is often a small, elite group within DevOps teams that builds tools to automate processes in a way that the rest of the DevOps team can more easily consume. Private orbs make it easier for anyone on the DevOps team to create a reusable process, said Rose.

    Many of those DevOps teams are, of course, attempting to accelerate digital business transformation initiatives launched in response to the pandemic. Those initiatives are generally based on applications being constructed using microservices that, ideally, developers will reuse across multiple projects. The CircleCI private orbs can be remotely invoked from anywhere via the CI/CD platform the company manages on behalf of DevOps teams that employ it.

    It’s not clear to what degree IT organizations are shifting CI/CD platforms to the cloud in the wake of the pandemic. In theory, a cloud-based platform is more accessible to DevOps teams that, like most employees, are working from home. Even after the pandemic eventually subsides, many members of DevOps teams will undoubtedly be working from home more regularly.

    In the meantime, a recent analysis of data from 11 million workflows on the CircleCI platform suggests that, after an initial spike of activity in the early days of the pandemic, DevOps teams settled into a more deliberate pace of application development after quickly adjusting to what has become the new DevOps normal. That new normal is likely to change again once COVID-19 vaccines become more widely distributed, but there will certainly be no returning to the way applications were built and deployed prior to the pandemic.

  • LinearB Aligns with Clubhouse to Advance Project Management

    LinearB Aligns with Clubhouse to Advance Project Management

    LinearB, a provider of an analytics application for software delivery projects, announced today it has allied with Clubhouse, a provider of a project management application that includes built in collaboration tools.

    LinearB’s analytics application uses code, Git logs and releases to automatically construct and visualize detailed timelines without requiring any manual input. That data can now also be automatically fed into the Clubhouse project management application.

    Dan Lines, COO, LinearB, said the goal is to make it possible for any project participant to keep track of updates without spending hours manually compiling and entering data into a project management application.

    In addition, DevOps teams can use both applications to gain visibility into every feature, bug and chore via a live feed of activity generated by branches, pull requests and releases for Clubhouse stories. Along with tools for tracking individual activity and collecting metrics, blockers, delays, high-risk code and branches merged without review for Clubhouse stories are also highlighted. There is also a shadow work detector that detects activity in a Git repository that is not attached to a Clubhouse story.

    Lines said one of the project management challenges organizations face today is the tools they employ only allow them to see segments of a DevOps pipeline. The LinearB analytics application unifies the data and metrics required to make an informed decision, without having to first collect and then normalize data residing in different tools. Instead, all the metrics required to make an informed decision about how best to allocate limited software development resources is automatically collected from the tools the DevOps team employs, Lines said.

    There is no need, then, for project managers to pester members of a DevOps team to enter data in a separate analytics or project management application, Lines added. Given the rate at which application development projects are now updated, much of the data manually entered into a project management application is usually outdated, anyway.

    LinearB employs the application programming interfaces (APIs) exposed by DevOps tools to gather the existing metadata about the status of any given project. LinearB then employs machine learning algorithms to construct and correlate that data to give DevOps leaders more accurate insights into the status of a project, instead of requiring that data to be entered manually.

    In fact, Lines said meetings should be shorter, too, because rather than the need to provide a status update, it will become apparent to everyone what the real status of a project is before a meeting even starts. That means more time can be allocated to determining how best to optimize limited application development resources, Lines said.

    In the wake of the COVID-19 pandemic, organizations have more application development initiatives underway than ever. Many of these projects are core to digital business transformation initiatives which a core group of developers are working on simultaneously. Being able to dynamically reallocate resources to a faster-moving project from one that is stalled is increasingly critical. The challenge has been that the more projects there are, the harder it becomes to collect data that accurately reflects their true status at any given time. The goal should be to eliminate the need to reenter data that, in one form or another, already exists in another tool.

  • Survey: DevOps Growing Pains in Salesforce Environments

    Survey: DevOps Growing Pains in Salesforce Environments

    IT organizations that have adopted software-as-a-service (SaaS) application platforms from Salesforce in part to jumpstart their digital business transformation initiatives are experiencing some DevOps growing pains.

    A survey published this week of 230 executives, managers, and members of Salesforce delivery teams conducted by Beagle Research on behalf of Copado, a DevOps platform for IT teams building custom applications, shows that while 82% have increased or maintained the size of their Salesforce development teams, the change fail rate jumped from 23% last year to 33% in 2020 when compared to a similar survey conducted a year ago. The survey also notes time to recover increased from 110 hours in 2019 to nearly 196 hours in 2020.

    Andrew Leigh, chief marketing officer for Copado, said those results generally reflect the nature of the more complex application initiatives being launched on top of Salesforce applications. The survey also noted 55% of respondents don’t consistently automate their Salesforce deployments, with less than a quarter of respondents (23%) having a way of automatically deploy a configuration from development through testing to production.

    In fact, in teams with fewer than 10 members, only 28% of survey respondents automated Salesforce deployments compared to 56% of teams with more than 100 developers that have automated their deployments, the survey found.

    The survey also noted that more than a third (34%) of respondents said features that development teams create may not be adopted. That issue suggests there is more work to done in terms of better aligning development teams with the goals of the business, Leigh said.

    Organizations building applications on top of Salesforce don’t typically embrace DevOps best practices until development teams have 25 or more members, Leigh noted. Salesforce environments generally require their own DevOps platform, because applications are constructed using a set of objects and tables unique to the platform. Those objects and tables are most often invoked using a set of low-code application development tools. Nearly three-quarters of respondents (73%) employed at least one commercial Salesforce development tool.

    Overall, the survey finds 82% of organizations have increased their budgets for or effort toward digital transformation, with 53% reporting they have exceeded their performance goals for the year. The survey also identifies a set of elite DevOps performers that are reporting 30x shorter lead times; 46x more frequent deployments; 5x lower change fail rate; and 37x shorter time to restore. Elite performing teams are 65% more likely to have adopted continuous integration than low performing teams, the survey notes.

    More than half of respondents (53%) said that they found it easy to find people who have DevOps experience in a Salesforce environment, with salaries typically in the $90,000-$140,000 range.

    The survey also finds the majority of organizations (82%) expect the systems integrators they contract to build applications to help define their DevOps processes.

    As organizations continue to adjust to new business realities in the age of COVID-19, there’s no doubt more custom applications will be built and deployed on top of SaaS application platforms such as Salesforce. The challenge is finding the best way to accelerate the development of those applications in a way that consistently produces the desired result.

  • EraDB Launches Log Management Alternative

    EraDB Launches Log Management Alternative

    EraDB today launched EraSearch to provide IT teams with an alternative to open source ElasticSearch software for managing logs at scale.

    Todd Persen, EraDB’s CEO, said EraSearch is based on a modern database that decouples storage and compute. In this way, EraSearch can scale more efficiently than ElasticSearch, while providing IT teams with the same familiar construct for managing logs.

    Persen said EraDB also uses true zero-schema data representation, and an indexing engine based on machine learning algorithms, to enable EraSearch to scale better. This approach also cuts cloud and operational costs by as much as 30% and 70%, respectively, by relying, in part, on object-based storage systems to store log data.

    As IT environments become more complex and continue to scale, organizations that standardized on ElasticSearch are discovering that a platform based on decades-old search engine technology is not keeping pace with their needs.

    EraSearch provides an alternative, based on a commercial time-series database platform that EraDB created, to address such use cases as an alternative to database platforms such as InfluxDB.

    The question is whether IT organizations, that have relied on ElasticSearch for years, are ready to abandon an open source platform in favor of a proprietary platform, even if it provides a similar user experience. EraDB, however, is betting there are enough IT organizations willing to make that shift to justify its investment in EraSearch.

    Persen said one issue with ElasticSearch is that, as log volumes increase beyond a few terabytes, the platform becomes inefficient and difficult to efficiently scale. That leads to higher costs in terms of both raw storage, as well as all the manual effort on the part of software engineers, required to maintain the platform.

    EraDB, of course, is making a classic case for a proprietary platform over an open source platform: that open source, while free, often proves more expensive to support. Persen said EraDB decided to eschew the open source model altogether, because it is too difficult to prevent nullification of the benefits open source could bring to a small company by open source project forks.

    The ElasticSearch community, meanwhile, is roiled by Elastic’s decision to change the licensing terms under which it makes ElasticSearch available. Elastic is shifting away from an Apache license to a dual-license strategy that includes a Server-Side Public License, or SSPL, along with an Elastic License. That shift was made to help ensure more of the revenue generated by Elastic search winds up in the hands of Elastic, rather than cloud service providers, such as Amazon Web Services (AWS), that routinely launch paid services based on open source code. AWS responded to the Elastic move by announcing plans to fork ElasticSearch code that it will now curate.

    Regardless of how that battle ends, however, EraDB is making it clear it views its platforms as too critical to its customers’ IT operations to get caught up in a similar maelstrom over business models and associated licensing terms.