staging-devopsy.kinsta.cloud

Category: DevOps Toolbox

  • Automated Certificate Management No Longer a Luxury

    Automated Certificate Management No Longer a Luxury

    On December 6, 2018, approximately 32 million customers of the UK telecom provider O2 woke up to a frustrating reality: The most widely used mobile network in the country was experiencing a day-long outage of its 4G mobile network.

    For some, not being able to check the weather forecast, traffic report or football scores was a minor inconvenience. For others, the impact was much more severe, as people throughout the country realized they couldn’t collect email, use their connected apps or perform any other task requiring data usage—no matter how critical to their business or personal lives. The world quickly came to realize that O2 subscribers in the UK were not alone: Other mobile providers around the globe, including Softbank in Japan, suffered similar outages in the same time frame.

    What caused this global outage? Was it a software bug? A poorly tested update, shared too soon? Maybe even a cyberattack targeting mobile network infrastructure? In the end, the explanation was much simpler: A digital certificate expiration in a back-end mobile data service from Ericsson caused a cascading systems failure, ultimately resulting in data outage for these mobile services. This issue highlighted the potential impact of certificate errors in our modern digital enterprise and the value of automation in managing such a critical process.

    Human Errors, Absences Lead to Outages

    While the O2 and Softbank service outages had serious consequences in their home countries—the U.S. equivalent would be a provider like Verizon losing data service for an entire day—they are hardly the only ones to experience this issue. Earlier this year, during the shutdown of the U.S. Federal Government, a number of important sites went down or lost functionality because no one was there to manually renew expiring web certificates. Even popular agencies such as NASA and critical services such as those provided by the Department of Justice were not spared, and both agencies and the citizens they serve were forced to watch helplessly as their certificates expired and services went down.

    Although it is of little comfort to the customers who woke up without mobile data or citizens who were unable to use government services, this scenario could easily have been prevented had automation been utilized to manage the certificates in question. When certificates are managed manually, it isn’t always easy to determine the cause of an outage like the one experienced by O2 and Softbank. An expiring certificate may not be the first answer that springs to mind, and may only be noticed after poring over code, looking at network monitoring and checking other potential culprits. Certificates are not always front of mind for IT teams, which underlines the value in simplifying their management.

    The technology sector has made significant strides in this regard, and organizations have tools available to them that can help streamline certificate issuance, management, renewal and revocation. Gone are the days when certificates had to be manually issued one at a time, and organizations kept spreadsheets of certificate expiration dates hoping they remembered to renew them. While we’re not able to say specifically why or how the root-cause Ericsson certificate was allowed to lapse, unexpected certificate expirations can be eliminated through proper use of certificate automation—making such a disaster unlikely to repeat itself if the proper steps are taken.

    Peace of Mind at Scale

    Getting started is easier than you might expect. The rise of automation in managing digital certificates presents organizations with a simple and exciting solution that extends well beyond just outage protection. Perhaps best of all, many organizations operating in a Windows environment may already have access to auto-enrollment and auto-renewal tools using the Microsoft CA, which has been a pioneer in the field of automated certificate services. Chances are, you may already be familiar with using Microsoft CA as an easy way to take care of your internal (private) certificates.

    But what about public certificates? True, Microsoft CA doesn’t handle those, so most organizations will need to find a third-party vendor to manage the SSL certificates for their web servers, load balancers, VPN devices and other networking gear. There are vendors capable of managing all certificate requirements under one roof, whether you operate in Windows, MacOS, Linux or other environments. The rise of standards like the Automation Certificate Management Environment (ACME) have made integrating different environments increasingly simple.

    Designed by the Internet Security Research Group, ACME has become a popular certificate management tool. More than 150 million websites and 130 open-source client tools use ACME. In fact, ACME is supported by a number of tools familiar to those who use DevOps to manage deployments, including Kubernetes, Chef, Ansible, Salt Stack, Terraform, Istio, Docker and others.

    As enterprises look to leverage DevOps with increasing regularity, effective certificate management can help facilitate container, micro-service and application security. Furthermore, orchestration tools such as Kubernetes support certificate management natively, leveraging ACME as the underlying protocol. The private keys may be stored in Kubernetes Secret or Hashicorp Vault so the certificate management system integrates seamlessly.

    Digitally signing your DevOps containers with a private CA helps verify the identity of a given container as it communicates over TLS while also preventing unauthorized applications—and the tools available to ACME users add significant value. Of course, this is just the tip of the iceberg—using automation to manage these certificates means plugging into a vast ecosystem of resources, reducing deployment time and eliminating the possibility of human error. It really is that simple.

    A New Automated Future

    Certificate management is hardly the only area where human error can have an outsized impact, but the O2/Softbank outage clearly illustrates severe effects such errors can have. Service outages, damaged reputations and frustrated consumers lie in the wake of poorly managed certificates, and organizations are increasingly turning to automation to solve the problem. As protocols such as ACME become integrated into an increasing number of commonly used web tools and outside vendors expand their ability to shoulder the certificate management burden, it will only become easier for organizations to incorporate automated elements into their own environments.

    — Abul Salek

  • Micro Focus Lays DevOps Foundation for RPA

    Micro Focus Lays DevOps Foundation for RPA

    Micro Focus has announced the general availability of Micro Focus Robotic Process Automation (RPA), which sets the stage for employing DevOps processes to construct highly automated business processes.

    Thanks to the rise of artificial intelligence (AI) and digital business transformation initiatives, interest in RPA has increased significantly. However, rather than merely using RPA to build bots to automate a narrow set of static processes, RPA is now leveraging machine and deep learning algorithms to optimize a wider range of dynamic business processes and adjust more easily to changes within an application.

    Travis Greene, senior director for IT operations management ay Micro Focus, said new capabilities such as object recognition technology included within Micro Focus RPA enables the platform to detect and automatically adapt to changes to the user interface.

    Other new capabilities now include the ability to visually or programmatically create workflows using a tool that records screen actions, the ability to more easily combine graphical and programmatic workflows spanning multiple platforms and the ability to deploy hundreds of thousands of bots as needed. A centralized dashboard manages the bots, all of which have unique IDs and encrypted, role-based credentials.

    As RPA and other forms of AI continue to be integrated and embedded with applications, Greene said it’s a matter of time before DevOps practices are extended to RPA platforms. Organizations will seek to train bots continuously as more data is made available, which in turn will drive updates to applications that are dependent on those bots. In most cases, depending on the complexity of the task, the bots will be built using a mixture of application programming interfaces (APIs), command-line interfaces (CLIs) and graphical tools. Organizations will need to define the processes to determine how and when each organization will opt to invoke one interface instead of another, either in isolation or combination, added Greene.

    Historically, organizations have been able to apply RPA to static processes with mixed success. With the rise of AI, the scope of those efforts has become more ambitious. Business leaders are too inclined to believe that just about every business process can be automated using a combination of AI and RPA. However, AI and RPA still are confined to relatively narrow use cases involving highly predictable outcomes.

    That’s changing, however, and RPA and AI are becoming more pervasive across the enterprise. DevOps teams will need to find ways to update both their applications and RPA/AI models without breaking one or the other. Most of the current RPA/AI models are highly sensitive to any change made to the application environment they are monitoring. And, unfortunately, the data scientists who are largely responsible for constructing those RPA/AI models don’t yet tend to have much of an appreciation for best DevOps processes. There will, however, come a day soon when these two worlds will collide.

    — Mike Vizard

  • How Serverless Is Impacting the IT Landscape

    How Serverless Is Impacting the IT Landscape

    Although most users will never realize it, with every digital data entry and exchange servers are consuming substantial quantities of energy to file, store and sustain that data’s availability.

    While the average consumer or office worker doesn’t consider the economic or environmental impact they have, IT and C-Suite teams are awakening to the problem and the awareness among energy concerns, legislators and thought leaders is growing. From the expanding troves of transactional data for AI to each video streamed on a smartphone or to a social media platform, hyperconnected companies are hungry to collect, understand, feed and leverage data on our digital habits. In today’s infrastructure models, data centers continuously burn through processing power to be ever-ready to those workloads—even when demand falls during regular off-peak periods.

    That’s one of the two real pushes behind the shift from legacy to new serverless infrastructures: waste, both environmental and economic.

    The world is waking up to the realization that the digital economy is consuming disproportionate levels of energy, putting new pressures on their organizations, their region’s energy infrastructures and, ultimately, on the planet.

    A recent report from Climate Change News predicts billions of internet-connected devices could produce 3.5% of global emissions within 10 years and the communications industry could use 20% of all the world’s electricity by 2025. In fact, data from statistics portal Statista notes the energy consumption of Facebook usage alone amounted to about 2.46TW hours in 2017, compared to 532GW hours in 2011.

    In the U.S. alone, data centers consume an astounding 90 billion kWh of electricity per year, the approximate annual output of 34 (500MW) coal-burning power plants. The data center consumption costs are baked in—organizations pay for always-on computing power to meet peak need regardless of how demand may fluctuate.

    The Promise and Potential of Serverless

    Cutting Energy Consumption: Serverless compute technology such as Amazon’s AWS Lambda or Microsoft’s Azure Functions are native cloud architectures designed to decrease consumption and increase agility and innovation.

    It provides a more efficient function-as-a-service (FaaS) consumption model that enables organizations to pay for what they use, rather than constant peak demand charges.

    Some of the economic impacts of serverless are obvious: Organizations that host their own servers have reconciled themselves to paying for those servers running at full power while sitting idle for much of the time. It’s like keeping your car constantly running, regardless of whether you’re going anywhere.

    In contrast, serverless not only promises far greater energy efficiency from the macro-infrastructure perspective, it also is event-driven. Organizations pay by the computing power consumed, so their costs reflect their actual server usage rather than paying for always-on capacity.

    The Shift Is On, From Managing Infrastructures to Solving Problems

    Reduced energy consumption and cost efficiencies are important for organizations and the planet. Two other major promises serverless offers are the ability to let organizations focus on creating applications and services without chaining them to infrastructure and innovation opportunities.

    Organizations shift the focus of their IT resources away from infrastructure maintenance—the provisioning, patching, operating system maintenance, managing runtimes and capacity planning overhead—to solving business problems. This means the focus of the organization’s IT resources can shift from managing infrastructure maintenance burdens to doing what IT was always meant to do: solve actual business problems.

    But there’s another aspect to serverless that’s even more exciting.

    Unlocking ‘Vendor Lock’ and Unleashing Innovation

    Every great technology breakthrough inevitably tears through old barriers and accelerates innovation. Serverless is no different.

    Furnace is a good example. It’s a new open source, serverless, streaming data processing platform for building big data solutions that are cloud platform and language-agnostic.

    It supports the use of applications that run natively in AWS Lambda, Microsoft Azure and Google Cloud, passporting data between platforms. Using coding languages including JavaScript, Node, Python and Golang, a growing community of developers are creating constructs to connect cloud-based applications.

    Some of the most urgently needed application construct types we’re likely to see in the near term will be for extracting and anonymizing old proprietary data for GDPR compliance, and for honeypot data collection and manipulation, with Elasticsearch and Redshift. Other application foundational on-ramps are in development.

    According to Danny Waite, CTO for the Furnace Ignite organization, the Serverless Furnace source platform was architected to allow deployment and data traversal into and across various cloud, on-premises and hybrid environments, putting the underlying efficiencies of serverless to work on practical applications.

    The application development platform is optimized to ingest massive volumes of data from different sources and in various formats. The Furnace open source community is soliciting feedback and community developers are working on various projects, with plans for constructs to harness and tap into the massive stores of now-untouched IoT data, among other sources.

    Greener, Cleaner, Streamlined Computing

    Serverless is a driver for ecologically greener, operationally cleaner and innovation-driving new computing. Its reduced data-center energy consumption, pay-per-use event-driven model, openness to innovation and seemingly unique ability to help align historical data with new regulatory requirements are all arguments for a shift in thinking and embrace of serverless. The improved economics are another compelling reason, though likely not a primary one for data-driven organizations.

    Serverless communities such as Furnace Ignite are emerging with tools and techniques for adoption. The Furnace application development platform has been architected in a way that allows it to be deployed into various infrastructures, such as cloud, on-premises or within hybrid environments, with the ability to ingest huge volumes of data from different sources, in various formats, so developers can take that data and make it useable. The whole process can be done with speed and agility and removes all the nuances normally associated with working with streaming data.

    Serverless platforms such as Furnace will continue to be developed to drive DevOps and meet the needs of the opensource community.

    — John Blamire

  • Low-Code Has Taken Development by Storm. Why Not the Web?

    Low-Code Has Taken Development by Storm. Why Not the Web?

    For more than a decade, development tools increasingly have made their way into the hands of less-technical people. On the web in particular, we’ve eliminated the webmaster as the middleman with content management systems and moved progressively toward a model where non-technical users could assemble their own sites with little to no coding experience of their own.

    Fast forward and today, the terms “no-code” and “low-code” are used to describe this idea. Furthermore, no-code and low-code tools have been on a decade-long rise, with an increasing impact on the web. Examples include:

    While this has been a strong, decade-long market trend, we’re only at the beginning. This trend won’t stop until new website experiences are automatically generated and optimized by machine learning.

    Trends Driving the Low-Code and No-Code Movements

    According to Wave, “In our survey of global developers, 23% reported using low-code platforms in 2018, and another 22% planned to do so within a year.”

    Major market forces driving this trend include a talent shortage among developers, with an estimated 1 million computer programming jobs expected to remain unfilled by 2020 in the U.S. alone. What’s more, the developers who are employed are often overloaded with work and struggle with how to prioritize it all. Some of this burden could be removed by low-code and no-code tools, allowing marketers and other non-technical professionals to do some of the development work themselves. In addition, the fact that technology has permeated every aspect of our lives, from our smartphones to our smart homes, has driven the need for more people to become creators.

    “As creating things on the internet becomes more accessible, more people will become makers. It’s no longer limited to the <1% of engineers that can code, resulting in an explosion of ideas from all kinds of people,” said Ryan Hoover, founder of Product Hunt, in a blog post.

    In other words, the supply of web creators must keep up with the demand for more digital experiences across more channels.

    But this does not only apply to individuals. Consider this: The typical large organization has to build and maintain hundreds of sites. These organizations need to build, launch and customize sites in days or weeks, not months. Today and in the future, marketers and other non-technical stakeholders can embrace no-code and low-code tools to develop websites rapidly.

    Abstraction Drives Innovation

    As predicted in my middleman blog post, developers won’t go away. Just as the role of the original webmaster has evolved with the advent of the CMS, the role of web developers is changing with the rise of low-code and no-code tools.

    Today, developers tend to focus on creating next-generation digital experiences that require new and emerging technologies not yet available in no-code solutions. In the process, they help invent and build new technologies such as React, Vue.js, etc.

    Successful no-code approaches abstract away complexity for web development. This enables less-technical people to do things that previously only could be done by developers. When those abstractions happen, developers can move on to the next area of innovation.

    Such abstractions are a good thing for both developers and non-technical website creators. When everyone is a builder, more good things will happen on the web. I was excited about this trend more than 12 years ago and remain excited today. I’m eager to see the progress no-code and low-code solutions will bring to the digital world in the next decade.

    — Dries Buytaret

  • Chef Brings DevOps to Windows Platform Migrations

    Chef Brings DevOps to Windows Platform Migrations

    Chef and Microsoft revealed today that they are working together to apply DevOps processes to accelerate the migration of Windows Server 2008 and SQL Server 2008 to more current versions.

    Announced at the Microsoft Inspire conference, the goal is to enable organizations to leverage Chef Enterprise Automation Stack (EAS), which includes an EAS Migration Accelerator for Windows that will scan existing .NET applications running on legacy Microsoft platforms to discover configuration and dependency details, then encapsulate that information as code in a way that enables the application to run on the latest version of a Microsoft virtual machine or as a container.

    Ewan Dalton, partner development manager for Microsoft, said there are roughly a half-million .NET applications running on platforms that are scheduled for end-of-life support beginning in January. In an ideal world, most of those applications would be modernized to be deployed on the Microsoft Azure cloud, said Dalton.

    Vikram Ghosh, vice president of business development at Chef, said rather than merely lifting and shifting legacy applications onto a new platform, Chef EAS will make it possible to employ DevOps processes to modernize how those applications are updated and maintained. Chef EAS makes it possible to, for example, scan, extract and package legacy applications and their dependencies, using source control repositories such as GitHub, and convert them into cloud platforms using pipelines such as Azure DevOps. Customers can deploy these application packages in Azure and run continuous security and compliance scans, added Ghosh.

    Chef EAS

    The biggest challenge many organizations face when trying to migrate applications is the developers who created the application haves long since left the company. As such, most organizations only have documentation, which is usually sparse. They then must determine whether it is worth modernizing the application or whether they should simply replace it. In cases where it is deemed that the business logic contained within that legacy application is still critical enough to continue running, they need to find a way to automate that process as much as possible.

    Longer-term, of course, most organizations will work to modernize legacy monolithic applications even further by slicing them up into a series of more manageable microservices. But many organizations are already running out of time when it comes to existing .NET applications. Once Microsoft ends support for the platforms they are running on, patches to address cybersecurity issues no longer will be provided as part of the standard licensing agreement. In effect, that means there is enough time between now and January to re-engineer an application. Chef EAS essentially throws organizations a lifeline to determine what they want to do next in terms of replacing a legacy application, said Ghosh.

    Of course, there will always be some subset of the IT community that will continue to run .NET applications on outdated Microsoft platforms. It’s also unlikely that any of those organizations will be embracing DevOps processes to maintain those platforms anytime soon.

    — Mike Vizard

  • AWS Advances DevOps Agenda at Summit Event

    AWS Advances DevOps Agenda at Summit Event

    Amazon Web Services (AWS) has added services to its cloud platform designed to automate the creation of virtual private clouds (VPCs) and made available additional tools to build applications and integrate external data.

    Speaking at the AWS Summit event, CTO Werner Vogels said these latest services are part of an ongoing effort to automate DevOps processes to the point at which developers will need to write only the code for business logic that runs on top of the AWS public cloud.

    One of the latest tools the company is making available in pursuit of that goal is AWS Cloud Development Kit (CDK), a software development framework for defining cloud infrastructure in code as constructs and provisioning it through AWS CloudFormation. AWS CDK currently supports TypeScript, JavaScript and Python with support for C#/.NET and Java available in preview. That capability will eliminate the need to write low-level code using a different tool just to spin up a VPC on AWS, Vogels said.

    AWS CDK also allows organizations to design and compose their own custom components for managing infrastructure alongside their application code.

    The cloud services provider also announced AWS EventBridge, a programmable event bus for integrating data on AWS with data residing in software-as-a-service (SaaS) applications on any platform based on a serverless computing framework, and the general availability of AWS Toolkit for Visual Studio Code, a plug-in for the Microsoft integrated development environment (IDE) that makes it easier to build, debug and deploy code on AWS.

    Finally, the company announced that Amazon CloudWatch Container Insights is now available in preview. This offering provides access to automated dashboards summarizing the performance and health of their Amazon Elastic Container Service (ECS) and Fargate clusters by tasks. Those offerings come on the heels of AWS Fluent Bit, a tool for container logging being made generally available earlier this week.

    Vogels stressed that the way applications are built is now fundamentally different in the age of the cloud. Developers are making greater use of containers and serverless computing frameworks. The challenge with containers is that in the absence of a cloud service provider to manage the environment, the complexity of the underlying middleware and infrastructure conspires to reduce the amount of time developers spend on writing actual application code, he said.

    To address that issue, AWS is investing more in container and Kubernetes services as well as in on-demand serverless computing services such as AWS EventBridge that don’t require organizations to pay for idle processes, Vogels explained.

    In general, Vogels said AWS continues to advocate for the elimination of separate IT operations and cybersecurity teams in favor of a more unified approach leveraging automated services provided by AWS that ensure, for example, the entire continuous integration/continuous deployment (CI/CD) platform is secure. However, while the number of organizations embracing best DevOps and DevSecOps processes continues to increase steadily, it’s much less clear how many of them will standardize those processes on a single cloud platform at a time when reliance on multiple clouds also continues to increase steadily.

    — Mike Vizard

  • Workflow Automation Poised to Accelerate DevOps

    Workflow Automation Poised to Accelerate DevOps

    Automation is taking the market by storm. Research firm Gartner predicts that by 2020, more than 40% of data science tasks will be automated, and that is just the tip of the iceberg. Businesses are turning to automation to reduce costs, increase productivity and gain a competitive advantage. Add DevOps to the mix and it becomes readily apparent how DevOps teams can benefit by leveraging a little workflow automation. After all, the DevOps process is framed by repeatable workflows, wherein the agile manifesto dictates progress or other frameworks define how change moves through the development and deployment cycles.

    “Most development projects are hindered by numerous manual steps and the lack of defined collaborative communications methodologies, leaving far too much room for error,” said Raj Mehta, CEO of Raj Technologies. “Defining a workflow and bringing automation into the picture could go a long way toward reducing errors.”

    The DevOps community has come to rely on many manual processes and numerous tools to keep the pace of rapid application delivery and change management. Those solutions often have points of failure, where important tasks can slip through the cracks. Even so, many developers are resistant to change and embrace the processes they have already established. Yet, intertwining manual processes with automation and management tools has been demonstrated as rather inefficient, slowing the perception that workflow automation can actually achieve the productivity promises claimed.

    Perhaps the challenge comes from the fact that many DevOps teams require assistance to achieve any type of automation goals. That indicates there may be a lack of qualified professionals to bring forth workflow automation. “You have to crawl before you can run, and that means you must become an expert in designing workflows, before deploying one, or at the very least, turn to an expert,” Mehta noted.

    A workflow is a fundamental process that dictates the steps to achieve a goal. Workflows have been used for centuries by businesses and have become digitized over the last few decades, with numerous workflow tools coming from leading vendors.

    Many technologists may have first encountered the workflow ideology back in the days when Microsoft’s Sharepoint first came on the scene, which offered rudimentary tools for routing content through various steps. Of course, the workflows in use today have become much more complex, requiring designers to not only know how to script actions but also understand the overall goal of those actions.

    That’s a simple truth understood well by Workfront, a Lehi, Utah, based vendor of workflow solutions which is looking to better service various organizations, as well as DevOps professionals, to create workflow automation opportunities. The company’s primary product goes by the name of Workfront as well and uses a platform-based approach for creating workflows that can be integrated across multiple technologies, bringing centralized workflow orchestration to many use cases.

    Tools like Workfront fit well into the automation equation by creating the foundation of workflows. However, most DevOps professionals need more than workflow design to achieve improved automation. Other elements must be considered, such as demand management, resource management, project templates, policies and scripted actions. Efficient workflow tools should also incorporate task lists, collaboration capabilities and reporting.

    Of course, Workfront is only one of many workflow automation companies on the market. Many other vendors are established in the market, with Market News Reports mentioning leaders such as Pegasytems, Tibco Software and Sourcecode Technology Holdings dominating the market.

    HNY Research projects workflow automation market size will grow to $16.58 billion by 2023 from $4.68 billion in 2017, at an estimated CAGR of 23.48%. Key factors such as ease in process with the installation of workflow automation tools, convergence of workflow automation with traditional business processes, focus on streamlining business processes and cost efficiency through workflow automation are driving the workflow automation market.

    “Business leaders must carefully analyze different processes and identify which workflows can automated. Therefore, business leaders should avoid rushing into workflow automation adoption and analyze the various potential use cases and choosing which vendors best fit in their organization,” said Naveen Joshi, founder and CEO of Allerin.

    Many businesses are leveraging workflow solutions on their path to digital transformation and bringing once-manual processes into a world where automation can be applied. While some may argue over the intricacies of digital workflows and how those workflows fit into business process management (BPM), one truth remains evident: The goal is to digitally transform and introduce automation, which in turn indicates robotic process automation (RPA), becomes the next logical step.

    According to Grand View Research, the global RPA market is expected to register a CAGR of 31.1%, from $597.5 million in 2018, driven largely by demand for business process automation via artificial intelligence (AI) and software robots.

    RPA paired with workflow design may prove to bring full digital transformation to DevOps, and potentially grow beyond just a niche in those challenging environments. Workflows, BPM, BPA and RPA are becoming increasingly important to DevOps organizations seeking to lower error rates, boost productivity and improve communications.

    — Frank Ohlhorst

  • The Linux Foundation Breathes New Life into Osquery

    The Linux Foundation Breathes New Life into Osquery

    Facebook’s useful but neglected DevOps tool Osquery has gotten a new lease on life, thanks to The Linux Foundation.  

    Anyone who has been tasked with monitoring the security of server instances in a data center or cloud knows how laborious and time-consuming it can be. Osquery, a project started by Facebook, aims to lessen this burden by reframing how developers engage with their infrastructures. DevOps professionals can use Osquery to expose an operating system as a high-performance relational database, making it possible to use SQL commands to access data about a system, just as they would for a database. 

    Osquery works on Mac, Linux and Windows systems and is provided as an open source download via GitHub. Although Osquery was developed by Facebook to monitor and safeguard the security of its own platform, the social media giant quickly realized the utility of the platform would extend to other enterprises that depend upon insight into the low-level behavior of operating systems. 

    Facebook publicly released the project’s code and documentation in 2014, in response to feedback from a small number of other companies that tested Osquery. The public availability of Osquery resulted in a number of major companies including Airbnb, Dropbox, Netflix, Etsy and Uber to bring Osquery into their software development environments. Nevertheless, users have voiced frustrations with Facebook’s handling of Osquery, and have accused the internet giant of neglecting the project.

    To address those complaints, The Linux Foundation has taken over the mantel of Osquery and recently announced the formation of a new foundation to support the Osquery community. 

    The new Osquery Foundation brings together engineers and developers from Dactiv, Facebook, Kolide, Trail of Bits, Uptycs and many others committed to supporting the project with technical contributions and long-term stewardship. A Technical Advisory Board will be created to handle the shift in management from Facebook to the Osquery Foundation and will oversee the priorities initiated by the foundation’s members.

    This restructuring of management promises to extend the usability of the project as a number of developers contribute to the codebase.

    “Osquery has drastically simplified the process of operating system monitoring, which has unlocked new methods for securing infrastructure, detecting anomalies and more. We look forward to working with the Osquery community to develop the foundation in a manner that will continue to foster the growth and adoption of Osquery while supporting the community’s diverse needs,” said Jim Zemlin, executive director at the Linux Foundation.

    Developers working to maintain the security of their systems believe it would be wise to incorporate Osquery into their workflow. “Although Osquery will never do everything that a security team needs, it gets about 80% of what you need for endpoint insight in one package,” said Doug Wilson, director of security for Uptycs.

    Although the platform requires some configuration, developers familiar with SQL will feel accustomed to using Osquery to retrieve information about their systems. Furthermore, Osquery’s cross-platform support and customizability make it especially attractive to businesses with newer types of infrastructures and ones that work at scale. 

    The move away from Facebook toward a more community-friendly foundation is sure to breathe new life into the project, and as development on Osquery expands, it is likely to become a more valuable resource for security teams moving forward.

    This article was co-authored by Tyler Ohlhorst, research analyst and freelance copy editor and copywriter at Magnum Consulting.

    — Frank Ohlhorst

  • HashiCorp Tools Bridge DevOps, Legacy IT Environments

    HashiCorp Tools Bridge DevOps, Legacy IT Environments

    At its HashiConf EU 2019 conference this week, HashiCorp extended the reach of its Terraform Enterprise tools for automating the management of IT infrastructure beyond the realm of DevOps into legacy IT environments.

    Via integration with the Service Catalog from ServiceNow, IT administrators working with a traditional ITIL-based framework now can invoke Terraform Enterprise to provision IT services. HashiCorp CTO Armon Dadgar said rather than requiring organizations to hire an infrastructure engineer to automate the provisioning of IT infrastructure, IT administrators now can do the job.

    At the same time this week, HashiCorp announced that policy sets may now be configured using the source policies available in multiple version control systems (VCSs), including GitHub, GitLab and Bitbucket. The goal is to make it simpler to enforce policies that would limit, for example, the number of virtual machines a developer might be allowed to spin up within the context of a set of GitOps processes, said Dadgar.

    Finally, HashiCorp also announced that HashiCorp Consul service mesh networking software has been enhanced to include Layer 7 application traffic management capabilities alongside integration with the HashiCorp Nomad application and cluster scheduler software.

    In general, Dadgar said HashiCorp is moving toward enabling its portfolio of offerings to span multiple types and classes of IT environments. HashiCorp Consul, for example, can be run across both legacy IT environments as well as Kubernetes. Most organizations will need to support both legacy monolithic and microservices applications using a combination of DevOps and legacy ITIL processes, noted Dagar. Rather than having to master new tools for each environment, HashiCorp is making a case for a common set of tooling that can be applied across multiple IT classes of processes.

    While some organizations have fully transitioned to DevOps, many more organizations are still struggling to make the transition. Reasons for not making that transition faster range from simple inertia to a lack of skills. Rather than having to take an all-or-nothing approach to modernizing IT processes, Dadgar said HashiCorp is enabling each IT department to embrace IT automation wherever they happen to be on that journey, including being able to embrace new platforms as they become available.

    Dadgar noted that level of extensibility is critical because most tools are still being brought into organizations from the bottom up by individual members of the IT staff, rather than being championed from the top down. In the absence of an executive mandate, Dagar said the path of least resistance is to make IT automation tools as accessible as possible to members of the IT team regardless of skill level.

    There will, of course, come a day when IT leaders will try to rationalize processes around a common standard set of practices. The degree to which any IT organization has been able to achieve that goal, however, has been mixed at best. In the meantime, IT organizations will always need to keep their options as open and flexible as possible.

    — Mike Vizard

  • source{d} Delivers Enterprise Edition of SDLC Platform

    source{d} Delivers Enterprise Edition of SDLC Platform

    source{d} has made available an enterprise edition of its software development lifecycle (SDLC) platform that includes visualization and analytics tools along with additional management capabilities. A free community edition of the core source{d} platform, which is stripped of most of the enterprise-class tools, also is now available in beta.

    Company CEO Eiso Kant said the enterprise edition of source{d} provides the additional tools enterprise IT organizations require to employ the platform as the equivalent of a business intelligence (BI) application for managing DevOps processes.

    The tools can launch queries against a big data repository based on the Apache Spark in-memory analytics engine running on a Kubernetes cluster, which can be populated with actual data or metadata pulled from multiple DevOps tools and platforms. That capability enables IT leaders to then query data that has been joined together in a way that surfaces actionable intelligence, said Kant.

    source{d} Enterprise achieves that goal by including Apache Superset, an open source business intelligence web application. The platform also can be extended to support SQL queries launched from other BI tools and project management applications. The enterprise edition also provides access to advanced queries and user-defined functions to address tasks such as code duplication analysis or identity matching.

    Finally, source{d} Enterprise provides authentication integration with LDAP and OpenID to limit who can edit or view queries, data sources, charts and dashboards.

    Longer-term, source{d} plans to leverage predictive analytics and machine learning algorithms more aggressively as the amount of SDLC data that gets collected by enterprise IT organizations increases.

    Kant said many enterprise IT organizations have tried and failed to build the equivalent of source{d} platform mainly because bringing together and then maintaining all the required technologies often becomes too complex an undertaking. Those efforts also have been confined largely to managing processes such as bug fixes versus including all the BI tools needed to manage software development in a way business and IT leaders can align technology initiatives to specific business goals, he said, noting the source{d} approach brings together enabling technologies such as Spark and Kubernetes to enable IT organizations to deploy an SLDC platform on-premises or in a public cloud.

    As software development increasingly grows more complex, the days when it could be managed efficiently through a project management application alone have come to an end. At the same time, more organizations realize their success ultimately comes down to how consistently they can deliver a high-quality application experience. For all the hype surrounding digital business process transformation, the core tenet is that most organizations are indeed becoming software companies. As such, they need to borrow many of the SDLC processes that successful software companies employ routinely. The only real difference is they don’t need to develop those processes from the ground up when more turnkey SDLC platforms are emerging.

    — Mike Vizard