staging-devopsy.kinsta.cloud

Tag: devops

  • DevOps and Retail: Transforming Brick-and-Mortar to Brick-and-Click

    DevOps and Retail: Transforming Brick-and-Mortar to Brick-and-Click

    Brick-and mortar-retail is rapidly transforming with the digital age. As physical storefronts turn digital, it’s safe to say that traditional retail is quickly becoming a mesh of “brick-and-click.”

    So, how are retail giants surviving in this new landscape? Interestingly enough, the remaining brick-and-mortar retail are reinventing themselves by applying helpful DevOps principles.

    In this post, we’ll peek into cases of innovation and horror stories from historically brick-and-mortar retail chains and brands. From Target, to Balian Group, Sears and other companies, we’ll discover the importance of DevOps in this new brick-and-click multi-channel retail environment.

    Target

    In an effort to increase agility, Target engineering teams embrace something they call “the Dojo”—immersive, six-week training sessions with hands-on DevOps and Agile coaches. As Target CIO Mike McNamara recently told the Enterpriser’s project: “The Dojo has been fantastic in getting teams engaged with agile and DevOps, removing the natural resistance and fear of change, and then supporting the team through the changes while maintaining productivity. It’s been a huge success for Target.”

    The result of these sessions is increased comfort with methods including containerization and tools that enable continuous integration and continuous deployment. McNamara also noted their grassroots approach; they encourage open source activity and listen to their engineers when they petition the use of new tech.

    “Our engineers have fully embraced DevOps,” he said.

    Bailian Group

    For another case study, take Bailian Group, a large Shanghai retailer that operates 6,000 grocery and department stores. By the time Bailian entered the online marketplace, it suffered from a severe lack of agility, non-native web apps and underutilized server assets.

    “Our transition from traditional brick and mortar to omnichannel business presented a great opportunity but an equally large challenge,” noted Lu Qichuan, director of IaaS and Cloud Integration Architecture at the Bailian Group.

    Qichuan and Bailian realized that to stay relevant, they had to cut long development cycles and embrace “new cloud application development tools and processes that foster a CI/CD and DevOps culture and increase innovation and time-to-market,” he said.

    They ended up implementing a private cloud using OpenStack. Now, midway through their cloud transition, they estimate to have 10 million virtual machines by the end of 2020.

    Sears

    While the previous two examples had positive outcomes, digital transformation has not gone over well for other retailers. Sears declared bankruptcy in October 2018.

    In its final years, Sears was in a sticky situation. To the chagrin of former CEO Eddie Lambert, surveys found their customers preferred to shop in stores as opposed to online. Disregarding customer desires, however, Sears attempted a big data play with its “Shop Your Way” program, under which Sears gathered user data to then sell. The program floundered, resulting in consumer irritation and loss of sales as reported by Business Insider.

    Writing for InformationWeek, Emily Johnson reminded us that evaluating DevOps using the wrong metrics will ultimately lead to failure, citing customer satisfaction as the No. 1 goal.

    In its final years, an abrasive Sears leadership halted any sense of DevOps to flourish. A disengaged Sears CEO Lambert created “a climate of fear,” according to Busines Insider reports. It appears the retailers that stay afloat encourage agility and intrapreneurship—this means encouraging innovation, not fear.

    DevOps Aids the New Omnichannel Retail Benchmark

    Brick-and-mortar are evolving into omnichannel retailers that bridge physical and virtual spaces. Take IKEA’s augmented reality app, which visualizes items in a house. From Disney, to Startbucks, Nordstroms and elsewhere, physical storefronts embrace digital offshoots of their core model to augment the traditional retail experience.

    Surprisingly, physical storefronts aren’t completely dead. They still offer conveniences for urban environments with high walking scores. See the emergence of Amazon physical locations for ironic evidence of their benefit. Other storefronts and pop-up shops have taken on a museum-like quality. The flagship Funko store, for example, creates impressive displays and interactive exhibits, pulling in flocks of weekend shoppers that want more than a dull e-commerce shopping cart.

    DevOpsGroup similarly describes a “reverse showrooming” movement; consumers research products online to then purchase in person. This enables brands such as ASOS, which is emphasizing social media and free worldwide deliveries, to stay competitive.

    “Retailers have been forced to re-evaluate their business strategies and understand how new technologies will affect their offering,” according to the DevOpsGroup website.

    DevOps Turns Bricks to Clicks

    Online shopping has sufficiently disrupted retail, and brick-and-mortar must reinvent themselves to stay relevant. An open-minded leadership and a commitment to fast development, dynamic scaling, uncompromised availability and open source technology underpin a successful evolution.

    Of course, this increased reliance on technology necessitates a hardened look at how development and operations communicate, and brick-and-mortar retailers must adopt the behaviors of tech companies to retain agility.

    This may come in many forms: new user-facing digital experiences, microservices for composability in niche omnichannel environments, containerization to package code that scales well, distributed data stores, heightened user data security or smart incident management techniques, to name a few.

    Traditional retailers, then, should dissuade their doubts against agile. DevOps can aid retail by reducing redundancy, cutting costs and streamlining production and distribution.

    — Bill Doerrfeld

  • Feature Flagging Poised for Wider Adoption

    Feature Flagging Poised for Wider Adoption

    Feature flags, also known as feature toggles, provide a means for DevOps teams to test and add new capabilities to their applications without having to take the entire application offline. The downside is they generally result in more branches of applications that need to be carefully managed within the context of a DevOps process.

    A recent survey of 550 software developers and decision-makers conducted by the market research firm Vanson Bourne on behalf of Atlassian and Rollout.io, a provider of a platform for managing feature flags, suggests that IT organizations are getting more comfortable with managing feature flags.

    The survey finds 95 percent of survey respondents said their organizations have implemented, have begun implementing or plan to implement features flags. Those that have implemented feature flags said they add 30 new features on average to each application per year, compared to those that have only begun implementing (21) or plan to implement (18) them. A full 81 percent said use of feature flags will increase over the next 12 months, and 88 percent said modern software development should assume to include feature flagging.

    The primary driver for implementing feature flags are a need to reduce risk (46 percent), increase speed of development (46 percent) and increase speed of deployment (45 percent). Eighty-seven percent of respondents said enabling an on/off or “kill” switch for features also reduces the risk of application downtime for customers.

    A total of 97 percent of respondents said they are under pressure implement new application features faster, and yet 65 percent said it is difficult to accomplish that goal safely. More than 61 percent of respondents said their organization releases new versions or updates for applications once a week or more. However, 72 percent admitted they have lost customers or users due to application development errors.

    Rollout COO Ronit Belson said building out the functions required to manage feature flags doesn’t make much sense when platforms such as Rollout.io are now emerging. The survey finds nearly 90 percent of respondents who have implemented feature flags in-house would consider moving to a third-party supplier for feature flagging management software. A total of 37 percent of respondents said the cost of in-house development is higher and 32 percent said they encountered more complexity than anticipated.

    Belson added most organizations not only fail to appreciate the cost of implementing feature flagging on their own, but also the impact feature flagging has on the culture of the IT department.

    Thanks to the rise of microservices, feature flagging should become more prevalent as monolithic applications give way to a more granular approach to application development that makes it easier to isolate functionality. In fact, the ability to apply feature flagging might be considered one of the primary benefits of the agility inherent to any microservices architecture, especially it if relies on containers that can be easily ripped and replaced. The challenge organizations will face is implementing the DevOps processes required to embrace feature flagging at scale.

    — Mike Vizard

  • How to Become a DevSecOps Engineer

    How to Become a DevSecOps Engineer

    The popularity and success of DevOps means it’s now an increasingly common feature in IT job descriptions.

    One of the most important roles within DevOps, however, is the Security Engineer or DevSecOps Engineer. This deeply rewarding career requires a specific technical skill set, current knowledge of cybersecurity trends and a decent amount of experience.

    Perhaps the best thing about this role is that demand is rising. You’ll find the need for such engineers is sky-high right now and will continue to rise as cyberattacks grow in frequency and sophistication.

    What Does Being a DevSecOps Engineer Involve?

    The work of a DevSecOps Engineer is like many other IT security professional roles. Both use a variety of best practice tools and methods such as cybersecurity software, threat modelling and risk assessments to detect and analyze threats.

    However, in keeping with DevOps practices, there are some key differences when compared to a typical IT security role.

    On DevOps projects, security isn’t an afterthought but is built into the software while it is being created, by using secure coding. During development, the software is attacked to find vulnerabilities, as opposed to running scans once it has been created.

    Collaboration is a core practice of DevOps, and therefore DevSecOps roles work alongside DevOps Engineers to ensure that security vulnerabilities are assessed and fixed during development. Automation tools to detect vulnerabilities play a key role, so DevSecOps need a good understanding of such toolsets.

    Knowledge of threats is shared with the whole team, instead of keeping it within the Ops silo team. Therefore, DevSecOps Engineers require great communication skills.

    What Skills Are Required?

    DevSecOps Engineers require a broad set of skills. They need the technical skill set of an IT security professional, as well as knowledge of the DevOps approach. They’ll also need a passion for cybersecurity, with sound awareness of the latest threats and trends. These are the main skills required:

    • Knowledge of the DevOps culture and principles.
    • An understanding of programming languages such as Ruby, Perl, Java, Python and PHP.
    • Strong teamwork and communication skills.
    • Knowledge of threat modelling and risk assessment techniques.
    • Up-to-date knowledge of cybersecurity threats, current best practices and latest software.
    • An understanding of programs such as Puppet, Chef, ThreatModeler, Checkmarx, Immunio and Aqua. They may also need to know Kubernetes, Docker or AWS.

    Such skills can be acquired on the job, either in formal employment or through an internship or work placement. You can also take courses to learn DevOps principles, programming languages or automation tools. Of course, you can also teach yourself such languages and tools, as many IT engineers tend to do.

    What About Experience and Qualifications?

    Many DevSecOps Engineers have experience in a non-DevOps IT security role. If you have no experience as an IT security professional, it is recommended you start there first before getting into DevSecOps.

    Starting a career in IT security generally requires a degree in computer science, cybersecurity, math, engineering or science. It is then recommended you gain experience on an internship or graduate placement. You can also enter the field without a degree by gaining practical industry standard certification, such as that offered by Microsoft, CompTIA, Cisco or Certified Ethical Hacker (CEH).

    Making the move into DevOps requires IT security experience, and you’ll need to gain knowledge of the languages and automation software commonly used on DevOps projects. It is also advised that you gain accredited DevOps qualifications from the DevOps Institute, specifically the DevOps Foundation and DevSecOps Engineering (DSOE) qualifications. These will equip you with a solid understanding of DevOps principles and DevSecOps methods.

    Summary

    You might think DevOps or DevSecOps are just trendy words for developers, operations and security teams who work closely together. But this is wrong. There is so much more to DevOps than that. Ask Netflix, Amazon or Sony—all hugely successful companies that use DevOps culture to get things done.

    DevSecOps is a crucial part of DevOps, especially given the current cyberattack climate. As the need for such individuals grows, so will the job openings. If you have security experience and want to boost your career or are thinking about which degree and career path to take, keep DevSecOps in mind.

    — Stephanie Rowe

  • Upskilling: The Enterprise DevOps Skills Survey and Report

    Upskilling: The Enterprise DevOps Skills Survey and Report

    There has been a lot of recent reference to “T-shaped” IT professionals—individuals whose core competence (the stem) is supplemented by a broad range of other knowledge including soft skills (top of the T). T-shaped talent is in such high demand with a limited supply that recruitment efforts have become very competitive for those with upskilled and cross-skilled transformational know-how.

    Unfortunately, the migration from a deeply skilled specialist to a T-shaped practitioner is leaving many confused about what skills they should be acquiring and hiring and when. That’s why DevOps Institute (DOI) recently launched the 2019 “Upskilling: Enterprise DevOps Skills Report” community survey to help individuals and enterprises better understand the current and future state of DevOps skilling across the global IT community. Using data science coupled with hiring manager interviews, DOI’s goal in fielding the report is to provide tangible insight into why and where IT practitioners, managers, leaders and business stakeholders should focus their efforts in growing personal and organizational skills portfolios.

    Eveline Oehrlich, a former vice president and research analyst at Forrester, is bringing her data science expertise as the lead analyst for the report. Founding Platinum sponsor Electric Cloud and founding Gold sponsor Cloudbees are also sponsoring this important project. The “Upskilling: Enterprise DevOps Skills Report” will be a year over year project, and the results will be published in Q1 2019.

    The global survey will remain open until Dec. 7 and can be accessed through the following link: https://www.research.net/r/K9WD2XC

    We encourage everyone in the DevOps community to complete the survey so that the data samples truly reflect the global state of DevOps skills.  Any organization that has at least 20 people complete the survey will get a personal snapshot of the data results. All respondents who complete the survey will be entered into a drawing for four $100 Amazon gift cards.

    — Jayne Groll

  • InfluxData Time Series Database Extends Industry Lead with Performance and Data Compaction Enhancements in Latest Release

    Company Also Launches Technical Preview of Industry’s First Purpose-Built Functional Query Language for Time Series Data Ready for Developer Community Trial and Feedback

    SAN FRANCISCO — Nov. 7, 2018 — InfluxData, the modern Open Source Platform built specifically for metrics, events and other time series data that empowers developers to build next-generation monitoring, analytics and IoT applications, today announced the release of InfluxDB 1.7, delivering important query performance and data compaction enhancements to the leading Time Series Database. Announced at InfluxDays SF 2018, this new functionality gives developers powerful tools they have been asking for to query and process time series data used for monitoring IoT, DevOps, cloud, containers, and real-time analytics data.

    InfluxDB is InfluxData’s Time Series Database used as a data store for any use case involving large amounts of time-stamped data – including DevOps monitoring, application metrics, IoT sensor data, and real-time analytics. These enhancements announced today, as recommended by the community, further the gap between the functionality of InfluxDB – recognized as the overwhelming leader among Time Series Database management systems, according to DB-Engines’ latest results – and the competition.

    InfluxData has also announced the technical preview of Flux, the industry’s first purpose-built functional query language for time series data. Flux combines data query and processing in a single powerful and extensible open source language, which is enriched through user contributions. Developers are encouraged to use Flux and provide feedback to help shape the future of the time series data scripting.

    “InfluxDB is one of the most popular data sources for Grafana, and Flux is a powerful query processing engine, so we were excited to develop a first-class integration for Flux,” said David Kaltschmidt, Director, UX, Grafana Labs. “The Flux plugin comes with a new query environment which includes syntax highlighting, tag completion, raw table preview, and inline function documentation to help you get the most out of using Flux with Grafana.”

    The open source InfluxDB 1.7 delivers major improvements around query performance and throughput when using the Time Series Index (TSI), which supports large numbers of unique time series data that is ephemeral and stored in Time Series Databases.

    The technical preview for Flux is available today for developers’ trial and feedback. A getting started guide is available now from https://bit.ly/2qph9Xx which provides specific examples along with a reference for the language semantics. Developers are encouraged to use the InfluxData community site (https://community.influxdata.com) to ask questions and share experiences. To launch the Flux technical preview, InfluxData will host Office Hours today at InfluxDays SF 2018 to continue efforts towards finalizing the language, address questions, and take additional feedback.

    “Performance, data compaction and Time Series Index improvements help InfluxData maintain its leadership position in this category by delivering superior performance and meeting the advanced needs of the developers,” said Tim Hall, VP of Products. “InfluxData has been working closely with the developer community in defining the Flux language. We look forward to developers downloading Flux, using it and providing us with useful feedback to help make the language a success.”

    Metrics, events, and other time-based data are being generated at an exponential rate, as there is a growing requirement for analyzing today’s complex environments. The InfluxData Platform provides a comprehensive set of tools and services to accumulate metrics and events data, analyze the data, and act on the data via powerful visualizations and notifications.

    InfluxData has rapidly built its developer and customer base across industries – including manufacturing, financial services, energy, and telecommunications – by delivering the fastest growing Open Source Platform that enables customers to derive better business insights, data-driven real-time actions, and a consolidated single view of their entire infrastructure – from applications to microservices, and from systems to sensors. More than 450 customers, including Cisco Systems, Coupa Software, IBM, Houghton Mifflin Harcourt, Nordstrom, and Tesla, have selected InfluxData as their modern data platform for metrics and events. InfluxData is pioneering the shift to time series in a modern metrics and events platform, and is making it possible for customers to become data-driven and take on digital transformation initiatives.

    Availability

    InfluxDB 1.7 is available today. It is available for download at https://portal.influxdata.com/downloads .

    About InfluxData

    InfluxData, the creator of InfluxDB, delivers a modern Open Source Platform built from the ground up for analyzing metrics and events (time series data) for DevOps and IoT applications. Whether the data comes from humans, sensors, or machines, InfluxData empowers developers to build next-generation monitoring, analytics, and IoT applications faster, easier, and to scale delivering real business value quickly. Based in San Francisco, InfluxData’s more than 450 customers include Cisco, eBay, IBM and Siemens. Visit https://www.influxdata.com/. Twitter: @influxdb.

     

  • DevOps and Cloud: A Symbiotic Relationship

    DevOps and Cloud: A Symbiotic Relationship

    Most companies want to increase their competitiveness in today’s swiftly changing world, and so they cannot ignore digital transformation. DevOps and cloud computing have become two of the ways companies can achieve this needed transformation, though the relationship between the two is not easily reconciled—DevOps is about the process and process improvement, while  cloud computing is about technology and services. It’s important to understand how the cloud and DevOps work together to help businesses achieve their transformation goals.

    DevOps and Cloud

    Different organizations outline DevOps in different ways. This article does not debate which definition is correct, but rather presents them both to focus on the cloud’s benefit to DevOps. That said, DevOps definitions generally fall into two terms:

    1. In organizations it is defined as developer-friendly operations—IT operations are run separately yet in a way that is much more friendly to developers (e.g., self-service catalogs are provided to developers for stipulating infrastructure or providing technology-enabled pipelines for deploying new code).
    2. DevOps as a single consolidated team is habituated in organizations—developers take on operations responsibilities and vice versa.

    While cloud computing itself has several definitions, the most basic explanation of the cloud is a system that allows the provisioning of infrastructure (e.g., VMs or routers). Thus, it allows to being defined as code or templates, enabling the creation of repeatable processes that are not possible without cloud technologies.

    The aesthetics of cloud technologies and services become so easily amalgamated in DevOps vocabulary because they complement DevOps processes, regardless of how your organization defines them—or, regardless of which DevOps routes you travel to digital transformation with the cloud integration.

    Companies that focus on developers for operations often use cloud computing to speed developer productivity and efficiency. Cloud computing permits developers more control over their own components, resulting in smaller wait times. This application-specific architecture makes it easy for developers to own more components. By using cloud tools and services to automate the process of building, managing and provisioning through the code, service teams speed up the development process, eliminate possible human error and establish repeatability.

    Cloud computing also enables users to create self-service methods for provisioning infrastructure through AWS Service Catalog. Developers are able to quickly try new things, fail fast and just easily succeed in getting new products to market faster, without having to wait for IT operations to provision services for them.

    DevOps as a single term is really a mixture of these approaches, in which developers and operations work together using the cloud as a single common language. In this way, both the DevOps and cloud are able to work together, as everyone is learning new definitions and approaches at the same time. Developers and operations are equally cushy with the new language of the cloud, as developers often teach operations about the code aspect and operations can teach developers about infrastructure and security, developing a meeting point that leads to strong team dynamics.

    No matter the definition, the importance of cloud computing to DevOps can’t be accentuated enough. Cloud computing advances IT transformation, and with advanced tools and automation, it can enable companies to double down on their work to streamline and embed DevOps processes for greater efficiencies that are truly transformative.

    Inherently, as process improvement, DevOps also requires a culture change. Cloud computing can play a role here, as it can help codify and automate new processes. For example, if there is a new way in which developers should cluster the components they need to deliver a service—such as code, configuration, libraries and pipeline definitions—advanced automation tools can streamline the process and make it distinctly repeatable.

    The business case for DevOps combined with cloud technology for a successful IT transformation is fivefold:

    1. Bring products to market faster through faster access to development environments and streamlined developer processes.
    2. Automation and architecture as code reduce cloud complexity and even system maintenance.
    3. Add high security with automated, repeatable processes that serve to eliminate inaccurate error and, even more importantly, develop security controls from the very beginning.
    4. Reduce downtime through cloud-based continuous operations. Moreover, in the process of applying automation, developers can build stateless cloud application development, which increases availability and failover ability, in the process increasing business reliability and customer satisfaction.
    5. Increase scalability. One of the main reasons organizations look to cloud computing in the first place is its scalability, which allows organizations to increase capacity with the click of a button. When combined with DevOps, scalability becomes an integral part of applications as they are developed, while reducing the cost of infrastructure and increasing global reach.

    Get Past the Changes

    Together, DevOps and cloud computing are a powerhouse. While each offers greater effectiveness and business impact, together they are able to drive meaningful IT transformation that directly impacts business goals, regardless of the DevOps definition.

    At the same time, culture within the enterprise and among developers must change around the nuances of DevOps and its role in driving cloud development. Finally, IT companies must invest in these technologies and do so without an immediate goal—which, of course, drives corporate leaders and shareholders crazy.

    However, the alternative is doing nothing, which means certain failure. Competitors are likely to pull ahead in terms of time-to-market with solutions and application services. When you can hold up applications and processes in near real-time and do so within an elastic and efficient environment, the market will reward you for putting that effort. If you haven’t commenced on the process yet, it’s time to get started.

    — Jaymin Vyas

  • IBM Survey: Rise of Multi-Cloud Computing to Force DevOps Issue

    IBM Survey: Rise of Multi-Cloud Computing to Force DevOps Issue

    A survey of 1,106 business and technology executives published this week by the IBM Institute for Business Value finds 85 percent of companies are already operating in multi-cloud environments, with 98 percent forecasting they will be using multiple hybrid clouds within three years. But the survey finds that only 39 percent of the respondents have implemented some level of DevOps processes to help streamline the management of multiple cloud computing platforms.

    Blaine Dolph, vice president of Global Business Services for IBM, said it’s clear that DevOps will play a critical role in enabling IT organizations to centralize the management of multiple clouds. But the IBM survey suggests that the management of multiple clouds within organizations is not nearly as efficient as it should be.

    Only 41 percent of survey respondents said they currently have a multi-cloud management strategy in place, and just 38 percent have the procedures and tools in place to operate a multi-cloud environment. Only 30 percent of enterprises have a multi-cloud orchestrator or other multi-cloud management platform to choreograph workloads, while fewer than 40 percent of organizations have cloud configuration management tools that provide information about resource configuration and relationships between resources, the survey found.

    All told, the IBM survey is only able to identify 20 percent of respondents as “multi-cloud maestros” capable of employing multiple clouds to smoothly and routinely deliver one or more business functions. The survey found almost two-thirds of multi- cloud maestros report they employ cloud management platforms.

    The survey also found that multi-cloud maestros are advanced in terms of adoption of microservices and containers. A full 61 percent of multi-cloud maestros predicted that within three years at least 80 percent of new applications will be developed using containers. Currently, 50 percent of multi-cloud maestros are using Docker containers and 63 percent are using containers as a service (CaaS). By 2021, 65 percent expect to use container orchestration. More than half (56 percent) of the multi-cloud maestros are already microservices and 59 percent plan to use serverless cloud services in their cloud environments, according to the survey results.

    Dolph said IT organizations in general would be well-advised to focus on more than just “lifting and shifting” applications into the cloud. They may need to leverage Docker containers to move an application into the cloud. But once there, an effort should be made to modernize that application by turning large swaths of it into a set of microservices that ultimately are easier to maintain and consume by developers building complementary applications, he said.

    The IBM survey makes it apparent that multiple clouds are the new IT normal. Almost half of respondents said they are establishing a multi-cloud architecture to develop new and enhanced products and services. A total of 62 percent of respondents require multiple clouds to create innovative business models, while 52 percent said they need them to produce new revenue streams. Sixty-six percent need them to enhance margin and 46 percent said they need a multi-cloud environment to support agile business processes.

    From a DevOps perspective, more than half (51 percent) of respondents use multiple clouds to cultivate a flexible, modular infrastructure that quickly absorbs and leverages technological advances.

    Obviously, one of the great paradoxes of multi-cloud computing is that in the pursuit of becoming more agile, many organizations are making their IT environments more complex by employing multiple platforms. The challenge they will face next is maturing their DevOps processes to the point where some order is finally brought to that multi-cloud computing chaos.

    — Mike Vizard

  • Log Management: What DevOps Teams Need to Know

    Log Management: What DevOps Teams Need to Know

    One of the most important shifts of this decade is the rise of interconnected data across distributed systems. This explosive growth not only sparked the need to rethink cloud and global IT strategies, but it also disrupted traditional development, DevOps and ITOps practices. These changes are pushing requirements for higher-velocity development of features and products—together with more responsive support requests—onto technology professionals. These new requirements have prompted teams to evaluate new tools and agile processes that help support the needs of a competitive modern business.

    Regardless of the pace of change within your group, basic functions such as infrastructure monitoring are crucial to operating production applications. At this new speed of business, concepts such as observability are important for enabling successful deployments with fewer user interruptions—not only in production, but also in the early stages of your continuous integration/continuous delivery (CI/CD) pipeline. As part of “shifting left,” log and event management in your CI/CD pipeline empowers developers to monitor and observe application behavior before releasing it to production. While some might say this adds additional layers of process, time, effort and due diligence in dev environments, it can help you evolve your organization’s fundamental software development behaviors while reducing preventable issues from post-deployment into production. It can create a smoother, more seamless user experience and reduce the need to fight fires or rearchitect your solution after the application is live in production and at scale.

    Setting the Testing and Dev Stage

    The rapidly evolving technology landscape has increased the need for log management and observability across distributed systems and containers. Changes to how applications and services are created, the ability to deploy applications across combinations of global logging-as-a-service (LaaS) providers and the pervasiveness and volatile lifespan of containers, along with the capacity to build services using various development languages, has increased the need to collect, monitor and trace data points across connected systems, which provide critical end user-facing value.

    Traditionally, many considered log management a tedious task because each instance of a new server required you to run a search command across the logs local to that server. That approach isn’t scalable when every cluster of more than 20 servers is built up in minutes, and each application has independent logging formats. In this scenario, you’d need to run 20 individual searches across multiple systems, compare timestamps across servers residing in different time zones and so forth. With the decreasing cost and increasing prevalence of containers and virtual systems, the number of systems running within a typical organization has grown exponentially as the business prospers.

    In addition to increased usage of virtual systems and containers, independent scrum teams of developers can build in parallel using technologies which are the best fit for the services they are producing, while also sharing environments for builds, staging and production. With these distributed services, aggregated log systems can quickly determine the root cause for debugging their application’s errors, exceptions or performance bottlenecks. A log aggregation system can reduce the element of “personalization” that can often happen when developers are using different technologies with various logging formats, and also reduces the noise by filtering out information that delays the team’s ability to find the source issue. This can cause confusion or slowed productivity when one developer is left to understand another’s non-standardized, yet shared, workload.

    The reduced cost, increased collaboration and prevalence of container services are collectively driving an uptick in virtual environments and containers, often requiring more logs to be created in testing and dev environments, which can force organizational shifts to enable full observability across their infrastructure.

    The 3 Pillars of Observability

    Businesses are in the midst of a shift toward adopting these principles—the three pillars of observability—and using a three-pronged method of monitoring that takes data from several vantage points to depict a more accurate and consumable view of overall health and stability:

    1. External Monitoring: An example of this is health checks run against your internal and external applications and websites to see the “digital experience” of your users.
    2. Metrics and Distributed Tracing: This enables you to trace communications between applications distributed across systems/containers to identify errors and exceptions from your applications and resolve latency quickly.
    3. Events and Logs: Data, which helps provide contextual information about events, enables you to identify issues in the code when combined with information from the first two pillars.

    The above types of information are critical to DevOps to provide elastic and resilient services. Bringing application, system and infrastructure events into logs together with external monitoring and metric collection is quickly becoming the standard. With powerful log-parsing features such as live-tail search accessible from a web browser and the ability to visually pair log insights with added health metrics, the real value of log data is made apparent with little effort.

    Site reliability engineering (SRE) is another area where significant evolution has occurred. ITOps has always been asked to react to fire and run from fire to fire. The evolution from ITOps to DevOps to SRE is forcing the alignment of different types of data to be used very early in a CI/CD pipeline, and enabling SRE teams to get involved in architectural discussion at project inception instead of in the latter stages or at release. The SRE is more involved in monitoring, understanding the development and architectural principles used to develop application services and build in automation, and high availability and circuit breaker patterns from inception.

    This approach has allowed teams to be proactive and responsive rather than reactive. It has equipped our teams (which keep our business running successfully and our customers loving what we do) with the ability to focus on being scalable, elastic and resilient. Instead of troubleshooting systems in production, we auto-scale and replace to minimize disruptions and investigate that incident offline when possible.

    Best Practices

    Given the increasing pace of application development required to support the needs of the business, the spread of containerized applications and virtual environments and the emergence of new disciplines for observability (due to the lowered costs and increased reliability), log and event management has become a critical aspect for all involved in building, supporting and even using mission-critical applications. It’s moving to a defined area where logs should be compartmentalized, and access control should be defined:

    1. Compartmentalizing Logs: Critical as a best practice so we can differentiate development, staging and production environments and segregate logs based on practical grouping. Using a log aggregation service provides you with a consistent experience and a common set of capabilities across all your environments.
    2. Access Control: Another best practice that should be implemented more broadly; for example, developers don’t need access to all of your production logs if approximately 95 percent of their work happens prior to reaching production environment. In other words, even though tech pros are working to “de-silo” log management, we still want to silo access control and manage visibility. This can also help to ensure personally identifiable data (PII) isn’t exposed to anyone it shouldn’t be.

    Looking Toward the Future of Log Management

    As the landscape evolves and best practices become more defined, we should expect revitalized strategies with elevated expectations for the traditional scenarios where basic/free monitoring is leveraged. While the pillars of observability are leveraged more broadly in the ITOps and DevOps communities, we’ll be able to move more efficiently from an end user-facing incident—for example, an application error traced across distributed systems—straight into the logs to find the specific error causing the issue. We’ll be able to see greater correlation between logs and other custom metrics, as well as much broader use of these techniques outside of production environments.

    DevOps teams need an intuitive event and log management system that is as de-siloed as we are. No longer can log management systems be relegated just to one on-premises storage system or one virtual environment; they need to fit the reality of how work is increasingly done today and will be in the future.

    — Keith Kuchler

  • DevSecOps: Critical in Building the Bank of the Future

    DevSecOps: Critical in Building the Bank of the Future

    It is no secret that change in a large organization can be difficult. This is often true within banking, an industry that often has sought predictably instead of agility in software delivery. However, with the right tools, guidance and teams, creating a bank that can be both complaint and responsive may be more attainable than initially it may seem. For a digital bank of the future to emerge from a heritage of independent application, information security and operations departments, an organization should embrace new technologies that can unify these groups and improve the ability to respond to change more quickly.

    Read on to learn why the inclusion of information security within DevOps practices can play a key role in establishing a bank of the future, and why it is important.

    Digital-Only Banks Gaining Ground

    One of the shifts in the distribution of banking products is the movement from primarily branch-based banks to digital-only banks. In fact, Citizens Financial Group launched a digital-only bank in July of 2018 to appeal to a growing number of digital-first customers. JPMorgan Chase also has a new digital banking offering, Finn, aimed at millennial customers.

    While digital-only is meant to simplify the end user experience, it complicates security because it increases the number of attack vectors within the digital channel. In effect, bank security is judged not by how well physical transactions are protected within the branch (think bank notes and checks), but how well digital transactions are protected over the internet. This can require a commitment of applying security controls while also honoring agile delivery practices. In turn, this developmental agility can be more effective when it is part of a responsive organization—one that is willing to adapt and adjust quickly, often requiring institutional culture change to an organization that is more transparent, open and ultimately innovative.

    Incorporating Security from the Get-Go

    To be more responsive while at the same time hardening security, adapting to DevSecOps methods is becoming important in the journey to be a bank of the future. DevOps should never be only about the development (Dev) build and operations (Ops) implementation—it should have security in the middle (Sec). Information security should play an integrated role in the full life cycle of apps. DevSecOps can remove the isolation and boundaries that previously existed between the security team and the rest of the organization, and it brings security smack dab into the conversation, rather than as a part of a deployment checklist. With service level expectations from release cycles going faster—from months to weeks or even days—integrating and automated security into the delivery process can be even more critical. In a DevSecOps application methodology, digital bank initiatives would build security into every touch point from the very start.

    How is Information Security Evolving in Banking?

    Banking, like all other aspects of our lives, seems, to be  transitioning to a smartphone transaction. Deposit and savings accounts can now be fully opened and closed through the digital channel. Information is exchanged digitally and the process is often painless. Information security becomes even more critical to help make  information safe and secure from potential fraudsters.  So while these types of transactions are often expected by consumers, it is important to have proper security measures in place and that can be fully automated and throughout the development process.

    The Organizational Change to Embrace the Future of Banking

    The future of banking will not happen exclusively just by changing technology and tools. It can often be successful when it is adapting organizational change at the same pace as technology and Agile best practices, such as DevSecOps, can also be applied to cultural transitions. To prepare organizations to be successful, there should be clear commitment by leadership across applications, operations and information security; otherwise, the change may never be fully absorbed into the organization. The bank of the future will be more likely to be successful if it functions as part of an ecosystem of interdependent and more securely connected people, processes and technology to have the structural support in place to successfully engage.

    — Eric Marts

  • How to Fix ‘Dark Debt’ in Your IT Organization’s Culture

    How to Fix ‘Dark Debt’ in Your IT Organization’s Culture

    Poor decisions, bad metrics, lack of communication can accrue dark debt in any company

    Buying a low-quality microwave meal from a gas station, kicking a sock under the bed before guests arrive, jaywalking … We all take shortcuts. But unlike enterprise software delivery, these decisions rarely cause any collateral damage. The ramifications of taking shortcuts in IT, however, are much more serious.

    These decisions—i.e. “technical debt,” the “rework” debt owed after opting for a convenient solution instead of a better approach that takes longer—has implications on your company’s ability to grow and adapt to change. Fortunately, since the 1990s, when the term was coined by Ward Cunningham, we’ve learned how to identify and manage most forms of technical debt, understanding how to limit its impact and manage the interest.

    Your organizational culture can accrue debt, too. For example, a company that finds it easier to hire a homogeneous team up front might discover the costly cultural trade-off of hiring diversity later. When it comes to this “cultural debt,” we are much less disciplined. We often make decisions in the dark, without understanding how they are going to play out over the long term. “Dark debt” builds up over time and creates a toxic work environment in which people are not set up for success.

    Common issues indicating dark debt include:

    • Failures lead to finger-pointing.
    • Metrics incentivize undesirable behavior.
    • Lack of candor.

    Ways to Combat Dark Debt

    Call Out Psychological Safety Indicators

    Failures are inevitable, whether we are discussing power outages, data breaches or false missile alerts. A “safe to fail” mentality promotes learning and support adaptability. If employees don’t feel safe telling the boss bad news, the latter might be the last person to know what’s going on—not a good situation for a company working to disrupt an industry, or one expecting to stay listed on the Dow Jones.

    A Net Promoter Score-like survey can help identify areas where employees experience dissatisfaction. Here are some sample statements for your NPS survey that are based on Westrum’s organizational culture model as introduced in the “2014 State of DevOps Report.”

    • On my team, failure causes inquiry and not blame.
    • Our leadership is open to hearing bad news.
    • In my organization, failures are learning opportunities and messengers are not punished.
    • People on our team trust one another.

    The survey results will shine a light on the forces shaping your culture. Instead of finger-pointing, you can learn to address issues earlier and ask questions such as, “Why did it make sense for someone to do that at that time?”

    Measure Thoughtfully

    Tell me how you’ll measure me and I’ll tell you how I’ll behave. Gaming metrics are fairly easy to do, especially when an employee is incentivized to hit an unrealistic target or avoid embarrassment. Take activity metrics, for example (lines of code, number of bugs): These metrics focus on keeping people busy. But busyness does not equal business value delivered. People can be 100 percent utilized sprinting from meeting to meeting while making scant progress on business goals.

    Instead, consider metrics that reveal progress toward important business objectives or ones that help you make better business decisions. Flow metrics such as flow load, flow time, flow efficiency, and flow velocity reveal trends that measure four key areas: workload, speed, waste and throughput (respectively). These metrics are helpful in numerous ways, such as:

    • When others imply that things take too long, teams work inefficiently or people aren’t fully utilized, it’s useful to test their opinion against the data.
    • Having a balanced set of metrics allows you to see how changes in one area impact other areas. If the amount of work-in-progress (flow load) goes up, things will take longer to do (because of context switching and conflicting priorities), which results in slower speeds (longer flow times).

    Think about the behaviors that occur in your organization by how metrics incentivize people. What we measure impacts people because people value what is measured. Think twice before automatically defaulting to free and easy metrics available in your existing toolset.

    Build Social Capital

    Ideas start out flawed and incomplete. High levels of social capital, debate and discussion are the means by which ideas flourish. Social capital is what enables people to collaborate more effectively. The absence of social capital makes it impossible for people to speak openly (due to the lack of trust), resulting in withheld feedback and invisible work.

    Not much happens without discussion. When we engage in debate, our capacity to see each other’s perspectives is realized. TED talk speaker Margaret Heffernan’s insight on social capital is compelling: “Time compounds social capital.” The longer teams work together, the more trust, helpfulness and candor they accrue. Longstanding teams, therefore, benefit from increased productivity and reduced risk.

    Unlike the costly and time-consuming transitions observed in project-managed work, where people disband after dumping their work on the operations team before starting another project, organizing teams around a product allows the people who developed, tested and delivered the functionality to not only stay in their area of expertise, but also grow and gain from social capital.

    — Dominica DeGrandis