staging-devopsy.kinsta.cloud

Category: Features

  • DevOps and Retail: Transforming Brick-and-Mortar to Brick-and-Click

    DevOps and Retail: Transforming Brick-and-Mortar to Brick-and-Click

    Brick-and mortar-retail is rapidly transforming with the digital age. As physical storefronts turn digital, it’s safe to say that traditional retail is quickly becoming a mesh of “brick-and-click.”

    So, how are retail giants surviving in this new landscape? Interestingly enough, the remaining brick-and-mortar retail are reinventing themselves by applying helpful DevOps principles.

    In this post, we’ll peek into cases of innovation and horror stories from historically brick-and-mortar retail chains and brands. From Target, to Balian Group, Sears and other companies, we’ll discover the importance of DevOps in this new brick-and-click multi-channel retail environment.

    Target

    In an effort to increase agility, Target engineering teams embrace something they call “the Dojo”—immersive, six-week training sessions with hands-on DevOps and Agile coaches. As Target CIO Mike McNamara recently told the Enterpriser’s project: “The Dojo has been fantastic in getting teams engaged with agile and DevOps, removing the natural resistance and fear of change, and then supporting the team through the changes while maintaining productivity. It’s been a huge success for Target.”

    The result of these sessions is increased comfort with methods including containerization and tools that enable continuous integration and continuous deployment. McNamara also noted their grassroots approach; they encourage open source activity and listen to their engineers when they petition the use of new tech.

    “Our engineers have fully embraced DevOps,” he said.

    Bailian Group

    For another case study, take Bailian Group, a large Shanghai retailer that operates 6,000 grocery and department stores. By the time Bailian entered the online marketplace, it suffered from a severe lack of agility, non-native web apps and underutilized server assets.

    “Our transition from traditional brick and mortar to omnichannel business presented a great opportunity but an equally large challenge,” noted Lu Qichuan, director of IaaS and Cloud Integration Architecture at the Bailian Group.

    Qichuan and Bailian realized that to stay relevant, they had to cut long development cycles and embrace “new cloud application development tools and processes that foster a CI/CD and DevOps culture and increase innovation and time-to-market,” he said.

    They ended up implementing a private cloud using OpenStack. Now, midway through their cloud transition, they estimate to have 10 million virtual machines by the end of 2020.

    Sears

    While the previous two examples had positive outcomes, digital transformation has not gone over well for other retailers. Sears declared bankruptcy in October 2018.

    In its final years, Sears was in a sticky situation. To the chagrin of former CEO Eddie Lambert, surveys found their customers preferred to shop in stores as opposed to online. Disregarding customer desires, however, Sears attempted a big data play with its “Shop Your Way” program, under which Sears gathered user data to then sell. The program floundered, resulting in consumer irritation and loss of sales as reported by Business Insider.

    Writing for InformationWeek, Emily Johnson reminded us that evaluating DevOps using the wrong metrics will ultimately lead to failure, citing customer satisfaction as the No. 1 goal.

    In its final years, an abrasive Sears leadership halted any sense of DevOps to flourish. A disengaged Sears CEO Lambert created “a climate of fear,” according to Busines Insider reports. It appears the retailers that stay afloat encourage agility and intrapreneurship—this means encouraging innovation, not fear.

    DevOps Aids the New Omnichannel Retail Benchmark

    Brick-and-mortar are evolving into omnichannel retailers that bridge physical and virtual spaces. Take IKEA’s augmented reality app, which visualizes items in a house. From Disney, to Startbucks, Nordstroms and elsewhere, physical storefronts embrace digital offshoots of their core model to augment the traditional retail experience.

    Surprisingly, physical storefronts aren’t completely dead. They still offer conveniences for urban environments with high walking scores. See the emergence of Amazon physical locations for ironic evidence of their benefit. Other storefronts and pop-up shops have taken on a museum-like quality. The flagship Funko store, for example, creates impressive displays and interactive exhibits, pulling in flocks of weekend shoppers that want more than a dull e-commerce shopping cart.

    DevOpsGroup similarly describes a “reverse showrooming” movement; consumers research products online to then purchase in person. This enables brands such as ASOS, which is emphasizing social media and free worldwide deliveries, to stay competitive.

    “Retailers have been forced to re-evaluate their business strategies and understand how new technologies will affect their offering,” according to the DevOpsGroup website.

    DevOps Turns Bricks to Clicks

    Online shopping has sufficiently disrupted retail, and brick-and-mortar must reinvent themselves to stay relevant. An open-minded leadership and a commitment to fast development, dynamic scaling, uncompromised availability and open source technology underpin a successful evolution.

    Of course, this increased reliance on technology necessitates a hardened look at how development and operations communicate, and brick-and-mortar retailers must adopt the behaviors of tech companies to retain agility.

    This may come in many forms: new user-facing digital experiences, microservices for composability in niche omnichannel environments, containerization to package code that scales well, distributed data stores, heightened user data security or smart incident management techniques, to name a few.

    Traditional retailers, then, should dissuade their doubts against agile. DevOps can aid retail by reducing redundancy, cutting costs and streamlining production and distribution.

    — Bill Doerrfeld

  • Upskilling: The Enterprise DevOps Skills Survey and Report

    Upskilling: The Enterprise DevOps Skills Survey and Report

    There has been a lot of recent reference to “T-shaped” IT professionals—individuals whose core competence (the stem) is supplemented by a broad range of other knowledge including soft skills (top of the T). T-shaped talent is in such high demand with a limited supply that recruitment efforts have become very competitive for those with upskilled and cross-skilled transformational know-how.

    Unfortunately, the migration from a deeply skilled specialist to a T-shaped practitioner is leaving many confused about what skills they should be acquiring and hiring and when. That’s why DevOps Institute (DOI) recently launched the 2019 “Upskilling: Enterprise DevOps Skills Report” community survey to help individuals and enterprises better understand the current and future state of DevOps skilling across the global IT community. Using data science coupled with hiring manager interviews, DOI’s goal in fielding the report is to provide tangible insight into why and where IT practitioners, managers, leaders and business stakeholders should focus their efforts in growing personal and organizational skills portfolios.

    Eveline Oehrlich, a former vice president and research analyst at Forrester, is bringing her data science expertise as the lead analyst for the report. Founding Platinum sponsor Electric Cloud and founding Gold sponsor Cloudbees are also sponsoring this important project. The “Upskilling: Enterprise DevOps Skills Report” will be a year over year project, and the results will be published in Q1 2019.

    The global survey will remain open until Dec. 7 and can be accessed through the following link: https://www.research.net/r/K9WD2XC

    We encourage everyone in the DevOps community to complete the survey so that the data samples truly reflect the global state of DevOps skills.  Any organization that has at least 20 people complete the survey will get a personal snapshot of the data results. All respondents who complete the survey will be entered into a drawing for four $100 Amazon gift cards.

    — Jayne Groll

  • DevOps Chat: Instrumentation with Influx Data’s Evan Kaplan

    DevOps Chat: Instrumentation with Influx Data’s Evan Kaplan

    Evan Kaplan, CEO of Influx Data, is someone who knows his way around the technology world. In a lifetime of experience that has seen him found and manage several successful companies from startups to public entities, Evan has had a great track record of success.

    Influx Data is yet another success for Kaplan. Building on the open source InfluxDB, the company boasts more than 200,000 users of Influx today.

    In this DevOps Chat, Evan and I discuss systems instrumentation today and what big data and velocity means for developers and everyone up the stack. It is a very different world from just 15 or 20 years ago.

    As usual, the streaming audio is immediately below, followed by the transcript of our conversation.

    Transcript

    Alan Shimel: Hey, everyone, it’s Alan Shimel, staging-devopsy.kinsta.cloud, and you’re listening to another DevOps Chat. Happy to be joined in this episode of DevOps Chat by, actually, someone who grew up in my neighborhood. We knew each other – well, we didn’t know each other way back when, but we’ve come from a very small area of concentrated – lot of people, you know, we know from that area, but, anyway, Evan Kaplan, CEO of InfluxData and of North Woodmere. Evan, welcome.

    Evan Kaplan: [Laughs] Thanks, Alan. Thanks for having me. Look forward to _____ –

    Shimel: My pleasure. So, Evan, it’s been a long, strange trip from North Woodmere. Why don’t you give our audience a little bit of your background and your journey? And  then we’ll jump into –

    Kaplan: Yeah. Sure. Sure. Very long for me. If those of you don’t know, that’s a suburb of New York City, right near Kennedy Airport, that we both grew up. My strange trip took me to Seattle, where I finished up college and worked as a climbing guide for years. And then worked in aerospace and started a company, the first company to develop SSL-based VPNs, called “Aventail,” back in the mid-90s. _____ through the bubble. Ran a public company called “iPass,” and, about two-and-a-half years ago, met Paul Dix, the founder of Influx. And Paul and I really connected about his vision and what he was doing, and we partnered together to start building Influx from 25 employees back then to north of 100 now. So growing fast.

    Shimel: Good stuff. Good stuff. And, Evan, I’m gonna assume most of our audience has at least heard of InfluxData, but, for those who have or even those who haven’t, just to set the level-set, tell us about what InfluxData does.

    Kaplan: Yeah, so InfluxData is an open-source time-series-based data platform. It’s the leading time-series platform out there. Time-series is the fastest growing market in the database, data category. And so Paul, our founder, spun up Influx back in 20 – I think it was late 2013. It received a lot of accolades on GitHub. It was a very popular project and it has grown dramatically since then, from roughly – I don’t know – 1,000 sites running it to, today, we’re approaching 200,000 sites running it.

    Shimel: Wow.

    Kaplan: And, you know, most of the large Global 2000 is running it somewhere, so.

    Shimel: That’s really impressive, man. That’s something to be proud of. So, Evan, we could probably spend our whole 20 minutes talking generally about Influx DB and what’s happening, but I’m wanted to kinda dive in a little bit and talk specifically today around instrumentation in modern software installs, in the modern software factory. This whole term “modern” has become sort of a catchall buzzword, dog whistle, if you will, for when we talk about  things like Agile and DevOps and serverless and microservices and all of these ways that we’re seeing software being written, managed, and deployed today. So talk to us a little bit about it, if you will, Evan. Where’s your take on this?

    Kaplan: Yeah, so I think we’re – we grew up in the industry, going back a couple of decades ago, the common myth for monitoring or instrumentation of systems was you would buy a package from HP OpenView or CA or a mainframe management package, and you’d buy a monitoring-for-a-service package. They were expensive and they were largely horizontal across the stack, and you’d implement those. And what’s happened over – certainly over the last ten years, during the DevOps revolution, is that stuff has fractured and you’ve seen companies, like Splunk and Elastic and a variety of other folks, arise to handle the modern workloads in an open-source sort of fashion. And now, if you take a look a look at DevOps world, most of solutions are dominated by open-source kinds of models.

    If you look at state of the art now, that people are really paying attention to, the revolution that brought you to logs and log views of stuff and the consolidation of logs has moved much closer to metrics and events. And what I mean by “metrics and events,” the instrumentation of real time of these complex systems. And they’re not just software systems that are DevOps things, so – those are obvious, but they’re also hardware systems; they’re IoT things. And so what’s happening is people wanna see these complex systems and they wanna interact with them in real time because the only way to characterize these systems is to watch them behave in real time. And “observability,” as you know, Alan, has become sort of a keyword here. And observability generally means real time and it means metrics and events; it doesn’t mean verbose logs.

    And so what we’re trying to do is help developers, specifically and, up the chain, architects and CTOs, build instrumentation or telemetry natively into these complex systems, so they can view these things in real time. As you can guess, the data loads with this are absurdly high. You’re talking about hundreds of millions of points a second in some of these systems and you’re talking about to visualize those in near real time, you know, control loops in near real time. And you’re talking about this sort of next generation of, lack of a better way to say it, system instrumentation and management.

    Shimel: Sure. Evan, if I could stop you there for a sec, you know what I find interesting? We’re talking about building this instrumentation and to give people insight and who are we giving insight? Well, you mentioned developers and managers and architects and CTOs, et cetera, but you didn’t mention ops. And is that –

    Kaplan: Yeah. Yeah.

    Shimel: Isn’t that a little bit of, you know – I don’t know if it’s Freudian or if it’s just insight into what today’s world looks like for these things.

    Kaplan: You know, that discrete ops, right? That discrete ops role is changing, right, into what you obviously cover, which is this DevOps. And so our view of how our – uptake of our platform into large corporations, companies like IBM and Cisco and SAP and Tesla and PayPal, has been from developer or DevOps people up. So we tend to think of “Make it super easy for developers, right? Get their time to value down to really, really minimal and then it will rise up and be taken as a complete operational system.” So we tend to think of it from developers up. Not to say the operations guys aren’t relevant – they’re super relevant – it’s just, when we enter into a large opportunity or business or that sort of stuff, it’s usually coming from a developer who figures out he can instrument his code, his service, his offering relatively quickly in Influx.

    Shimel: Got it. And I’ll tell you something I – and it goes back to what you mentioned earlier – the sheer volume and the acceleration of data, right, the – so, no, it’s how much data and the speed that it comes at us. With automation and everything that that brings, it easily outstrips the ability of a human, a lot of times, to stay on top of it, so you almost need to build automation, machine learning, AI into these types of instrumentations, that you bring what the human really needs to see and touch or make a decision on to his attention ’cause so much of it is just whizzing by right now.

    In the last week or two, Evan, I’ve probably had a couple of discussions – trying to think of a friend of mine who – CEO at ScienceLogic and another company called “Ravel Systems”; some friends of mine in security just launched something called “DisruptOps.” And they’re all talking about this term, “AIOps.”

    Kaplan: Yeah. Yeah.

    Shimel: And I’m wondering is this something relevant to what you’re talking about, with instrumentation? Where does that fit in? How do we account for just the big data nature of this?

    Kaplan: Yeah, I think, first of all, absolutely. So, if you think about – so the journey. And let’s call it the end state of all of this is true autonomy, which is very rare and, in fact, almost never the case, right? But, if end goal state is true autonomy, what’s that journey look like? It looks like, first and always and most importantly – and, by the way, the stuff that we skipped a lot, when we were in the early stages of our career – it is, in fact, instrumentation, is telemetry. Telemetry always was an afterthought before whether shipping a product or a service or that sort of stuff. Now the ability to drive that cost of instrumentation down so low that the ability to see what’s going on and create the data – not just see as in “human” see but create the data – has changed dramatically.

    So, if you follow that journey, you go from instrumentation; you go to observability; you go to automation in the form of “I’ve observed these sorts of things, whether it’s through AI capability or not. I built control loops because I’ve observed them, so I’ve built control loops to keep it in a relative stasis. Right? To autonomy.” All of that – the last two steps, right, even the last three steps – involves some form of machine processing of learning, whether it’s machine learning or AI or call it, but all of that gets increasingly sophisticated. But it means taking in, as you sort of articulated, you know, millions, hundreds of millions – in some case, billions – of points per second, right?

    Shimel: Yep.

    Kaplan: Think about it. Now that’s just completely – you know, what a different world, than doing server management with your CA Unicenter product 20 years ago.

    Shimel: [Laughs] To say the least.

    Kaplan: Right?

    Shimel: Yeah, I hear ya. I mean, it really is. If we transported – you know, if we could do that – if we could transport someone from not that long ago, Evan, an hour or _____ _____ –

    Kaplan: No. No.

    Shimel: From 1997, ’98, right? I was helping an early ASP, Interliant. We were building – opening data centers, managing Lotus Notes and PeopleSoft and stuff like that.

    Kaplan: Right.

    Shimel: If I could take a NOC guy from there and show him today’s velocity and the fact that we actually do have telemetry and we actually are building tools that allow us to manage at that level, it would be like taking someone from colonial times and showing them highways with cars. I mean, the supports –

    Kaplan: But, Alan, these solutions are still sitting out there. They’re a huge _____ _____ _____ –

    Shimel: No – well, that’s the scary thing. Yeah, I mentioned my friend, ScienceLogic, my friend Dave Link. He was saying that’s his – you know, and they’ve done very, very well, and his thing was just replacing the Tivolis and the Unicenters and that generation, if you will, of network management, of systems management.

    Kaplan: If I could tell you what, back in the late ’90s, is I was building an ASP also, based on the Aventail, the SSL VPNs, what we were standing on Netcool, back in the day, and things like that, _____ _____ –

    Shimel: And Netcool was a new program, right?

    Kaplan: Yeah.

    Shimel: Before IBM bought it, right? It was one of the cool, new kids at the time.

    Kaplan: Exactly. If I could tell you what somebody could take with Influx today and do, relative to what Netcool did back in the day, and the difference, right – first of all, they could do it for nothing. Forget paid licenses. And the amount of scale, the amount of data collection, and the ease of sort of visualization has dramatically – and all those _____ _____ –

    Shimel: It’s phenomenal. Phenomenal, man. It’s just crazy. So, Evan, let’s bring this back to Influx. How does all of this now manifest itself with your business today?

    Kaplan: Yeah. So that’s right. So we look at our businesses in two segments, right? We look at it as the instrumentation of and the collection of data around software, being the platform of record for this next generation of cloud software. And I don’t have to describe to you is is that, if, in the old days, it was an X Windows terminal talking to a Unix server app, today, there is no “there” there. There’s a virtual network; there’s VMs, containers, and microservices that are growing at exponential rate. The whole infrastructure is ephemeral, so the monitoring and the instrumentation workload has dramatically changed. So one side of our business is really focused on that, that opportunity, and we’d call that the “DevOps chain” of the business, right? Because it comes up into organizations through DevOps folks who bring in Influx and instrument.

    The other side is IoT, which is completely different. So, while DevOps is largely metrics-driven, IoT is largely event _____ driven. And that’s the instrumentation of the physical world, right? So our watches, our clothing, our health care, our cars, our homes, are all being instrumented with sensors. The amount of data pouring in through that is driving whole other systems into business. And, while most of our time was spent on the management of software systems back, now it’s this deep, deep integration of software and hardware. Deep. And so our business breaks out into those two categories, largely.

    Shimel: Got it. Got it. Evan, anything – any new news coming out of Influx you wanna share with our audience or –

    Kaplan: You know, where – we have – you know, we’re almost constantly releasing stuff into the community on a regular basis, so no news that I’d share at this moment. We can touch base in three or four weeks. We have our big InfluxDays event. We’ll make some announcements there, but you can look for us to build increasingly sophisticated solutions that allow people to do their machine-learning workloads, their application tracing, their log and metrics, their asset management, on top of Influx. What we want is people to use us as the platform of record for all of these metrics and events pouring in.

    Shimel: You know, we live in interesting times and I can’t wait to see where this all goes as well, so we’re about out of time, though, here. As I mentioned to you, the 15, 20 minutes goes so quickly.

    Kaplan: Yeah, it really does.

    Shimel: And you barely ever – you know, it is what it is. But, Evan, thanks for joining with us. Evan Kaplan, CEO, InfluxData. You know, so, in three, four weeks, you have your event coming up; maybe we’ll have you back. We can talk a little more about that. We’re doing a big – I didn’t even announce it yet, but I’m gonna announce it right here on the show with you – we’re gonna be doing a big virtual event, December 18th, called “Predict 2019,” and we’re gonna –

    Kaplan: [Laughs]

    Shimel: Yeah. We’re gonna be opening up the crystal ball and asking a lot of pundits and experts and so-called experts what their predictions are for what 2019 has in store. Maybe we can have you back then and we can do some crystal-ball gazing.

    Kaplan: That’d be great, Alan. Always a pleasure. Always a pleasure.

    Shimel: All right, my friend.

    Kaplan: Have a great day. All right.

    Shimel: Well, Evan Kaplan of North Woodmere and InfluxData, thanks for being our guest on this episode of DevOps Chat. This is Alan Shimel for staging-devopsy.kinsta.cloud, and you’ve just listened to another DevOps Chat. Have a great day.

    Kaplan: Take care.

    — Alan Shimel

  • DataOps: DevOps Plus Big Data

    DataOps: DevOps Plus Big Data

    In traditional DevOps, there are the complimentary forms of development operations (that I call DEVops), and development operations (that I like to call devOPS). Between them they automate the toolchain and bring the people working on getting an application out to users onto the same team—not necessarily the same organizational team, but using their strengths in tandem to meet the needs of the business.

    DataOps is an interesting extension of DevOps. You still need all of the coding side to get data into the system and queries consistently maintained. You still need all of the operations side to get your database/noSQL/whatever up and running. In fact, while the development side in a DataOps environment might be (but isn’t necessarily) lighter, the operations side is almost always more complex. No matter the big data engine in use, it is a complex system in addition to other supported systems in a normal environment. My first installation of a big data environment (Cloudera, as it happens) was a weeks-long learning voyage. Only after I’d completed it did I use an automation tool (which is no longer available) to make it easy. My second round was hours … But it assumed the knowledge I had gained in the first attempt.

    The other bit of DevOps—ongoing monitoring and management—is also more complex in a big data environment, but we’ll come back to that in a moment.

    Once the normal DevOps systems are in place, ETL/data import tools will need to be supported also. The volume that these tools crunch through in a day makes their inclusion in DevOps in a data-heavy environment critical. If the data uptake is slow or the data itself inaccurate, there is an impact on the organization. This step also requires inclusion of data scientists, people who traditionally have not been pulled into the DevOps model. But they are the ones that can gauge the accuracy of data, and normally are responsible for data acquisition anyway.

    Which brings us to the monitoring and management stage of DataOps. Normally when we talk monitoring and management in DevOps, we are talking about tools that ultimately help with things like availability, responsiveness, auto-scaling and recovery. DataOps needs all of these things, like any other application does. And we’ve already mentioned that it has a complex environment embedded in your complex environment. But it also (some would argue more importantly) needs data monitoring. A knowledge of the state of the data is imperative. Whether from a piece of code gone wrong or data poisoning, monitoring the health of big data archives is essential to keep results from getting out of whack. This monitoring will again involve the people who know the data—data scientists by whatever title. This iterative process will need to run relatively constantly, checking on whatever makes the data valuable, normally using cross-field checks to verify that data hasn’t somehow been corrupted, or comparing trending averages to current average to monitor for sudden swings in results.

    Of course, for a system (and in research there are many) that spins up, finds results and spins down, the monitoring part is less essential, but for anything that keeps accessing and adding to the data pool, it is the key to a healthy application or application portfolio based upon the dataset.

    If all that sounds a bit intimidating, you’re not alone. The phrase “DataOps” has been around for a few years, and while data scientists use it, it hasn’t yet gained serious traction in the enterprise. Here’s how I would approach it as CIO: Given that DevOps is running on other projects, that is the low-hanging fruit. Given the rate of change for servers (as queries—particularly long-running ones—increase and decrease, systems needs change is one example), traditional DevOps is a greater benefit to big data than many other corporate systems. So roll out DevOps first without serious concern for DataOps beyond how DevOps impacts ETL—not ETL data quality, but ETL data availability/performance. Only when the system and the team are in place and the software/hardware/code part of the overall system is fully in a DevOps environment, start looking at ways to move data into it. Data monitoring will be the big one, but ETL under the initial round of DevOps implementation will have shown places it could be improved and ETL data integrity can be added, so improving those areas also. A measured, step-by-step approach that builds off of organizational experience is key here. Data monitoring and management will require different tools than normal DevOps monitoring and management, so having the data scientists look into available toolsets and how they mesh with existing tools early on will help with eventual implementation.

    Make no mistake, it is a big job, moving large data driven environments to DevOps. But the benefits are increased awareness of data quality and data quality issues, along with more responsive systems and teams that DevOps offers every business application. The amount of data available to any give organization is currently on a massive increase as IoT, open datasets, partnerships and ML drive greater pools. It is worth getting the automation and cooperation of DataOps into the mix as early as possible.

    — Don Macvittie

  • Education in the Age of Automation

    Education in the Age of Automation

    Automation is changing the nature of education and the ways people learn. More schools are using online learning to supplement in-class activities. In fact, some schools are completely online. Distributed computing on the internet and the automation that drives it makes this possible.

    Everything changes when the formal education process becomes embedded in the internet. Courseware delivery becomes asynchronous; people learn what they want, when they want to. Test administration becomes more about the efficient acquisition of data and evaluation thereof. Human-to-human interaction as a basis for a meaningful educational experience becomes optional. Today, it’s entirely possible to learn all you need to know to make a good living in the modern world without ever needing to sit in a classroom or interact with a human directly. Let’s face it: You can do a lot of learning from YouTube.

    As a result, more people are beginning to question the conventional wisdom of spending four years and thousands of dollars to get a bachelor’s degree—particularly now that companies including IBM, Google and Apple no longer require one to get hired.

    Coincidentally, as companies eliminate the college degree requirement for new hires, we’re seeing significant growth in technical bootcamps. One study reports that this year, 20,000 students will complete a course of study at a coding boot camp and be “job ready.” Considering that the number of students who graduated with a computer science degree from a typical college in 2017 numbered about 93,000, there’s a good case to be made that boot camps are siphoning off a number of students that otherwise would be headed to academia.

    Going to boot camp instead of college is an upward trend: The study projects that the number of boot camps are expected to grow 20 percent this year. The career path for a technical professional does not necessarily involve going college. Going to a boot camp for a year—if not less—will suffice.

    Is this a problem? I’m not sure. On the plus side, I am reminded of the scene in the film, “Good Will Hunting,” when Matt Damon’s character Will Hunting, a genius with a photographic memory who walked the halls of MIT working as a janitor, makes the profound remark:

    “… you dropped a hundred and fifty grand on a f****** education you coulda got for a dollar fifty in late charges at the public library.”

    There’s a good argument to be made that when it comes to getting a good paying job, the $15,000 median price tag for attending a coding boot camp has a better much better ROI than spending  ~$200,000, the cost of a four-year degree from MIT.

    But then, I am reminded of Aldous Huxley.

    In his book, “Brave New World,” people are grown in test tubes according to a specific formulas, the result of which is a biological caste system. Those in the Epsilon caste are grown to be 3 feet tall and abhor the outdoors. Their job is to work their way underneath the machines on the factory floor to do simple maintenance. Deltas, the next caste up in the hierarchy, do the menial chores—janitors, for example. Gammas are the bookkeepers and data entry clerks. Then you get to the higher levels, Alphas and Beta. Betas are technicians who do very sophisticated jobs, such as mixing dangerous chemical together. One mishap can result in an explosion, killing all. Finally, at the top of hierarchy are the Alphas. Alphas possess advanced analytical skills and a broad knowledge base. They are able to plan and strategize. They have a high degree of creativity. Their role is to run the society.

    Thus, I have a concern about the increasing role of technology and  automation education, and the emergence of tech-focused boot camps as the prime post-secondary educational experience for many.

    As more of those entering the workforce forgo a formal college education, we run the risk of creating a society in which most of the technical workforce is Betas. They’re able to do highly complex tasks, yet have no idea why they’re doing them.

    Let me be specific. Let’s talk about Kubernetes: It takes a lot of knowledge and experience to be good at getting Kubernetes to work. It’s a complex, evolving technology that you need to keep your eye on. Companies are spending a good deal of money making sure their IT staff are competent with Kubernetes—as well they should be. One little misstep can result in the digital equivalent of a chemical explosion. Yet, I wonder how many of those “working Kubernetes” are thinking about the bigger picture—for example, the purpose and value of the enterprise using Kubernetes. Are these engineers questioning if the technology they’re working is doing good or harm?

    Or, let’s turn the dials down a bit and ask a question that’s less loaded philosophically. How many of those working Kubernetes today wonder if an alternative orchestration technology such as Docker Swarm might be a better choice to meet the need at hand. Or are they so concerned with getting Kubernetes to run that they don’t have the bandwidth to think about anything else? Do they just defer such critical thinking to “the system architects”?

    Critical thinking, although an acquired skill, is not one that’s new on the cognitive landscape. It’s been taught for thousands of years, since the time Socrates sat under an olive tree asking simple questions that forced his students to transform their assumptions and allusions into thoughts based on reason and reflection.

    Developing the ability to think critically traditionally has been one the aims of a college education. The romantic ideal is that college is the place where students are exposed to a variety of ideas from a variety of viewpoints. The hope is that such exposure, along with instruction aimed at developing one’s ability to engage in higher-order thinking, will create graduates who will contribute to making a better-functioning, more prosperous and more humane society.

    But, that’s the traditional perception of college. The reality is changing. Maybe college is just the place you go to get the skills required to get a good job. If that’s the case, given that a college education is becoming less affordable to more people, maybe the boot camp approach is best. Yet, there is a risk.

    A good while back when I was considering enrolling in a doctoral program, I attended a required orientation at Harvard University. I sat in an auditorium composed of about 50 aspiring doctoral students. One of the department heads on stage said the following, “The objective of our doctoral programs is simple. We want to train you to run the world.”

    Yep, that’s right. And, I can say with a good deal of confidence that this objective is the same today—to create thinkers who have the skills and knowledge necessary to run the world. The Alphas, if you will.

    So, we’re at a crossroads. Automation is a liberating force in education in that it makes infinite knowledge available to anyone who wants it at a fraction of a cost of a traditional college education. All that’s required is the time and desire to take advantage of the opportunity at hand. However, if the result of such education is a workforce in which the concern of most is to ensure that the technology works and does not blow up, then the risk we run is that bigger decisions will be made by those who had the time and money for an increasingly elite education. In other words, as more people spend time focused on turning the dials and setting the levels to make the technology go, a very small minority actually will be determining the purpose and direction of the machine the technology supports.

    Still, the fact is, most of us need to make a buck to survive. This means being good at getting the tech to work. Yet, we will do well to understand that there is a Big Picture and if don’t pay attention to it, someone else who has been educated to run the world will.

    — Bob Reselman

  • When DevOps and ITSM Collide: Mistakes to Avoid

    When DevOps and ITSM Collide: Mistakes to Avoid

    In many ways the DevOps movement evolved as a reaction against the worst elements of IT service management (ITSM). IT leaders wanted to get from under the yoke of overly complex processes, bureaucracy and red tape to speed up software delivery, and to just get stuff done. Now that DevOps has kicked into full gear at so many organizations, the question remains: What do we do about ITSM?

    While the instinct in some may be to completely ditch ITSM or to just let the ITSM team operate in a separate orbit from DevOps practitioners, experts say that would be a big mistake.

    “DevOps aims to deliver high-quality software to the business faster and in shorter cycles,” explained Krishna Mohan Reddy, vice president and deputy global head of Cognitive Business Operations at Tata Consultancy Services. “However, it still does not eliminate the need for controls.”

    In many ways ITSM is the yin to DevOps’ yang—the two operating philosophies do offer contrasting viewpoints but when the operate in a single system they can be extremely complementary and synergistic.

    “Maintaining the proper balance between the ITSM and DevOps helps the organizations to scale and grow and become more organized,” said Sachin Agarwal, DevOps engineer and solution architect for Mantra Labs. Operating at DevOps speeds with ITSM frameworks in place keeps finger-pointing at a minimum and reduces the risks of failed deployments. “Both in harmony will increase the efficiency and reduce the blame-game within the organization.”

    As organizations try to figure out how to achieve that harmony, here are some pitfalls they should seek to avoid:

    Letting Each Tribe’s Preconceived Notions Hold Back Overall IT Progress

    It can be easy to get wrapped up into dogmatic fervor that your particular philosophy is the One True Path for IT success, but that really does nobody any favor. Organizations that want to pick up the best of both ITSM and DevOps need to encourage stakeholders to approach the table with an open mind.

    “You need to compromise. A lot of the new, young practitioners in DevOps and Agile will pretty much automatically discount opinions coming from people who have got ITIL certifications,” said Andi Mann, chief technology advocate for Splunk. “And vice versa—a lot of traditional operations and development teams will look at the people in DevOps and Agile and think they’re all cowboys. But they all have something to learn from each other.”

    For DevOps practitioners, it’s important to keep in mind that ITSM controls such as service requests, change management or structured management of incidents and problems don’t have to stop a team from delivering quality software at speed, said Troy Vetter, COO of Coda Global.

    “In fact, speed and quality are actually helped by being closely monitored. In IT, just like in science, if it isn’t seen, measured, and “reproducible, it doesn’t really count,” he noted. “So, ITSM is actually the perfect complement to DevOps because it aids in the visibility, measurement and reproducibility of key DevOps processes.”

    Failing to Address the Elephant in the Room: Change Management

    Analyst Roy Atkinson agrees that ITSM has a lot to bring to the table for DevOps workflows. ITSM’s rich history in thinking about and tracking operational best practices offers tons of value in improving the customer experience with the software being delivered. But on the flip side, ITSM teams can’t continue to operate as they did in the past. They need to compromise and learn new ways of accommodating DevOps approaches—particularly in the area of change management.

    “Change management has often been where DevOps and ITSM collide,” said Atkinson, who analyzes the service management market for HDI. “The stereotypical Change Advisory Board must radically revise its approach to be in sync with the enormous amount of change that DevOps can produce in a day.”

    The good news is that thought leaders in the ITSM world have been thinking hard about this issue and have been evolving practices so they can embed change management controls into a DevOps framework without holding back continuous delivery.

    “There is some excellent recent thinking in the ITSM world about how change management can serve its intended purpose without being a roadblock by acting as an enabler of change and being involved all along the way, rather than a bolt-on quality check at the end of the line before the production environment,” Atkinson said.

    Maintaining Separate Toolchains

    Often times one of the big reasons ITSM will remain a roadblock for DevOps isn’t because the ITSM folks aren’t willing to collaborate, but simply because each team relies on completely separate toolchains.

    “You might have a tool that developers use for tracking bugs or product backlog and then there’s a completely separate IT service management tool,” said Donna Knapp, curriculum development manager for ITSM Academy and author of the DevOps Institute’s course on ITSM for DevOps. “So what you end up doing is forcing developers to use some process that’s outside of their norm in order to address issues.”

    As Vetter explained, a lot of the knee-jerk DevOps reaction against ITSM has to do with an aversion to the historical view of ITSM requiring manual inputs in monolithic ticketing systems. But the technology of ITSM controls has advanced—it just needs to be appropriately implemented.

    “Self-service requests, automation flows and IT orchestration are now our front door to actions, and IT service management can and should be happening in the background, automagically created, maintained and measured by our DevOps tools,” he said.

    — Ericka Chickowski

  • DevOps: Build a Continuous Improvement Culture

    DevOps: Build a Continuous Improvement Culture

    DevOps was built around the concept of continuous everything. We even use it in many of the names—continuous delivery (CD), continuous testing, continuous integration (CI)—and it is a worthy goal to want to improve the places that form bottlenecks in your software delivery systems and processes.

    But what about the bottlenecks in the DevOps systems and processes?

    Like any other system, DevOps involves tools. We choose those tools, invest in learning those tools and use them to do great things. But then we have investment in the tool. If our knowledge and systems for deployment are all based around CD tool X or ARA tool Y, we are just as unlikely to want to move from it as we were unlikely to want to move away from the previous toolset.

    Even if you discount the number of shops that have done pilot or small-team DevOps but kept the core of their business waterfalling, there are still a large number of organizations that stepped into DevOps and stopped improving. Granted, it is a huge job to move a significantly sized application portfolio into the DevOps mold and toolchain, but it is contrary to the process/goals to stop just because you did.

    The market is nowhere near finished evolving: New tools are coming available and massive improvements to existing tools are being released. Keep taking advantage of those facts.

    The more projects/portfolios you have moved into DevOps, the more weight there is to resist change. That is the simple reality, just as the larger a rock, the harder it is to move (assuming the same type of rock, of course). The answer: Watch for those improvements that will make moving the rock worth the force required to do so.

    Working containers into the overall DevOps process, for example, can benefit every step in the toolchain. But if they’re not already in the process, it is a lot of work to get them folded in. I chose this example because for most shops, a little research will show it is well worth the effort. From preconfigured Dev/Test/Prod servers to run-and-done compile instances, containers can make the process more manageable and more agile. The benefits outweigh the costs for most shops.

    So keep an eye out. Don’t implement and spend all of your time on the tools you’ve chosen. Stay aware of what is out there, and what is coming. Understand how it would impact the DevOps process and try to pull in those items that fit the environment.

    And, as always, keep rocking it. The end goal is to keep the systems up, running and usable. And most of you are doing a great job.

    (This, and all blogs, brought to you as much by the geeks behind the scenes as the author and editor.)

    — Don Macvittie

  • Tricentis, QASymphony Merge: The Age of Continuous Testing Is Upon Us

    Tricentis, QASymphony Merge: The Age of Continuous Testing Is Upon Us

    Big news in the world of continuous testing today as continuous testing market leader Tricentis announced it is merging with QASymphony. Tricentis, a leader of continuous testing, is adding QASymphony’s leading testing automation platform to the mix. The full text of the press release is below.

    With the worldwide testing market pegged at more than $34 billion, this is a big move in a very big market. Tricentis raised $165 million last year from Insight Venture Partners and has put this money to good use: The QASymphony merger is Tricentis’ latest move. Previously, it acquired Q-Up, in June, and last year acquired Flood IO.

    Tricentis’ merger with QASymphony is happening at a time when continuous testing is becoming one of the hottest areas in DevOps. The demand for automated, continuous testing is driving a revolution in the QA market as companies recognize that quality is as important as speed when it comes to deploying new apps. It is a key cog in the modern software factory.

    I had a chance to sit down and speak with Tricentis CEO,  Sandeep Johri, about today’s announcement, the testing market and what the future may hold. Sandeep, who has previous experience at HP and other large providers, has a great vision of the market and where it is heading as well as the details that escape many “professional manager” CEOs. His hand is firmly on the wheel as he drives Tricentis forward in this space.

    Have a listen to our conversation below, and stay tuned—we will update and continue this breaking story as more details emerge.

     

    Tricentis and QASymphony Merge to Drive Innovation in Continuous Testing for Enterprise DevOps

    Next Gen platform disrupts the $34B testing services spend and transforms software testing for DevOps

    Mountain View, CA— June 20, 2018 — Tricentis and QASymphony announced their agreement to merge under the name of Tricentis. Tricentis’ market-leading Continuous Testing platform has earned the company unanimous “leader” recognition among the top industry analysts. QASymphony is recognized as the industry’s leading test management solution. The combined solution represents the most modern platform for transforming software testing to keep pace with digital transformation.

    “Tricentis and QASymphony are both recognized for their innovative testing solutions,” explained Sandeep Johri, CEO of Tricentis. “Tricentis is known for helping enterprises transform testing from a burdensome manual process that delays software delivery into an automated, continuous process that drives ‘quality at speed.’ The addition of QASymphony’s test management solution strengthens our leadership position and builds on our momentum replacing legacy testing tools.”

    “Today’s CIOs are responsible for ramping up innovation through accelerated software delivery,” explained Barry Libenson, Global Chief Information Officer at Experian. “Amid this demand for rapid innovation, CIOs are recognizing that software testing often delays their digital transformation goals. In fact, testing is now commonly cited as the #1 bottleneck to delivery speed. When testing lags behind development, the organization is forced to delay their innovation and that’s no longer an option. It’s inevitable that software testing must be transformed to keep pace with Agile development and DevOps.”

    “Global 2000 organizations remain locked into legacy testing platforms that were designed decades ago to enable manual testing processes,” explained Dave Keil, CEO of QASymphony. “Scaled agile and enterprise DevOps initiatives have changed the game. Yet, legacy testing technologies such as those by HPE (now Micro Focus) and CA are throttling speed and innovation. QASymphony has experienced triple-digit growth over the past year—allowing us to aggressively invest in our technology, infrastructure and people to meet the emerging needs of our customers.”

    QASymphony’s 100+ employees will be joining Tricentis’ 450+ employees. Sandeep Johri will lead the organization as CEO and Dave Keil will assume the role of COO. The combined company will have a global presence with offices in the US, EMEA and Asia. The transaction is expected to close within 90 days. EMBARGO UNTIL WEDNESDAY, JUNE 20, 2018 @ 8:00 AM ET USA

    The integration will reflect the commitment to innovation and customer success that both companies share—to learn more, register for the upcoming webinar. Additionally, the company will be showcasing the roadmap and solution at Accelerate 2018 in Vienna.

    DevOps Impact

    The market for software testing (recently estimated at $34 billion)1 has reached an inflection point. DevOps adoption is rising rapidly, and both Gartner and Forrester report that transforming testing is a primary factor for DevOps success.2 Yet, software testing remains a predominantly manual activity that delays releases and consumes 30-40 percent of an organization’s application budget3,4 Tricentis Continuous Testing solutions accelerate delivery and free up significant budget that can be reallocated to underfunded innovation initiatives.

    Gartner’s Joachim Herschmann and Thomas Murphy report: “As DevOps principles gain significant traction in enterprises, testing and QA is no longer a stage in delivery but an integral DevOps activity that exists in various forms through all areas. Automation has become an indispensable element in these efforts, and there has been renewed focus on software test automation tools as part of a larger toolchain that enables DevOps.”5

    About Tricentis

    With the industry’s No. 1 Continuous Testing platform, Tricentis is recognized for reinventing software testing for DevOps. Through risk-based testing, scriptless end-to-end test automation, and the industry’s most extensive technology support, Tricentis breaks through the barriers experienced with conventional software testing methods. Our innovative technologies simplify testing for even the most complex enterprise applications—transforming testing from a roadblock to a catalyst for innovation. The result is accelerated software delivery speed, improved cost efficiency, and reduced business risk.

    Tricentis is the only vendor to achieve “leader” status in all three top analyst reports (i.e., the “Triple Crown.”) This honor is based on our technical leadership, innovation, and a Global 2000 customer base of 800+ companies, including global enterprises such as Allianz, ANZ Bank, Experian, First Data, HSBC, Lexmark, Merck, Sentry Insurance, Starbucks, Telstra, UBS, Vantiv (now WorldPay), Vodafone, and Whole Foods. These customers rely on Tricentis to achieve and sustain test automation rates of over 90 percent—increasing risk coverage while accelerating testing to keep pace with Agile and DevOps. EMBARGO UNTIL WEDNESDAY, JUNE 20, 2018 @ 8:00 AM ET USA

    Tricentis has a global presence in Austria, Australia, Belgium, Denmark, Germany, India, Netherlands, Singapore, Switzerland, Poland, United States and the UK. To learn more, visit https://www.tricentis.com or follow us on LinkedIn, Twitter, and Facebook.

    About QASymphony

    QASymphony accelerates digital transformation with software testing solutions that help enterprises prioritize quality, develop reliable software and increase speed to market. The qTest platform streamlines software testing in agile and DevOps environments and centralizes testing efforts across the enterprise. By taking a strategic approach to quality, QASymphony enables enterprises to develop higher-quality software, faster, to better support new business models and customer experiences.

    The qTest platform streamlines development workflows by seamlessly integrating testing with leading automation, development and delivery tools including Atlassian’s Jira, CA Agile Central, Github, Selenium, Cucumber, Jenkins and many more. QASymphony is trusted by leading enterprises including Forcepoint, PFSweb, Cisco, Samsung, Verizon and Office Depot.

    1 Nelson Hall, https://research.nelson-hall.com/sourcing-expertise/it-services/software-testing/

    2 Gartner, Build Continuous Quality Into Your DevOps Toolchains: https://www.gartner.com/doc/3868367/build-continuous-quality-devops-toolchains

    3 Capgemini, HPE, Sogeti, World Quality Report: https://www.capgemini.com/service/world-quality-report-2017-18/

    4 Capgemini, HPE, Sogeti, World Quality Report: https://www.capgemini.com/service/world-quality-report-2017-18/

    5 Gartner, Magic Quadrant for Software Test Automation: https://www.gartner.com/doc/3830082/magic-quadrant-software-test-automation

    Media Contact

    Victoria Brown, Revere

    +1 (650) 850-2009

    Victoria.brown@revereagency.co

    QASymphony Contact Information Kira Mondrus

    QASymphony +1 (844) 798-4386 Kiramondrus@qasymphony.com

    Tricentis Contact Information

    Noel Wurst, Tricentis +1 (904) 673-4427 n.wurst@tricentis.com

    — Alan Shimel

  • Using CALMS to Assess an Organization’s DevOps

    Using CALMS to Assess an Organization’s DevOps

    Of the tried-and-tested frameworks that allow enterprises to assess DevOps in their organization and how it can be improved, the CALMS model remains particularly useful.

    CALMS, which stands for Collaboration, Automation, Lean, Measurement and Sharing, is particularly helpful for analyzing an organization’s DevOps structure, and ultimately, its utility in any organization. The CALMS framework covers all stakeholders within DevOps, including business, IT operations, QA, InfoSec and development teams, and how they collectively deliver, deploy and integrate automated processes that make business sense.

    “DevOps as a technique is slowly but steadily maturing, so it’s good to see general assessment methods like the CALMS model,” Holger Mueller, an analyst for Constellation Research, said. “The CALMS model gives a good frame of reference to compare the maturity of a DevOps team, and as such,  is invaluable for assessing the state of teams for the transformational change that goes with it.”

    Applying CALMS

    Here are a few ways CALMS can be applied to DevOps for enterprises either thinking about shifting to a DevOps structure or seek to make improvements to an existing deployment.

    Culture

    It is long-established that technology adoption must be to serve a business need rather than investing in technology for its own sake. This shift in mindset applies to how the “culture” component of CALMS supports how the forecasted ROI associated with automating a process must be championed across the DevOps team. This mindset is especially important for getting the non-technical business teams onboard, Mueller said.

    “DevOps must have the backing from the business executives. That’s a crucial factor in the culture part,” he said. “One thing software teams can do is use successful software deployments to help convince the business executives in DevOps to make the investments. Those are the kinds of tribal events that every DevOps organization needs.“

    Automation

    Netflix is well-known as an exemplary case study for DevOps automation. Among other things, it can deploy code and software updates in a matter of minutes across its entire massive infrastructure, many times throughout the day. But Netflix’s example of automation does not take into account what can go wrong at firms that want to follow Netflix’s example when automating their software deployments and updates, Mueller said.

    “The risk is always deploying too fast and you end up deploying bad code,” Mueller said. “Automation, of course, needs to take this into account and compensate. But not everybody can do that.”

    Lean

    Once completed, it is relatively easy to document cost savings by eliminating resources. Within the CALMS framework, the corresponding “lean” component in DevOps is thus relatively easy to assess, Mueller said.

    The good thing about DevOps, Mueller said, is companies can come to a consensus ahead of time regarding what they consider to be a lean operation.

    However, he added, “There is always the risk of being so lean you begin to err on the QA side.”

    Measurement

    The success of software deployments is relatively easy to measure once completed. However, attempting to measure future deployments as part of a forecast model is, of course, much more challenging.

    “It can be surprisingly tricky to measure how good DevOps is ahead of time,” Mueller said. “Measurements should thus allow errors to be pinpointed  quickly during the post-deployment stage to fix the code if needed very quickly.”

    Sharing

    Sharing, in many ways, overlaps with culture. This is where assessing whether all of the different teams within DevOps are working together and communicating comes into play.

    “Thanks to  DevOps, the days of the lone developer are long gone. But sharing is not always easy,” Mueller said. “Even within separate teams, it is easy to fall into the trap of wanting to get your own work done without necessarily thinking about the team as much as you should.”

    The End Result

    While applying CALMS to assess DevOps never hurts, the framework is still not as important as one of the main objectives of DevOps—namely, using technology to make organizations’ operations more agile and reactive to customer needs, Mueller said.

    “At the end of the day, assessment methods such as CALMS are great, but they are never as important as focusing on implementing next-generation applications,” he said.

    — B. Cameron Gain

  • The ‘Blockbuster Effect’ in the Age Of Automation

    The ‘Blockbuster Effect’ in the Age Of Automation

    Saying that new jobs will always be created to replace those that have been eliminated by automation is a mechanism for avoiding a real problem that’s lurking on the horizon.

    In 2004, at the height of its popularity, Blockbuster Video had more than 9,000 stores and employed more than 80,000 people worldwide. About 50,000 of the total workforce was employed in the United States. Going down to Blockbuster to pick up a movie was a Friday night ritual for many, this writer included.

    Blockbuster and other video rental stores, along with the advent of affordable Big Screen TVs, were significant contributing factors in the growth of the home theater. People now had an alternative to $10 movie tickets and battling weekend traffic to get to the cineplex. An entire family could view a recent movie in the comfort of their own home for about $5. The only thing that was needed to complete the movie going experience was the popcorn, which frugal families bought at Costco by the caseload. Those who were less thrifty could buy a box of microwave popcorn at the checkout counter of the video store. The $25 cost of a Friday night date was reduced to roughly $6.

    Blockbusters kept people on the couches and out of the movie theaters. Television had already put a damper on movie-going. In 1930, 65 percent of the population went to the movies once a week. By 1964, the number dropped to 10 percent. Despite the introduction of 3D projection technology and movie theaters offering added features such as in-seat dining, movie attendance is now at its lowest point since 1992.

    Blockbuster was a transformational business on the entertainment landscape. Today, the company is gone. Streaming technology combined with an “all you can eat” subscription model killed it. (See Figure 1.)

    Figure 1: As Blockbuster stock declined, Netflix’s grew.

    Just as Blockbuster did away with the need to go to a movie house to see a film, Netflix, Amazon and other video streaming services did away with the need to leave the house to get the evening’s entertainment. America stayed home and enjoyed movies on the internet for a fixed monthly fee. Blockbuster went out of business.

    Netflix employs around 5,500 people with gross yearly sales of $11.69 billion. Blockbuster’s gross sales in 2004 were $6 billlion. It’s astounding that Netflix does almost twice as much business as Blockbuster did at its height, with less than 10 percent of the Blockbuster workforce. Such is the benefit of automation.

    Whereas Blockbuster needed to staff a store in every neighborhood to do business, Netflix and other streaming services work from a central, virtual location without the added expense of redundant staffing. Blockbuster had 9,000 cash registers to man. Netflix has one, which is run by some sort of AI.

    So, where did all the out of work clerks and store managers go? Hopefully, they went on to other jobs, maybe ones that were far better than the day-in, day-out tasks that go with processing video cassette rentals. After all, that’s the conventional wisdom: There will always be new jobs to take the place of the ones eliminated by automation. But, let’s face a cruel truth: It takes a lot more knowledge and skill to work on video delivery at Netflix than it does to check out customer purchases at Blockbuster. And Netflix requires fewer people to make the business go. Would a Blockbuster employee in Sioux City, Iowa, have the skills and mobility necessary to go on to to work at Netflix in Los Gatos, California? Seems pretty far-fetched.

    Right now, things are pretty rosy in terms of the economy. So the conventional wisdom prevails: As automation eliminates one job, another new type of job takes its place.

    Yet, there is another way to look at it. There are good arguments to be made:

    • As automation eliminates jobs in the modern world, new jobs for humans will NOT be created at levels required to keep a workforce fully employed. Rather, AI will do more and require less.
    • The lead up to large-scale unemployment will not be slow. It will happen fast, over a period of a year or two. And, it very well might happen within a decade rather than decades in the future.

    Blockbuster’s demise came about pretty quickly. The company was at its peak in 2004. Six years later it filed for bankruptcy with $1 billion in debt. The music retailer Virgin Megastore closed its last North American store in 2009, a year after music streaming Spotify went public on the internet. (Spotify generated $3.8 billion in revenue in 2016 with an employee count of 2,162.)

    Productivity due to automation continues to accelerate. In the early 1980s, it took 10.4 man-hours of labor to make a ton of steel. By 2014, the number was down to 1.9 man-hours per ton. The increase in productivity is due to increased levels of automation (think: robots). Given the choice between buying a machine and hiring a worker, companies are more likely to choose the machine. Thus, it’s entirely possible to imagine that the day when a few humans will be supervising the work of many robots making steel is close. Not too soon after that, a single human will be supervising an entire robotic workforce. Then, finally, as AI continues to improve and further dominate the production landscape, no humans will be required. The robots will supervise themselves.

    How far off in the future is this time? Dunno. But I can tell you that robots are now assisting with surgery on humans. How long will it be until one surgeon can use automation to do the work of 10? Then, how long will it be until AI can do surgery unsupervised, five years? Ten years? Only 10 years ago driverless cars were still on the test tracks; today they are roaming the streets of Mountain View, California. An autonomous robot with a scalpel is no longer the stuff of science fiction. Once AI can do surgery unassisted, where will all the displaced surgeons find work? Maybe a few will go into research. What of the others?

    At one time there were 200,000 horses in the City of New York. They provided the labor for a good part of the city’s transportation system. Today there are fewer than 1,000 horses. Cars, trucks, planes and trains have taken over. There’s no benefit to employing a horse when a car can do the work better. As General AI continues to develop and improve its ability to connect the dots as only a human can, how long will it be until the labor of most humans is not needed? What will it be like when there is no benefit to employing large parts of the human population because a machine can do the work better? Yes, the ramifications are scary. But, the possibility needs to be considered. The sad fact is that few people are considering such a scenario, let alone coming up with ways to address the impact on the horizon. Saying there will always be employment for humans is an avoidance mechanism, particularly given the fact that modern machines are going beyond purely mechanical labor and doing more thinking work.

    In 1990, people thought there would always be movie rental until there wasn’t. In 2000, people thought there would always be a Virgin Megastore until there wasn’t. Today, Sears keeps closing stores. It will soon be gone from the face of the earth. Yes, Amazon is opening brick and mortar stores. This might be good news. But the fact that these stores are highly automated and have few employees cancels out the good feelings of the moment.

    These days when a company comes to an end, it’s not a drawn-out process. It happens fast, over a period of a year. Many of the people left behind are confronted with low-wage, “show up and do it” jobs. The more educated have access to a pool of better paying work that comes with an excruciatingly steep learning curve. Whether the job is low-wage retail work or high-paying tech work, each worker is—or will be—competing against AI-powered automation that can—or will—do the job better.

    It’s not a pretty picture. But it is one that can avoided if we decide to think about alternatives and put those ideas into action. Sadly, there are more people thinking about how to get to Mars than solving the problem of unemployment on a massive scale due to the proliferation of industrial automation powered by general AI. The problem is that we still seem to be stuck at, “There will always be jobs to replace the ones eliminated by automation.” To say otherwise is deemed Cassandra talk. Maybe so. But to not give the problem consideration is beyond folly. It’s a danger that will exact a heavy price from everyone who has to work to earn a living.

    — Bob Reselman