staging-devopsy.kinsta.cloud

Tag: devops

  • Apache Hop Integrates DevOps and DataOps – Techstrong TV

    Apache Hop Integrates DevOps and DataOps – Techstrong TV

    Matt Caster, chief solution architect for Neo4j, explains how open source Apache Hop projects advances integration of DevOps and DataOps. The video is below followed by a transcript of the conversation.

    Mike Vizard: Hey, guys, thanks for the throw. We’re here with Matt Casters, who is Chief Solution Architect at Neo4j and we’re talking about a new open source project called Apache Hop, that helps you move data round more easily. Matt, welcome to the show. 

    Matt Casters: Thank you very much for having me, Michael.

    Vizard: So walk us through what exactly is Apache Hop, what’s the problem we’re trying to solve, and it seems like we’re trying to move more data than ever, and when I was a kid people told me moving data was not necessarily a great idea, but we’re doing it anyway. So what’s the challenge and what are we trying to solve?

    Casters: Oh yeah, yeah. So that’s a great observation. I remember when everything was going to be the new mainframe, do everything in one box, right? And the opposite happened; we saw more and more services on prem, in the cloud, and all mixed up. See, I think, you know, more specialized services, in the cloud, on prem, graph databases, you know, advent of containerizations, virtual machines, and containers. So it’s become like a really complicated ecosystem out there and I think a lot of organizations are struggling to find expertise to service and cater to all those needs.

    I think what we’re trying to do is make the life of the developer easy by wrapping up user interfaces and easy-to-use tools around all these technologies so that you get like UMLI diagrams that you can work with and that can be easily understood by not only the experts but medium-level educated people on subjects, right? For example, if you’re working with a Kafka queue you might need to know how it works in general, but you don’t need to dive into the Java or Python APIs to get the best out of it. I mean that’s kind of it. 

    So lower the maintenance costs. And this comes down to the DevOps side of things. And while we try to make the life of the developer easy we position ourselves exclusively on the side of the organization. We’re not made to just make life fun for the developer, but we really try to make the life fun for the developer, but we really try to make life for the company or the organization that uses the software better. 

    So with that in mind you come to the conclusion that, okay, we need to also focus on configuration management, life cycle management, version control, unit testing, security investment, and that companies put into often many years of work of data orchestration, data integration, ETL work, right? So that is what I think Hop has been trying to focus on the last couple years by making sure that all these technologies are standard in the tool and easy to use. 

    Vizard: Do you think that we’re reducing the need for specialists, ’cause back in the day you had to have somebody who knew how to program with ETL and that capability and now with this we’ve got more of a visual tool, and maybe some people still prefer CLIs and APIs, but this is designed for developers to just easily handle this task themselves without any outside intervention, right?

    Casters: Yeah, that’s the lore. We fully acknowledge that it’s like a 95/5-percent kind of deal, right; you’re never going to do the last 5-percent without some scripting or coding. But solving the 95-percent is a big deal, it means that all of a sudden you open up possibilities to people that we would never have expected of using Apache Hop. Right? I’ve seen salespeople use it to extract the sales numbers from Salesforce. I would’ve never expected that in the past. It’s like, “Oh, they can figure it out.” “Oh, I can just enter the details: username, password.” “Oh, I can read this” and before you know it people start to use that. 

    Vizard: Citizen developers are here with us today. One of the things about Hop is that it’s lightweight and I don’t think people appreciate just how much heavy lifting went into ETL historically. So why does it matter that this thing is lightweight and metadata-driven and what’s the impact? 

    Casters: Well, at some point we saw the advent of LANDesk, you know, microservices, but also in its previous life, Kettle, before that was used was for plotting trains in more like an IOT settings. Call these Edge devices, maybe Raspberry PIs or very tiny devices. And there’s a real use case out there for a device that doesn’t have to do it or anything; it’s just trimmed down to the bare essentials for what it needs to do. Maybe it would be like measure temperature and air humidity and pressure and then send it off every 30 seconds or something, over some – using some Rescall. 

    And then this whole Hop library trims down to something like 30-40 megabytes and starts up fast and uses next to no memory. That is not in contrast to saying like, oh, maybe I run it on an AWS LAN in a small services. And it also gives us the ability to create new packets, right? You can only remove them, remove functionality, but you can add functionality quite easily as well. And that is really important. We’ve also already seen people implement machine learning or libraries or Python iteration and plug-ins for Hop. So this will be our next goal, is to create a marketplace around that plug-in kernel kind of architecture. 

    Vizard: Do you think that the functions of DevOps and DataOps are starting to converge more? Is this becoming more of a team sport or is DataOps just going to be folded into DevOps? What’s the relationship?

    Casters: Well, what I’ve seen personally is now that we have the ability to lock everything that a workflow or pipeline does into a graph, a Neo4j graph, it makes it easy to see where the arrow was, right? Can use graph algorithms to figure out not just where an arrow was, but figure out what the execution pat was in milliseconds. So the convergence of, “Oh, the developer has an easy time figuring out what went wrong, but also on Monday morning when you come in the office and see that the Saturday night run failed for some reason, or the monthly load of the data warehouse or whatever people are doing, they don’t have to troll through millions of lines of loading texts for a large workflow, but just see straightaway what went wrong, where it went wrong, saves a lot of time. 

    So is that DevOps? For sure. For sure DevOps. It’s also ease of development. There’s really like an overlap and we need to do better in providing a lot of these tools. They’re still too primitive in a lot of our software, As if locking text is enough, right? We can do so much more. [Laughs] We’re executing stuff; we know where it went wrong. Why guess? Why go through the effort? [Laughs] 

    Vizard: There’s a lot of focus these days on the volume of data that people are trying to work with, but it seems to me it’s also the speed in which that data is starting to emerge more, and people are trying to work in near-real time, and the data is going to be updated. We’re moving beyond batch. So are we really prepared for not just the volume of data, but the speed at which that data is starting to emerge. And for that matter, there’s a lot more types of that data, right?

    Casters: Yeah. So being from the Hop from the beginning opted to choose Beam to work as an abstraction layer so that we could execute on Spark, Flank, and Dataflow for the really fast use cases, but also, yeah, like real-time streaming, and just like playing, just streaming, Spark streaming and cloud Dataflow are really cool. And this comes along with the advent of all the streaming back-answer messaging services: Kafka, Pulsar, jewel clouds pops up. I think repertoire, right? I probably forgot like Azure. [Laughs] 

    But yeah, so these services have become really popular in making it possible to create cool Hop and Spoke architectures, where you can send stuff in all directions. Yeah, so it falls under the sort of like streaming category, and you can do all sorts of cool load-it-and-see exercises there. And opening that up is quite easy for us in Hop; all the pipelines are streaming, whether it’s in batch or not. So yeah.

    Vizard: So what’s your best advice to folks as they kind of look at these DevOps/DataOps challenges? What do you see as the best practice that people do the right thing? And for that matter, what kind of mistakes are people making that maybe they should avoid?

    Casters: So I think when it comes to these best practices, and then we’re talking about the classical list, put everything in version control. You know, keep your passwords and usernames and everything separate. Don’t hardcode anything. Like these are not specific to ETL data orchestration, but they’re genera programming best practices – development best practices. So even if it is like graphical programming, it still applies. Same with unit testing and integration testing.

    What we’ve seen in the past is that if you want to do, for example, unit testing, there’s a lot of plumbing that you need to put in place. So specialized Docker containers and so on and so on and so on. What we’ve tried to do is always look at the return investment of metadata. So what is the return investment for setting up like a lot of infrastructure and a lot of manual plumbing? It’s very low, right? So you see some benefit from a unit test like that, but it’s not great. So by building the integration testing and unit testing into Hop, into GUI, making it very easy to create, we’re saying like, “Okay, now you really don’t have any reason not to do it anymore. Right? [Laughs] Well, apart from laziness or not knowing what the value is. But at least those best practices are not different from any other development job that you might have.

    Vizard: All right, so we finally reached a point where we’re going to automate the orchestration of the data, just like the rest of any other DevOps process for that matter. So it’s onwards and upwards. Matt, thanks for being on the show.

    Casters: Hey, you’re welcome. It’s good to talk to you, Michael. 

    Vizard: All right. Back to you guys in the studio. 

  • Plutora Releases Test Environment QuickStart Bundle – Techstrong TV

    Plutora Releases Test Environment QuickStart Bundle – Techstrong TV

    Bob talks about Plutora’s new “Test Environment QuickStart Bundle”. It provides an easy, immediate fix to the Achilles’ heel of software development: environments that cause delays, produce bugs, and waste time and effort. The video is below followed by a transcript of the conversation.

    Alan Shimel: Hey, everyone. Welcome to another Tech Strong TV segment. I’m really happy to be joined today – I don’t know if he’s actually been on Tech Strong TV with us, but I’ve spoken to him many times; it’s my friend Bob Davis from Plutora.

    Hey, Bob, how are you? 

    Bob Davis: I am great, Alan. It’s so good to be here. I haven’t had the pleasure of being on the Tech Strong TV spotlight and it’s great to be here.

    Shimel: Fantastic to have you. So, Bob, our audience has been fed a steady diet of Plutora for years already, right? So most of them know Plutora, one of the leaders in the value stream management space, a DevOps pioneer leader. But I don’t know if they really know you. I’d like to introduce them to Bob Davis. 

    So why don’t we start there? Give them a little bit of your history, Bob.

    Davis: Sure thing. Alan, I’ve got a few years under my belt, and I started in the semiconductor business back at Intel as a chip designer. And I ultimately said, “Gee, you know, I think I understand why I want to build things bigger, better, faster and cheaper, but I don’t understand the business.” So I went and got an MBA and I moved into more of a product marketing orientation, and from there became a serial entrepreneur. 

    This is, Plutora is my tenth startup. I never joined another large company after I joined Intel, but I found my way into some through acquisitions and so forth: Novell, CA. Always on the software side, always on the, you know, typically the infrastructure, you know, a lot of enterprise IT, networking, compression, security, a lot of that, and ultimately back up in recovery and continuous data protection and those kinds of things, which ultimately led me to really understanding, wanting to understand more about the developer side. 

    And so about five or six years ago I found my way into Plutora and I became chief marketing officer and away I went. Along the way I became a pilot, I play golf, and love the water. So I’ve got a lot of things. And six grandchildren, so good stuff.

    Shimel: Six children. God bless you, man. Whoo. You know, I’ll tell you off camera; I don’t even want to get into it now.

    But anyway, that’s great, man. It’s great to have you here. And it’s interesting, ’cause you know what, a lot of people will hear “Bob Davis, CMO Plutora” and say, “Ah, he’s a marketing guy.” Well, yeah, but you’re a marketing guy who started off designing chips at Intel when a lot of people watching this were playing computer games, not even computer games, Show & Tells or whatever the heck they were. Right?

    Davis: Yeah. Well, I’ve always been on the product side and I’ve always been more on the “What’s the next idea?” In fact, the best example of that was right around 2000 – you know, we started Kaseya. 

    Shimel: Oh, I remember that.

    Davis: So I was the original founder of Kaseya and started that in my kitchen and it’s …

    Shimel: Really?

    Davis: Yeah. To being what it is today. And there wasn’t even a managed service provider market at the time.

    Shimel: No, I remember that early on. Because I had started a company called StillSecure and in 2005 we pivoted into MSSP. ‘Cause I just thought MSSP was the way to go. It was too hard for too many companies; they needed an MSSP. Actually Kaseya bought friends of mine, Carl and Billy Austin, they’re serial entrepreneurs as well. They were acquired by Kaseya. And then of course Kaseya was acquired, what was it, a year or two ago maybe? 

    Davis: Yeah, Inside Venture Partners bought Kaseya in 2013, which is when I left actually.

    Shimel: Right. Oh, okay.

    Davis: That’s when the new management came in. And we sold the company and kind of went off on our own journeys. I went to a company called Atlantis, which we were doing hyper-converged storage, and then ultimately Dalibor Siroky, CEO of Plutora, found me and brought me over to Plutora in 2017. 

    Shimel: Very cool. So, Bob, we were talking off camera a bit. We’re at an interesting sort of boundary layer here again. I think many of us are let’s call it cautiously optimistic that we are going to emerge from our kind of COVID cocoons and venture out again en masse. 

    Davis: Yes.

    Shimel: Many of us are already.

    Davis: Yes. 

    Shimel: The interesting thing is the world we left two years ago is a different place today than it was two years ago. We’ve learned a lot of things, we’ve experienced a lot of things. Digital transformation – no pun intended – has transformed many a business – many a business model. For us in the tech industry, for our audience, you know, the way they shop for products, try products, use products, measure flow in terms of value stream and stuff like that, it’s really changed in two years. They’re different. You can’t put your foot in the same water twice, right? And so companies like Plutora – someone once said that, I’m pretty sure. 

    But companies like Plutora, they’ve had a change. You’ve had to adapt. And you guys have. You have that sort of a whole new program you’re coming out in market with. Let’s talk about it.

    Davis: You bet. You know, in the beginning, right in the beginning of the pandemic, early pandemic, early 2020 we saw as the world was trying to find its axis again we saw an incredible increase in people wanting to learn. So they spent the first part of that 2020 just in this learning mode. And value stream management was one of the things that people were really focused on learning. And the notion of being able to say “Wow, if I could really see a pipeline, a value stream from the beginning to the end and understand if I’m getting better.” And that was really the question that the CIOs were asking, “I’ve spent millions of dollars on these tools. Am I getting better?”

    The point I’m making is that everybody was trying to understand how they could tell if they were getting better. And all these flow things became almost an MBA project or a PhD-type project to really understand all that’s possible. And that was awesome and we got a lot of interesting traction from that. 

    What we found, however, is that that transformation is a daunting transformation and the world changed around everybody. We went from distributed teams to distributed teammates. So now that autonomous team was distributed in and of itself, and the communication and collaboration became very difficult to achieve. So processes, organizations were changing. So many things changing all at once that people were really slowly doing like they’d be happy to go to an event, sit on a virtual show. What do I need to understand to get better?

    That’s not what’s happening today. Some of it, sure. Some of it still happens. But people are moving to more execution. They want to get value. They want to start seeing the value of that journey and they want to see the value in real-time. They don’t want to see it a year from now; they want to see it right now, this year. So we have seen our customers struggle with saying “How can I get started in value stream management? I may not even want to go all the way to the end of the value stream management journey, depending upon my specific tastes.”

    So we’ve developed an entire program around doing just that. How can we get people into a value stream process mentality, into a value stream platform, and do so quickly so that they can value right away. And so we have this program called the Quick Start Program and we released in January the first of several Quick Start products. This one is on environment management, which is a core component of value stream platform that Plutora sells. And in so doing we’re able to provide people solutions to a very complex test environment problem in three or four weeks, whereas today it can take 30 days just to configure and make an environment available on some of the prospects that we’re talking to.

    So in the time it takes to get an existing manually configured test environment ready for a test event you can have a full test environment management program and go from manual management of test environments to an automated process very quickly. Now I see value. Now I can see value in terms of the efficiency of utilization of expense and test environments. I can see better quality. I can see saving tons of money. 

    We find that people with average levels of releases, well UMA did a study and they came out and said the average number of releases in an enterprise environment was 76 a year. That was in a survey. They did a survey and they came back and said that was the average. There was certainly a distribution, but that was the average. And if you’re manually managing your test environments you’re losing money. They determined based on all the issues of managing these things almost $1.5 million a year you’re wasting. 

    And so we can save you that with a product like this. So imagine I’ve got a bottleneck in my DevOps tool chain, the Achilles’ heel of my tool chain is waiting for test environments as I shift left and try to test earlier. I solved that problem, I removed the wait states there, I become more efficient, I save money, and I can do that in about three to four weeks worth of setup. And now I’m rolling. Wow, I’m getting value. Now I go to the next problem and so forth. And I get quick starts, I can come into the platform pretty soon. I’ve got a full value stream management platform down the road. I’ve gotten value every step of the way and I never had to really think about the prospect of doing the whole thing. It’s like one step at a time; every journey is one step. So it’s like let’s give people the ability to get value in that same way, one step at a time. 

    And that’s really, it’s been met with tremendous success already. We’ve seen a lot of responses, a lot of people wanting more information, and our SDR team is very busy fielding calls on it. So it’s been a real successful launch so far.

    Shimel: That’s great. You know, Bob, I think part of the reason for its success is there’s two kind of trends you’re riding there. First of all it’s this, I call it “deliver delight,” but I mean that’s just two words, right? It goes to when I was an entrepreneur founder of companies we were hiring enterprise salespeople who sold to the C-levels and all that stuff. In today’s world we hire DevRel evangelists, and the idea is get the software into the hands of the people who are going to use it, deliver delight, and if it works they’ll tell their bosses and so forth to buy it. So it’s a different sales model. But it’s become dominant, especially in DevOps cloud native, you know, the kind of markets we play in. So this really is kind of playing into that. 

    Davis: Totally.

    Shimel: It’s delivering delight. Because synonymous for delivering delight is delivering value. 

    Davis: Yes. Well, that’s the point, right? Now it’s the whole point of value stream management in the first place. You know, it’s interesting; it’s really an agile approach to delivering the value stream management solution. There’s a parallel there. You know, saying “I’m going to buy a value stream management platform and implement it” is a little like deciding I’m going to build a waterfall process. ‘Cause it’s everything at once, this is what I want to do.

    Shimel: No, it’s like trying to eat an elephant.

    Davis: Exactly. 

    Shimel: You’ve got to eat it one bite at a time. [Laughs] That’s the only way you can eat it.

    Davis: Right. Well, you’re crossing – you know, by the very definition value stream management embraces both, you know, everything from the ops person on the far right to the strategic planning office on the far left, and so you’ve got lots of people, you’ve got lots of moving parts. It’s hard. We actually have found hard when we market the product we get the interest from the pain point, I’m having trouble. In this case I’m having trouble managing my environment. If you’re selling the entire platform that environ manager, in order to take advantage of that feature, has to sell the platform to the planning person, to the ops person, ’cause they’re all going to get affected. 

    There’s value there, but they’re like, “Show me.” Now we’re giving them a tool they can do that. It’s an on ramp into the platform, a smaller bite of the apple, as you said, an agile approach to delivering value stream management and value at the pain point level. And so we’re doing everything around it to make it super-clear if you have this pain, this is what it looks like in our product when you solve it, and you can get there by the end of the month. Does that sound reasonable? And it’s way cheaper.

    And so that’s the other thing, these platforms can be expensive. But what if I can buy something that’s in my sign-off authority, so it’s more approachable to me? The whole notion of S-a-a-S is all about product life growth and let me get something in my hands for free and try it out and then I can grow from there. That’s the beauty of what so many people have enjoyed with S-a-a-S-style software products. 

    Value stream management is hard in that sense, ’cause it’s complicated. There’s integrations into the tool chain, there’s process changes as people evolve from more legacy processing to DevOps tool chains and DevSecOps tool chains and so forth. There’s that. There’s organizational change, budget-holder changes, people moving from silos and project-oriented things to product-oriented things. So we’ve run into prospects that get so frustrated, they want the solution but there’s a lot going on in the organization. We’re answering that question.

    You can each have had a success, show the value, and go from there. It’s a great on-ramp. It’s a freemium approach almost in a complicated environment and its value stream management, and we think it’s really going to be awesome when people can have multiple. We’ll do probably deployment, we’ll have a quick start for master deployment plans. You know, Jeff was on in November talking to you about our progressive release management and some of the new stuff that we did with the Insights dashboard. That’s another screaming for of Quick Start, “Hey, wouldn’t it be cool if you could get this up and running very quickly?” 

    So all of those things is where we’re headed. We started with environment management and it was made available on the 18th of January and it’s already in the market, so we’re excited about it.

    Shimel: So no pun intended, but where can we get a quick start on this Quick Start? [Laughs] Where can we go, Bob? 

    Davis: Well, we have, there are online demos. You know, there are all kinds of spots on our website. We have a spot where you can say “Talk to me and you can sign up for that and have it by the end of the month.”

    Shimel: So just go to Plutra.com and check out the stuff for Quick Starts. 

    Davis: Yep. There’s a banner right on the top of the home page that will take you right to the information on it.

    Shimel: Love it. Good stuff.

    Davis: Yep. It’s all right there.

    Shimel: Hey, Bob, we’re about of time. I want to thank you for coming on and telling us about the Plutora Quick Start program. You know, the first one, as you mentioned, came out the 18th. Any timeframe on subsequent quick starts?

    Davis: We’re going to bring out the next one in the beginning of the next quarter, so right around April. 

    Shimel: Excellent, man.

    Davis: So you heard it here first, Alan. 

    Shimel: Well, that’s why we’re here. Thank you for letting us know. We’ll be watching. Good luck with this one. Maybe we’ll see you show up on one of the Plutora webinar workshops that you guys do with us so often. Love to have that.

    Davis: We’re going to do that.

    Shimel: Cool stuff. All right. Bob Davis, CMO Plutora here on Tech Strong TV. We’re going to take a break right now. We’ll be right back with our next guest. 

  • A New Development Model: Bringing SaaS Apps to Data

    A New Development Model: Bringing SaaS Apps to Data

    The last few decades have seen multiple major shifts in how applications are built and deployed, and now we’re seeing the start of a new shift that opens up enormous opportunities for all kinds of developers. Instead of bringing data to applications, developers are bringing their applications to the data.

    Let’s back up. In the 1990s, as data grew too large for local machines, applications moved with the data to data centers—a model more like the mainframes that preceded them. Then, in the 2010s, those same applications migrated from data centers to the cloud, bringing the data with them again. Now we’re seeing a third major shift, where customers take back control of their data and centralize it in the cloud. And developers are picking up on this trend, building their applications to not control their own data, but instead to connect to the customers’ data, letting them have and control a single copy.

    In a traditional SaaS application, the SaaS vendor stores and processes data in their own platform, even if that platform is running on one of the major public clouds. A traditional CRM (customer relationship management) tool, for example, will use its own data platform to store all of the data that all of its customers are managing with the tool. When a customer wants access to their data, often to join with other data sets, they either have to export the data (via APIs or CSVs) or upload the other data set into the tool and use whatever analytics capabilities the tool has built. This is painful for customers and also painful for the developer who has to support use cases that are further afield from traditional relationship management.

    In this new world, the customer runs and the applications all connect to the same data platform, storing and processing their data there. This means that joining two data sets is a single query away, with no movement of the data required. It also means that the customer owns and controls the data, making security and governance hurdles much easier for developers to overcome. And, because the customer pays for the data platform, the developer doesn’t have to mark up the costs and is able to command significantly higher margins.

    While it’s early, this shift is already underway. Fast-growing startups like Supergrain, Anecdotes.ai and Rudderstack, to name a few examples, are taking full advantage of this shift and leading investors have picked up on it as well.

    Why Now? Access to the Best Data Platforms Available

    There are two key trends that are leading to the emergence of these connected applications: The wide adoption of cloud data platforms and the prioritization of control over data by customers.

    In the last ten years, a new class of highly scalable and cost-efficient data platforms has emerged, allowing enterprises of all sizes to adopt technology that was previously only available to the most sophisticated vendors for use inside their own products. Previously, SaaS vendors had to roll their own complex data platforms that were very specific to their use case and charge high prices to cover the cost of running those platforms. Today, we’re not only seeing SaaS vendors migrate to cloud data platforms but also enterprises adopt the platforms themselves. This means that, for the first time, every enterprise has access to the best data platforms available.

    We’re also seeing a huge change in how enterprises think about and secure their data. Partly because of regulations, and partly because of publicity around data breaches, large and small companies are paying significantly more attention to how secure, how controlled, and how well governed their data is. When your data is spread across dozens or more different SaaS applications, it’s nearly impossible to guarantee its security and access controls. Centralizing that data into a single platform, and connecting your applications to it, allows for a single point of control and security for every piece of data.

    What’s In It For Developers?

    It’s clear that connected applications are a big win for customers: A centralized source of truth with strong security and control. But this model has enormous benefits for application vendors as well.

    First, it changes the go-to-market dynamics by letting you tap into work that customers have already done to centralize their data. According to George Xing, co-founder and CEO of customer engagement platform Supergrain, CEPs built with traditional SaaS models suffer from several drawbacks including long implementation times, poor reporting, and higher costs. The connected app model makes Supergrain more competitive, he said.

    “Instead of moving your customer data to a third-party data platform, we bring the marketing software to your data warehouse, unifying source-of-truth customer data and marketing product experiences into one cohesive system,” Xing said.

    It also accelerates time to market. Developers no longer need to build and maintain a data platform for their customers and can focus instead on their core product differentiators. They also get to leave responsibility for data security and compliance to the customer and their data platform.

    With the customer managing and paying for storage and compute directly, vendors can significantly increase their margins, charging for the value they provide, rather than trying to mark up the compute they’re consuming.

    What’s Next for SaaS?

    The incredible growth in the amount and types of data available to companies is only accelerating this shift in how applications store and operate on their customers’ data. We’re going to see more and more SaaS vendors support connected application deployment models, but it won’t replace vendor-managed models. Instead, there will be opportunities for both models to flourish depending on the customer and specific application. We expect that many vendors will offer both models, just as we’ll see others go even further, bringing their entire application—including its UI, logic, authentication, etc.—fully into the customers’ data platform.

    The opportunity for companies that take advantage of these new models is enormous, and we’re in the very early days. But the applications that take full advantage of this shift will be the winners of the next generation of data apps.

  • Survey Shows Steady DevSecOps Progress

    Survey Shows Steady DevSecOps Progress

    A survey of 200 DevOps and IT/information security professionals published this week by Mezmo, a provider of an observability platform, conducted in collaboration with the market research firm Enterprise Strategy Group (ESG), finds only 22% report their organization has a formal DevSecOps strategy to integrates security into their software development lifecycle (SDLC) processes.

    Among those that have adopted best DevSecOps practices, however, 60% saw an acceleration in incident detection, while 53% saw a reduction of incidents in production environments.

    Overall, the survey finds 62% of respondents report their organization is actively evaluating use cases for, or has plans to implement DevSecOps.

    Mezmo CTO Rob Fry said the primary challenge organizations face when it comes to DevSecOps remains cultural. In fact, 99% of survey respondents said establishing a culture of collaboration between development and cybersecurity teams is either critical (51%) or important (48%). Building cross-functional DevOps teams that have the required level of security expertise, however, is a major hurdle when there is a general shortage of cybersecurity expertise, noted Fry.

    The survey finds that on average it takes 17.5 person hours to triage and understand security incidents, an amount the survey finds that 82% of respondents desire to reduce.

    However, organizations are also struggling with the sheer volume of data collected by DevOps teams. More than half the respondents (52%) said the variety of data to be captured, analyzed and be acted on throughout the software development lifecycle (SDLC) hinders security efficacy/efficiency. A total of 43% said the scale of data to be captured, analyzed and acted on throughout the SDLC also hinders security efficacy/efficiency.

    Organizations surveyed capture several (54%) or even hundreds (32%) of terabytes per month, with 6% capturing a petabyte or more per month. Well over two-thirds (69%) do not capture certain data sources because of the high cost of storage/retention, which Fry noted is going to be problematic if there is an incident. Log data (35%), followed by metrics (29%) and traces (22%) account for most of that data, the survey finds.

    In all, 84% of respondents said getting the right data and tools to developers is key to achieving success. A full 91% of respondents are using multiple tools to get the most value out of their data. Two-thirds (66%) have dedicated logging analysis tools, while 59% have a security information event management (SIEM) platform. Cloud to two-thirds 64% have a data lake hosted in the cloud, while 55% have one in an on-premises IT environment.

    A full 87% are using open source tools as part or all of their observability stack because they are more customizable. However, 84% noted that over time it will become challenging to manage, adapt and scale with these solutions. Nearly all survey respondents (98%) said they will likely investigate a managed observability solution over the next 12 months.

    The application security issues that plague organizations today have been around for decades, so it’s not likely they will be resolved in a few short weeks. On the plus side, at least, there’s now a lot more awareness of the root cause of the problem than ever.

  • Building a Platform for DevOps Evolution, Part One

    Building a Platform for DevOps Evolution, Part One

    Winston Churchill once said, “To improve is to change; to be perfect is to change often.” While nearly everyone will confess that change is necessary to grow and succeed, we all know it’s much harder, especially when human behavior is involved. Professionals who work in DevOps environments know this all too well.

    Adoption of DevOps principles and behaviors should be viewed as a gradual process. Many organizations make false starts or attempt to implement a change only to see it fail. Sometimes this is due to changes in leadership, market conditions or technology, but then organizations catch their second wind, build momentum and reaccelerate their evolution.

    DevOps can Change

    With a shared vision, mutual goals and hard work, DevOps can change to keep up with user demands. DevOps professionals will always confront two potent barriers: Cultural and process debt and technical debt.

    Cultural and process debt occurs when leaders refuse to delegate authority, distrust their teams and lack a value stream approach to work. Technical debt develops when monolithic systems are tightly coupled, automation is delayed for testing, environments and version control and there is no access to cloud technologies.

    Although these obstacles are challenging and complicated, many organizations still meet them head-on. Evolution depends on a habit of making improvements every day through the use of an improvement kata such as PDCA (plan, do, check, act), which involves long-term strategies and objectives and continual experiments between the current and next target states.

    An evolving DevOps approach maintains moving pieces across a large organization of hundreds, thousands or even tens of thousands of people who cannot change overnight. Many lack the will and the cognitive capacity to embrace and enact change.

    This environment requires release managers to delegate power and transition from centralized commander to an enabler, understanding different value streams have varied skills and requirements. Mark Peters, a cybersecurity specialist and DevOps leader with years of military operations experience, outlined his top release management traits in the context of DevOps ways of working as automated processes, distributed accountability and accelerated delivery.

    PDCA Improvement Kata

    DevOps proposes that we adopt an improvement kata in our day-to-day efforts as an improvement habit, consisting of four steps that are repeated:

    1. Set the long-term vision
    2. Understand the current state
    3. Identify the next target state
    4. PDCA between the states

    Experimentation is at the heart of the PDCA cycle. The most practical experiments are empirical, data-driven and date-specific. Using a value stream management platform to give data to support the experiments leads to data-driven discussions on real and measurable success.

    In the next part of this series, we will begin exploring different experiments DevOps organizations can conduct to drive change and improve their release management procedures.

  • Five Great DevOps Job Opportunities

    Five Great DevOps Job Opportunities

    staging-devopsy.kinsta.cloud is launching a weekly DevOps jobs report today through which opportunities for DevOps professionals will be highlighted as part of an effort to better serve our audience.

    Our goal in these challenging economic times is to make it easier for DevOps professionals to advance their careers.
    Of course, the pool of available DevOps talent is still relatively constrained, so when one DevOps professional takes on a new role it tends to create an opportunity for others.

    The five job postings shared this week are selected based on the company looking to hire, the vertical industry segment and, naturally, the pay scale being offered.

    We’re committed to providing additional insights into the state of the DevOps job market in the weeks ahead. In the meantime, here are five DevOps jobs for your consideration:

    Indeed.com

    Mayo Clinic
    Phoenix
    Senior MLOps/DevOps Engineer
    $121,701 – $176,446 a year – Full-time

    Dice.com

    AWS – Professional Services
    Anywhere
    DevOps Engineer
    $131,000 Base Pay

    LinkedIn

    Kasisto
    New York
    Director of DevOps
    $160,000 to $180,000

    Monster.com

    University of North Carolina
    Chapel Hill, North Carolina
    DevOps Developer
    $80,000 to $95,000

    SimplyHired

    Tide Cleaners
    Cincinnati (Preferred)
    Senior DevOps Engineer
    $120,000 to $160,000

  • Techstrong TV: Scratching the Surface of Testing Through AI

    Techstrong TV: Scratching the Surface of Testing Through AI

    Mark, Torsten and Alan discuss the findings of the “Disrupting the Economics of Software Testing Through AI” report. The report identifies critical factors that hinder software engineering and DevOps teams including the escalating costs of quality control, and growing complexity associated with the increasing release velocity, number of smart devices, operating systems, and programming languages. The video and a transcript of the conversation are below.

    Announcer:                 This is Digital Anarchist.

     

    Interviewer:                Hey, everyone. Welcome to another segment here on Techstrong TV. I am really happy to tell you, introduce our guests for this next segment. First of all, he’s a returning guest with us. He’s from our friends at Applitools. It’s Mark Lambert. Hey, Mark. How are you, man?

     

    Interviewee 1:             Fantastic. Thank you, Alan. Great to have you – for you to have me on. Yeah, great to talk to you.

     

    Interviewer:                Absolutely. And then joining Mark and I today is Torsten Volk. Torsten is – I just drew a blank – EMA, right? Enterprise Management Associates?

     

    Interviewee 2:             Yes, exactly.

     

    Interviewer:                Yep. And we were talking off camera. Torsten is actually a friend of our friend, Andi Mann, who he knew or was his predecessor at EMA 11 years ago.

     

    Interviewee 2:             Yeah.

     

    Interviewer:                And, Torsten, welcome to Techstrong TV. It’s great to have you on.

     

    Interviewee 2:             Thank you very much. It’s awesome to be here.

     

    Interviewer:                Thank you. So, guys, look, Applitools, leader in visual testing. We’ve had, beyond Mark we’ve had many Applitools people on over the last almost two years now of doing Techstrong TV, so I think our audience is familiar. But just in case people aren’t, Mark, how about a little quick Applitools backgrounder?

     

    Interviewee 1:             Sure, yeah. So Applitools is known for inventing Visual AI, so helping software development teams accelerate their releases by mimicking the human eye and brain and how we as humans analyze computer screens, highlighting differences and issues within your UI so you can do not only visual testing, but also fundamentally change and modernize your functional test automation. So it’s really the glue that plugs right into your existing practices, so Selenium, Cypress, we support over 50 SDKs. There’s a free account as well, so you can get in there and get going, kind of feel the water, so to speak. But what we’re seeing is that it’s helping leading brands accelerate their digital experiences, and really it ties directly into the research obviously that Torsten’s been doing around AI and how that can help organizations disrupt the economics of software testing.

     

    Interviewer:                Love it. And, Torsten, look, I know EMA from before 2010 even, but for those folks who maybe, you know, a lot of our audience are technical people. They may not necessarily follow the analyst beat. Why don’t you give them a little EMA background?

     

    Interviewee 2:             My boss describes it always as the industry and analyst firm for enterprise grade solutions, and in the beginning – I’ve been here for a while. In the beginning I was always a little bit wondering, you know, what does that really mean? And what it at the end of the day means is we have all of those emerging technologies coming up, especially in the area of artificial intelligence and cloud native applications and intelligent automation, and what we are looking at is really how can we leverage those capabilities within an enterprise setting. So once the dust has settled and we’ve all gorged ourselves, you know, by consuming the leading edge of all of those tools, the question then is how do I expand that footprint of those tools across the enterprise so that the enterprise as a whole is more competitive in the end. So basically projecting those constraints of all of those cool new products and technologies into the enterprise, that is really what EMA is all about. And my specific practice is focused, as you can see in the background here, as one of my research graphics is focused on looking at machine learning, deep learning, reinforcement learning, and how those capabilities can enhance the enterprise.

     

    Interviewer:                I get it. Great use of the background, Torsten. Thank you. All right. So, Mark, we’ve got all that out of the way. Let’s jump into what we really want to talk on today, and I’m gonna let you kick it off if it’s okay.

     

    Interviewee 1:             Yeah, I’d love to. Thank you. So yeah, so recent report from EMA, and Torsten is the author and I’ll kind of throw the ball to him very quickly, obviously, but really talking about how artificial intelligence and machine learning is revolutionizing the software testing space, and is really important right now because there’s so much buzz around AI, machine learning, and kind of where it comes in. You want to understand where does the rubber meet the road? What is impactful today? What’s impactful for the future? And then what’s, what are the drivers that are getting us there? So I mean, Torsten, I’ll turn over to you, and maybe kind of like talk about some of the capabilities that you talk about in the report and the challenges that they, that you’ve seen come into play.

     

    Interviewee 2:             We started that report by looking at the key pain points, like we start almost all of our reports. Right? What is not sustainable anymore in a specific discipline, and in this case it’s test engineering. And we have pressure factors. One of them is that we, the business needs us to release software a lot more often, and in a really granular manner where somebody’s asking for a feature capability and the software development team is expected to release that thing very quickly, ideally in an ongoing manner. So that is one part, and the second part is that the end users are very, have become very much more diverse than they even were in the recent past, where we have multiple charts where we show the escalation of the number of devices with different screens, different processors, a different network connection, different interfaces for your fingers and for voice control. And all of those things make testing much, much more difficult, and one of my favorite parts always is the little war stories that you get told. Right?

    So in the beginning when we look at al the material that we have, we talk to a lot of developers and test engineers and DevOps guys, and my war stories or their war stories always started with, “Oh, yeah, regression testing, Torsten. That is really not doable anymore. This is kind of a dreamland that you are living in as an analyst if you think that we can regression test all of those releases. It’s just too much, and it would take forever to be able to do that.” So if we can’t regression test, we are running a significant risk, Mark, that something goes awry. Right? We always are sure that in theory things should work, but in real life there are combinations of technologies, of code functions, that they then don’t work. And that’s where we start with artificial intelligence and this whole pair of eyes, Mark, that you are describing, right, where you say, “Oh, yeah, my test engineers don’t have to be that pair of eyes that check through dozens of boxes and just say yes, yes, yes, yes, yes everywhere, and then also overlook some things.

    But that pair of eyes is something more. It’s something that has a certain degree of – when I say understanding, I think I’m gonna embarrass myself, but it has a certain degree of ability to comprehend – maybe that’s the same word – but to pick up things of a similar character and to categorize and cluster things in a manner that makes sense for human beings then to review, Mark. Right? And that is –

     

    Interviewee 1:             Yeah.

     

    Interviewee 2:             – where a tool like Applitools is very interesting, and also unique in the marketplace, right, where it’s a pair of eyes that you guys are building.

     

    Interviewee 1:             Yeah, and I think that talks – I mean your research touches on several different areas where AI comes in, and visual inspection, which is obviously where Applitools fits, is really a highly impactful and the most impactful today of those techniques because it does so many different things. So first of all, obviously there’s the streamlining of the manual review effort. Right? So I’ve got this explosion of devices. How do I check them all? Well, because of the high level of accuracy of Applitools Visual AI, and it’s been trained on over a billion images and has got four nines of accuracy, so what that means is as a human I don’t have to review tens or hundreds of screenshots. I’m just focused in on a very small subset. But that’s just the first part of it, because then you layer on top of that and this explosion of release velocity, increasing device combinations, and as well as the, just the complexity of the application and so many things out of your control, you need help in reviewing the volumes of data and really saying, okay, all of these things kind of look like the same.

    So and then bringing them together. I talk about basically sorting your visual review into different stacks of paper, so for example I might have 76 screens that have a visual defect, but I can break those down into two categories of 40 and 36. And those types of techniques go beyond just the basic pixel dipping approach, which is the one that would be a traditional visual testing technology, and really the ability to categorize that you talk about, Torsten, is the key driver that, or the key foundation that all of these things get built on top of. So it’s a really powerful technique, and as I said earlier plugs right into existing, so you don’t have to rip and replace. That’s one of the things that a lot of AI technology today is say, okay, it’s great, but you have to build it all new in this track. The thing that is really powerful with Applitools is we plug right into your existing frameworks, open source as well as third-party commercial tools, and then we elevate the process with the dashboard to really enable cross-team collaboration. So it’s not just your testers or your test automation or developers that are engaged. Your domain experts, even other parts of the team like UI, UX design, for example, get to be involved in that collaboration process.

     

    Interviewee 2:             Yes, and we see often that communication is key, and it’s not happening in testing where we have those more and more decentralized product teams that are doing their own thing with their own toolkits and pipelines, but they’re not really benefiting from what other teams are finding. And so coming back to that horrible list approach, right, where you have, you check those boxes or you don’t check them. Right? So it’s a very monotonous affair as a human being to review this, but then once you’ve reviewed it usually it comes back to you again and again, and then it comes back and hits other people in that organization because there is no communication and there’s no unified platform to retain that knowledge. Right? And with reinforcement learning, you can teach the system what it needs to come back to a human with, and then what it shouldn’t. So if one team makes decisions, all the other teams are benefiting from that and it’s less and less of this toil type of task where you just also as a human tend to make mistakes and it’s just not scalable. Right? So that whole scalability aspect is an interesting one that deep learning and reinforcement learning brings us there, Mark.

     

    Interviewee 1:             Yeah, you hit on the key things. Right? You’re scaling to meet the current demands of release velocity, shorter release cycles. But the other thing I always think about when I think about AI machine learning and how it can help us from a software quality perspective, it’s an assistive technology so you’re reducing or removing mundane and repetitive tasks, which in themselves are error prone. Humans are great at creative work. We’re terrible at boring, monotonous things  because we miss stuff. Whereas leveraging AI, and especially Applitools Visual AI, it’s highlighting those things. So that same example I talked about, 76 screens, you’re gonna miss stuff. If you’re looking at them all and you’re like, oh, it’s the same. It’s the same. It’s the same. But the Visual AI will identify key differences between the 76 images to really kind of like focus and improve the efforts.

     

    Interviewee 2:             Yeah. And that gets even when you look a little bit into the future, Mark. I know that you can’t share roadmap items, but that whole idea of having a Visual AI can go even beyond just user interface testing, right, where you can do sanity checks in certain areas, where you say, “Oh, yeah, no, that number can never be a billion because of what I know about the data in the background.” Right? And a lot of the things that come through that we continuously see coming through when you release and then you don’t want to be the one in the test team to hold off a, to hold back a release in the middle of the night.

    It’s funny that when I was young and had a ton of hair still, that was what happened, and it’s still happening today just as much, right, where people think, “Oh, no. I shouldn’t be the one who says that. I think it’s all fine. Let’s sign off on that.” Right? So the release data kind of getting mushy and mushy and mushier in those areas, and that ability to say, “Oh, yeah, I had somebody look over it, and something horrible had happened,” some compliance violation or some crazy numbers in the shopping cart that can never add up. Can be even simple things where it’s, oh, this cost a billion dollars, or it’s euros instead of dollars but we are in a certain country. Those are things that you can catch more and more.

     

    Interviewer:                Fair. Guys, I feel like you didn’t let me, I didn’t get much of any words in here –

     

    Interviewee 1:             We feel the same way.

     

    Interviewer:                – but it’s okay. You know, I would – but I try to be a good host and let the guests talk, and you guys had it. Right? This is something, Torsten, you’re obviously very, very passionate and involved in and, Mark, it’s kind of, it’s your life. Right? And so I let it go. However, we are just about out of time, believe it or not, and we went on here for 15-plus minutes on this. But –

     

    Interviewee 1:             Can I just let people know where they can get the report? Would that –

     

    Interviewer:                That’s what I was going to ask you.

     

    Interviewee 1:             Sorry, I just wanted to make – yeah. So –

     

    Interviewer:                Go ahead.

     

    Interviewee 1:             – yeah, go to applitools.com/ema and you’ll be able to download the full report and access all the information there. So yeah, thank you, Alan.

     

    Interviewer:                Thank you, Mark. So it’s applitools.com/ema.

     

    Interviewee 1:             Yes.

     

    Interviewer:                Which also happens to be where Torsten works, at EMA, Enterprise Management Associates. Guys, it sounds like it was a really worthwhile research project here with some really good results and really good findings and insights. Look, this whole, there’s a reason Applitools has been as successful as it has. The whole idea of AI and testing is like peanut butter and chocolate in some way, and we can – but we barely scratched the surface in a lot of ways, right? There’s so much more yet to be done, and looking forward to hearing more about it in the future.

     

    Interviewee 1:             Yeah.

     

    Interviewer:                All right.

     

    Interviewee 1:             Thank you, Alan. Look forward to talking again soon.

     

    Interviewee 2:             Thank you.

     

    Interviewer:                All righty. Excellent. Torsten Volk, EMA, of course Mark Lambert from Applitools here on Techstrong TV. We’re gonna take a break and we’ll be right back with our next guest.

     

    [End of Audio]

  • Applying TestOps to Cultivate a Quality Culture

    Applying TestOps to Cultivate a Quality Culture

    Quality assurance is a universal concern, especially when it comes to application development. Enterprise applications are faster and more reliable than ever before. However, in addition to performance, application developers also must consider data security, regulatory compliance and cost-effectiveness, among other things. Organizations are embracing quality engineering (QE) as a foundation to create a software quality culture to address these challenges. Creating a quality engineering or TestOps environment is not simple, especially when QE departments must test for hundreds of applications across multiple enterprise environments. Whether it is a solution provider or a multinational corporation, applications often need to be tested across platforms, industries and geographies.

    TestOps encompasses a broader concept than just QE. With TestOps, there are testers and engineers using automated and manual testing to validate software for flawless application performance. In enterprise DevOps, the TestOps team serves as the bridge between development and deployment since they have a good understanding of the applications and a better perspective on the context and variables that can affect performance.

    Continuous testing is an integral part of any software development cycle. Quality engineers must be brought in early as part of Agile development to keep development on track and head off any problems that could escalate or become harder to resolve later. In Agile, testing becomes more crucial and frequent to ensure flawless performance.

    It pays to think horizontally when setting up a TestOps approach. For example, testing should be performed across different vertical environments such as media, manufacturing and finance. The goal is to create a test environment that can accommodate various industries.

    TestOps Across Environments

    In the age of SaaS and universal data access, it is essential to test across multiple platforms. In addition to ensuring enterprise performance, mobile web and remote data access must be tested. For streaming services, it is vital to test across mobile platforms, over-the-top (OTT) platforms such as Roku, Apple TV, Android TV and Fire TV, and even gaming consoles like PlayStation and Xbox.

    When shipping software updates, geography matters (especially when shipping across time zones), as one may never know when the software will be used or have control over the environment where they plan to use it.

    For instance, consider the challenges a leading mortgage lender faced with mobile application testing and how geography was a factor. First, Google Analytics was used to assess specific device and user requirements and ascertain performance in various parts of the United States since different regions use different carriers. Testing was then done across mobile platforms such as iOS, Android and combinations and across carriers since most banking and mortgage applications required one-time passwords (OTPs) and SMS-related capabilities. Further tests were implemented for new identity authentication strategies such as biometrics and facial recognition. Finally, performance testing by geography was conducted to determine whether a new feature or version would create issues.

    Testing for Geographies and Regional Compliance

    Localization testing is important since every region and every country has unique compliance issues and laws. To ensure proper localization, it is essential to perform complete testing, including lower environments and production systems.

    In any TestOps environment, the philosophy of continuous testing should already be part of the design environment, including understanding the nuances of production. Release readiness does not mean there are only a few known bugs in certain regions. It means the application has been tested for compliance in every geography where it could possibly be utilized.

    Testing Across Industries

    Testing application performance in specific industry settings brings along its own set of unique challenges. In manufacturing, for example, each sensor in an IoT device must be modeled against a conformance specification to determine if it can be automated using simulations. In media, one needs to test for different platforms, content management systems and business workflows.

    To accelerate time to market, one would want to support a dual-shift development lifecycle to gauge the consumer experience and get continuous feedback and also need a test environment with multiple automation tools such as Selenium, Appium, Cypress, Tricentis, UiPath, Test Complete and Winium.

    Artificial intelligence (AI) plays a larger role in testing, shortening time to market, and ensuring superior quality software. AI-driven test tools are optimized for domain-specific applications, such as testing for industries like media, manufacturing and fintech. AI can fill in the gaps to achieve optimization and offer a quicker turnaround with limited manual intervention.

    Any organization cultivating a TestOps culture needs to have specific technologies and tools in place. When assessing your testing technology stack, it is important to create an end-to-end testing ecosystem. This means the environments must be inclusive and capable of incorporating testing for unique applications, environments and scenarios. To support the needs of today’s Agile application development environment, one must plan for an agile testing infrastructure to keep development on track.

  • Techstrong TV: Value Stream Management & Value Creation

    Techstrong TV: Value Stream Management & Value Creation

    Lance Knight, ConnectALL President and COO, joins us directly from the Agile 2022 conference. Lance discusses the concepts of value stream management and value creation. The video and a transcript of the conversation are below.

                                         

     

    Mitch Ashley:            I have the pleasure of being joined by Lance Knight. Lance is COO and president of ConnectALL. Lance, good to be talking to you.

     

    Lance Knight:            It’s great to talk to everybody here again. It’s great to meet you as well.

     

    Ashley:            Nice to meet you. I know you’ve had a lot of conversations with Allen and great opportunity for me to have a conversation with you, too. For folks that might not know you or have seen you before, tell us about yourself and of course ConnectALL. Tell us a little bit about that.

     

    Knight:            Well, you know, I hardly ever done that before, told anybody about myself. But anyway, I’m Lance Knight. I’m president and CEO of ConnectALL. And you know, we’ve been around ConnectALL for a while. We’ve been spearheading in maybe in the forefront of this value stream management market and we’re excited to be part of this fun ride that’s value stream management. It’s been an interesting ride for me for years. I’ve been in this space since even before it was value stream management for years now. Just excited to see it grow and excited to be a part of___.com and we recently got nominated. It’s part of the 12 for value stream management solutions. So appreciate that. Yeah.

     

                                     As for myself, more about myself, you know, I’ve got a wife, a kid. I’ve got three kids. I’m just kidding now. But lots of cool things happening in our industry now.

     

    Ashley:            It’s fascinating I think one of the things that is fascinating about value stream management is it’s emerging but it’s been around for a long time. It’s not like it’s new. It’s in the manufacturing and you go back to the goal and all. You know, a lot of other things and Toyota. So it isn’t a new concept, but it is as we apply it to software. What do you think it’s now is the time for value stream management? I’m really curious. Obviously you made a big decision of you’re CEO of this company and that’s what you’re focused on. Why is now the time you’re going to be able to make this happen?

     

    Knight:            First, you know, you’re so right and I’ve said this quite a bit, is, you know, I started my career in manufacturing, dong value stream management work, understanding leans, systems thinking, all of those things to improve how manufacturing happened. And what I think is now as you take a look at what happened with Agile, what’s going on with DevOps, and the way that people are developing software, they’re thinking how do I make this even better. And I have a whole bunch of opinions why now. One of those is maybe a little controversial, is the goals, aims, and means, and the great vision of what an Agile transformation was supposed to do with my organization was never truly achieved.

     

    Ashley:            You mean it got overhyped and oversold?

     

    Knight:            Overhyped, oversold. It only really talked about how developers to work better. It never, you know what I mean?

     

    Ashley:            Oh, I totally know what you mean.

     

    Knight:            My organization has spent millions of dollars on these coaches and transformational people, and now you want me to do that with DevOps too and build a community and feel good about it, and sing Kum-Ba-Ya.

     

    Ashley:            [crosstalk] solve that for us, didn’t- Agile solved that for us too before that and you know, this succession.

     

    Knight:            Yeah, right.

     

    Ashley:            It’s an evolution, right.

     

    Knight:            And I see a little of that bleeding into value stream management, but I think why V.S.M right now, because it makes you look at your end to end flow. It makes you understand how you’re deciding what to do. Makes you look at how co goes through things. It’s a little more logical, not that those other ones aren’t. But it’s more specific. It’s got an age old set of things. And business people, the ones that make decisions on things can wrap their head around it. All right, so someone puts a request in here, it goes here. It goes through our agile processes. What do mean? I can finally get these reports to tell me how efficient my teams are? I think that’s what’s driving a lot of it. And ConnectALL we call it see, measure, automate. I can see my value stream now. I can measure and I can automate.

                                       

                                     And I think that’s just one reason why it’s taking off. And I think there’s some other ones. You know, well, we do agile, DevOps, value stream, what are we doing and how does this relate? And at ConnectALL we say this all the time, so it’s really interesting. Value stream management is a human thing. It’s a human endeavor. We talked earlier it’s been around forever with manufacturing. We didn’t have machines and computers out there measuring our value stream. We went and did that. We improved the flow. We looked at how parts moved around the shop floor. We mapped it. And we had some tools to help us. And a value stream management platform that’s what they’re there to do, help humans be more effective with managing requests to code in production, connecting it and automating the processes within it.

     

    Ashley:            yeah, there’s this phrase I heard, I don’t know where from. You know, you want to work on the business not just in the business. In other words, you have to sort of, that’s about continuous improvement, whatever that might be. But you have to understand it first. You have to be able to measure first. Where do we go improve. Another thing I wanted to run by you is I think one of the reasons why value stream management gets a lot of attention is we’ve elevated the value, the strategy behind software and how much that is importance of that to businesses, how much they’re counting on our ability to deliver. But you know, gone are the one year, 90 day whatever projects where you know, we think we’re going to get this spec, but we never do and something else pops out. And now we’re doing lots of little smaller races. How does a business know if I put a quarter in here I’m going to get my quarter’s worth of what I thought I was going to get there and how we can manage that process to help achieve those results? Is that fair do you think?

     

    Knight:            That’s fair too. And I think that’s another reason why it’s taking off right now, right. You know, my controversial point, but then there’s this other point of I’m never- I’m an executive. I’m over an organization. I’ve never seen what I’ve had move quick enough, move through. What are the bottom numbers? It’s invisible to me. Right? I have an initiative and I know I need these capabilities. I need this software. I need all these things. And as I’m saying that I’m watching just this initiative that’s an artifact in some system, I don’t know if that’s moving forward. I don’t know if I’m blocked. It’s a black box to me.

     

                                     And value stream management kind of opens some of that up as you kind of look at flow. And then also like you said earlier, how am I prioritizing that? I’ve got technical done. That’s really important to repair. But how do I prioritize that rather than somebody saying I need to fix this technical debt. Where you could look at it and say, not yet. This is more important than that. This new capability to make me competitive in the market is more important than if I fix that technical debt today. And those decisions aren’t able to be made at the right levels. And it’s also strategy execution where I can go and to sign my outcomes I want and my initiatives. And then have those get executed across all the organizations, because we have all this visibility and track it.

     

                                     So that’s the see part. And also the measure part where you talk about measuring. I can say, all right, now I’ve improved our processes by 25%. So now I can move quicker. The only thing that I think value stream management we have to be careful of is I don’t- and I’m seeing some of the other analyst firms do this. It’s not DevOps value stream management. It’s value stream management. And DevOps things are a part of that value stream.

     

    Ashley:            We have a value stream whether you’re doing DevOps or not, right?

     

    Knight:            You know, at ConnectALL my head of products says there’s a value stream now. Every company has it. You may not be aware that that is a value stream, but it is.

     

    Ashley:            I think it’s just been more elevated to our consciousness, right, because of DevOps.

     

    Knight:            Or more aware of the flow of value through it. Right. So, it’s interesting as it takes off and goes next stages of it and so on. I just want to make sure it stays the highest level thing from an organization perspective and that’s your value stream, right. You have DevOps tools of course and those are part of your value stream also. I just did a speaking slot somewhere about value stream management is not a feature and I went into that whole thing about how value stream management isn’t a feature of a tool or another tool. It4 is what you do and the tools and the platforms how you be more efficient. So it’s a different take I think than a lot of organizations have on it as they try to figure out how to build the best value stream feature in their product.

     

                                     So ConnectALL we have value stream management platform. It lets you see, measure and automate but if you’re not educated on systems thinking, lean principles, we can do some little stuff, but you would have to map it, use area waste principles, lean principles to think it through, right. I’ve got some mooda, if I used the word right. Maybe we’ll get some text about he said that wrong, which I probably did. Which anyway, that’s waste. And then you could go figure out how to fix that. Why am I having these things?

     

                                     So yeah, I mean I think it’s going to take off pretty well in organizations need to do this now because software is no longer that thing that that guy in the corner called Lance programs so that you’re more efficient in some of the manufacturing stuff you’re doing.

     

    Ashley:            Reminds me of, well, Jim and I first worked together in security. Security we evolving from sort of the people nobody knew that did this thing behind the scenes and couldn’t talk to them because they didn’t know what it was about, to more everyday people, system administrators, network administrators. And so you had to really think about the barrier to entry, right. Do you have to be a lean expert to do value stream management? Or can you step your way into it so there’s the whole idea for barrier to entry for your products? And how do you make that lower to get people started? And I know you had I think an announcement recently for ConnectALL. Can you tell us about that?

     

    Knight:            Yeah. So first, so you don’t need to be a lean expert to do it. Just have to maybe read a little white paper on it. And understand just the basics of it and it’ll help you through. So what we have done if you’re referencing to the fact that, and one of the reasons why we wanted to do this is that ConnectAL is going to offer a free online value stream management visualizer, designer where you can log in, put all your tools on there, draw the boxes of flow, decide what kind of flow loops you want, and just lay out your whole value stream, that’s actually yes, that’s’ what we have. That’s our big announcement coming next week is the free online value stream design. And it’s a nice little drag and drop environment where I can just drag on my tools. It’s going to have all the different tools on there, and then I can draw lines between them. I can set up my swim lanes, my boxes. We can do all kinds of things in order for you to lay out your value stream. You can name the links. You can name it so it’s personalized. You can put notes in there. And then it’ll give you a  PDF view of your value stream that you can export, you can go around and share that with your manage- here’s how our workflow works. And it comes from the years that I’ve been doing value stream management workshops on a big whiteboard, but we’re not getting together these days for these big whiteboard sessions. So you know, it’s got these different pallets and you can do all these things so that you can actually get a good vision to see your value stream and how things are flowing. And we’d be happy to help you lay that out if you wanted to get on with a session with one of our session architects to walk you through it. You can save them because maybe multi-session thing. Export them and share them with other people that can load it into the free value stream from their computer. And we’re really excited about it because I think it’ll really help people understand how they can use just this one part of value stream management.

     

    So you know, at ConnectALL we’re all about helping the humans be more effective at this, right. And it’s just a great opportunity for anybody’s interested in getting involved in value stream management to have a place where they can go just lay it out. It’s pretty robust. There’s some cool things. Yesterday I was messing with it. I drew a turkey in it. So, you know, Thanksgiving’s coming. So, we drew this little turkey in there, right. Somebody will probably do a Christmas tree as well. But the cool thing is I know some people will probably be like well, I can do that with this tool and this tool. That’s true you can. However, this one’s also free and we’re going to have all of the images and icons of the tools and the flow icons and all of those things preloaded so you can just drag and drop on there. And you’re off to the races of being able to start to tell the story around how you can improve value stream within your ___.

     

    Ashley:            Looking at things systemically, even if you think you know the system, the process really well, at least my going through things like that, you just go, I don’t really know what happens here. I kind of do but I don’t really know. So you can learn that or figure that out or somebody who’s mapping this out. And then to your point is once you’ve got that, now you’ve increased your understanding of what we do or the flow of what we do, where do we work? Right, you’re going to go with the theory of constraints and let’s look for a bottleneck. Let’s look for you know, we need a future over a technical debt is where value is. And you start to connect it into a system of creating software, which is I imagine more of your- the paid version of ConnectALL.

     

    Knight:            Well, so paid version will operationalize your connections. We can actually, you’ll see the flow of work and we do the integrations, the connections, the triggering, the capturing. We offer all the tough stuff the V.S.M. platform can do at ConnectALL. And normally for these talks I don’t talk much about our great product which is really, really great. We got a brand new release, 2-11 coming out that has some great- I’m using platitudes by saying great, but it has some really great features coming out in the future.

     

                                     Usually when I talk I want to help the industry. I want to help the space. And that  helps me of course. But here this free tool will get you up and running. You can see, measure, you can do some measurement stuff in there. And actually you can put your measurements in there and actually tell a story to your boss. So I was on a conference recently and someday said, well, to do value stream management you need to you know, go talk to your boss and have him approve the project and build a little community and figure out how you’re going to adopt value stream management principles and do all this stuff. And I started to feel like here we go again with community and adoption and the feel good part of it.

     

                                     And all of that is really relevant when you think about agile, when you think about DevOps, that is relevant. DevOps is a culture shift. And a culture shift, I’ve spoken on this before, is transforming operations and DevOps to work so well together and when operations is normally commanding control, right, can’t have access to those servers because this reason. I used to be that guy, so I can speak to it with great understanding. And having people change that mindset and culture so they can collaborate better, this is not. This is value stream management. You map the value stream. You look for waste. You point it out. You try to figure out how to remove it. And you work with people to do it, don’t get me wrong. But there’s no sprints. There’s no ceremonies. Once you’ve done it, you look at it, figure out how to improve it. You improve it. You modify your map. And by the way you can save these and come back. And then you look for the next place to improve flow. I’m for the other stuff. I’m just saying this is not. This is value stream management. It’s lean thinking. It’s been around forever. Doesn’t require all this other stuff, go do it.

     

    Ashley:            I think from an end user perspective I would think that now having a way of dipping my toe in the water, maybe more than that to really understand what’s going on. And I can decide. Now I can decide to go to the next place, whatever that looks like with ConnectALL.

     

    Knight:            Yeah. I go map it out and then I can talk about it.

     

    Ashley:            Not making a big bet that this thing is going to do stuff for us, but I just don’t know. Right. You know, you’re going to get value unintended.

     

    Knight:            So we’re really excited about it.

     

    Ashley:            Well, congratulations. When this airs actually I think it’ll be when this is released. So, how do folks get ahold of this? Is this ConnectAll.com?

     

    Knight:            Right, ConnectAll.com you’ll see a link there in the home page to go to designer and it’ll come up. We’re not looking for tons of lead information. We don’t want to know how many people run your company and you know, got control over that. I mean all we want is maybe an email and you’re off to the races and you can save them and do all the stuff. And just use it to take that first step. And we have some partners who are using it to run consulting sessions and different things like that. So, I think it’s something that nobody’s done yet. I think it’ll really help move value stream management forward.

     

    Ashley:            Great. Well, congratulations on the launch of this entry point, the value stream designer. And definitely I’m going to check it out. So, appreciate-

     

    Knight:            I’d be happy to walk you through it. I’ll send you a quick copy of the turkey we made.

     

    Ashley:            I would love to see the turkey. All right, Lance. Lance Knight, CEO and president with ConnectALL. Thanks for joining us today, Lance.

     

    Knight:            Thank you.

     

    Ashley:            Take care.

     

    [End of Audio]

  • Future of DevOps: Trends to Watch

    Future of DevOps: Trends to Watch

    Technology is transforming every aspect of industry, and digitalization and automation have flourished in the past few years. DevOps has established itself as an indispensable software development methodology for successful digital transformation.

    According to a recent market survey, the DevOps market will exceed $20 billion by 2026, growing at a CAGR of 24.7% from 2019 to 2026. DevOps has facilitated rapid and dependable software development, delivery, improved quality and higher customer satisfaction.

    DevOps, too, is continuously evolving to meet the changing demands of the corporate environment. Let’s look at the future of DevOps and the trends you should expect to see.

    Serverless Computing

    DevOps teams will see a rise in the adoption of serverless architecture, mainly because of the massive capital and operational costs of server infrastructure. Numerous DevOps groups are working with modular components offering an overview of the pipeline. Moving these modular components to serverless architecture will empower groups to mitigate a few pieces of errands identified with pipeline architecture and focus on deployment and development of the product.

    Serverless computing streamlines overall DevOps operations, including the complete software life cycle, from development to deployment, testing and maintenance in one place. Also, serverless computing reduces the developer’s workload by taking care of server maintenance, such as cloud monitoring and system updates. Other prime USPs include flexibility, reliability, speed and cost-efficiency.

    Microservices Architecture

    Microservices architecture has been gaining popularity as an alternative to traditional monolithic architecture in the IT industry, and this trend is expected to continue. Microservices architecture is a pattern of organizing computer systems into segments that can scale on demand. A microservices architecture pattern is the next evolution of service-oriented architecture (SOA) that structures an application as a collection of coupled services. If one piece of the application fails, microservices make it easy to identify and contain the issue without disrupting the whole application.

    DevOps and microservices architecture enable decentralized teams to innovate rapidly, maintain control over their tech stack and standards, track performance metrics, supervise the development and release cycles and reduce overall consumption time by achieving shorter software release intervals.

    Low-Code/No-Code Applications and Platforms

    To keep up with the demand for new applications and features, many organizations have turned to low-code/no-code development to deploy applications swiftly. Adopting a low-code approach with DevOps is a game-changer. It enables developers to experience higher agility and gives the organization a competitive advantage in the fast-paced, demanding software market.

    Low-code applications allow non-technical ‘citizen developers’ to contribute to software development with the help of visual interface integration. In addition to end users, developers and DevOps engineers are set to play a crucial role in this trend, using low-code platforms and tools that offer drag-and-drop elements and other extensions. These low-code application development tools support every aspect of software development, including app ideation, analysis, software design, code development, quality assurance, deployment and documentation.

    DevSecOps

    Many large enterprises are integrating and automating security into DevOps processes. The transformation from DevOps to DevSecOps is expected to gain even more momentum in the future. According to a forecast, the DevOps market size is expected to reach $5.9 billion by 2023 at a CAGR of 31.2% during the forecast period.

    More companies will embed security controls earlier in the software development cycle to ensure application security. Tools and frameworks will simply integrate with CI/CD pipelines and testing tools will see a rise in 2022. DevSecOps empowers developers to consistently track, monitor and remediate security defects during the various stages of development while improving the speed of delivery and quality.

    Kubernetes and GitOps

    Kubernetes is an open source container orchestration platform emerging as an infrastructure trend on the operations side. Kubernetes will expand further into the software development space in 2022 and beyond. It will ease the task of starting a local Kubernetes cluster on a developer’s machine. Kubernetes security is expected to take center stage as well, through infrastructure-as-code (IaC) scanning.

    GitOps is a Kubernetes-related infrastructure trend that allows developers and IT operators to use Git to manage clusters and deliver efficient, secure applications without misconfigurations. It automates releasing incremental updates, and consistent delivery will enable developers to build, test and deploy software rapidly.

    AI and ML

    Artificial intelligence and machine learning have already transformed the way businesses operate nowadays. In 2022, it is anticipated that AI/ML will further streamline and accelerate every phase of the SDLC. It enables DevOps teams to enhance their performance through intelligent automation, improving feedback and managing alerts.

    Advanced analytical methods like predictive analytics are becoming increasingly important as the market evolves. AI can enable proactive investigation of the DevOps pipeline to identify problems and offer possible solutions. AI and ML integration also enable deep learning, data mining, statistics and modeling, becoming an integral part of your business’ workflow.

    Conclusion

    DevOps continues to evolve and pivot to drive significant IT transformations that directly bolster business goals and objectives. It will assist in enhancing the company’s ability to design, produce, launch and maintain high-quality software solutions. The above trends will help organizations move rapidly beyond automation, focusing on continuously improving outcomes. These trends are catalysts for creating a regular and dependable release pipeline and building better communication between development, IT and business teams.