Category: DevOps Toolbox

  • SnapLogic Extends DevOps Reach via GitHub

    SnapLogic Extends DevOps Reach via GitHub

    SnapLogic has extended the reach of its SnapLogic Enterprise Integration Cloud into the realm of DevOps by making it possible to store integrations creating using Snap connectors in GitHub.

    The company also announced integration with the DC/OS platform from Mesosphere along with a new service catalog and several updates to the Iris artificial intelligence (AI) software it uses make recommendations concerning the construction of integration pipelines. At the same time, SnapLogic has enhanced both its monitoring tools for application programming interfaces (APIs) and search tools for discovering broken integration pipelines.

    Craig Stewart, vice president of product management for SnapLogic, said support for GitHub makes it easier for developers at all skill levels to share preconfigured integrations via GitHub alongside all the other code they already share via the widely employed repository. That capability also makes it easier for DevOps teams to include preconfigured Snap connectors within a continuous integration/continuous development (CI/CD) pipeline, he noted, adding those integrations now can be more easily discovered within a Patterns Catalog to help developers discover and reuse integration patterns.

    Support for Mesosphere, Stewart said, is being provided by customer request. That effort will then be extended to include support for Kubernetes, which SnapLogic expects will become the de facto standard for container orchestration engine.

    Finally, SnapLogic updated its Iris AI software to provide an additional option for users to build their integration workflows starting with the last Snap of the integration pipeline. Previously, the Iris recommendation engine started with the first Snap connector. Now, when a user first places the last Snap on the canvas, the Integration Assistant will display suggested previous Snaps until the pipeline is complete. Users also have the option to start with the first and last Snaps of the pipeline and the Integration Assistant will work from the outside-in to recommend Snaps to build out the entire pipeline.

    SnapLogic is making investments designed to facilitate self-service capabilities that promote reuse of integrations, Stewart said. A recent survey of 500 IT decision-makers conducted by Vanson Bourne on behalf of SnapLogic finds on average organizations expect to generate a 547 percent return on their data investments, increasing revenue by an average $5.2 million by using data more effectively. But the survey also finds that on average organizations are using only half (51 percent) the data they collect or generate, and data drives less than half (48 percent) of the decisions being made. Organizations will spend $1.7 million on an average to operationalize data over the next five years to rectify that problem, according to the survey results, which is more than double what they currently spend.

    Much of that effort will be driven by so-called citizen integrators that reside within various lines of business outside the traditional IT department. In many cases those citizen integrators will be leveraging prebuilt integration patterns created by developer working within the internal IT department. The challenge now is finding a way to make sure all those integration patterns can be slipstreamed into a larger set of DevOps processes that are increasingly spanning the entire enterprise.

    — Mike Vizard

  • OverOps Brings Machine Learning to DevOps

    OverOps Brings Machine Learning to DevOps

    OverOps has launched a namesake platform employing machine learning algorithms to capture data from an IT environment that identify potential issues before a DevOps team decides to promote an application into production.

    Company CTO Tal Weiss said the OverOps Platform is unique in that, rather than relying on log data, it combines static and dynamic analysis of code as it executes to detect issue. That data then can be accessed either via dashboards or shared with other tools via an open application programming interface (API). The dashboards included with the OverOps Platform are based on open source project Grafana software.

    That approach makes it possible to advance usage of artificial intelligence (AI) within IT operations without necessarily requiring that every tool in a DevOps pipeline be upgraded to include support for machine learning algorithms, Weiss said.

    OverOps also includes in the platform access to an AWS Lambda-based framework or separate on-premises serverless computing framework to enable DevOps teams to also create their own custom functions and workflows.

    Weiss said OverOps is designed to capture machine data about every error and exception at the moment they occur, including details such as the value of all variables across the execution stack, the frequency and failure rate of each error, the classification of new and reintroduced errors and the associated release numbers for each event. Log data is, by comparison, relatively shallow in that it is challenging to determine precise root cause analysis when trying to troubleshoot an issue, he said, noting the OverOps Platform offers visibility into the uncaught and swallowed exceptions that would otherwise be unavailable in log files.

    DevOps teams spend an inordinate amount of time analyzing log files in the hopes of discovering an anomaly. But as IT environments continue to scale out, the practicality of analyzing millions, possibly even billions, of log files becomes impractical. OverOps is making the case for employing machine learning algorithms to analyze events before the log file is even created, which eliminates the need to find some way to store log files before they can be analyzed.

    There’s naturally a lot of trepidation when it comes to anything to do with machine learning algorithms and other form of AI to manage IT. But as the complexity of IT environments continues to increase, it’s clear DevOps teams will need to rely more on AI to mange IT at levels of scale that were once considered unimaginable. For example, while microservices based on containers may accelerate the rate at which applications can be developed and updated, they also can introduce a phenomenal amount of operational complexity. Most DevOps professionals would rather automate as much as possible the manual labor associated with operations, especially if that leads to more certainty about the quality of the software being promoted into a production environment.

    Of course, while making use of machine learning algorithms to analyze code represents a step forward in terms of automation, it’s still a very long way from eliminating the need for DevOps teams altogether.

    — Mike Vizard

  • When AI Met DevOps: Machine Teaching

    When AI Met DevOps: Machine Teaching

    One of my favorite quotes about artificial intelligence (AI) isn’t from a data scientist or tech industry analyst. It’s from a doctor.

    When asked if AI would eventually replace radiologists, Dr. Curtis Langoltz of Stanford University pithily replied “No, but radiologists who use AI will replace those who do not use it.”

    This great insight is highly applicable to IT. Yes, there are many ways we can use AI methods to fully automate a broad range of DevOps tasks. Companies, including CA Technologies, are actively involved in the development of solutions that enable our customers to do exactly that.

    Algorithmic machine learning, however, doesn’t just empower systems to perform tasks and solve problems autonomously. It also makes them great active partners with human beings. In fact, much of what machines learn they also wind up teaching.

    The synergy between AI and human radiologists, for example, stems in part from the fact that digital systems can differentiate about 200 levels of gray in a diagnostic image—compared to only about 16-20 that are discernable by the human eye. Train an AI system with enough images, and that precise cognitive power can more effectively detect that something is going on.

    But for the most effective diagnostic process, you don’t just depend on that detection alone. You use that detection to empower a human diagnostician who can apply a broad understanding of pathologies and deep experience with the complexities of individual patients to deliver the highest quality care.

    In DevOps, we can do the same. We can use AI to capture insights that teach us how to continuously optimize our workflows and processes. We can also use our AI learnings to push our work up higher on the value chain.

    More specifically, the synergy between AI and human intellect can:

    Make development smarter. The speed, quality, and efficiency of development pipelines can be affected by all kinds of subtle factors. A less-than-optimally designed API, for example, can be a small but chronic stumbling block to everyone who has to use it. Scrum outcomes can be undermined by anything from a particular type of technical challenge to a nascent personality conflict.

    By capturing a rich set of DevOps metrics and applying machine learning to those metrics, development leaders can discover process bottlenecks and skilling shortfalls. They can better coach individuals and promote team collaboration. The result: a better working environment that facilitates digital agility for the enterprise and higher satisfaction/retention for valuable employees.

    Make ops smarter. Enterprises are running increasingly volatile and complex workloads on increasingly hybridized infrastructure. At the same time, the tolerance of internal and external users for latency and outages continues to approach zero. There are also real costs associated with performance problems.

    The elastic capacity of public and private cloud does much to help with workload volatility. But adding cloud capacity also has its costs—and end-to-end application performance often depends on back-end systems that are not cloud-based. So not every performance issue can be solved by simply throwing more capacity at it. Nor should it be, if a rearchitecting can fix a bottleneck less expensively.

    Here again, AI can teach us a lot. We can uncover opaque interdependencies in processing load and data throughput. We can spot conditions when it may make business sense to throttle cloud costs that aren’t cost-justified. We can even better understand the real-world conditions—whether patterns in customer behaviors or our own marketing programs—that are driving our demand spikes and troughs. All of this helps us deliver consistently responsive digital experiences at a cost that makes good business sense.

    Make security smarter. AI is already being broadly implemented in security solutions such as endpoint protection and threat response to automate the detection and neutralization of anomalous activities in the enterprise environment. But effective multi-layer security isn’t just about finding and stopping exploits. It’s also about building applications that are themselves inherently less vulnerable to hacking. This is the essence of DevSecOps.

    AI has huge potential value here. We are writing a rapidly growing volume of increasingly sophisticated code. It is very easy for subtle vulnerabilities to hide in that code. As our development practices become more complex—often including multiple contractors—it becomes more difficult to understand exactly where and why these vulnerabilities were introduced into our code. Machine learning can teach us the answers to these questions, so we can more proactively secure our data and our businesses.

    It’s especially interesting to consider what may happen as we start to apply AI to DevSecOps across our organizations, as well as within them. More diverse inputs enable machine learning to discover more factors that impact code pipeline performance. By aggregating our knowledge about how we build, deliver and secure our code, we are all likely to benefit with better practices and stronger guardrails.

    — Cameron Van Orman

  • 6 Ways AI and ML Will Change DevOps for the Better

    6 Ways AI and ML Will Change DevOps for the Better

    There’s been a lot of media attention in recent years about how artificial intelligence (AI) and machine learning (ML) are going to change the world—how they’re going to create new and interesting applications in fields as diverse as education, law, health care and transportation. This may happen. But if I had to bet on a use case where AI and ML will create a tangible, lasting impact, I’m putting my chips on DevOps.

    DevOps is all about automation of tasks. Its focus is on automating and monitoring every step of the software delivery process, ensuring that work gets done quickly and frequently. While it doesn’t eliminate human tasks—far from it—it does encourage enterprises to set up repeatable processes that promote efficiency and reduce variability.

    AI and ML are perfect fits for a DevOps culture. They can process vast amounts of information and help perform menial tasks, freeing the IT staff to do more targeted work. They can learn patterns, anticipate problems and suggest solutions. If DevOps’ goal is to unify development and operations, AI and ML can smooth out some of the tensions that have divided the two disciplines in the past.

    Here are six ways AI and ML can and will change DevOps for the better.

    Promoting Feedback on Performance

    One of the key tenets of DevOps is the use of continuous feedback loops at every stage of the process. This includes using monitoring tools to provide feedback on the operational performance of running applications. This is one area today where ML is impacting DevOps already. Monitoring platforms gather massive amounts of data in the form of performance metrics, log files and other types. Advanced monitoring platforms are applying machine learning to these datasets to proactively identify problems very early and make recommendations. These recommendations go to the DevOps team members so that they can ensure that the application service remains viable. Machine learning is enhancing the continuous feedback loops that are critical to DevOps.

    Enabling Communication

    Communication and feedback is always one of the biggest challenges when organizations move to a DevOps methodology. Human interaction is vital, but with so much information flowing through the system, teams need to set up a wider variety of channels to set and revise workflows on the fly. Using automation technology, chatbots and other systems initiated by AI, these communications channels can become more streamlined and more proactive.

    Correlate Data Across Platforms and Tools

    To operate efficiently, DevOps teams need to simplify tasks. This is getting more difficult as environments get more complex. Start with monitoring tools: Teams tend to use multiple tools that monitor an application’s health and performance in different ways. Machine learning applications can absorb these data streams and find correlations, giving the team a more holistic view of the application’s overall health.

    Manage a Flurry of Alerts

    Since DevOps encourages teams to “fail but fail fast,” it’s critical to have an alert system that spots a flaw quickly. This tends to create scenarios where alerts are coming fast and furious, all labeled with the same severity, making it difficult for teams to react. Machine learning applications can help teams prioritize their responses based on factors such as past behavior, the magnitude of the current alert and the source that specific alerts are coming from. Humans can set up rules, but machines can help manage these types of situations when too much data overwhelms the system.

    Evaluating Past Performance

    AI/ML also has the potential to help developers during the application creation process. By examining the success of past applications in terms of build/compile success, successful testing completion and operational performance, machine learning algorithms could make recommendations to developers proactively based on the code they are writing or the application that they are building. The AI engine could direct the developer in how to build the most efficient and highest-quality application.

    Software Testing

    In the future, we could see AI/ML applied to other stages of the software development life cycle to provide enhancements to a DevOps methodology or approach. One area where this may happen could be in the area of software testing. Unit tests, regression tests, functional tests and user acceptance tests all produce large amounts of data in the form of test results. Applying AI or machine  learning algorithms to these test results could identify patterns of poor coding practices that result in too many errors caught by the tests. This information could then inform the development teams so that they can become more efficient in the future.

    Similarly, leveraging historical data, AI/ML could be used to fine-tune deployment strategies as applications are moved from Dev to Test to Production environments.

    — Mike Hughes

  • Choose Your Toolstack — and Manage It — Wisely

    Choose Your Toolstack — and Manage It — Wisely

    Every DevOps toolstack has a control point, be it a bus, an app, a management app or whatever. Something always is in control and driving the other pieces of the architecture to do their part. It is a basic tenet of automation that jobs are mostly kicked off by something in the software hierarchy.

    In DevOps, most of us have defaulted to Jenkins for a large chunk of that automation. It was the center of the continuous integration/continuous delivery (CI/CD) side, and increasingly it is used to run both sides of DevOps (as I have called it previously, both DEVops and devOPS). CloudBees and recognized this trend, and an increasing amount of control and process functionality has been turning up in Jenkins.

    On the other side, there are organizations running all of devOPS from deployment tools such as Ansible or Puppet. Again, that’s not a terrible solution, since they can configure the software on the server and make sure all is ready to run.

    But neither of these toolsets was originally envisioned with the idea it would control as much of the DevOps stack as they currently do.

    If you read me regularly, you know I’m not a huge fan of this type of default control structure. There is a lot of functionality that can be laid over the CI/CD and environment automation tool. But to do some of it, the tool needs to be above the day-to-day functionality these tools specialize in.

    ARA can do it—and honestly, for a first brush at connecting the worlds represented by CI/CD and configuration management tools, is pretty impressive. Other vendors have come at it from a service bus perspective, allowing everyone to communicate through their tools and doing roll-up reports based upon what has been integrated. That’s not a terrible solution either, if it works with or can be made to work with your tools.

    ARA as a market is short on the monitoring and management end of the process when the app is deployed and clipping along. It’s getting better, and hopefully ARA that is fully integrated with monitoring tools will be the next step in this evolution, but for now most companies are handling tooling, monitoring and response to that monitoring outside of the ARA system.

    The queue systems tend to lag behind ARA in their ability to correlate and coordinate, but again this is getting better. They know only what is reported through them, so as they gain add-ons, expect they’ll handle the overall stack better.

    The long-term goal is, after all, to run from commit to release with only designed “go/no-go” checkpoints involving humans, and then to monitor, with humans getting notifications only as needed. How we get there may not be so important—or it might.

    In the market, the sand is still shifting too fast for vendors to keep up. In any given implementation, how fast you adopt all the new products is a function of choice. Use controlled chaos, changing things out for the better only when you have a quantifiable benefit and they fit into your overall architecture. Most importantly, no matter which toolset you choose to use to control the check-in-to-release cycle, new tools you introduce should integrate with it.

    So what am I recommending? That you choose a control tool that fits your specific environment and needs, and that you make certain when choosing new tools you have a path to get them integrated with that control tool. Nothing more. Don’t put a tool in charge of half or all of your DevOps toolstack just because it was there. Have a real reason, then use that reason as a gauge when adopting new tools.

    — Don Macvittie

  • DevOpsGroup Academy Launch: Interview with Head James Harvey

    DevOpsGroup Academy Launch: Interview with Head James Harvey

    DevOpsGroup Academy launched in July with a mission to eradicate the UK’s tech skill shortage and invest in the high-performing IT professionals of the future. I recently had a chance to speak with DevOpsGroup Academy Head James Harvey about the mission and vision for DevOpsGroup Academy and what he sees for the future of technology learning.

    Hartley: Can you tell us more about the launch of DevOpsGroup Academy in July 2018, your university connections and addressing the skills gap?

    Harvey: I think the biggest challenge that we’ve seen is that, even though you are getting hundreds of thousands of students graduating with computer science degrees over the years, the gap that we are experiencing between what people are learning at university versus what the commercial world is demanding is astronomical.

    We’ve seen, with our intern program, for example, if you compare an intern who has spent a year with us directly with someone who is freshly graduated—the difference in terms of ‘work readiness’ beyond comprehension. It’s insane!

    The skills gap is one of those things that is very difficult for universities to keep on top of, because things in the tech world change rapidly and dramatically. So ultimately what you are learning is often outdated by the time you finish anyway. It’s a different skill set that you learn at university: It’s very lecture-based, it’s very academic and it’s very blue-sky. And it’s very much “in an ideal world” setting.

    Whereas what we are trying to do with The DevOpsGroup Academy is to say, when things aren’t quite so perfect and when you do have to be a bit more pragmatic and apply some of the things you have learned on a bigger scale, then we can help bridge that gap.

    The first instance of the launch of The DevOpsGroup Academy is really going to be focused on the fact that these are commercial courses that we are selling to people who are looking for a career change, looking to enhance the skills of their team or whatever it might be. And the fact that we run these courses with students, interns, graduates, associates, apprentices and placements even will become a huge selling point for us. But the initial target for us is going to be those small businesses, SMEs, and individuals looking to skill up.

    Hartley: Can you talk about The DevOpsGroup Academy and what you’re offering that’s unique?

    Harvey: There are, of course, plenty of DevOps courses that you can find out there from many other suppliers. The big USP (unique selling proposition) for us when it comes to public training is that we are not this “one-stop shop” for agile DevOps or tech training. What we pride ourselves in is having a carefully selected portfolio of courses that align with our company values, and aligns with the way in which we go in and facilitate a transformation. And critically, these courses and our portfolio have been chosen so that they form this kind of learning journey, this development pathway, so that if you take one course there is something else that is related that makes sense to go and do elsewhere.

    I mean, it would be very easy for us to sit down and say, “Right, we are going to run these 50-odd training courses that we can offer.” But then, as I said, the difference there is that we would just become another training company, which we are not trying to do.

    The other USP that we have got is that we don’t typically use trainers to deliver the training. We use our practitioners and the people who are doing this day in and day out to deliver the training, because again we feel that it allows them to give you a more realistic demonstration of what it is we are trying to teach you. Rather than somebody who has been training for 10 years and has lost touch with what the pragmatic, real-world looks like.

    So it allows us to take all of the experience that we have got in DevOpsGroup and regurgitate that, as not just textbook training, but as this real-life experience that we’ve been through. So learn from our mistakes and learn from our successes. That’s the wisdom that we can impart.

    Hartley: So are there any specific new courses that you are launching with DevOpsGroup Academy?

    Harvey: The main focus at the moment will be around the Agile Foundation certification, and we are in the process of getting toward the end of launching a certified DevOps Foundation training course.

    There are a few other DevOps training courses out there, but it’s definitely not a crowded marketplace and it’s something that we’ve developed internally, because we didn’t feel that there was a suitable option out there for that kind of agnostic learning, to get a DevOps Foundation certificate for.

    Hartley: Going back to universities and government, can you talk more about those partnerships?

    James Harvey: You can look at our partners’ page on the new DevOpsGroup Academy website for a bit more information on our current university partnerships.

    We are very much focused on working with all of the local universities in Wales at the moment, so we do a lot of work with the National Software Academy that’s based at Cardiff University. They do things a little bit differently in that their computer science students are working in what is a much more commercial environment, where they do a lot more hands-on with workplace technologies and things like that.

    DevOpsGroup Academy has introduced a summer placement scheme with the National Software Academy, to help their students take the learnings that they’ve got from university to a different level. Part of what made that happen is that, because we’ve had that relationship with them up until now, the level that the students are at means that they can come in for a few months and actually do some work that adds some value, both for us and for them.

    Beyond that, we are doing things like giving guest lectures at universities across Wales and just trying to get involved as much as we can. The end game for us, really, is to help facilitate and extend the learning that the students get in the classroom. So how can we ultimately become a finishing school for students wanting a career in IT? And it’s about doing that while working as closely as possible with our university partners and without completely condescending the university system.

    Hartley: Going back to addressing the skills gap. What can you say about the limitations of university computer science courses?

    Harvey: It’s a different type of learning. Universities are crippled somewhat by the overarching governance around getting syllabuses together. And what we have found is that a lot of the time there is a resistance to change those.

    So, for example, if you were ever going to have a commercially relevant computer science degree, for example, you would be looking at revising that syllabus every year at the very least, considering how technologies moves on and what becomes relevant and not relevant.

    And I think it’s very difficult then, because there is this emphasis at the university level on broad learning and not specializing in a specific thing. So it’s difficult then for students to pick up things and to get in and get their hands dirty and add value to what it is they are doing. So there is a lot of general learning that goes on, and the general feedback that we get is that they just don’t have that space to get their hands dirty. To try things out. To fail, if need be.

    And that’s the difference between the university and us. The whole point of our intern program, for example, is that the whole point of their learning over the year is continued. And their learning is accelerated.

    So I do think that it’s probably fair to say that they learn more from the 12 months they spend with us, than they do from their first two years at university. But that’s because we are able to accelerate that academic learning by ultimately letting them get hands on with the tech. And that’s one of the most difficult things for universities to be able to do at scale: to allow students to get into the nitty-gritty, if you like, because it’s very difficult to govern that and test that and so forth.

    Hartley: In terms of teaching agile and DevOps processes at university, how does that work?

    Harvey: Really badly! Those that are doing it, generally, will take agile or DevOps and they will treat it as a module. And that is taught for a semester.

    Instead, what you should be seeing is that agile and DevOps should be this underpinning thing that runs throughout your entire three years at university. So everything that you do, you are constantly learning about how to implement that in an agile way.

    I’ve been doing guest lecturing at universities in Wales for the past five years and that has been my underlying theme—that there are some things that shouldn’t be taught as modules. They should be underpinning everything that you are doing. And that’s one of the big things we learn from our interns—on the first few weeks they are here we will give them an Agile Foundation training course and then that underpins everything that they do from that point onward whilst they are here.

    So time and time again, with regards to specific agile and DevOps practices and ways of working, we are seeing that it is being shoved in as a module and, ironically, more often that not towards the end of a degree course. Rather than at the beginning, when students could benefit from learning about it off the bat.

    But then again, a lot of university work is done in isolation and it’s working on small solo projects, which doesn’t lend itself well to a lot of agile and DevOps practices and processes. The types of projects that they are working on at university simply don’t mirror what you find in a commercial world. Universities work in isolation; they are not working with businesses and local people; the curriculum is all in isolated modules that are not interlinked in anyway. The barriers need to be brought down if we are going to work in a collaborative way.

    Hartley: Is breaking down those barriers the long-term mission of the DevOpsGroup Academy?

    Harvey: Yeah, definitely! I think the long-term goal for us is all about that accelerated learning. Learning as you are working and as you are going along. And allowing and giving people the space to do that, to learn from each other, to collaborate.

    We are ultimately disrupting and changing the norm of what goes on right now, because the one thing about university learning is that it doesn’t set you up for starting your career once you’ve graduated. You are learning about these small, isolated examples that don’t really set you up for work in the commercial world.

    So if you give these students the space and environment to do this accelerated learning, then within six months they are very close to what you would consider to be an associate-level engineer that comes into our organization. Which is extremely powerful for us. And it sends a really strong message about the educational methods that we use and the partners that we work with as far as training is concerned.

    Ultimately we are doing this, as we’ve always said, because we want to invest in the next generation of talent. We are expanding rapidly and we are doing this because we want to give these kids jobs, in the end.

    Hartley: What about the start-up scene in Wales and Welsh government funding for DevOps training for SMEs?

    Harvey: Yeah, one of the pilot schemes that we are working on at the moment are “boot camps”—the idea being that an SME would send one or two people along to spend a week with us, in which time they would do two training courses, agile and DevOps together, and then they would get a final day of workshops and bringing everything together. The learners would then go away with two commercial certifications and a DevOpsGroup Academy diploma. And the Welsh government will be funding 50 percent of the cost of that course for any Welsh SME that takes part.

    The other thing is that we are working with the local universities across Wales to run these boot camps with students as well. So the idea is that we are not watering these down, and creating a boot camp specifically aimed at students. We are giving them the full commercial experience.

    And it also a great opportunity for SMEs to come to the boot camps to meet some really exceptional students that they may think about recruiting at some point down the line. And it’s a great opportunity for the students to work with industry professionals.

    Hartley: So how does an SME get the funding? And how do the boot camps work in practice?

    Harvey: Getting the funding is easy. It’s a case of paying for the course and reclaiming 50 percent of that back. You simply fill out a two-page form, which we can help with, and it means that Welsh startups get a fantastic deal out of it.

    Practically, we have great training facilities in Cardiff. And we have access to similar facilities across Wales and the rest of the UK, for when we take the boot camps on the road. Ultimately we get people together to run through the two training courses and multiple different activities and games.

    And then there’s a day to bring everything together, to take all that learning that you’ve had in the classroom and apply it with some workshop activities or some discussion about how this might help solve some of the challenges you have within your organization. And again, the idea of hosting it here is that we have all these different practitioners who can jump in and out of sessions to offer their advice, as an engineer or as a consultant or from whatever role you can imagine.

    So it really is that next step on from just coming in for a training course. It has the emphasis on accelerated learning, so you tick three boxes with one trip, plus you get all of the networking and the collaboration that comes along with the boot camps as well.

    Initially we’ll pilot the boot camps with DevOps and agile, as a combination. Then ultimately the sky’s the limit, really. There’s a huge potential for technical-based training alongside the DevOps and agile courses. Both in terms of agile working practices and how we deliver this training, this is all too often the neglected part of IT, both at university and at a commercial level.

    — Adam Hartley

  • HPE Brings Storage Deeper into DevOps Realm

    HPE Brings Storage Deeper into DevOps Realm

    Hewlett-Packard Enterprise (HPE) today moved to bring storage further into the realm of DevOps by integrating its 3PAR storage systems with VMware vRealize Orchestration software, open source Kubernetes container orchestration software and the Red Hat OpenShift platform-as-a-service (PaaS) environment built on top of Kubernetes. HPE 3PAR storage systems already support Docker containers and the DC/OS private cloud platform from Mesosphere.

    In addition, HPE is making available pre-built blueprints for Chef, Puppet and Ansible automation frameworks, as well as software development kits written in Ruby and Python that can be used to create custom integrations.

    Finally, HPE announced that HPE InfoSight, a cloud-based service that employs machine learning algorithms to predict when a drive might be about to fail or when application performance might be adversely impacted by I/O contention. HPE gained the core technology for that service when it acquired Nimble Storage last year.

    Vish Mulchand, senior director of product management and marketing for HPE Storage, said machine learning algorithms and other forms of artificial intelligence (AI) will soon be playing a much larger DevOps role. Research indicates that more than half of all support issues involve storage systems, and AI technologies will be able to identify most of those issues long before they impact applications, he says.

    Mulchand notes HPE has been adding integrations with various DevOps and container technologies to make it easier to scale applications as needed. The goal, he says, is to make storage management a more natural extension of any set of DevOps processes.

    In general, providers of storage systems have been investing heavily in automation frameworks designed to make it easier for storage administrators to manage ever increasing amounts of data. The next challenge facing IT organizations is to be able to provision storage resources in a way that doesn’t hamper the rate at which those applications might be deployed.

    It’s not clear to what degree storage administrators will be subsumed into DevOps teams. In most cases, storage administrators are looking to make available a pre-determined set of storage resources available to developers on a self-service basis. The amount of those resources is determined by the characteristics of the application, which storage administrators can gain earlier access into by participating in the DevOps process.

    Going forward, Mulchand says HPE expects competition between storage vendors increasingly will be defined by the quality of experience they provide IT organizations rather than the performance of the storage systems. Every storage vendor has embraced Flash storage systems that can address the performance requirements of just about any type of application workload, he says, so making it easier to access those storage resources and make sure they are constantly available is the next major opportunity.

    As more automation gets introduced into IT environments, there’s no doubt the role of the average storage administrator will change. Rather than manually provisioning resources, storage administrators will be able to develop policies for classes of applications that are automatically implemented whenever storage systems need to be provisioned. As storage administrators learn the characteristics of those application in a production environment, those policies will be further fine-tuned. The good news is that storage itself no longer needs to be seen as a bottleneck from either a performance or DevOps perspective.

    — Mike Vizard

  • CloudBees Expands Visibility into DevOps Processes

    CloudBees Expands Visibility into DevOps Processes

    CloudBees has moved to bring some order to the potential DevOps chaos via an update to CloudBees DevOptics, a monitoring tool for tracking stages of the software delivery process in real time.

    The update to the software-as-a-service (SaaS) application adds capabilities that identify everything from potential bottlenecks to periods of inactivity that might be better allocated to maintenance activities.

    Michael Baldani, product marketing manager for CloudBees, said CloudBees DevOptics makes it possible to create benchmarks that track DevOps performance across the entire software delivery life cycle, including deployment frequencies, lead time to committing code into production environment, mean time to recover from failures and change failure rates. The “2017 State of DevOps Report” identifies these four most important metrics for a DevOps team to track.

    Armed with those insights, a DevOps team is capable of drilling down into a specific issue to make more informed decisions concerning how best to optimize processes, said Baldani, noting that longer term, CloudBees DevOptics also sets the stage for eventually applying machine learning algorithms to processes.

    The rate at which organizations have been making the transition to DevOps has been uneven at best. Much of that issue can be traced back to a lack of visibility into how specific processes are being employed, Baldani said. The analytics capabilities provided via CloudBees DevOptics should help organizations accelerate the transition to DevOps by enabling organizations to more easily identify areas where they need to optimize DevOps best practices, he said.

    Traditional enterprise IT organizations are still wedded to legacy approaches to software development largely because of inertia. CloudBees DevOptics provides an analytics application that, in addition to helping optimize DevOps processes, provides access to dashboards through which a DevOps process can be shown conclusively to be more efficient than, for example, an existing waterfall approach.

    Most organizations are embracing DevOps from the bottom up as individual teams launch various application development projects. As those teams start to deliver applications faster, it’s usually not too long before senior managers want to replicate that success across the rest of the organization. CloudBees DevOptics provides a mechanism through which the best practices implemented by various teams can be captured to facilitate sharing them with other teams. The biggest challenge facing most organizations today is not so much being convinced they can benefit from DevOps, but rather how precisely to go about it.

    It may take a while longer for DevOps processes to become the most widely used approach to developing software in the enterprise. But as more organizations continue to appreciate how dependent they are on software to manage increasingly digital processes, the need to accelerate the rate at which software is being developed becomes more apparent. CloudBees is betting that access to real-time analytics will be a major factor in any decision to standardize on a continuous integration/continuous deployment (CI/CD) platform by providing visibility into how software development ultimately impacts the business. After all, it’s now only a matter of time before some new or existing rival leverages software to create a competitive edge. In fact, it’s already been proven time and again that the rate at which organizations can create and deliver quality applications is critical to the overall customer experience.

    — Mike Vizard

  • The State of the Kotlin Programming Language in 2018

    The State of the Kotlin Programming Language in 2018

    Pusher, the communications and collaboration API provider, recently released the findings of its “State of Kotlin 2018” research report. Over the past 24 months, adoption and use of the Kotlin programming language among developers has seen tremendous growth. Pusher has seen this growth firsthand, with more developers using Kotlin while building applications with the company’s products.

    Earlier this year, Pusher surveyed more than 2,500 developers to get a better understanding of the ecosystem, specifically around usage, features and demographics.

    Key report findings:

    Kotlin is used both in work and personal projects, as confirmed by 60 percent of respondents. Kotlin is even more prominent in side projects, as they tend to be smaller, easier to convert and more experimental.

    Android is hugely popular across the board, used by professional developers and students alike. However, when it comes to backend applications, Kotlin’s users tend to be more experienced developers.

    Most Kotlin developers come from a Java background, or also work with Java. More than 87 percent of respondents have migrated existing Java code to Kotlin. However, more than a quarter of respondents who migrated Java to Kotlin needed to revert back due to both technical and organizational reasons. Tools that use reflection or generate code were mentioned most often as technical reasons to revert to Java.

    Null safety is the favorite feature for 80 percent of respondents, whereas co-routines and multi-platform support are the least important.

    The report concluded that Kotlin is overwhelmingly popular in the Android circles. This raises the question, Will Kotlin break into communities outside Android?

    “We expect to see new generations of developers taking their first steps into programming with Kotlin,” said Pusher Developer Evangelist Zan Markan in an email. “It’s a modern, versatile, cross-platform language that can cross between OO, functional, scripting, and declarative paradigms with ease. This could mean that Kotlin will affect the wider programming language landscape by becoming a benchmark for what a programming language should be able to do.”

    Read the full report here.

    — Malia Powers

  • CA Technologies Looks to Shift Automation Left

    CA Technologies Looks to Shift Automation Left

    CA Technologies, at its recent CA Automation Summit, advanced its case for leveraging automation to create a software development factory via an update to the CA Automic One automation platform that adds automation-as-code functionality.

    Gwyn Clay, vice president of product management for the Automation Business Unit at CA Technologies, said the latest release of CA Automic One make is simpler to “left shift” automation artifacts by making it possible for developers to embed those artifacts within application code versus having to wait for a DevOps teams to run them out of band.

    The latest version of CA Automic One also now includes version control and lifecycle management for automation artifacts that enables automation objects such as jobs, workflows and scripts to be promoted from development to production as part of an integrated DevOps process, said Clay.

    CA Technologies also revealed it is embedding intelligence into workflows enabled by CA Automic One, which enables dynamic runtime estimations that can predict potential issues within certain paths in the workflow execution.

    Other new capabilities added to the automation platform include support for PostgreSQL databases, as well as new integrations with CA Workload Automation, CA Continuous Delivery Director and CA Jarvis analytics software delivered as a cloud service.

    CA Technologies has been on a multi-year campaign to embed automation across the entire DevOps processes. With this latest release, that campaign effort is being extended to developers themselves. Clay said the rise of digital business transformation—much of which is being driven by cloud-native applications built with technologies such as Docker containers, Kubernetes orchestration software and serverless computing frameworks—is finally forcing the automation issue in the enterprise. Not only are applications being built faster than ever, but the complexity of those applications has increased exponentially. And yet, the size of the overall IT staff has remained stagnant. Without some form of automation being embedded within the applications itself modern software development practices becomes difficult to sustain, she said.

    Clay noted that most organizations are not yet at the point where they are deconstructing monolithic applications into a set of microservices. But it’s already clear most new applications will be built on a microservices architecture. CA Automic One represents an opportunity to apply automation across both legacy monolithic applications in addition to embedding artifacts directly within new applications, she noted.

    It’s not clear the impact the ability to embed automation artifacts will have on how enterprise IT is managed going forward. IT operations teams still will need to be able to set policies and constraints concerning the level of automation a developer might be able to invoke. Most developers don’t have much regard for the need to share limited infrastructure resources with other applications. But as the ability to infuse automaton into almost every application becomes pervasive, the locus of power within enterprise IT organizations will continue to shift to the left. The challenge now is determining the best way to strike a balance between all the competing factions looking to control who gets to automate what exactly when.

    — Mike Vizard