staging-devopsy.kinsta.cloud

Tag: cloud and DevOps

IT Ops Trends: Cloud, Containers and COVID

Almost all companies are modernizing their IT infrastructure to remain competitive, with the focus firmly on migration to the cloud and the use of dynamic infrastructures. In a recent Trends in IT Ops 2021 survey, more than 160 IT professionals were asked how the implementation of IT strategies affected their day-to-day work. The results provided an impressive insight into the current work priorities of system administrators and IT Ops in general.

According to the survey, most administrators believe that monitoring cloud (79%) and container (73%) environments will take on a more important, or much more important role for them in the future. Additionally, 59% of those surveyed agree that their job role has evolved over the years due to cloud computing and containerization.

Not all IT Teams are Equal

However, the data also shows that the changes are not affecting every IT team equally. For example, 37% of all respondents say that cloud solutions are not relevant—or less relevant to them on a day-to-day basis. This response shows that on the one hand, system administrators see cloud computing as an essential topic. However, on the other, the impact in practice is not felt to the same extent by every administrator. These responses show that not every administrator has to deal with cloud platforms on a day-to-day basis, with some being ‘siloed’ into dedicated teams that may not interact with the broader corporate cloud strategy. In addition, not every organization has fully committed to the cloud, with some finding their migration hindered by applications that are not suited to cloud migration.

In the survey, 45% of respondents also said they believe the importance of monitoring physical infrastructure will not change in the next few years. However, 43% believe that monitoring physical systems will become more important or much more important. Conversely, a mere 1% replied that they thought physical infrastructure monitoring was less critical.

Taken together, these responses demonstrate that, despite the excitement around the cloud, many organizations take a hybrid approach to infrastructure. IT Ops and administrators recognize the importance of monitoring cloud systems and dynamic assets while, at the same time, the monitoring of local systems remains relevant or is even expected to gain in relevance.

Network Management and Automation

Cloud and containers enable faster growth of IT environments and promote digitalization. However, they also lead to a sharp increase in data traffic and more complex networking of systems. As a result of these changes, the demands on local networks are growing. This increasing demand demonstrates the importance of network management tools. 81% of respondents said that solutions in this area are relevant, or very relevant to them in their day-to-day work.

This year, the move to a home office joined the trend towards migration to the cloud, containers, and network monitoring. In the wake of the COVID-19 pandemic, many IT departments had to adapt their infrastructure to allow employees to work from home offices. However, most administrators were relatively stress-free, with 54% saying they had no trouble adapting infrastructure to meet remote work requirements.

Automation is also a key issue. For example, 81% of all respondents say projects that deliver deployment and configuration automation have been relevant in their organizations in the past 12 months. In addition, 92% say that automation tools are relevant, or very relevant to them on a day-to-day basis.

Cloud, Containers and COVID

The Trends in IT Ops 2021 survey shows that system administrators are concerned with cloud computing and containers and several other topics. In addition, most administrators recognize the increasing importance of monitoring cloud environments and containers. That said, administrators continue to see monitoring networks and other physical infrastructure assets as critical, and some system administrators see increasing importance in monitoring this vital area.

In addition to the topics outlined here, respondents also provided feedback on topics such as virtualization, application performance monitoring and network management.

January 12, 2022
How to Achieve AWS Competency Accreditation

For businesses in the early stage of their ventures, proving that they have the expertise and technical know-how when it comes to cloud technology can be difficult. An acceleration toward digitization due to COVID-19 and an increasingly crowded cloud market, dominated by the big players, means that simply shouting about your skills in cloud engineering and DevOps isn’t enough to get noticed.

But what if you can work alongside these giants? Joining forces with expert brands and entering into endorsement partnerships can propel your business forward by supplementing it with seasoned expertise. Accreditation, if you can achieve it, can give you a stamp of approval and reassure prospects that you have the relevant knowledge to support their digital transformation efforts. Collaborating with industry leaders on skills accreditation programs is a great way to demonstrate your capabilities and provides clear goalposts to work toward when it comes to building and learning.

So, how do businesses achieve this status and what are the long-term benefits of the process?

Become an AWS Partner

Achieving accreditation as an AWS DevOps Competency Partner takes a lot of hard work and investment into building the technical skills required to reach such a high standard–logging and evidencing every action your team takes is no mean feat. From beginning to end, the course takes around six months. The process begins with researching and selecting which specific competency your team is interested in achieving. It’s important to understand which competency your business is most suited to, so doing your homework is key. From there, there’s a lot of work to evidence, certificates to achieve and reviews to undergo. Becoming an accredited AWS partner brings a multitude of benefits to your brand, so it is well worth the investment.

Top Tips for the Accreditation Process:

Create detailed accounts of your work: To become a certified DevOps partner, businesses need to evidence four case study examples; two public and two private. As such, it can be useful to bring in a tech writer to consolidated your notes and observations throughout the process. We began, for example, with a technical exploration from the team to understand the best case studies to put forward; our work had to be clearly evidenced with strong examples and the documentation to back it up which took a considerable amount of time to pull together. Maintaining clear and detailed accounts of previous work can prevent any delays. This information is firstly analyzed by AWS to check that it met the specific requirements before being passed on to be examined by a team of external auditors, so every detail counts when it comes to showcasing your work.

Prepare the marketing team: As part of the process, businesses are required to have certain marketing collateral in place across their website to aid with lead generation. For the DevOps competency certification, we needed to evidence three technical blogs–businesses applying for the certification need to ensure that the content best reflects their expertise in the given competency. Once the accreditation is finalized, the partnership should be displayed on the site. Even if a business is tech-agnostic, demonstrating your skills in a particular type of cloud engineering is highly valuable to your brand and AWS is a respected platform to use.

Advance technical training in the workplace: Having members of the team qualified in cloud engineering is another requirement for accreditation. For DevOps, we needed eight people on the team to have achieved their associate certifications and four to have achieved professional certifications. To fulfill this requirement and ensure we’re on top of our game when it comes to cloud training, we’ve invested in third-party tools, provided access to LinkedIn Learning programs and we give our people additional time off from client-facing work to prepare for their exams. Building and learning in tandem is the fabric of our culture, and a similar approach would benefit any company looking to achieve certification status.

The Benefits of AWS Accreditation

Being able to showcase a close association with a major player such as AWS opens up a number of doors for smaller businesses. Operating within this kind of business ecosystem provides the opportunity to demonstrate a business’ DevOps and technical skills to an enormous global audience which can drive new business conversations and drive quality leads. Confirmation that your business is a trusted partner for a major cloud organization will ensure that prospects have confidence in your capabilities and you’re taken seriously as a technical player in the market. Building confidence within your teams and supporting them in their learning is an investment that will pay huge dividends.

Working alongside AWS as an accredited DevOps partner gives us a springboard to talk about our work and engage with other players in the market. The accreditation is a fantastic talking point and definitely gets you noticed in the cloud community–plus, you will be taken more seriously as a partner.

Investment in Cloud Tech is Key for the Future

As businesses everywhere begin to operate in an increasingly digitized environment, technical skills and cloud expertise are a necessary investment for any business. Working toward coveted industry accreditations while prioritizing long-term skills training is imperative. It’s a competitive, buoyant market for cloud engineers and DevOps at the moment; opportunities are at their fingertips and if they aren’t feeling fulfilled by their current work, they’re likely to look elsewhere. Investment in training and nurturing growth in this way breeds employee engagement and loyalty. For a company to thrive technically, it needs to host a wide range of diverse talent and be able to retain it. Learning opportunities should always be the key focus.

The shortage of this talent means it’s critical to advance the business’s cloud expertise by pursuing recognition from major global players like AWS. Providing advanced training and certification opportunities for your people will ensure long-term talent retention while making the business a leader in cloud engineering. That’s pivotal for long-term success.

August 30, 2021
Cloud Misconfigurations Threaten Cloud Migration
If we’ve learned one thing during the COVID-19 pandemic, it’s that digital-native business models are essential to survival. That means cloud adoption is more important than ever before. Research shows that the public cloud market is expected to grow to $308.5 billion in 2021, an increase of 16% from 2020.

But while the cloud holds incredible potential, we have an outstanding security issue to address. That issue is cloud misconfigurations. Through 2025, 99% of cloud security failures will be the responsibility of the customer, quantifying the need for security professionals to turn their attention to security hygiene issues like cloud misconfigurations.

Digging Into the Cloud Misconfiguration Problem

To get to the root of the problem, we must fully understand the risk of cloud misconfigurations and how they differ from more traditional vulnerabilities. While traditional vulnerabilities can be patched, cloud misconfigurations can create vulnerabilities in otherwise secure applications and infrastructure. Imagine cloud infrastructure like a highway system and cloud misconfigurations as road hazards. They can be incredibly dangerous and lead to accidents.

As a specific example of cloud misconfigurations, let’s look at identity and access management (IAM). Poorly configured IAM, such as misconfigured roles or a lack of multi-factor authentication (MFA), can lead to compromised administrative accounts. If a threat actor hijacks a legitimate administrator account, they potentially can take full control of an entire cloud environment. Because IAM sits above the cloud infrastructure layer and all workloads and data within, once it is exploited a threat actor will often be able to circumvent your other security barriers, such as network segmentation, leaving you defenseless.

Cloud Misconfigurations Can Easily Scale

Where misconfiguration risk dramatically increases is through the adoption of cloud-native applications and practices like infrastructure as code (IaC) templates used by DevOps teams. These templates offer teams greater speed and scale for building and managing applications, but the downside is that misconfigurations can be unknowingly replicated from development environments to production environments (where sensitive data is stored) at greater velocity.

In short, the biggest risk to organizations right now is scaling these misconfigurations through the cloud. As the cloud grows in adoption and scale, we’re witnessing these misconfigurations scale right alongside it. Where before, a misconfiguration might have been limited to a siloed application or environment, today, that same misconfiguration can impact the entire organization without checks in place to catch it. Worse, many cloud storage buckets have logging disabled, meaning once threat actors are able to identify a misconfiguration and access an internal cloud bucket, an organization won’t even be able to see what data was accessed.

Understanding the Shared Responsibility Model

This all points to a higher-level, hygiene-related issue to keep in mind when securing cloud environments: responsibility. Particularly, organizations struggle to understand the shared responsibility model and how it applies within their own organizations. While some may fail to delineate between cloud provider responsibilities and their own, the major issue at stake is the shared responsibility between various internal teams that often goes undiscussed and undocumented until there is a security event.

When thinking about software-as-a-service (SaaS), to a much lesser degree the responsibility falls on consumers. However, when considering infrastructure- and platform-as-a-service (IaaS/PaaS) and all the moving parts such as network, user credentials, resource configurations, workloads, identity configurations and more, cloud consumers become responsible for much more. A key consideration to note is that one can never outsource accountability, no matter which cloud model is used. To put this more simply, if one puts data in any cloud provider, they are still accountable for that data.

Enabling a Holistic Cloud Security Strategy

When advising security and business leaders on how they can better secure their cloud environments and applications, I highlight what my team calls “The Big Cloud 5,” a set of recommendations developed to help organizations adopt a holistic cloud security model that accounts for proper security hygiene and shared responsibility.
- Gain awareness and deep cloud visibility
  The very first step to ensuring cloud security is understanding how teams are using cloud technologies, leveraging shadow IT and cloud provider APIs. This allows you to get situational awareness and make informed decisions today as well as in the future. This is not a one-time event, but something you’ll need to do continuously.
- Set guardrails to automatically prevent the most serious cloud misconfigurations
  Drawing lines in the sand around the most offensive (and potentially destructive) misconfigurations that should never exist in an environment is key to automating protection in the cloud. This will help keep templates and practices controlled, so that poor hygiene doesn’t inadvertently take root and spread. Think of this as your “dirty dozen.” What configurations should never exist in your cloud environments?
- Standards are the precursor to automation
  One can’t automate what hasn’t been standardized, and while there aren’t widely accepted security standards yet, key stakeholders in an organization must be in agreement about how to secure cloud infrastructures.
- Train and hire security engineers who code
  To fully leverage APIs, security teams must have engineers who know how to code and automate security processes. An assessment of skills that exist across your security team (e.g., knowledge of coding in the likes of Python or Ruby) can point to areas in training and hiring that need further investment.
- Embed security in the development pipeline
  Map out who, what, when, where and how your organization pushes code into the cloud. Once mapped, identify the least disruptive insertion points for security processes and tools, so that they can exist in as much of the development pipeline as possible.
If you feel your organization is a step behind on proper cloud security hygiene, know that you’re not alone. Earlier in 2021, the Cybersecurity and Infrastructure Security Agency (CISA) released a report highlighting the importance of strengthening security configurations based on the uptick in successful cloud attacks, often attributed to poor hygiene and the mixed use of computing devices in the remote work environment. While this is a growing problem, it is also a reversible trend. Best practices, cyber hygiene and a shared responsibility model can help companies safely and securely migrate to the cloud.
February 19, 2021
DevOps and Cloud: A Symbiotic Relationship
Most companies want to increase their competitiveness in today’s swiftly changing world, and so they cannot ignore digital transformation. DevOps and cloud computing have become two of the ways companies can achieve this needed transformation, though the relationship between the two is not easily reconciled—DevOps is about the process and process improvement, while cloud computing is about technology and services. It’s important to understand how the cloud and DevOps work together to help businesses achieve their transformation goals.

Different organizations outline DevOps in different ways. This article does not debate which definition is correct, but rather presents them both to focus on the cloud’s benefit to DevOps. That said, DevOps definitions generally fall into two terms:
1. In organizations it is defined as developer-friendly operations—IT operations are run separately yet in a way that is much more friendly to developers (e.g., self-service catalogs are provided to developers for stipulating infrastructure or providing technology-enabled pipelines for deploying new code).
2. DevOps as a single consolidated team is habituated in organizations—developers take on operations responsibilities and vice versa.
While cloud computing itself has several definitions, the most basic explanation of the cloud is a system that allows the provisioning of infrastructure (e.g., VMs or routers). Thus, it allows to being defined as code or templates, enabling the creation of repeatable processes that are not possible without cloud technologies.

The aesthetics of cloud technologies and services become so easily amalgamated in DevOps vocabulary because they complement DevOps processes, regardless of how your organization defines them—or, regardless of which DevOps routes you travel to digital transformation with the cloud integration.

Companies that focus on developers for operations often use cloud computing to speed developer productivity and efficiency. Cloud computing permits developers more control over their own components, resulting in smaller wait times. This application-specific architecture makes it easy for developers to own more components. By using cloud tools and services to automate the process of building, managing and provisioning through the code, service teams speed up the development process, eliminate possible human error and establish repeatability.

Cloud computing also enables users to create self-service methods for provisioning infrastructure through AWS Service Catalog. Developers are able to quickly try new things, fail fast and just easily succeed in getting new products to market faster, without having to wait for IT operations to provision services for them.

DevOps as a single term is really a mixture of these approaches, in which developers and operations work together using the cloud as a single common language. In this way, both the DevOps and cloud are able to work together, as everyone is learning new definitions and approaches at the same time. Developers and operations are equally cushy with the new language of the cloud, as developers often teach operations about the code aspect and operations can teach developers about infrastructure and security, developing a meeting point that leads to strong team dynamics.

No matter the definition, the importance of cloud computing to DevOps can’t be accentuated enough. Cloud computing advances IT transformation, and with advanced tools and automation, it can enable companies to double down on their work to streamline and embed DevOps processes for greater efficiencies that are truly transformative.

Inherently, as process improvement, DevOps also requires a culture change. Cloud computing can play a role here, as it can help codify and automate new processes. For example, if there is a new way in which developers should cluster the components they need to deliver a service—such as code, configuration, libraries and pipeline definitions—advanced automation tools can streamline the process and make it distinctly repeatable.

The business case for DevOps combined with cloud technology for a successful IT transformation is fivefold:
1. Bring products to market faster through faster access to development environments and streamlined developer processes.
2. Automation and architecture as code reduce cloud complexity and even system maintenance.
3. Add high security with automated, repeatable processes that serve to eliminate inaccurate error and, even more importantly, develop security controls from the very beginning.
4. Reduce downtime through cloud-based continuous operations. Moreover, in the process of applying automation, developers can build stateless cloud application development, which increases availability and failover ability, in the process increasing business reliability and customer satisfaction.
5. Increase scalability. One of the main reasons organizations look to cloud computing in the first place is its scalability, which allows organizations to increase capacity with the click of a button. When combined with DevOps, scalability becomes an integral part of applications as they are developed, while reducing the cost of infrastructure and increasing global reach.
Get Past the Changes

Together, DevOps and cloud computing are a powerhouse. While each offers greater effectiveness and business impact, together they are able to drive meaningful IT transformation that directly impacts business goals, regardless of the DevOps definition.

At the same time, culture within the enterprise and among developers must change around the nuances of DevOps and its role in driving cloud development. Finally, IT companies must invest in these technologies and do so without an immediate goal—which, of course, drives corporate leaders and shareholders crazy.

However, the alternative is doing nothing, which means certain failure. Competitors are likely to pull ahead in terms of time-to-market with solutions and application services. When you can hold up applications and processes in near real-time and do so within an elastic and efficient environment, the market will reward you for putting that effort. If you haven’t commenced on the process yet, it’s time to get started.

— Jaymin Vyas
November 7, 2018
The Cloud and DevOps: Like Peanut Butter and Jelly

Cloud is practically designed for DevOps. No matter which cloud provider you are using, each has set up systems designed to spin up/spin down on demand, has the ability to run pretty much what you need and offer APIs/command lines to achieve your DevOps goals on the infrastructure side.

That means that the infrastructure side of DevOps can be handled by careful choice of base image, some scripts to configure network and other supporting infrastructure, and some scripts to install what’s needed once the instance is running and protected.

Not All Roses …

The problem is one of breadth and complexity. Yes, cloud is the perfect place to spin up DevOps, at least on the infrastructure side and, once you have a Jenkins instance, on the coding side, too. But the complexity masked by “spin up the infrastructure side” can be daunting. Different sets of APIs for servers, storage and networking, and sometimes different APIs for security versus accounts … It can get snarled quickly.

So, What to Do?

What can you do to make life in cloud automation/DevOps easier? There are numerous options available to you to simplify the operational part of cloud DevOps and allow you to focus on discussing issues and resolving them.

First on the list is the growing amount of automation of cloud automation. Sound crazy? It’s not—just my way of expressing that cloud vendors provide you with APIs and tools such as Puppet are getting better at wrapping those APIs for you, so you can focus on problems, not what is supposed to be the solution.

This idea is extended by companies that are tying disparate APIs together into a unified whole in one segment or another. One example of this is CloudCoreo, whose tools can do security audits of cloud deployments. The company takes the time to wrap the various security-related APIs and let you pick what is important for your team to monitor.

There also is an entire market segment growing up of both consultants and VARs that will help an enterprise move to the cloud. Most of these companies focus on one vendor or another, but all share a desire to help make the shift to the cloud easier. By way of examples to get you started, consider Neoxia (in the EU) for GCE, Accenture (in North America) for AWS or Atmosera (in North America) for Azure. (Note: I have never used the cloud services of any of these companies; I’m merely offering you a sampling of what is available. Check your cloud providers’ partners page for more partners.)

Another option that will reduce, but not eliminate, cloud-based complexity is to use containers to set up the application. The container can then be run on whatever cloud platform suits your needs (or in your data center, if that’s the preference). Any instances spun up still must be protected and given access, but most of the networking, security and instance complexity can be handled at the container management level.

In fact, if you implement container management and choose for interoperability, a container management tool such as DC/OS can let you choose at deployment time where to put the container.

Seeing what enterprises are starting to do with containers and entire build/test environments, it is possible that very soon you will be able to just kick off a job, have it build containers that will run your DevOps processes, run tests, deploy to other containers and then go away once the relevant data from them has been saved. The containers used along the way, by taking advantage of a multitarget container management platform, could be anywhere.

To Sum It Up: Cloud is Good for DevOps

The issue at hand—as it has been for DevOps and cloud for a while—is the rate of change. Here we’ve discussed a cross-section of ways to make DevOps in the cloud easier and more complete. But no doubt we’ve missed some great ideas and products, because there is so much going on.

The point is, tools are coming available to make it easier. Don’t write a billion miles of scripts that are useful for only one provider if you can avoid it. Look into these options (and others that will crop up) to lighten your load of technical debt moving forward, and make getting DevOps roaring in the cloud easier.

And as always, keep rocking it. The change has been constant, and you’re still cranking. That says a ton.

— Don Macvittie

November 14, 2017