staging-devopsy.kinsta.cloud

Tag: devops

  • Understanding Cloud APIs

    Understanding Cloud APIs

    Six out of every 10 businesses had moved their workloads to the cloud as of last year. More steady traffic in the cloud and more direct cloud instances mean more to juggle–and the need for greater customization and integration. That’s where APIs come in. APIs are doing for the cloud what they did for on-premises resources: Segregating, simplifying, specializing and integrating services across a network–only this time, it’s future-proofing the architecture by building it in the cloud. As one cybersecurity researcher quipped, “Everyone wants to use APIs to build innovative solutions, but no one wants to invest in building them.” Cloud APIs will soon spell the difference between companies that can keep up with digitization and those that can’t.

    Ultimately, it will come down to which organizations take the time to understand them and use them to their best benefit.

    What is a Cloud API?

    Simply put, “a cloud API is a software interface that allows developers to link cloud computing services together.” APIs allow services to share functionality and information for other programs to use, and nowhere is this more important than in the interconnected ecosystem of the cloud.

    Because APIs act as a waiter in a restaurant, the better the waiter, the simpler ordering will be for you and for your customers. The best cloud API connects backend cloud components, services and platforms and does all the busy work so all your users see on the front end is a seamless customer experience. Since most of the work of tomorrow will be in the cloud (and is moving there in ever-increasing numbers), to not build up this functionality now is to take a deliberate risk.

    Types of Cloud APIs

    There are several types of cloud APIs based on the four major areas where cloud computing will require integration–either with another platform or another cloud provider.

    ● PaaS APIs (Service level): These platform-as-a-service APIs are designed to integrate with databases, portals, storage components and messaging systems, providing access and functionality in the cloud.
    ● SaaS APIs (Service level): Referred to as software-as-a-service APIs, these connect application layer resources with the cloud and underlying IT infrastructure. A CRM is a good example of where these types of APIs can be used to create a “cloud application extension” for your ecosystem.
    ● IaaS APIs (Service level): Infrastructure-as-a-service APIs control the distribution and deployment of specific cloud resources, performing such tasks as rapid provisioning and de-provisioning, network configurations and workload (VM) management in the cloud.
    ● Cross-platform APIs: Because many environments use multiple cloud providers and platforms, cross-platform compatibility is required. Cross-platform APIs streamline data management by allowing tenants to access their cloud-based resources and workloads from any of their cloud providers, not just their primary one. They provide a higher level of abstraction by translating a specific provider’s API calls into a more generic form, making accessibility, manageability and interoperability in the cloud easier.

    Cloud API Security

    In many respects, the security threats faced by cloud APIs are the same that any cloud-based resource would encounter and present the same challenges. That being said, malicious API traffic surged over 117% last year, from an average of 12 million to over 26 million calls, according to one industry report. The same study revealed that over a third of respondents experienced authentication and data exposure-related incidents with their APIs, and 54% delayed rolling out a new application because of API security concerns.

    Unguarded, a cloud API could present the keys to the kingdom and allow an attacker to access the hub of not just one but several cloud providers and their workloads, resources and assets. Protecting cloud-based APIs with standard authentication and authorization practices such as OAuth2.0 and OpenID Connect can reduce the chance of a lateral attack.

    Why Cloud APIs

    Cloud usage is not set to slow down and will soon become ubiquitous. Companies that aren’t on it are migrating their workloads in greater numbers and entrepreneurial ventures are spinning up as cloud-native. Cloud APIs are simply the logical response to the complexities that come with the terrain. Without cloud APIs, the user experience will be negatively impacted and will only become more apparent when compared with companies that do use cloud APIs.

    For companies looking to innovate in 2023 and release new applications, APIs are the way to cut-and-paste the process safely. Similarly, as more cloud-based services become API-reliant, application reusability is another factor requiring cloud APIs. And lastly, the complexity of cloud products, platforms and ecosystems (even within one organization) makes cloud APIs a necessity for smooth interoperability. And they help with scalability–multiple components of a cloud-based service can be parsed out, updated and patched individually and refined separately so that one area can develop without being dragged down by the whole. Cloud APIs allow for that granular level of customization that can help an organization refine its edge.

    Simply put, organizations will need these APIs to not only compete but stay relevant as businesses continue to take their fight to the cloud.

  • DevSecOps Provides a Modern Security Model for Modernization

    DevSecOps Provides a Modern Security Model for Modernization

    Developers and security experts are now tasked with bolstering, extending and adjusting cloud and Kubernetes security to protect against cyberattacks that are ever more complex, volatile, and frequent. To foil attacks and create a secure foundation for applications and infrastructure from the beginning, DevSecOps (development, security and operations) has become the trending development and operations practice. In the DevSecOps model, security becomes a shared responsibility. Thus, DevSecOps requires a mindset shift to enable collaboration between development, security and operations teams. This shift, commonly referred to as shift left, involves culture, processes and tools. With DevSecOps, whether it’s testing for security vulnerabilities or building security services that can be used directly for business purposes, the goal for all participants is to build security into applications from the start and throughout the continuous integration and continuous delivery (CI/CD) workflow of DevOps.

    DevSecOps to the Rescue

    In the past, security considerations and practices were often introduced at the end of the development cycle by a separate security team and tested by a separate (QA) team. This was manageable when software updates were released every few months or even years. But in today’s landscape, consumer habits and expectations are shaped by smartphones and digital commerce, which has fueled the demand for software services that are real-time and available 24/7. This requires modern enterprises to find ways to improve the efficiency of application developments, releases, and updates.

    Because of these factors, enterprises realize that security can no longer be a small part of an afterthought in the production environment. The purpose of DevSecOps is to ensure that through collaboration, security can be carried through to every point in the development cycle. This enables teams and enterprises to deliver secure, high-quality applications in a more efficient manner without extensive security checks and fixes occurring during post-production.

    To meet these more extreme demands, cloud computing, containers, and microservices have made it possible to accelerate the development and delivery of software releases. Developers adopting agile and DevOps practices can reduce software development cycles to days and weeks, thus meeting the diverse needs of enterprises and users.

    This fast-paced development and upgrade frequency has created new security concerns and the need for companies to be more agile in responding to security issues. DevSecOps has been introduced into the DevOps framework to meet these needs by make security a shared responsibility. Today, continuous testing and integration, including security scanning of pipelines, is becoming the norm.

    DevSecOps Business Benefits

    In principle, DevSecOps shares the DevOps ideal of multiple teams working together to improve team efficiency and achieve secure, continuous software delivery.

    In addition to the security benefits DevSecOps provides, there are significant business advantages to be gained, including:
    ● Efficiency–Under the DevSecOps practice, security is integrated into all periods of development to help all teams be more agile in responding to security risks, eliminating the need for teams to spend a lot of time tweaking and fixing during the production cycle.
    ● Cost reduction–By discovering security vulnerabilities before they enter production, organizations and teams can significantly reduce the time and labor costs of fixing them.
    ● Ensure compliance–DevSecOps can ensure compliance with industry-standard regulations, such as the General Data Protection Regulation (GDPR). DevSecOps gives teams a holistic overview of these measures that makes compliance easier.
    ● Establishes collaborative culture–Integrating security practices into DevOps enhances the value of DevOps and improves the overall security posture as a culture of shared responsibility. When everyone is involved in the process, it increases their awareness of security fundamentals and best practices and provides a sense of ownership in the results.

    Meeting New DevOps Challenges

    Kubernetes offers many advantages but also poses unique security challenges that can be difficult to address for organizations lacking in Kubernetes talent and experience. This is why organizations will increasingly see the need to reevaluate their security practices and prioritize a more advanced security-focused culture.

    Because DevSecOps requires security to be addressed throughout all development stages, it requires developers to have security expertise while coding and operating. We are currently seeing a growing skills gap in the DevOps industry, and developers are feeling burned out. Security training is a way for teams adopting DevSecOps to acquire additional knowledge.

    Deploying Kubernetes platforms with security built in by default also will be recognized as a means to reduce the burden of security on IT teams. This will reduce the pressure and burnout on all sides.

    Platform Engineering Provides Additional Help

    With the rapid development of Kubernetes and cloud-native applications, organizations are realizing the inadequacies of their IT teams to leverage DevOps practices. Just as platform engineering eases the burden of DevOps by providing an Internal Development Platform (IDP) that serves as a “golden path” for developers, an IDP can simplify the practice of DevSecOps.

    We’ve seen that a DevOps workload is difficult to practice in small and medium-sized enterprises, as well as in large enterprises that lack sufficient talent. The gradually accumulated cognitive load ultimately leads to a less agile and efficient collaboration between teams. Given these issues, more organizations will adopt platform engineering as a better alternative.

  • Good Things Happen When DevSecOps and Cloud Collide

    Good Things Happen When DevSecOps and Cloud Collide

    The marriage of cloud and software development is arguably the heart and soul of digital transformation. Providing pathways to greater efficiencies, lower costs and greater enterprise performance, the duo is poised to fundamentally transform industries as we know them. The compounding complexities of hybrid cloud operations and software supply chains, however, have led to a greater risk of vulnerabilities, errors and attacks.

    Increasingly wrapped around microservices, containers and orchestration platforms, the complexities of cloud expansion and software development couldn’t be more intertwined or expose more complicated risks than they do today.

    Software: A World of Consistent Change

    The state of software development is one of consistent change. Whether we call it agile or adaptive matters far less than the need to infuse operations with a new mindset and culture about the need for continuous development. Increasingly that includes the infusion of security disciplines and technologies directly into the main artery of that development.

    Enter DevSecOps, an approach to software that integrates security initiatives at every stage of the software development lifecycle to deliver high-performing secure applications.

    The cause is not lost on the National Institute of Standards and Technology (NIST) and its cybersecurity center of excellence (NCCoE) which has announced a DevSecOps project dedicated to “developing and documenting an applied risk-based approach and recommendations for secure DevOps and software supply chain practices consistent with the secure software development framework (SSDF), cybersecurity supply chain risk management (C-SCRM), and other NIST, government, and industry guidance.”

    This is a far cry from software development of old. According to CTO Emeritus, Hu Yoshida, “Software, once the product of developers confined to writing long lines of code for specific applications, is now an exercise of integration – an amalgamation of third-party components provided by a variety of vendors, cloud providers, and open source software (OSS) groups. In fact, today, as much as 90% of code comes from such outlets, creating a software supply chain.”

    Although these new processes provide great advantages in terms of access to innovation and faster development cycles, the addition of external touchpoints and third-party software increases the risk of flaws, mistakes, and vulnerabilities. These risks only rise further when considering their deployment, usage, and management across complex hybrid cloud environments.

    How to Take Advantage of the Collision

    Cloud-based data is accounting for 39% of successful cyberattacks. Containerized applications, which have been a boon to both migration and management can also lead to vulnerabilities – which is fitting for security to be cited as a top concern for more than half of the organizations surveyed.

    From business continuity to business performance, organizations must begin assuming a security posture that considers the rising confluence of cloud management and software development.

    The idea is simple: You must find a way, a process, a method, and the right partners to help secure all workloads across any cloud environment, regardless of the platform or the amount of data and application real estate needed.

    By establishing this model, organizations are able to create a fundamental layer of protection against the ever-evolving threat of cybercriminals. Take one of our large banking customers, for example, who runs critical applications on AWS with stringent security and compliance requirements. We implemented a secured framework to protect their applications running on modern, cloud-native services like containers and Lambda functions using DevSecOps principles and cloud-native SIEM solutions. This enables them to scale their business in the cloud without worrying about compliance violations.

    One of our large pharmaceutical customers with a very large presence in both AWS and Azure experienced challenges keeping their cloud security posture in pace with the growing business demands on the cloud, increasing risk to the environment. By establishing a security process using cloud security posture management, we were able to significantly reduce cloud misconfigurations and auto-remediate several cloud vulnerabilities proactively to improve the security posture of the environment.

    The bottom line is that building security into the workloads will provide a fundamental layer of protection. The collision of cloud and DevSecOps is real. Organizations would be wise to realize it and take advantage of it to dramatically improve business performance and resiliency.

  • DevOps Flow: Accelerating Velocity With Software Factory Best Practices

    DevOps Flow: Accelerating Velocity With Software Factory Best Practices

    In the cloud era, senior executives are investing in key capabilities like DevOps with the expectation this will accelerate innovation and bring new digital products to market faster. To deliver on these goals, enterprise organizations seek to deploy high-quality code faster and more frequently, but they face a organizational, process and technology complexity that can hinder this objective, slowing throughput and incurring high error rates. Often when DevOps is discussed, the focus is concentrated on the technology layer, particularly the latest tools like Kubernetes, and also on the productivity of each individual developer.

    However, for many organizations, the main challenges will be with team practices and organizational models.

    Writing for staging-devopsy.kinsta.cloud, Mike Vizard highlighted the reality of DevOps uptake; that most adoption issues are related to organizational challenges, not technology. The top two barriers to adoption are slow processes and speed of adaptation (29%) followed by budget and funding (21%). Only 18% identified technology limitations as an issue.

    As Mike writes, it is the broader organizational transformation that is more challenging. ‘Addressing bottlenecks’ might be a function of confronting the situation of a department manager who zealously guards their turf and acts politically to resist any changes required.

    Implementing DevOps for one team is a relatively straightforward exercise, but for enterprise organizations, they face a much larger magnitude of complexity, as they typically have multiple teams spanning multiple geographies and suppliers with hundreds of developers all contributing to the same software development process.

    Therefore, a DevOps methodology that also addresses the organizational level is required, one that maps processes end-to-end and identifies the improvements needed to speed production across this entire life cycle.

    Software Factories

    Transformation leaders are turning to a ‘digital factory‘ approach to streamline this complexity into a production line model that yields consistently high throughput and quality using Lean principles derived from the manufacturing world.

    McKinsey explored the dynamics and successes of the digital factory model, a new approach to organizing digital teams. This model involves a self-organizing structure rather than rigid departmental hierarchies, and they described the KPI improvements the digital factory enables: “We see reductions in management overhead of 50% for technology teams in the digital factory, 70% in the number of business analysts needed to write technology requirements, and, as test automation becomes the norm, a drop of 90% in the number of testers. Finally, we see top engineering talent performing at eight times the level of their peers, as measured with metrics such as code commits.”

    In short, McKinsey described a development team that improves quality and finds efficiencies as a direct by-product of one that leverages the latest tools and team methods to produce better digital products faster. The concept of a ‘software factory’ is a very powerful metaphor because much of the insights are drawn literally from the world of manufacturing, where over many decades they have perfected the science of optimizing the efficiency of a production line.

    To close the loop and map these development improvements to business value, management can leverage organizational performance practices, notably Six Sigma and the theory of constraints, to define their ‘DevOps algorithms,’ a systematic formula for quantifying and improving the rate of software development throughput.

    DevOps Flow

    DevOps Flow is a methodology for implementing high-performance, cloud-centric software development based on science developed by pioneers like Toyota to optimize factory lines and apply this to software engineering.

    Organizations need the ability to measure the entire system end-to-end to understand how value flows and where it is constrained and, most importantly, to correlate those metrics with desired business outcomes. This approach allows for continuous optimization in the pursuit of delivering greater and greater value to the organization faster.

    It can be applied to speed the deployment of new software releases, notably identifying and removing constraints, reducing batch sizes and eliminating waste. These are all steps taken within manufacturing to increase production throughput and are improvements that software teams can adopt to achieve equivalent benefits.

  • Survey: More Cybersecurity Pros Embedded in DevOps Teams

    Survey: More Cybersecurity Pros Embedded in DevOps Teams

    A survey of 2,500 C-level executives published today by Palo Alto Networks found 81% of organizations have embedded cybersecurity professionals within their DevOps teams.

    Despite the presence of those cybersecurity professionals, however, the survey also suggested there is much work to do in terms of optimizing DevSecOps workflows. A full 90% of organizations cannot detect, contain and resolve cybersecurity threats within an hour, the survey found.

    Bob West, chief security officer for Palo Alto Networks, said most of the challenges associated with DevSecOps are directly related to persistent cloud security issues. More than three-quarters of organizations (78%) said they had distributed responsibility for cloud security, but almost half (47%) said a majority of their workforce still does not understand their security responsibilities.

    In many cases, developers with little to no cybersecurity expertise are responsible for provisioning cloud infrastructure. As a result, misconfigurations that create security issues are commonplace, noted West. That’s especially problematic because three-quarters of organizations (75%) are deploying new or updated code to production weekly, with almost 40% committing new code daily, the survey found. Given that developers far outnumber cybersecurity professionals in most organizations, it remains challenging to ensure application code is secure, noted West.

    In addition to improving the overall state of cloud security, organizations need to focus on fundamentals such as training developers to be more mindful of cybersecurity issues, he added. Most developers never had any formal cybersecurity training, so it’s up to organizations to make sure that particular skills gap is closed, he noted. In addition, organizations need to make sure they have a robust set of patch management processes in place alongside a capability to manage permissions and entitlements, added West.

    Overall, the survey found organizations are using more than 30 security tools, on average, including six to 10 dedicated to cloud security. More than three-quarters of respondents (76%) said relying on multiple security tools created blind spots that affected their ability to prioritize risk and prevent threats. A full 80% said they would benefit from a centralized security solution that sits across all of their cloud accounts and services.

    One way or another, a rising tide of regulations focused on improving the security of software supply chains will force organizations to improve cloud security, said West. The survey showed there is a strong commitment in terms of making cybersecurity professionals an integral part of a DevOps workflow. The issue is that there are still a raft of training and process issues that need to be addressed at a time when cybercriminals are becoming increasingly adept at compromising applications both as they are developed and after they are deployed.

    Of course, training and modifying processes take time. DevSecOps is as much about changing culture as it is about acquiring new tools. DevOps teams are still highly committed to building and deploying applications as quickly as possible. The challenge is finding a way to insert a set of cybersecurity gates within those workflows without slowing down the pace at which applications are developed.

  • The Impact of Developer Happiness on Productivity

    The Impact of Developer Happiness on Productivity

    Even in an era of unprecedented tech layoffs, most companies are still eager to find and retain top software developer talent. What’s more, many firms are discovering that they lack direct insight into their development teams’ needs and priorities. As a result, some of them are struggling to keep their developers happy and thus productive. A recent Zenhub report found, for the first time, a quantitative link between developer happiness and productivity, including what keeps developers in the same position instead of moving on to another job at the first sign of perceived trouble.

    Software Development Happiness

    The 2022 Software Developer Happiness report is based on a survey of 380 software professionals from a host of different organization types. In brief, the survey found that two of the biggest contributors to devs being unhappy in their roles were a lack of work/life balance and not working on what’s perceived as “quality work.”

    Even with a looming recession and thousands of layoffs, there remains a dearth of senior software development talent. For more junior talent, there will be more available than in previous, pre-pandemic years. As companies strive to do more with less, they still require development teams to build their products and/or the applications that drive their business. Setting up more junior teams for success via collaboration and productivity tools will be key to navigating this new environment.

    Given these findings, if organizations want to retain developer talent, they still need to make sure developers are working on meaningful projects and that they’re able to achieve that work in a way that balances the needs of the business with the needs of the individual.

    An ideal example of this phenomenon is the developer tool ecosystem companies make available to their teams. While the growth of this ecosystem has been a net positive for developers, it has also created a lot of tool sprawl and significant integration challenges.

    Less Time on Application Development

    Stephen O’Grady, an analyst at RedMonk, an analyst firm focused explicitly on software developers, talked a lot about this through a term he coined called the “developer experience gap.” He is referring to developers spending more time integrating tools than working on application development or application code. That integration work comes at a real cost. It means developers are spending less time on application development, are having to work longer hours or do the integration work off the side of the desk.

    The real bottom line to these survey results is that developers are happiest and stay at their jobs longer when empowered to focus on building code. This means spending less time interacting with management, attending meetings or resolving and integrating multiple solutions. The report’s results also emphasized the importance of work-life balance in encouraging developers to stay at their current job longer, while poor work-life balance accelerated the hunt for another job. These insights into developer retention are especially critical as, according to the report, 71% of developers haven’t decided how long they’ll stay with their current employer. The report also suggested that 33% of developers have stayed in a job longer than usual due to the pandemic, which could mean an increase in developer turnover when the pandemic comes to an end.

    Another key finding is that collaboration challenges are viewed as the biggest impediment to developer productivity. Delays in feedback result in lost productivity; nearly 70% lose a minimum of three hours per week per developer and 75% spend at least an hour per week on reporting instead of writing code.

    Finally, developers said their biggest priority is ensuring their software meets business demands, and most developers want high-performance productivity tools. In fact, 93% of developers believed these tools are critical to their success.

    Increase Developer Happiness

    So what can team leads to alleviate these issues? For both team leads and more senior management, the biggest way to increase developer happiness is simply to let them focus on coding. Developers like to showcase their skills, so encouraging them to work on interesting and impactful projects that challenge their skill sets is a great way to boost morale. Ensuring an appropriate work-life balance and paying the developers competitively are the other two recommendations for improving developer retention.

    Additionally, senior management should take another look at how they are compensating their teams. Financial compensation will always be a critical element, but we should expect to see companies shift their focus to programs and benefits that their employees truly need. Particularly given the somewhat chaotic year in 2022 and continuing in 2023, those in the technology industry will shift to valuing organizational stability over just a paycheck. An improved focus on mental health, conflict resolution and work/life balance will also be a big part of attracting the appropriate talent to an organization.

    The full report provides further detailed analyst recommendations for project managers, product owners, team leads and executives. The entire report can be downloaded for free here.

  • Five Tips for Moving IT Ops to DevOps

    Five Tips for Moving IT Ops to DevOps

    The era of siloed development and operations functions is rapidly coming to an end. The concept of DevOps has already closed the gap between IT and developers, but as I speak with customers around the globe, one theme continues to arise. Many systems administrators are ill-equipped to handle the shift from IT Ops to DevOps and, in some cases, don’t even know the right questions to ask to ensure they are planning accordingly.

    Having the appropriate equipment and software in place means nothing without people trained and ready to implement the shift from IT Ops to DevOps. As IT strives to support and protect developers continuing to innovate with wild-west-style creativity, how can we empower them with the tools and knowledge to ease the transition? This starts with asking the right questions, but requires IT to think about new things: Automating processes, revamping config management, building efficiencies in data orchestration and standardizing a set of APIs with enterprise-level protections.

    Asking the Right Questions About DevOps and Infrastructure-as-Code

    Enterprises need to look holistically at how infrastructure services are created, delivered and managed. Continuing current practices, such as manual provisioning complicated by different interpretations of imprecise instructions, will continue to lead to snowflake architectures and brittle systems. Organizations must modernize or replace this existing infrastructure to exploit cloud architectures and implement true platforms, actualizing the promise of the cloud.

    Infrastructure-as-code is an underlying framework that removes the manual process of managing and provisioning resources and opens doors to enabling DevOps. Using code to define the infrastructure configuration means that there is greater consistency between platforms. That consistency allows IT to seed the agile processes, methods and technical approaches needed to evolve a DevOps culture.

    To get to a point where infrastructure-as-code is facilitating a DevOps culture, it’s important for IT Ops to understand how to develop an early-stage continuous delivery and deployment pipeline. They also need to begin to seed practices and values native to DevOps for software engineering principles and practices with IT Ops. It’s also important to define success criteria–referred to as the “definition of done” in agile and begin to think about how to implement the infrastructure-as-code using DevOps best practices.

    Don’t Expect Developers to Become Expert Sysadmins

    Developers don’t want or need an extra layer of oversight or management, nor do they want to ask for permission before doing what they do best: Innovating. DevOps can more rapidly provision compute and storage resources and provide infrastructure-as-code while automating DevOps processes, empowering developers to become an innovation engine for their businesses.

    So much of what we’ve seen puts pressure on IT to be proactive when protecting developer cycles in containers and beyond. The NSA, as well as other industry data protection companies, offer tips for IT to stay ahead of these threats, but manual patching and scanning is not only handcuffing IT but stifling innovation in DevOps. The challenge is that IT can’t move as fast as developer cycles and often lose sight of what developers are putting into containers or other IT environments.

    While global data protection restrictions might have an impact on things like machine learning and AI from a consumer experience standpoint, it does not affect how IT might use these technologies to automate protection. It’s also not about extending protection to developer workloads, it’s about having an environment that recognizes and automates protection against threats in all environments. This way, developers and IT don’t have to become system admins or data protection experts. They can focus on the innovation at hand.

    Config Management is Critical to Empowering Developers

    As we continue to prioritize the automation of processes in the developer environment, it’s also important to consider how to move beyond just automating protection and infrastructure while allowing IT Ops to still have an eye on identifying, documenting and tracking changes in the DevOps stack. To do this, DevOps should look to configuration management tools to help manage and automate the architecture through strong community support, customization, scalability and the simplification of complex configurations.

    These tools, such as Ansible, Terraform, etc., can be deployed to meet specific needs and preferences. The point is, once you create infrastructure-as-code, you need a way to configure and manage it that fits the model that’s appropriate for your business. While you can use IaC to configure and manage, you still need to determine how to best implement automation and configuration in the most appropriate environment.

    Data Orchestration Eliminates Repetitive Cycles

    Data orchestration tools bring all data together regardless of where it originates or its format. An ecosystem that can collect data efficiently and convert it to a format that is easily understood by any and all endpoints is critical to the app development process. There’s no need to iterate at multiple levels, on multiple platforms or in different environments. IT can enable developers to innovate and learn from any environment, data warehouse or platform and build applications that can do the same.

    In the same way data orchestration can drive efficiency, there are additional benefits. Automating the collection, analysis and formatting of data saves IT countless hours of doing these processes manually. While there are obvious cost benefits, there are also a few other important benefits.

    To allow developers to continually innovate at a wild west pace, IT/DevOps must also ensure they can meet data privacy compliance and governance policies. Data orchestration allows businesses to delete or store data based on policies preset in the environment. It also allows businesses to ensure that internal policy is upheld throughout the development process. Orchestration then takes that data and makes it workable so that any iterative process can be applied without fear of bottlenecks or additional cycles.

    Standardized APIs Trump Those Created by Unaffiliated Coders

    Today’s developer profile expands far beyond the enterprise. In fact, according to Developer Nation, we can expect a whopping 45 million developers in the world by 2030. These numbers don’t include the unaffiliated coders working out of their basements or garages. That’s not to say that creativity isn’t appreciated or eventually adopted by industry leaders. But the code and software created don’t necessarily abide by enterprise governance and privacy standards. Yet we continue to see wild west developers pulling in anything and any kind of API to ensure it meets the needs of the next business application they are developing. How does IT police this?

    They don’t have to. For IT/DevOps to identify a set of APIs that meet business imperatives and certify them for use within the organization is imperative. Think about a scenario in which a developer has not only a comfortable environment to sit in without learning new skills but also an environment that automates protection, orchestration and gives them a preset library of helpful and open APIs. Businesses can begin to focus on delivery rather than policing and upskilling developers to work within the confines of the business ecosystem.

    As data continues to flow in and out of organizations and more data is being created and stored at the edge, a DevOps culture built for tomorrow’s developers is critical. These tips are a great starting point to ensure you’re installing a future-proof culture of innovation. What are some of the obstacles you’ve faced and overcome during the transition to DevOps?

  • How to Achieve Data Mesh With Four Key Principles

    How to Achieve Data Mesh With Four Key Principles

    Today’s world is becoming increasingly data-driven, which means businesses need to refine their practices and decision-making processes with the use of trustworthy data. However, this has proven challenging as organizations often don’t know what to do with the abundance of data available to them. According to a recent survey from Snow Software, 89% of IT leaders receive actionable data, yet 60% feel overwhelmed by the volume at their fingertips. These companies are both thriving in data collection and are bogged down by the sheer quantity available. While traditional data strategies have not yet adapted to meet modern business needs as the amount of data continues to grow, there’s a strong opportunity for companies to transform their data warehouses and data lakes and optimize the flow of data to consumers through an emergent approach called data mesh. Data mesh forges a path for companies to improve their business strategy and make better, faster decisions by leveraging the insights buried within their data.

    Data Mesh and its Benefits

    At its core, data mesh is an approach to data management that enables organizations to evolve from the pitfalls of legacy, centralized data architecture toward a decentralized, domain-driven design at scale. Breaking that down, data mesh is designed to help companies manage complex data networks by focusing on decentralized data ownership, treating data as a product, and providing a uniform infrastructure capable of sharing data across the company landscape. It improves upon legacy data management, which is often time-consuming, error-prone, unsustainable and unscalable. Unlike these traditional methods, data mesh gives domain-driven ownership to experts who strategize how it can best serve the company, as opposed to just dumping all data in one place and waiting for a centralized function to process it and make it available to the business.

    A data mesh approach also enables organizations to democratize their data by allowing users to acquire trusted data when they need it, all while automating the complexity and being shielded by security and governance guardrails. It removes bottlenecks and helps businesses glean the data-driven insights they need at the scale and speed of their organization. This way, organizations can maximize the value of their data by ensuring it is clean, vetted and readily available to support business initiatives.

    The Four Principles of Implementation

    So how can organizations embrace this approach? The framework of data mesh is centered on four guiding principles to help organizations get the most out of their data. Leveraging these principles will help your organization unlock the benefits of a data mesh.

    ● Domain-driven ownership: As opposed to separating data from your organization’s subject matter experts, the first pillar of data mesh implementation involves putting domain experts in charge of their data ecosystem. This way, data will be cleaned and enriched at the source before it’s made available throughout the organization. Domain owners will maintain the quality of the data and keep all necessary facts and documentation available for audits, effectively combining data with business talent.
    ● Data as a product: The job of the data team is to provide the data that the company needs for whatever purpose, be it making decisions, building personalized products, or detecting fraud. By treating data as a product, the company is giving its data a vision, strategy and product roadmap, ensuring its value is executable.
    ● Self-service data infrastructure: A business is made up of logically autonomous domains; each domain not only supports a business function, product or process but also produces data products and therefore requires a data infrastructure. Self-service infrastructure ensures business domains don’t have to deal with the underlying complexity of networking, security, compute and storage requirements.
    ● Federated computational governance: Standards for data governance are defined centrally, but local domain teams are empowered and equipped to apply to these standards however is most suitable for their specific environment within predefined guardrails. The key to achieving this for policy, security, quality and definition at scale is automation and integration throughout the landscape of the data infrastructure.

    By enacting these principles within your organization, your data mesh architecture is sure to provide the most value for your data.

    Data mesh is an optimal solution for any business currently struggling to find value in its data. Traditional methods are time-consuming, unsustainable, unscalable and error-prone. Data mesh is an approach designed to transcend these barriers, decentralizing much of the IT team’s heavy lifting and putting the responsibility of managing data on individual business domains. With clear ownership, domain experts can develop deep data knowledge and expertise and ensure that it’s trustworthy and valuable to your organization before making it available for consumption. This way, enterprises can prioritize self-service and empower data consumers with access to accurate, high-quality data when needed.

  • How GitHub Actions Simplifies Your CI/CD Workflow

    How GitHub Actions Simplifies Your CI/CD Workflow

    In today’s fast-paced world, being able to deploy applications quickly is essential for success. However, setting up a CI/CD pipeline can be a tedious and time-consuming task. Fortunately, GitHub Actions can help simplify and automate your workflow, making it easier than ever to get your applications up and running in no time. In this blog post, we’ll take a look at how to set up a React application with GitHub Actions and explore the benefits of using this tool for your CI/CD needs.

    Continuous Integration and Continuous Deployment

    Continuous integration (CI) and continuous deployment (CD) are essential components of any product development workflow. CI is the process of automatically building and testing code changes, while CD is the process of automatically deploying code changes to a production environment. Both CI and CD help to ensure that code changes are deployed quickly and efficiently, with minimal errors.

    Setting up a CI/CD pipeline can be a tedious and time-consuming task, particularly if you are not familiar with the required tools and technologies. However, using GitHub Actions can help to simplify and automate your CI/CD workflow, making it easier and faster to deploy your code changes.

    GitHub Actions is a powerful tool that can be used to automate many different tasks, including building, testing and deploying code changes. In addition, GitHub Actions can be used to trigger other actions in response to specific events, such as when a code change is pushed to a remote repository. This makes it an ideal tool for automating your CI/CD workflow.

    There are many advantages to using GitHub Actions for CI/CD. First, it is easy to set up and use, even if you are not familiar with the underlying technology. Second, it integrates seamlessly with GitHub, making it easy to track and manage code changes. Finally, it is highly customizable, allowing you to tailor the automation process to meet your specific needs.

    Setting up a React Application

    React is a popular JavaScript library for building user interfaces. Setting up a React application is not complicated, but there are a few important steps you need to follow.

    1. Choose your development environment. You can use create-react-app to set up a development environment with all the necessary dependencies and scripts.
    2. Install React and ReactDOM. React is the library that handles rendering of UI components while ReactDOM takes care of interaction with the DOM.
    3. Create a component file. In this file, you will write your UI code using JSX syntax.
    4. Include the component file in your HTML page. Now you can see your component rendered on the page!

    Development Mode and Production Mode

    In software development, there are two primary modes that code can be run in: development mode and production mode. Development mode is typically used by developers when they are actively working on code. In this mode, code is usually not optimized and may include debugging features that help developers identify and fix errors. Production mode, on the other hand, is used when code is ready to be deployed to end-users. In production mode, code is typically optimized for performance and security.

    One of the benefits of using GitHub Actions is that it makes it easy to manage both development and production workflows from a single platform. With GitHub Actions, you can create separate workflows for development and production, or you can create a single workflow that automatically detects whether code is being pushed to a development or production branch. This flexibility can simplify your CI/CD pipeline and help you avoid errors that can occur when manually managing multiple workflow files.

    To learn more about how to use GitHub Actions to simplify your CI/CD pipeline, check out the following resources:
    – [GitHub Actions documentation](https://help.github.com/en/articles/about-github-actions)
    – [Tutorial: Use GitHub Actions to deploy a static website](https://help.github.com/en/articles/configuring-a-workflow#creating-a-workflow-file)
    – [CI/CD with GitHub Actions blog post](https://blog.github.com/2019-08-08-introducing-github-actions/)

    Setting up GitHub Actions

    Setting up GitHub Actions is simple and only requires a few steps.

    First, you’ll need to create a new file in your project’s root directory called “.github/workflows/ci.yml”. You can name this file whatever you like, but for the purposes of this article we’ll call it “ci.yml”.
    In this file, you’ll need to specify the language your project is written in and the name of the action you want to run. For example, if your project is written in JavaScript and you want to run the “lint” action, your file would look like this:

    name: CI
    on: [push]
    jobs:
    build:
    runs-on: ubuntu-latest
    steps:
    – uses: actions/checkout@v2
    – name: Use Node.js ${{ matrix.node-version }}
    uses: actions/setup-node@v1
    with:
    node-version: ${{ matrix.node-version }}

    Breakdown of the Actions File

    In the “ci.yml” file, there are three main sections: “name”, “on”, and “jobs”. The “name” section is where you specify the name of your CI workflow. The “on” section is where you specify the events that will trigger your workflow to run. In this example, the workflow will run every time code is pushed to any branch. The “jobs” section is where you define the actual jobs that make up your workflow. In this example, there is only one job called “build”.

    The “build” job has two steps. The first step checks out the code from GitHub. The second step runs a Node.js linter on the code to check for syntax errors. If any errors are found, they will be displayed in the GitHub UI.

    You can also add additional jobs to your workflow, such as testing or deployment jobs. For example, you could add a job that automatically deploys your code to a staging environment whenever code is pushed to the development branch. This would allow you to test new features in a safe environment before deploying them to production.

    Adding additional jobs can help simplify your CI/CD pipeline by automating more of the process. This can free up time for other tasks, such as writing documentation or working on other features.

    Running the Action

    Assuming you have already set up a workflow file (ci.yml) and added a job to it, running the workflow is simple. All you need to do is push some code to your GitHub repository and the workflow will automatically run.

    You can also manually run a workflow by going to the Actions tab of your GitHub repository and clicking the “Run Workflow” button. This can be useful for debugging purposes or if you want to run a particular job without pushing code.

    If you want to see the output of a workflow, you can go to the Jobs tab and click on the job that you want to see. This will show you the log output of the job as it ran.

  • Why You Need to Break the Observability Data Silo

    Why You Need to Break the Observability Data Silo

    In my role as a developer advocate for Coralogix, I speak with organizations across almost every conceivable industry, and there is a pattern across almost all of them. In the world of observability, the challenge is battling scale. At scale, observability becomes a complex problem to solve. Surprisingly, the most common pattern for handling observability data is also holding people back from unlocking the true power of their data.

    What is the Data Silo in the World of Observability?

    To understand the data silo, we first need to understand the typical architecture of an observability solution, whether it’s homegrown or SaaS. Observability data can be divided into three types: Logs, metrics and traces. These three types come together to give engineers a complete picture of the health and performance of their system.

    These three data types are typically held in different storage mechanisms. A popular solution for logs is open search, while Prometheus holds the mindshare for metrics, and Jaeger is a very popular tracing solution. This means three solutions, three interfaces and, subsequently, three problem-solving workflows.

    What’s wrong with separating data like this? Even though there are three types of data, there is still only one goal: To understand the health and performance of the system. This is the only use case. The data silo is the dominant trend for observability solutions to provide a fragmented view of the data. It requires engineers to piece together stories as they go, rather than correlating across all of this data at once, to provide a complete picture of system performance.

    This slows down troubleshooting. If there is an issue, an engineer will first review one of the data types. Then, they will run different queries against other data types to confirm. So they may detect a 500 error in the logs, then move to the traces to see the impact on traffic, and finally, review the application metrics. This means three minimum queries when attempting to assess only one problem.

    Breaking the Data Silo

    Breaking the data silo can be a daunting challenge. There are some key points to focus on when a company starts out and some finishing touches that make for an excellent developer experience.

    Unify data storage as much as possible. For example, traces can be stored in Prometheus rather than in a separate solution. This will enable engineers to query, for example, trace latencies alongside CPU metrics. It will also minimize the UI work necessary to join together data. If an organization has the expertise, it’s possible to store everything inside an OpenSearch cluster. This means a single interface, with a single query language, to query all available observability data. Anyone assessing this option should simply decide what their company can maintain and what skills they have in-house.

    Present observability data as one. If storage unification isn’t an option, then focus on the presentation layer. Tools like Grafana can consume most observability solutions as external data sources and render them all on the same dashboard. While an engineer will still have to query them using the specific language of the tool, it means that they can render things alongside one another, making for an insightful dashboard.

    Correlate data in one user interface. This means metrics are rendered in a single dashboard alongside logging and tracing data. This will give users a static view of the data at once, even if there is a limit to how queryable the data is.

    Create universal values. If it’s impossible to make any of the previous recommendations, then this is an easy, quick win that will at least make querying observability data more consistent. An easy way to do this without making your code messy is to use a library like MDC (or your tech stack’s equivalent). Add some tags or values that will exist in all logs, metrics, and traces. For example, the name of the application and the feature it belongs to, i.e. application is credit-card-service and feature is payments. This means that if something goes wrong, there is already a set of key-value pairs that translate across every data type.

    At the very least, this will mean an engineer can issue consistent queries across the three solutions, albeit in different syntaxes. This represents the least value, but also the least rework and effort, and will lower the cognitive load of engineers who are already trying to debug a potentially serious outage.

    The Data Silo Can be Beaten

    A siloed observability strategy drives waste and inefficiency in engineering teams, precisely when they need efficiency the most. By breaking the data silo, observability data can be queried, analyzed, and correlated together to form a single, cohesive and complete picture of your system health.