staging-devopsy.kinsta.cloud

Tag: devops

  • Considering Scriptless Automation: iOS App Dev in 2023

    Considering Scriptless Automation: iOS App Dev in 2023

    According to Statista, over 31,000 apps were released on Apple’s App Store in March 2023. As impressive as that is, it’s considerably smaller than the number of apps submitted to the Apple store during the same month.

    And it’s probably less than the number of apps removed from the App Store.

    Why?

    According to Apple, most submissions fail because of common issues like bugs, crashes, incompleteness, placeholder data and broken links.

    But getting app approval is just one of many challenges faced by iOS developers today.

    Contemporary iOS App User Expectations

    The truth is, even if you slip by Apple, there’s an even greater critic to be concerned about: Your audience. Users are much less forgiving and have high expectations. If your app isn’t perfect, you’ll lose your chance at meaningful adoption and your users will just go elsewhere.

    For iOS users, perfect is simply the status quo, and users have grown to expect it. In practice, this means that apps must be stable across devices and versions, see regular improvements. and provide a great user experience at all times.

    Seamless experience between devices, versions and iterations
    iOS users expect the same experience across multiple devices. From one iPhone to the next, from version to version and from screen size to screen size. It’s the responsibility of developers to prioritize a consistent experience across devices.

    Fast release cycles and regular updates
    Contemporary app users demand feature-rich applications that go above and beyond the simple functions they expect from an app. Given the statistics above, chances are that the competition is fierce. Achieving success usually requires an efficient and speedy release cycle with more planning, more deployments and more testing.

    Easy and frictionless user experience
    User experience (UX) and functionality are one of the big reasons iOS adoption is so strong. People have come to expect high quality from anything under the Apple banner: Users expect an intuitive and trouble-free user experience on every app with which they interact. Every user’s experience needs to be intuitive, issue free and consistent with other iOS applications.

    Challenges Abound

    Meeting those escalating user expectations comes with a wide variety of challenges. For example:

    Device landscape fragmentation
    With so many device types and form factors in circulation, how do you test across all of them? With dozens of iterations of the iPhone combined with an array of versions, screen sizes and configurations, app developers need a way to test on enough devices to cover the majority of their user bases.

    This leads many developers to turn to simulators and emulators to cover devices they lack access to.

    This poses another problem. Simulators aren’t perfect. A simulator doesn’t replicate the functions of the actual device; it can only mimic certain aspects of an iOS device. And it simply can’t effectively mimic hardware, component functionality or network.

    Using a simulator, you can’t test how an application will respond to factors like low battery, a rising network spike, or limited memory, not to mention GPS and camera functionality. You also can’t monitor usage metrics or run meaningful performance tests.

    Real devices present their own challenge. The vast majority of the time, developers will have to worry about the cost of acquiring, storing, maintaining and updating devices. In addition, they’ll have to centralize and control access to them. In an increasingly remote workforce, it’s not a great strategy.

    Manual tests
    Today, most organizations are over-reliant on manual testing. The result? Running test cases becomes the bottleneck that holds up releases. Sure, manual testing is unavoidable in some cases, but their ad-hoc, inconsistent nature leaves room for error. No matter how detailed the instructions may be, people test differently, and that’s a disaster waiting to happen in today’s complex software world.

    Maintaining complex test scripts
    If you’re automating test cases, you’ve experienced the pain of maintaining code-based test scripts. Every release requires coinciding updates to countless unit tests, integration tests and UI tests. The smallest changes can break test cases, causing false negatives and more deployment delays.

    On top of that, you’ve also got to worry about retaining talent that understands your language-specific testing framework. Given the economic uncertainties of today, that’s certainly not a good situation to be in.

    Constant iOS updates
    Each year sees Apple deliver iOS major releases and a variety of minor releases, patches and bug fixes. Organizations have to test after two release cycles: Their own and Apple’s. Failure to do so can lead to compatibility and App Store compliance issues.

    App store guidelines
    As I mentioned above, Apple maintains strict guidelines for any apps published on the App Store. Excellent testing can mean the difference between releasing an app or waiting another few weeks for the next approval. When time is money, this adds up fast.

    Security at scale
    As requirements change, scope increases and features add up, it’s easy to see how it can become more and more challenging to monitor for potential security flaws. In turn, testing needs to be continually updated to reflect the latest security risks.

    A Shift Toward Scriptless Solutions

    Scriptless automation is part of a growing industry of no-code automation tools, often dubbed the no-code revolution. It’s everything test automation is without the reliance on code-based frameworks.

    Scriptless test automation involves recording user interactions with app elements, asserting expected outcomes and assessing the results. The test is then automated, templatized and extended to test other scenarios.

    With scriptless test automation, you can walk through test cases step by step. Some tools even allow you to test on live devices and choose from real iOS devices that are hosted in the cloud.

    Scriptless testing allows you to:

    ● Easily reuse and templatize test scripts. Scriptless automation makes it easier to reuse test cases, create templates and compose new test cases.

    ● Enable faster time to value. Write test cases quicker, make changes without knowing how to write code and integrate directly with your CI/CD tool. As soon as you make changes to your application, your testing team is right there. Some no-code testing tools can analyze your source code and identify testing gaps so that you can address them right away.

    ● Untangle the web of code-based dependencies. With scriptless iOS test automation, testers don’t have to lift and shift their code every time they make a big change. Instead, the no-code framework provides a visual, easily-navigated UI that tells you simply what’s passing, what’s failing and what needs to be updated.

    ● Improve collaboration between teams. Scriptless iOS test automation improves collaboration between testers and developers by making it easy for all teams to visualize test cases.

    Real Device Testing

    When the pain points of fragmentation meet the shortcomings of simulation, developers are wise to test on real devices. At the end of the day, live device testing is the only way to accurately capture an end user’s experience.

    Yet it’s no secret that managing your own device lab is expensive and unrealistic for modern remote work environments. A real device cloud setup solves this problem: A device cloud hosts devices for you that you can access from your desktop, from anywhere.

    Given the ever-increasing complications of today’s mobile app landscape and the incredibly cut-throat mobile app landscape, it’s worth keeping in mind that you may only get one chance at making the right impression on your audience. Getting testing right is the key.

    Into the Future

    Everyone expects more. As a competitive organization, you have to do more with less: Fewer resources, less time and less room for error. Scriptless automation and real device testing won’t guarantee that your app stands out from the crowd, but it will certainly help.

  • Automation in DevOps: Advancements, Platforms and Examples

    Automation in DevOps: Advancements, Platforms and Examples

    Automation is the backbone of DevOps. By facilitating seamless integration and continuous delivery, it revolutionizes the ways organizations deliver software, products and services. When applied correctly, automation can significantly enhance productivity, reduce time-to-market, improve software quality and minimize the risk of human error.

    A range of automation tools support these goals, each tailored to a specific aspect of the software development life cycle (SDLC).

    From code integration to testing and deployment, these tools optimize processes and enable the rapid and reliable release of software. In this article, we will explore the role of automation in the DevOps landscape.

    The Importance of Automation in DevOps

    Automation in DevOps is the process of designing, developing and deploying software and systems with minimal human intervention. The goal is to streamline the software development life cycle from design, development and testing to deployment and management.

    Automation provides several benefits:

    1. Speed and efficiency: Automation enables quick development and deployment of software, which results in faster delivery of features and fixes to customers.
    2. Reliability: Automated systems are less prone to human error, resulting in more reliable code deployment and management.
    3. Consistency: Automation guarantees that routine tasks are performed the same way every time, ensuring consistent results.
    4. Reduced cost: Automation eliminates the need for manual intervention in routine tasks, leading to a reduction in operating costs.
    5. Increased productivity: By taking over routine tasks, automation frees up the team to focus on more critical, high-level tasks.

    Automation in DevOps: An Overview

    Automation in DevOps encapsulates various stages, from development and testing to integration, deployment, and monitoring. The process is often referred to as a DevOps pipeline, a set of practices aimed at reducing development times and improving software quality.

    ● Development: Automation begins at the development phase with tools for source control (like Git), which help developers manage and track code changes.
    ● Integration: The next step is continuous integration, where tools like Jenkins automate the build process, allowing teams to identify issues early in the development life cycle.
    ● Testing: Automated testing tools (like Selenium) streamline the testing process, ensuring that the code is error-free and ready for deployment.
    ● Deployment: Continuous deployment tools (like Kubernetes) automate the deployment process, providing fast, consistent and repeatable deployments.
    ● Documenting: Technical documentation is automated with tools like Scriversi to ensure updated, error-free documentation through each version and release.
    ● Monitoring: Finally, automated monitoring tools (like Prometheus) offer real-time insights into the application’s performance, facilitating proactive troubleshooting.

    Automation’s main objective is to minimize manual intervention, thus reducing the risk of errors, improving efficiency and fostering a culture of shared responsibility.

    Examples of Automation in DevOps

    1. Continuous Integration/Continuous Deployment (CI/CD): CI/CD is a critical element of DevOps automation. With CI, developers merge their changes back to the main branch as often as possible. The CI/CD server automatically builds and tests every change. For example, Jenkins, a popular open source automation server, allows developers to automate various stages of their build pipeline.

    2. Infrastructure-as-Code (IaC): This practice involves managing and provisioning computing infrastructure with machine-readable script files rather than physical hardware configuration or interactive configuration tools. For instance, Terraform, an open source IaC software tool, provides a consistent CLI workflow to manage hundreds of cloud services.

    3. Configuration Management: This is the process of maintaining a system’s configuration in a known, consistent state. For example, Ansible, a configuration management tool, helps to automate software provisioning, configuration management and application deployment.

    4. Monitoring and Logging: Automated monitoring and logging systems gather data on how software and hardware are performing and alert the team to any problems. One example is Prometheus, a popular open source monitoring and alerting toolkit.

    5. Technical Documentation: Proper documentation is crucial in the software development process. Yet, it is often overlooked or not updated frequently due to time constraints or the tedious nature of the work. When automated, it simplifies and automates the process of creating and maintaining technical documents.

    Final Thoughts

    In the world of DevOps, automation is not a luxury but a necessity. By automating repetitive and error-prone tasks, teams can focus on what they do best: creating innovative solutions that meet customer needs. From CI/CD, IaC and configuration management to monitoring and logging, automation simplifies the software development process, reduces human error and enhances speed and efficiency. Tools like Jenkins, Terraform, Ansible, Prometheus and Scriversi support this trend, providing various automation capabilities that make the DevOps approach more efficient and effective.

  • Breaking Boundaries: Integrating Generative AI for Developers

    Breaking Boundaries: Integrating Generative AI for Developers

    Many organizations are experiencing deficits in technical skills and gaps in productivity that ultimately harm their bottom line. To overcome those obstacles and drive growth, businesses can use generative AI tools like ChatGPT and Bard to handle large volumes of data and increase the speed and output of basic tasks without draining valuable human capital. Already, businesses are finding that saving time is one of the most valuable benefits generative AI can deliver. However, there must be oversight of deployments to avoid recurring pitfalls, as the technology contains just as much risk as it does opportunity.

    Generative AI provides developers with helpful task automation while simultaneously creating new challenges. Security, bias and IP questions have DevOps teams scratching their heads at how to best ethically and operationally implement generative AI. Developers must embrace these challenges to remain competitive in an increasingly automated and technologically-driven world.

    Maintaining Confidentiality of Company Code

    Generative AI’s ability to understand and solve code-related issues makes it an ideal tool to help software developers optimize code for faster feature development and quicker releases. By identifying where it can contribute to tasks, like legacy code conversion or automation of processes, companies can free up valuable developer time for building the tools that can grow the business or drive new initiatives. An added benefit is that development teams can automate the coding of new applications or systems by updating old code or re-using existing examples.

    ChatGPT, for example, can also turbocharge development time by scouring swathes of information on GitHub and other forums to find examples of whatever a coder is trying to build. A 2022 study published in the Proceedings of the Association for Computing Machinery on Programming Languages (PACMPL) found that developers could code faster and work more efficiently thanks to generative AI. Tedious searches in tech forums and manual copy-and-paste procedures are replaced by simply asking the right questions (via prompts).

    However, generative AI tools are not closed systems. As users enter data into ChatGPT, it is saved and aggregated to continuously train the language model, opening the door for potential leaks of sensitive company information. There is a serious, potential risk of employees using AI tools as sensitive data already makes up 11% of what employees paste into ChatGPT.

    As a third-party system, AI tools absorb every query, comment and question without any privacy guardrails for users. If an AI bot’s security is compromised, businesses can unintentionally hand over data to bad actors. Even without a hacker’s involvement, sharing confidential customer or partner information may violate contractual agreements and create long legal battles. Developer teams must remain conscious of AI’s security risks.

    Avoiding Bias Creeping Into Results

    It is of utmost importance that businesses make reliable, fair and traceable decisions based on the latest facts. Currently, ChatGPT-4 hasn’t reached the maturity level necessary to be trusted by most companies without significant verification measures. Already, testers have persuaded ChatGPT to make statements that are either inaccurate, biased or both.

    One key concern in generative AI is the ingrained and accidental bias in answers due to the human-generated content used to train large language models. ChatGPT works by searching for textual content relating to the question and recombining it for a distillation. In an almost democratic way, the majority opinion will dominate. For instance, if most people believe that the Swedish are hard drinkers, ChatGPT will say so. While the technology provides remarkably accurate responses in an alarmingly convincing way, it’s important to remember that it’s not always the most reliable or balanced source that wins.

    Furthermore, OpenAI, the creator of ChatGPT, has explicitly stated that its goal is to create a system where queries can be pre-processed before they are fed to the neural net. Answers can then be post-processed before being shown to users. But this safeguard lacks clarity around its objectives and opens the door for ideological or political manipulation.

    Companies considering implementing AI tools must carefully examine their potential applications to ensure they avoid scenarios where bias can result in insights and output that could end up harming the company and its reputation.

    Minimizing Developer Function

    Enterprises are looking for tools to integrate generative AI into existing business processes to enhance numerous functions and deliver greater value to their customers. While the technology still isn’t capable of replacing human developers responsible for updating and maintaining largescale software, updates to generative AI tools and integrations are occurring at a rapid pace, leaving businesses to determine which programs are most viable for developer function.

    As the economy continues to tumble closer to a recession, companies seek ways to minimize costs. And those looking to use generative AI to work with the source code and technical documentation the company uses to run its own operations—rather than the open source code tools like ChatGPT provide—face budgetary decisions. Implementing generative AI programs into the business model can be extremely costly if done wrong, leaving executives to weigh the benefits of adopting the tech versus the cost of trying to hire a full team of developers.

    Connecting the Enterprise to Avoid AI Pitfalls

    For businesses looking to build the connected experiences their employees, partners and customers desire, generative AI is a great resource to accomplish this. However, it’s just one piece of the puzzle that is a modern enterprise. These tools can only deliver smart answers if they get smart questions and businesses connect their internal processes and applications to enable machine learning on accurate and holistic datasets.

    Although AI poses many benefits for developers, such as increased efficiency, accuracy and the ability to handle large amounts of data, they must be wary of the potential security, ethical and budgetary concerns that impact employees, customers and the enterprise as a whole.

    However, once the concerns are overcome or managed, the power of large language models will be leveraged in all kinds of technology and business use cases. Certainly a truly transformational shift and a step change towards a world where technology and software play an unprecedented role in business and society.

  • Automation or Concatenation?

    Automation or Concatenation?

    We’ve been in a period of mergers and acquisitions for a while in the DevOps and DevSecOps spaces, and it seems to be heating up even more. Combined with products expanding to cover a more broad segment of the software development life cycle (SDLC) and strategic partnerships to cover more of the SDLC, it is starting to look like our automation efforts are going to be increasingly driven by vendors. Those efforts will be more concatenation than automation–though technically the same in perception, they vary wildly in implementation, and we should be planning for it.

    Build Versus Buy

    We have always struggled with build versus buy, and we have always struggled with best-of-breed versus overarching solution set. The entire hype of Every. Single. Vendor. Claiming to be a platform is partially driven by the latter argument. Putting “platform” in your marketing does not make a vendor better at those things outside its historical strengths, though, so that movement is doomed to fail for most organizations. Some were already moving to be a platform and had assembled–by M&A or by internal development–an impressive platform already, and those were successful because they were good at a breadth of things. The best of them are continuing this trend. I try to avoid plugging vendors here, but I think an example is in order in this case.

    Akamai’s recent purchase of NeoSec is the poster child for vendors doing the platform thing super well but not resting on their laurels. I’ve followed Akamai’s security offerings for years, and the platform has impressive functionality, but NeoSec is exactly what they needed. It plugs a couple of holes in their security offerings and brings a whole customer base into the Akamai fold. That’s what I’m talking about when I refer to the good ones.

    The problem is the acquisition and expansion of offerings trend is starting to gain a lot of steam, and not every vendor jumping on the bandwagon does the job as well as Akamai does.

    This leaves enterprise IT staff responsible for DevOps and DevSecOps – even straight security staff – with a new set of issues. Best-of-breed is taking a hit again, and overarching solutions are on the rise. The difference is the breadth that some of these new “platforms” offer is massive. And we know that even the good ones will have better and worse parts of the overall solution. So IT now has to start considering what vendors call a feature and how it individually compares to the competition. This will require more up-front work for evaluators. Now, a larger (and growing) list of “features” that were once “products” has to be considered piece-by-piece against the competition, and organizations must evaluate the feasibility (and cost) of implementing two (or even multiple) massive platforms to get the most critical functionality needed.

    As of this writing, point solutions that do one thing extremely well are holding up, but the trend is obvious. Those markets will be subsumed into larger offerings that may not do as well at other parts of the overall problem. Evaluators will have to be more careful and consider their options with an eye to the larger infrastructure and problem domain. Do you want a single vendor doing all of the steps of DevOps or responsible for every aspect of application security? That will be an individual decision and, if not, how the final solution/architecture looks will be a more complex discussion.

    Integration and Automation

    In our quest for integration and automation, many vendors are responding with the concatenation of tools to solve the automation part. You’ll be stuck figuring out if they’re doing it well enough for the org.

    But that’s not a problem you can’t resolve. You’ve been rocking it through staffing issues, lockdowns, growth and shrinking… Just keep rocking it. Choose the solution that makes the overall best sense for your organization, get it running and take a break; head to the beach for some relaxation time. Or whatever you do to clear your head – because the next problem will still be there when you get back, and it’s better to be relaxed when you tackle it.

  • Using Generative AI for Issue Remediation and Software Reliability

    Using Generative AI for Issue Remediation and Software Reliability

    In today’s fast-paced digital landscape, keeping your services up and performant, especially for cloud-based and online services, is crucial for staying competitive. Downtime or latency issues can drive customers away, especially when all it takes is a click to switch over to a competing SaaS solution. DevOps and site reliability engineering (SRE) teams face the constant challenge of minimizing mean-time-to-remediation (MTTR) when dealing with errors and issues. As valuable as search engines like Google are, the sheer amount of information available can make finding the right solutions a time-consuming task. What if there was a way to automate and streamline this process, making error investigation more intelligent, focused and efficient?

    This article will walk you through the journey of error resolution from the first obscure log line to identifying the problem hidden within it. We’ll resolve the issue using several tactics, ultimately leading to the use of generative AI to cut MTTR and provide precise IT recommendations. This journey includes useful measures and principles that can be applied by DevOps teams and explores the potential for applying generative AI.

    The Current Experience of DevOps/SRE

    Picture this: A log line pops up with an obscure error message, and the first instinct is to search for this message on Google for potential solutions. After all, someone else must have encountered a similar problem, right? However, the abundance of resources and search results can be overwhelming. The search results are often displayed based on website relevance, not necessarily the relevance of the error itself. Consequently, precious time is wasted sifting through countless search results, leading to a longer time to understand and remediate the error. For DevOps and SRE teams responsible for maintaining system stability, reducing MTTR is a crucial key performance indicator (KPI). This begs the question: How can we leverage automation and artificial intelligence to improve the search process and accelerate error resolution?

    Our First Step: Cognitive Insights

    In our organization’s initial attempt to tackle this challenge, we focused on leveraging crowdsourcing techniques to get more relevant results than the brute-force Google search. Moreover, we wanted to automate the process and run it offline so that when an incident occurs, we can offer useful insights rather than starting the search as the system is down or malfunctioning.

    The approach involves an offline phase and an online phase. The offline phase involved analyzing all our ingested logs and identifying common log patterns. This also allowed us to count how often this pattern occurred in our system and how prevalent it is. Then we crawled relevant technology forums like StackOverflow, Google Groups and Bing Groups for relevant discourse around these log patterns that could offer relevant solutions. The next step was to rank the search results by relevance and take the top five most relevant links.

    The offline phase resulted in a library of known log patterns and, for each one, a cognitive insight containing these links, as well as additional information such as the severity level, how many times this pattern occurred in the logs, the date of first occurrence and additional tags of involved technologies, tools and domains.

    The online phase occurs in real-time as new logs come in. Upon ingesting the log, it is automatically matched against all known patterns; if it matches one, it gets the cognitive insight for that pattern. This means that as soon as the problematic log comes in, the DevOps engineer already has focused and ranked search results and context to start from, which accelerates the investigation process.

    The Next Step: Why Don’t we ask Generative AI?

    After reflecting on our initial approach, we had an epiphany. Large language models (LLMs) like ChatGPT had already crawled the web and absorbed a vast amount of information. So, why not leverage their capabilities and ask them directly for insights? The idea was simple: Let the AI “read the posts” for us and provide us with recommendations. We started by formulating specific questions such as “What could the possible errors be?” and even went a step further by asking for further investigation steps. However, implementing this seemingly straightforward approach came with its own set of challenges. We needed to run preprocessing before asking the generative AI, as well as running post-processing on the answers it returned, to get what we expected. Let’s see what that process entails.

    How we Did It: Analyzing, Sanitizing and Validating

    Before asking ChatGPT (or any other generative AI tool) for assistance, several preparatory steps were necessary. First, we had to analyze incoming logs, identify patterns and score them based on severity. This allowed us to prioritize the most critical issues.

    Secondly, we had to carefully design our prompts for the generative AI using prompt engineering, ensuring that we framed our questions in a way that yielded precise and relevant responses. Prompt engineering is the practice of strategically designing effective prompts or instructions for language models to achieve more accurate responses. It involves carefully specifying the input format, context or constraints to guide the model’s generation process and obtain more accurate and desired results. This is true for any use of generative AI, and for our case in particular. Beyond accuracy, prompt engineering helped us tune the requested format and length of the answer to fit the “executive summary” paragraph we popped up for the user. This ensured it wasn’t too lengthy nor too short to provide meaningful insights.

    Additionally, we took great care in sanitizing the queries and removing any sensitive data to maintain privacy and security. This step was especially crucial for those working with public services, highlighting the importance of protecting users’ personally identifiable information (PII), such as names, emails, phone numbers, IP addresses or GPS coordinates.

    With all this preprocessing done, we sent the prompt to the generative AI. It is important to note that public services may not always be available (as some returned a “service is busy” message when working with ChatGPT and other services), so automation should take care of delays and employ retries as needed.

    Finally, as responses were received from the AI, it was essential to validate those. We ran an analysis and filtered out non-relevant answers from different domains (after all, a log is also a product of a tree, and Kafka is also a novelist), ensuring there was no offensive content or misleading information. The semantic integrity of the responses was also carefully evaluated.

    These AI insights have proven to be a powerful troubleshooting tool, and we’ve made it an integral part of our service.

    This journey is not unique. In fact, DevOps teams can employ similar measures for their own telemetry data to serve their own operational workflows. These principles, such as data sanitization and validation, even apply to other types of data and use cases.

  • Complete Guide to AWS Savings Plans

    Complete Guide to AWS Savings Plans

    AWS Savings Plans are a flexible and cost-effective way to save on your Amazon Web Services (AWS) usage. These plans allow you to commit to a specific amount of compute capacity over a one- or three-year term in exchange for a significant discount on your AWS usage. Savings plans can help you reduce your AWS bill by up to 72%, depending on the type of plan and your level of commitment.

    Why Do You Need AWS Savings Plans?

    With the growing reliance on cloud infrastructure in modern IT, it is crucial for organizations to optimize cloud computing costs without compromising on performance or scalability. AWS Savings Plans address this need by providing a flexible and cost-effective solution that allows you to commit to a certain level of usage at a discounted rate.

    With AWS Savings Plans, by committing to a specific amount of compute capacity over a set time period, you can save up to 72% on your AWS usage. This can result in significant cost savings for your organization, freeing up resources to invest in other business-critical initiatives.

    Another benefit of AWS Savings Plans is the flexibility they offer. Unlike Reserved Instances, which require you to specify instance type, operating system and region, Savings Plans are more flexible and can be applied across different instance types, regions and operating systems. This flexibility makes it easier for you to optimize your cloud computing costs while still meeting your business needs.

    Difference Between AWS Savings Plans and Reserved Instances

    AWS Savings Plans and Reserved Instances are both designed to help you save on your AWS usage, but they offer different levels of flexibility and cost savings.

    Reserved Instances require you to commit to a specific instance type, operating system and region for a one- or three-year term in exchange for a discount on your AWS usage. While this can result in significant cost savings, it also limits your ability to optimize your cloud computing costs based on changing business needs.

    AWS Savings Plans offer a more flexible solution that allows you to commit to a certain level of usage across different instance types, regions and operating systems. Additionally, in most scenarios, Savings Plans can provide a higher level of cost savings compared to Reserved Instances.

    Types of AWS Savings Plans

    There are two types of AWS Savings Plans: Compute Savings Plans and EC2 Instance Savings Plans.

    Compute Savings Plans

    Compute Savings Plans are the most flexible type of Savings Plans, providing a significant discount on your AWS usage across any instance type, region or operating system. This flexibility makes it easy for you to optimize your cloud computing costs based on your changing business needs.

    Compute Savings Plans offer a discount of up to 66% compared to on-demand pricing, depending on the level of commitment you make. They can be used across all AWS services that are powered by Amazon EC2 instances, including Amazon EMR, AWS Lambda and Amazon ECS.

    EC2 Instance Savings Plans

    EC2 Instance Savings Plans are a more specific type of Savings Plan, offering a discount on your AWS usage for a specific instance type within a region. These plans provide a higher level of cost savings compared to Compute Savings Plans, with discounts of up to 72%.

    EC2 Instance Savings Plans are ideal for organizations that have a predictable workload and require specific instance types within a region. However, they offer less flexibility than Compute Savings Plans, as they cannot be applied across different instance types, regions or operating systems.

    AWS Savings Plans Pricing Models

    AWS Savings Plans offer three different payment options to suit your organization’s budget and commitment level: All Upfront, Partial Upfront and No Upfront.

    All Upfront

    With the All Upfront payment option, you pay for the entire term of your Savings Plan upfront, resulting in the highest discount compared to on-demand pricing. This option is ideal for organizations that have the budget available to make a significant upfront payment in exchange for maximum cost savings.

    Partial Upfront

    The Partial Upfront payment option allows you to pay a portion of your Savings Plan upfront and the remaining balance on a monthly basis. This option provides a balance between upfront commitment and ongoing flexibility, with a lower discount compared to the All Upfront option.

    No Upfront

    With the No Upfront payment option, you pay for your Savings Plan on a monthly basis, with no upfront payment required. This option provides the lowest discount compared to on-demand pricing but offers the greatest flexibility for organizations with limited budgets or uncertain future usage requirements.

    How to Monitor and Manage AWS Savings Plans

    Monitoring and managing your AWS Savings Plans, as part of your overall cloud operations strategy, is critical to ensuring that you are optimizing your cloud computing costs and maximizing your savings. AWS provides several tools and services to help you monitor and manage your Savings Plans effectively:

    • AWS Cost Explorer: A powerful tool that allows you to analyze your Savings Plans usage and costs, helping you identify opportunities to optimize your cloud computing costs. With Cost Explorer, you can view your Savings Plans coverage, utilization and savings, as well as forecast your future usage and costs.
    • AWS Budgets: This service allows you to set custom cost and usage budgets for your organization, helping you track your Savings Plans usage and ensure that you are staying within your budget.
    • Savings Plans Recommendations: This feature analyzes your past usage and recommends the optimal Savings Plans for your organization. It can help you make informed decisions about which Savings Plans to purchase and how much commitment to make.

    Conclusion

    AWS Savings Plans are an excellent solution for organizations looking to optimize their cloud computing costs and save on their AWS usage. With their flexibility, cost savings and easy monitoring and management options, AWS Savings Plans are a valuable tool for any organization using AWS services.

    To unlock the full potential of AWS Savings Plans, it’s crucial to understand the different types of plans, their pricing models and how to effectively monitor and manage your usage. By doing so, you can maximize your savings and ensure that your organization is getting the most out of its investment in AWS.

  • Postman Acquires Akita Software to Gain API Observability

    Postman Acquires Akita Software to Gain API Observability

    Postman today announced it has acquired Akita Software to gain access to a platform for monitoring and observing application programming interfaces (APIs).

    The first integration of the respective products offered by both companies will arrive in beta this fall.

    Postman CEO Abhinav Asthana said the acquisition of Akita Software will make it much simpler for organizations to discover what APIs are being employed across an enterprise. A recent Postman survey found that 28% of developers are now deploying APIs at least once per week.

    The Akita Software platform will make it simpler for organizations to discover both rogue APIs that have been created by a shadow IT project and zombie APIs that have been inadvertently left exposed long after they are needed.

    Akita Software’s CEO Jean Yang said the integration with Postman’s API management platform will provide a rich source of data that can be used to add artificial intelligence (AI) capabilities to Akita’s API observability platform. For example, generative AI capabilities should make it simpler to create summaries in natural language that explain the calls an API is making to specific data sources.

    In addition, the combined company will be in a better position to enable organizations to better secure their APIs after deployment by surfacing anomalous behavior indicative of an API that is being used to exfiltrate data, she added.

    The merger with Postman will, in effect, enable the combined entity to provide a complete ecosystem for managing APIs, noted Yang.

    Most APIs are used primarily to interconnect internal-facing applications and systems, with a subset used to integrate partners. Only a relatively small percentage of APIs are externally facing in the sense that they expose data to the internet. Many organizations, however, fail to appreciate how quickly an internally facing API can become externally facing over the course of its life cycle. As such, IT teams need tools and platforms that can manage and secure APIs as they evolve.

    As APIs continue to play a crucial role in advancing business goals, there is a clear need for increased visibility as the types of APIs employed also continue to expand. Most APIs in use today are based on the REST format, but there are also APIs based on GraphQL, event-driven architecture and various messaging platforms. None of these newer APIs will supplant REST APIs any time soon. Rather, they are being applied to drive additional processes alongside REST APIs.

    In the longer term, it’s not clear how many APIs will be managed by the developers that created them versus a centralized DevOps team. However, the more APIs that are deployed in a production environment, the less feasible it becomes for developers to manage their life cycle. The challenge is making sure there is some type of structured workflow in place that, at the very least, makes everyone involved in the process aware that a new API has been created and deployed.

  • How Platform Engineering Bridges the IT and DevOps Divide

    How Platform Engineering Bridges the IT and DevOps Divide

    Ballooning lead and process time, with an opaque corollary relationship to fixed and marginal cost, is a significant and complex burden for any enterprise. And multi-cloud complexity and inefficient organizational and technical processes are some of the most wasteful areas companies face–so much so that Synergy Research reports organizations are cutting back on their cloud spend as tech budgets continue to fall under intense scrutiny, resulting in a Q1 2023 cloud spending growth rate of 19% compared to 32% just last year. With cloud cost containment a top priority, Forrester advised tech leaders to demonstrate to CFOs how they will stay on budget and demonstrate ROI at the board level. A key area where companies can eliminate expensive waste and increase time to value is the relationship and workflow between IT and developers, which, while being the DevOps charter for nearly 15 years, has gotten less attention among the viable cost-containment strategies for enterprises in 2023. Revisiting the latest iteration of this DevOps charter through the implementation of platform engineering strategies can help enterprises create a faster, leaner and more scalable dynamic. Let us examine why it’s now time for enterprises to consider platform engineering as a potent way to not only increase efficiency but also to bridge the divide between DevOps and IT.

    The Old Way: Forcing DevOps to Dance to the ITSM Tune

    In traditional enterprise organizations, IT typically dictates a top-down ITSM practice that limits the tools an organization can use for development and deployment; in many cases, it inhibits CI/CD and other core capabilities and practices of DevOps. This awkward attempt to accommodate modern DevOps practice through an aging ITSM paradigm has led to a suboptimal software delivery performance and poor developer experience (DX).

    McKinsey, highlighting the criticality of DX, found developers at a leading SaaS provider spent 20% to 30% of their time integrating different services such as firewall changes, DNS and security access to deliver code through its CI/CD pipeline. That’s a lot of time spent on low-value, repetitive tasks, ensuring a painful journey for a disappointing result. If the DX pain is high enough, developers often go around IT and create resources on their own, leading to shadow IT, adding new cost burdens and growing security exposures. Every minute and engineer count: Five minutes saved per day among 500 developers saves a week’s development time.

    Surely, there is a better way.

    Platform Engineering: Collaboration Through Composition

    Platforming engineering and “platform as a product” have been key to the PaaS ecosystem for years but are now gaining fresh traction in the industry. In Puppet’s State of DevOps Report, 51% of respondents said they had already adopted platform engineering and 93% said it was a step in the right direction. Gartner predicted 80% of software engineering organizations will have platform teams by 2026.

    The concept can be defined in several ways. Gartner reported platform engineering is “an emerging trend intended to modernize enterprise software delivery… designed to support the needs of software developers and others by providing common, reusable tools and capabilities, and interfacing to complex infrastructure.” PlatformEngineering.org’s recent blog post defines it as the discipline of designing and building toolchains and workflows for self-service capabilities in software engineering organizations during the cloud-native era.

    Regardless of definition, platform engineering is the latest iteration of IT centralization, though now attempting to retain all the benefits of distributed team empowerment through “composition” rather than converged control. As modernization continues, we experience the ebb and flow of composition and decomposition, both organizationally and technically. Microservices, Kubernetes, and GitOps patterns pushed the rational limits of decomposition and significantly raised the level of technical complexity in the enterprise. The natural counter pressure is a new cycle of composition that aims to simplify that complexity through platform engineering practices. Teams now look for these toolchains and workflows to be centralized in internal developer platforms (aka IDPs), exposing the development, deployment and maintenance of the entire life cycle of modern, cloud-native applications.

    IDPs can facilitate the flow, speed and innovation DevOps teams are demanding without sacrificing stability, security and control. They enable developers to configure their toolchain while meeting all controls through approved blueprints or “golden paths,” reducing lead and process time. Developers can focus on delivering business value instead of infrastructure tasks.

    Bringing IT and DevOps Back Into Alignment Saves Time, Money and Sanity

    Platform engineering creates a collaborative framework that connects IT and DevOps teams, so they can focus on outcomes rather than methods. Developers can use an IDP to follow conventions that speed up development and prevent anti-pattern creep. Guardrailing choices through golden paths reduce cognitive load and simplify execution context for security teams. The result? Developers can focus on what they were hired for and want to do–creating innovative, transformative applications and differentiated value. Security can sleep well knowing bespoke behavior isn’t opening new attack surface areas.

    On the Day 2 operations side, IDPs enable SRE and support teams to have a fleet-wide view and work effectively with developers to streamline the management of production needs across all applications, clouds and tech stacks.

    The reframing of composition breaks the false dichotomies between IT and DevOps practices. You don’t have to choose between control and flow, change isn’t triggered by serial requests or async events, safety isn’t found in planning or speed, value isn’t found in stability or innovation and administrative plane primitives aren’t servers or clusters. Platform engineering practices allow you to prioritize composition, orchestration, golden paths, holistic scalability and fleet management to get teams out of entrenched myopia and embrace systems thinking that gets your business delivering strategic gain instead of tactical win/loss tallies.

    Finally, platform engineering positively impacts software delivery and operational performance metrics by streamlining development processes and reducing manual efforts. The bottom line is faster time to market while ensuring more consistency, less human error and improved software quality. This maximizes productivity, minimizes downtime and optimizes resource utilization, leading to cost advantages and increased efficiency for enterprises.

  • A DevOps Guide to IoT Technology

    A DevOps Guide to IoT Technology

    Internet-of-things (IoT) is a term used to describe the network of physical devices—from everyday household items to sophisticated industrial tools—connected to the internet, sharing and collecting data. With the advent of cheap computer chips and the ubiquity of wireless networks, it’s possible to make anything a part of the IoT. Add to that the exponential growth in artificial intelligence and machine learning, and the IoT is a truly significant part of our lives.

    The IoT allows devices to communicate and interact over the internet, and they can be remotely monitored and controlled using technologies many of us have never heard of, such as MQTT, QUIC and AMQP. The proliferation of these devices has led to an exponential increase in the amount of data we can gather, analyze and use to make more informed decisions. We’ve already seen the transformative potential of IoT in our homes, workplaces and cities, and it’s only set to increase as more and more devices come online.

    At the heart of this technology is the concept of seamless connectivity and communication. However, the rapid development and deployment of IoT devices pose significant challenges. This is where the DevOps approach comes in, offering a solution to these challenges by improving efficiency and effectiveness in the development and operation of IoT systems.

    Importance of DevOps in IoT Development and Deployment

    In the context of IoT, DevOps is crucial. IoT devices are often deployed in the field, in homes or in other environments where physical access is difficult or impossible. As such, the ability to update and manage these devices remotely is a critical necessity. DevOps methodologies enable this remote management, providing mechanisms for continuous integration and continuous delivery (CI/CD) that make it possible to push updates to devices regularly and reliably.

    Moreover, given the pace at which IoT technology is advancing, it’s crucial to be able to adapt quickly to changes and new developments. In this regard, DevOps is invaluable. By bringing together development and operations, it allows for more agile response to changes, faster issue resolution and overall better performance.

    The Role of DevOps in IoT

    Speeding Up IoT Development Cycles

    One of the key benefits of using a DevOps approach in IoT development is the ability to speed up development cycles. With traditional development practices, the time between identifying a requirement or an issue and implementing a solution can be lengthy. In contrast, DevOps practices, such as continuous integration and continuous delivery, enable rapid prototyping, development, testing and deployment of new features and updates.

    With the DevOps approach, developers and operations teams work together from the beginning of a project, collaborating closely to ensure that the software being developed is robust, reliable and meets the needs of the business. This cooperative approach, combined with automated testing and deployment processes, significantly reduces the time to market for new features and updates.

    Ensuring High-Quality, Reliable IoT Applications

    Another significant advantage of utilizing DevOps in IoT development is the increased quality and reliability of the applications being developed. With traditional software development methods, testing and quality assurance often occur after the development process. This can lead to issues being identified late in the cycle, which can be costly and time-consuming to fix.

    On the other hand, DevOps practices emphasize continuous testing throughout the development process. This means that issues can be identified and addressed much earlier, leading to higher-quality, more reliable software. In the context of IoT, where devices may be deployed in critical applications such as health care or transportation, the importance of this cannot be overstated.

    Enabling Scalability and Flexibility in IoT Systems

    Finally, DevOps plays a crucial role in enabling the scalability and flexibility of IoT systems. As the number of IoT devices continues to grow exponentially, the ability to manage and monitor these devices efficiently is becoming increasingly important. DevOps practices, such as infrastructure as code (IaC) and the use of containers and microservices, provide the tools necessary to scale and adapt IoT systems as needed.

    With IaC, the infrastructure required to run an IoT application is defined and managed as code, making it easy to replicate and scale. Containers and microservices, on the other hand, allow applications to be broken down into smaller, independent components that can be scaled individually, providing a high level of flexibility and efficiency.

    Implementing DevOps in IoT Development

    Integrating CI/CD in IoT Application Development

    Continuous integration/continuous delivery (CI/CD) is a fundamental DevOps practice that plays a vital role in IoT development. It involves automating the processes of software delivery and infrastructure changes. With CI, developers frequently merge their code changes into a central repository where automated builds and tests are run. CD takes this a step further by automating the release of these changes to the production environment.

    For IoT, integrating CI/CD can help maintain the quality of software, increase efficiency, and speed up the process of delivering software updates or new features to devices. It also ensures that any changes made can be easily rolled back, reducing the impact of any potential issues.

    Applying IaC for IoT Infrastructure Management

    Infrastructure-as-code (IaC) is another key DevOps practice that is highly relevant to IoT. IaC is the process of managing and provisioning computing infrastructure through machine-readable definition files rather than physical hardware configuration or interactive configuration tools.

    In the context of IoT, IaC can simplify the process of setting up and managing the complex infrastructure required for IoT applications. With IaC, developers can quickly and reliably replicate infrastructure setups, making it easier to scale IoT applications as the number of connected devices grows.

    Using Containers and Microservices in IoT Application Architecture

    Containers and microservices are two more DevOps practices that can significantly benefit IoT development. A container is a standalone executable package that includes everything needed to run an application, including the code, runtime, system tools, system libraries and settings. Microservices is an architectural style that structures an application as a collection of small, loosely coupled services.

    In the context of IoT, using containers and microservices can greatly simplify the process of developing, testing and deploying IoT applications. Moreover, they can enhance the scalability and flexibility of IoT systems, making it easier to adapt to changing requirements and scale as the number of connected devices grows.

    Challenges of Implementing DevOps for IoT and How to Overcome Them

    Dealing with Device and Platform Diversity

    One of the main challenges in implementing DevOps for IoT is dealing with the diversity of devices and platforms. IoT devices can range from simple sensors to complex machines, each with its own capabilities, requirements and operating systems. Moreover, these devices may communicate using different protocols and have different connectivity requirements.

    The key to addressing this diversity is to adopt a platform-agnostic approach. This involves developing software that can run on any device, regardless of its hardware or operating system. Containerization can help achieve this by providing a consistent and reproducible environment for running applications.

    Ensuring IoT Security and Privacy

    Security and privacy are significant concerns in IoT, and they can be particularly challenging in a DevOps context. The increased frequency of updates and changes that come with the DevOps approach can potentially introduce new vulnerabilities. Moreover, the interconnected nature of IoT devices means that a security breach can have far-reaching consequences.

    To address these challenges, it’s essential to integrate security practices into the DevOps process. This is often referred to as DevSecOps. This approach involves continuously monitoring and testing for security vulnerabilities throughout the development process rather than treating security as an afterthought.

    Managing IoT Data Volume and Complexity

    The sheer volume and complexity of data generated by IoT devices can also pose a challenge in a DevOps context. Traditional data management and analysis tools may not be able to handle this volume and complexity.

    To tackle this, it’s important to utilize modern data management solutions that can handle large volumes of data. Additionally, employing advanced analytics and machine learning techniques can help make sense of complex data and extract valuable insights.

    Coordinating IoT System Updates and Maintenance

    Finally, coordinating updates and maintenance for a large number of IoT devices can be a significant challenge. This is particularly true in a DevOps context, where updates and changes are frequent.

    To address this, it’s crucial to have a robust system for managing and tracking updates. Automated deployment tools can be invaluable in this regard, allowing updates to be pushed out to devices in a controlled and coordinated manner.

    Conclusion

    In conclusion, implementing DevOps in IoT development can bring significant benefits, including faster development cycles, higher-quality, reliable applications and improved scalability and flexibility. However, it also presents unique challenges that need to be effectively addressed. By understanding and properly implementing the principles and practices of DevOps, we can harness the full potential of IoT technology and drive forward innovation.

  • Communicating Common Web App Security Threats to Developers

    Communicating Common Web App Security Threats to Developers

    Automated security testing tools enable us to identify web application vulnerabilities early and often, but what happens after a vulnerability has been identified? Does your security team have the coding skills to mitigate it themselves, or are your developers suddenly security experts once a vulnerability is discovered? Most likely not. So, how do we go from finding vulnerabilities to fixing them? Enter a key component of shifting left: Collaboration between security and developer teams. And a portion of that involves providing developers with the context they need to prioritize and remediate threats appropriately.

    Useful context includes (but is not limited to):
    – The vulnerability type
    – How it works
    – What malicious actors can accomplish by exploiting the vulnerability
    – Ways to mitigate

    The list below provides context around five common web application security threats to help you collaborate on resolutions with your developers. A shared understanding of these threats and how to mitigate them early in the development life cycle will significantly improve overall security posture without impacting software delivery.

    SQL Injection

    A SQL injection attack is executed when an attacker injects malicious code into an application’s database through user-controlled data. While these types of attacks can accomplish many different things, there are several common outcomes. SQL injection attacks can enable attackers to gain unauthorized access to sensitive data stored in the database, and largely depending on what data is stored in a particular database, can result in access to authentication information, financial information and personal data. The second key outcome is the manipulation or removal of data. Malicious actors can then leverage this outcome to access the larger infrastructure and find even more sensitive information.

    To help protect against SQL injection attacks, organizations should never implicitly trust and always validate user input and utilize input encoding, prepared statements, parameterized queries or stored procedures instead of dynamic SQL statements whenever possible. By combining frontend and backend standards to prevent SQL injection from happening, your application can increase its security against this type of threat.

    Cross-Site Scripting (XSS)

    Another common web application attack is cross-site scripting (XSS) which involves inserting malicious code or script into a website. The website will then execute on the given script, enabling attackers to steal sensitive user data, such as session tokens and cookies.

    There are three main types of XSS attacks we regularly encounter: Reflective, stored and DOM. Reflective XSS attacks insert malicious code into a website that is immediately executed, whereas a stored XSS attack involves injecting malicious code into a website that is stored and executed later. DOM XSS is an attack pattern that is used more in single-page applications but acts similarly to reflective XSS. If successful, a cross-site scripting attack can lead to the theft of user session IDs and/or website defacement and lead to redirection to malicious sites, thus enabling phishing attacks.

    Similar to SQL injection attacks, to protect against cross-site scripting, organizations must never implicitly trust and always validate user input and use output encoding techniques while also using auto-sanitization libraries such as OWASP AntiSamy and implementing a content security policy.

    Cross-Site Request Forgery (CSRF)

    Cross-site request forgery (CSRF|XSRF) attacks trick users into performing an action on a website without their knowledge or consent. This is often achieved by inserting a malicious link or form into a website where a user is already authenticated. When the user clicks the link or submits the form, the action is performed on their behalf, leading to potential data loss or unauthorized access.

    Mitigating these types of attacks requires leveraging CSRF protections which are built into the framework you are using, if applicable. Using CSRF tokens, which are unique, randomized values associated with a user’s session and can verify the authenticity of a request will also help migrate an attack. In addition, SameSite cookie settings can help prevent attackers from being able to send requests on behalf of a victim.

    Insecure Direct Object References (IDOR)

    Insecure direct object references (IDOR) occur when an application exposes direct object references, like URLs or database keys, which allow attackers to gain access to restricted data by manipulating these references.

    Safeguarding against IDOR requires the implementation of proper access controls and session management, ensuring that only authorized users have access to certain resources or data. Also, avoid using predictable references and instead utilize globally unique identifiers (GUIDs) to prevent attackers from guessing the direct object references they need to access restricted data.

    It is important to remember that IDOR-based vulnerabilities don’t occur singularly and are coupled with other vulnerabilities to become an effective attack vector.

    Remote Code Execution (RCE)

    Remote code execution (RCE) gives an attacker the ability to execute arbitrary code on a server, which can lead to the compromise of entire systems and offer unauthorized access to highly sensitive data. RCE attacks can occur through an array of means, including exploiting vulnerabilities in code libraries or injecting malicious code through user input fields. As illustrated by the critical zero-day exploit discovered in the Java logging library Log4j in December 2021, RCE can be a determinantal vulnerability.

    A successful RCE attack can cause denial-of-service (DoS) attacks, expose sensitive data, lead to illicit cryptocurrency mining and the execution of malware. In more severe instances, a successful RCE attack can allow an attacker to gain full control over the compromised machine.

    The good news is organizations can implement the same tactics used for protection against SQL Injection and XSS—never implicitly trust user data and adhere to strong user input sanitization and validation.

    Understanding Common Threats

    Safeguarding applications and ensuring a sound security posture requires both security and developer teams to have a shared understanding of common threats and the associated risks.

    The details provided in these examples only scratch the surface in terms of useful context for developers. When you can pinpoint when a vulnerability was introduced, which repository branch and code commit it’s associated with and whether it is exploitable from the outside, you create a powerful feedback loop that accelerates fixes.