staging-devopsy.kinsta.cloud

Category: DevOps Toolbox

  • 4 Benefits of Integrating a Language With a Database

    4 Benefits of Integrating a Language With a Database

    One of the hallmarks of a truly modern programming language is the coupling of a language—for creating applications, devising algorithms and solving business problems—with an underlying database. When there is practically no separation between a database and its programming language, developers immensely increase their productivity, maximize the effectiveness of the code they write and provide unparalleled speed and flexibility in supporting business needs.

    Predicating a programming language on a database delivers these advantages by enabling developers to treat data as if it is in-memory, liberating them from the burden of manipulating how data is represented in the underlying database. This one simple advantage exponentially increases the flexibility for addressing business problems and, when used with advanced logic languages, enables developers to write applications in a fraction of the time and with less effort than it would otherwise require.

    Specific benefits of tightly coupling a database with its programming language include eliminating the time-honored mismatch between data structures and database representations, smart caching with transactional technologies for committing or rolling back database changes and automatically changing object definitions.

    These benefits remove the database as an otherwise complicating factor in supporting business use of data by freeing developers to flexibly innovate solutions—and speedily implement them.

    In-Memory Manipulations

    Developers focus on manipulating data structures to solve business problems. Traditionally, this concern was hamstrung by the onus of translating those structures into how data are represented in the database, a time-consuming necessity detracting from programmers’ attention to business problems. By fully integrating an object-oriented database or graph database with its programming environment, users can manipulate data objects in memory and they’ll automatically persist to storage. Developers can solely focus on working with data structures for business objectives without spending just as much, if not more, effort wrangling data’s representation in the database.

    Programming languages create this effect when they are imbued with an intelligent caching layer that persists in-memory data manipulations into the database. This advantage requires basing the programming language on a specific database. It eliminates the mismatch between working on data structures and toiling to get the underlying data to and from disk.

    Smart Caching

    The caching layer also tames the inherent difficulties in data objects pointing to other objects, which is a hallmark of object-oriented systems and graph databases. These databases require intelligent caching because one object can conceivably include the majority of the database by pointing to objects that, in turn, point to additional objects. Tight coupling between that language and its database fixes this issue so developers cache what is needed to work on data in memory while automating its impact on disk. Such caching is significantly improved with transactional technologies so developers can specify when they want their modifications saved in the database.

    This way, if they make a mistake they can simply go back in time before it was made and undo it. Conversely, after successfully completing their work, they can easily commit changes to the database. Like the other advantages, this benefit boosts developer productivity to match that of business requirements. It adds to the flexibility programmers need to innovate solutions for mission-critical objectives by writing or integrating applications.

    Object and Class Definitions

    The third boon of basing a smart programming language on a database is the capacity to dynamically change the definition of data objects. With conventional approaches, this task requires developers to stop writing or operating applications and manually write code to adjust the classification, which may involve either reading or writing the data in and out again. But with no separation between the database and its sufficiently smart programming language, such coupling can effectively keep track of classifications while automatically modifying them as needed.

    Developers can seamlessly add or delete features without manual coding, which also furthers their productivity. For example, if they add an attribute to an object, the next time they see it it’ll magically have the new feature. Alternatively, if they create a new feature of an old classification and decide to delete it, they won’t see it anymore. Best of all, these changes occur without rewriting the classification.

    Advanced Logic

    There is one more capability of very smart programming languages: They can combine these advantages with advanced logic languages such as Prolog. Prolog, or a subset of Prolog called Datalog, can instantly be used as a query language. In addition, it becomes very easy to create rule-based applications with Prolog because one can write declarative, logic-based rules with minimal coding. For example, they can specify a series of event-based rules detecting customer churn, money laundering or any important business outcome without being impeded by unnecessary database concerns to deliver time-to-value that’s unmatched without this method.

    Basing a programming language on a database improves developer efficiency and effectiveness by letting developers work on data in memory and persisting changes to storage, intelligently caching data and rolling back changes as required, and dynamically adjusting definitions.

  • Avoiding Budget Overruns During Cloud Migration

    Avoiding Budget Overruns During Cloud Migration

    Many organizations have already started their migrations to the cloud, or soon plan to. It makes sense: The cloud is an attractive option for multiple reasons, including its potential for cost savings, scalability and support of continuous application development.

    However, benefits aside, it can be very easy for enterprises to lose financial control of a cloud migration. The potential for costly mistakes when shifting to the cloud is significant, and organizations often end up getting hit with surprising bills that ultimately stop them in their tracks. In fact, in a 2020 survey, 94% of participants claim that it is a challenge for them to take full advantage of public cloud resources, with 40% citing the need to control cloud costs as their biggest obstacle.

    These financial mistakes are typically made due to the pay-per-use model of cloud services and the growing cloud skills gap. Too often, IT team members are forced to work in cloud environments they are not proficient in, and as new features are released every week, it’s almost impossible for these individuals to keep up. One recent study claimed that 90% of enterprises found that their cloud skills gap doubled over a three-year period.

    The Two Different Types of Cloud Migrations

    There are two types of cloud migrations that typically occur, a lift and shift, or a replatform/rearchitecture migration. The lift and shift method is fast and relatively easy, but it doesn’t enable organizations to truly optimize their cloud usage. The rearchitecture method is the most popular as it gives an organization the opportunity to fix any outstanding issues by harnessing the benefits of cloud-native technologies and services.

    However, once a rearchitecting migration begins, organizations often find it takes significantly more time, effort and resources to complete than they originally thought. This can become a real challenge, especially when facing critical deadlines or experiencing major events like the COVID-19 pandemic or the wild weather events experienced around the globe.

    Understanding and Assessing Performance

    For both types of cloud migrations, understanding and assessing performance and underlying architecture is an ongoing task, as cloud providers are always releasing new or improved services or lowering prices. To make sure they’re achieving the best performance and building a resilient architecture, application developers and IT teams must always be looking for ways to improve their applications. Within the cloud, this is typically focused on four key areas: Networking, computing, storage and application architecture.

    From a network perspective, understanding where data is being used and where data is being accessed is critical and can help organizations cut costs exponentially. For example, cloud providers often have costly data ingress and egress charges, so assessing how best to access and use data when necessary can help a company navigate those charges.

    Regarding computing, there are multiple resources that can help manage system load, make tweaks to computing infrastructure and find performance and cost benefits. For instance, leveraging the data and metrics from native cloud reporting tools can help organizations understand and predict demand, and taking advantage of services like auto-scaling enables them to scale up or down appropriately depending on the changes in application use and demand.

    On the storage front, cloud providers offer a number of ways to save money and boost performance. From the release of GP3 volumes to tiered long-term storage options based on the frequency of access, storage is an easy way to find cloud savings opportunities. Additionally, cloud providers will usually make recommendations and set up tiering policies to help move data automatically into colder and deeper storage.

    As cloud providers continue adding new services, looking at your application architecture is essential to innovation and cost savings—especially when including labor and effort—and truly achieving the promise of the cloud.

    Solving Common Costly Mistakes

    While there are numerous examples of common financial mistakes companies make when managing cloud instances, lack of visibility and understanding are the leading pain points for the developers who stand up and maintain their organization’s architecture. With the demand for cloud so high and with their attention directed elsewhere, developers and engineers often take the mentality of “If it isn’t broke, don’t fix it.” This hampers the company’s ability to assess and optimize their cloud operations. They don’t have time to consistently review their applications for ways to save money or even know how much their decisions cost.

    This ubiquitous issue can be solved by bringing greater visibility into the cloud migration and management process and aligning real-time finances directly with established corporate budgets. In addition, organizations struggling with the cloud services’ pay-per-use model should take the necessary steps—whether internally or with a third-party vendor—to ensure cloud resource and service use is being effectively monitored. This makes it easier for developers to understand where and how their actions affect the financial posture of the organization before their boss comes knocking on the door.

    If enterprises provide the resources necessary for IT teams to become more knowledgeable about how their applications are being used within the cloud and address the cloud skills gap in their workforce, they will be able to prevent cost overruns, experience cost savings and take back control of their budgets. This can take a variety of forms, but enterprises need to establish continual training programs to ensure team members are up-to-date on the latest cloud features and services, as well as leverage tools and automation to reduce manual intervention. Providing this critical visibility throughout a cloud migration can often mean the difference between a successful, on-budget migration and one that is stalled and costly.

  • 3 Pillars of Autonomous IT Ops

    3 Pillars of Autonomous IT Ops

    Imagine a future where human IT operators and intelligent systems work together in a virtual operations room; the systems do the heavy lifting while the humans supervise. This may seem like science fiction, but the truth is we are already on the way to that future with AIOps. While actual AI-augmented teams are still quite a way off, AI/ML tools are slowly but surely becoming more ubiquitous within IT Ops, NOC, DevOps and SRE teams and leading us into this not-too-distant future.

    In the last five years, the number of internet users grew by 50%. In the last two years alone, more data was generated than in the entirety of human history up to that point. Today, 5 quintillion (that’s 18 zeros) bytes of data are produced by smart devices daily. These massive volumes of data are having a huge impact on digital operations.

    Looking at the enterprise, we see similar trends: More tools generating more data and with it, greater complexity. This is due to many factors, including the transition to the cloud, M&A activity that introduces new tools and data into an existing organization and the adoption of new technologies and methodologies such as CI/CD, microservices and more. At some point, there is so much data and everything becomes so complex that IT Ops teams reach a breaking point where they can no longer manually manage the data being generated.

    And when that point is reached, outages and service disruption become an everyday reality, as IT teams are pulled away from their strategic, business-critical projects into what is essentially firefighting. 

    Implementing more observability and monitoring tools simply exacerbates the problem, and scaling teams isn’t cost-effective in the long term. The more forward-thinking solution is automation, and even more so, autonomy. 

    The Road to IT Ops Autonomy 

    Autonomy is the ability of IT Ops to operate with minimal to no human intervention. It relies on the automation of the incident management life cycle so that humans are needed only for supervision, in the same way that a single operator can supervise a modern and highly complex robotic car assembly line.

    How can we hope to reach IT Ops autonomy?  The following three proposed pillars can act as a guide: 

    Pillar One: Automate

    Pillar Two: Predict and prevent 

    Pillar Three: Democratize

     Pillar One: Automate

    We must automate as much as possible across every step in every aspect of incident management. It won’t happen overnight, but with time and with the help of AI/ML it is doable.

    Consider the autonomous driving analogy.

    Just a few years ago, cars were manual, with little to no automation. Then we moved through partial autonomy to where we are today–conditional autonomy–where the car can drive itself in ideal conditions. This is a level of automation that we are beginning to accept; some of us cautiously, others with more enthusiasm! Meanwhile, progress continues toward higher levels of autonomy, in which vehicles will be able to drive themselves in most conditions. One day, maybe, we’ll get to the holy grail of full autonomy, where the car will drive on its own–perhaps we won’t even need a steering wheel!

     

    Similarly, in IT Ops, automation is a process and it will take time to build not only the capabilities required but also the necessary trust in AI/ML technology. Such a journey typically begins with human-driven automation, as “if-then-else” types of automation are added manually. For example, if a certain alert appears, a ticket will automatically be assigned to a specific person. As our confidence grows, the next stage could be AI-driven suggestions, whereby AI applies what it has learned from previous human actions to suggest what to automate, but the human expert decides whether to accept the suggestion or not. 

    At some point, we will progress to allowing AI to partially automate some tasks—perhaps those that are less risky, such as collecting diagnostic information and adding it to the alert, so that the incident responder automatically receives both together. We may even automate some parts of our system; perhaps within a staging environment that isn’t production-critical and, as such, is a good place to validate that the automation is working as expected. Over time, as our AI and ML capabilities develop and our trust in them grows, we may eventually reach full AI-driven automation, with no human involvement at all—the AI will simply detect the fault and resolve it. 

    Pillar Two: Predict and Prevent 

    Typically, when assigned an incident, a responder will take note of the incident’s priority. But they will also rely on their common sense and past experience in deciding how to respond. So, for example, if a P3 incident had recently escalated to a P1 in 30 minutes and a similar P3 incident now appears again, our responder will probably prioritize it, even over other incidents, to avoid a similar escalation.

    While this is one of the advantages of human logic, the flip side is that the IT environments in which we work are vast and complex—no human can possibly remember every potentially relevant detail. What’s more, if a different responder dealt with the earlier incident in our example, the current responder would not have the same insight. What is needed is an element of enrichment that can input leading indicators into the platform for the next operator to see and act upon to prevent an incident.

    This is where AI can help. 

    By identifying patterns in the data haystack—linking different incidents to the same root cause, for example—AI not only generates insights that help resolve these incidents faster but can also predict a certain outcome. This is done by anticipating a future chain of events so that appropriate preventative action can be taken before the event occurs. For now, this preventative action may be manually taken by the operator, but at some point in the future automation may enable the problem to be identified and fixed by the AI with no human intervention before it results in an outage.

    Pillar Three: Democratize

    The last pillar that will help us achieve full autonomy in IT Ops is democratization of AI. AIOps should not just be accessible to advanced, mature organizations with an army of data scientists working behind the scenes. Just as anyone can accomplish almost anything online these days with just a few clicks, any organization should be able to adopt AIOps and enjoy its benefits! 

    Democratization is about making relevant information accessible to anyone who needs it, when they need it, and in a manner that is easily applicable and actionable. AI should be accessible through simple user interfaces, with no need for coding; the benefits of machine learning should be available to people that don’t (need to) understand machine learning models. 

    This democratization can be achieved by making AIOps platforms simple and accessible to all—from the admin to the user. For example, an AIOps platform might automatically suggest how admins should prioritize incidents for operators, to make triage easier and provide instructions that even a non-expert can follow. The more such features are built into AIOps platforms, the more democratized AIOps will become.

    Data volumes will continue to grow exponentially in the coming years, and the only way to eventually conquer the tyranny of overwhelming data will be to reach autonomy in IT Ops. 

    If you’re an AIOps vendor, keep the three pillars in mind so your customers can continue to progress towards autonomy.  If you’re an IT Ops leader or tooling architect, and IT Ops autonomy is part of your vision, make sure that your AIOps vendors’ platforms and tools have been architected with these pillars and their principles in mind. 

    How far are we from achieving autonomy? Only time will tell. But one thing is for sure—we will get there, and if the COVID-19 pandemic has taught us anything it’s that we may be seeing machine-augmented teams sooner rather than later. 

  • Google Donates Sqlcommenter to OpenTelemetry Project

    Google Donates Sqlcommenter to OpenTelemetry Project

    Google announced today that Sqlcommenter, an open source project that provides access to an object-relational mapping (ORM) auto-instrumentation library, is being merged with OpenTelemetry, an open source project for building agent software to instrument applications.

    Nimesh Bhagat, a product manager for Google, said Sqlcommenter is designed to make it simpler to correlate application and database telemetry. OpenTelemetry provides a mechanism for collecting logs, metrics and traces that is being advanced under the auspices of the Cloud Native Computing Foundation (CNCF).

    The challenge organizations using OpenTelemetry to collect that data will face is the need for tools to query that data in whatever platform it ultimately resides, noted Bhagat. OpenTelemetry previously lacked a common standard by which application tags and traces can be sent to databases and correlated with an application stack. Google envisions that IT teams will either employ Sqlcommenter to explore data or access via platforms such as Cloud SQL Insights or a third-party application performance management (APM) platform.

    Google made Sqlcommenter available as open source software earlier this year. It is designed to enable object–relational mappings (ORMs) to augment SQL statements before execution by, among other things, providing access to comments containing information about the application code that caused its execution.

    Sqlcommenter also allows OpenTelemetry trace context information to be propagated to the database, enabling correlation between application traces and database query plans that ultimately should make it easier to, for example, determine the root cause of a performance issue involving a microservices-based application making calls to a database. Sqlcommenter is primarily designed to be employed by full-stack developers that have some familiarity with how to use SQL to query data residing in an observability platform, noted Bhagat.

    Specifically, Sqlcommenter is designed to provide deeper insights into the performance of backend databases by simplifying the process of correlating slow queries with source code. It currently supports Python, Java, Node.js and Ruby languages along with ORMs such as Django, Sqlalchemy, Hibernate, Knex, Sequelize and Rails.

    As the number of applications that are instrumented using open source software continues to steadily increase, the amount of data that a DevOps team could potentially query also will steadily increase. Cloud storage platforms have made it easier to store that data at an affordable cost but the next challenge is clearly going to be finding a way to surface meaningful insights from all that data. Undoubtedly, achieving that goal will require everything from query tools to machine learning algorithms capable of identifying patterns at a level of scale that would not otherwise be possible.

    In the meantime, DevOps teams can look forward to being able to attain a level of observability that previously could only be achieved at a cost that tended to limit the number of applications organizations would instrument. At a time when the dependencies between applications increases with each passing day, that level of observability will soon prove indispensable.

  • Progress Expands Chef Automation Portfolio

    Progress Expands Chef Automation Portfolio

    In advance of its ChefCon 2021 conference, Progress today unveiled a series of enhancements to its Progress Chef automation platform that includes managed services and a software-as-a-service (SaaS) edition of its offerings as well as tighter integration with cloud services and other third-party tools.

    Sudhir Reddy, vice president of engineering for the Chef business at Progress, said the SaaS platform, now available in beta, combined with the managed services provided by Progress will make the automation platform much more accessible to a wider range of organizations.

    Progress is now also extending the reach of its Chef Compliance Automation for Cloud Resources offering across Amazon Web Services (AWS), Microsoft Azure and Google Cloud Platform (GCP) to include Chef premium content that aligns with best practices for each cloud environment in addition to providing integration with Azure Policy, a suite of tools from Microsoft that compares resources to a set of business rules that define how cloud infrastructure should be employed.

    Chef

    Finally, Progress has enhanced Chef Enterprise Automation Stack to provide tighter integration between compliance and infrastructure automation along with adding infrastructure operations dashboard, support for Windows patch management, integration with third-party secrets management tools, and improvements to Chef Workstation and Chef Test Kitchen to simplify testing within application development processes.

    With the release of a SaaS platform and the availability of managed services, Chef is joining a long list of IT vendors that are removing responsibility for deploying and securing their platforms from internal IT operations teams. It’s clear more organizations now want to spend more time on building and deploying software than managing the platforms that automate those tasks on their behalf, said Reddy.

    It’s not clear how many IT teams that have already deployed a platform might shift over to a SaaS platform or rely on an IT vendor to manage that platform on their behalf. However, in the wake of the COVID-19 pandemic, many IT teams continue to work from home to help combat the spread of the virus. Many of those organizations have either shifted platforms to the cloud themselves, adopted SaaS platforms or outsourced the management of IT platforms altogether. There are also many small organizations that simply don’t have the internal expertise and resources required to stand up an IT automation platform on their own.

    That approach also makes it simpler to converge security, compliance and IT operations management within the context of a larger DevSecOps workflow, noted Reddy.

    The great paradox of IT automation has always been the level of expertise required to achieve it. Organizations that lack the expertise required to create automated playbooks, for example, don’t derive much value from an IT automation platform. It’s only been larger companies that have mostly derived the most benefit from investments in IT automation. However, as IT automation becomes more accessible via SaaS platforms and managed services, the level of expertise required to benefit from it naturally drops. In fact, with the advent of artificial intelligence (AI) for IT operations (AIOps), many manual IT tasks are increasingly being automated.

    It’s too early to say whether the IT industry as a whole is on the cusp of the next wave of automation, but the raw ingredients that make up the primordial soup needed to achieve that goal is, at this point, only awaiting a catalyst.

  • CircleCI Bets on Webhooks to Streamline DevOps Integrations

    CircleCI Bets on Webhooks to Streamline DevOps Integrations

    CircleCI today announced CircleCI webhooks, a feature that enables software engineering teams to build integrations that react to CircleCI job and workflow status notifications in near-real-time.

    Tom Trahan, vice president of business development at CircleCI, said this latest iteration of the company’s implementation of webhooks, which send messages any time an event occurs, will become the foundation upon which CircleCI will expand the ecosystem surrounding its continuous integration/continuous delivery (CI/CD) platform.

    For example, the CircleCI platform can now employ webhooks to integrate with a CI visibility tool from Datadog that can now be used to surface CicleCI metrics. Sumo Logic is similarly employing webhooks to enable its Continuous Intelligence observability platform to track events on the CircleCI platform. Those capabilities are now available via the software-as-a-service (SaaS) edition of the CircleCI platform.

    Trahan said the primary goal is to reduce the amount of friction within DevOps workflows that span multiple platforms by increasing the level of orchestration DevOps teams can take advantage of without having to create and maintain integrations using low-level application programming interfaces (APIs).

    Webhooks make it possible to streamline developer workflows while also increasing overall engineering velocity, he added. In addition, it should also become simpler for DevOps teams to triangulate events in a way that results in incidents being resolved faster, noted Trahan.

    CircleCI is relying on webhooks to strengthen its case for a best-of-breed approach to DevOps that enables organizations to mix and match tools as they best see fit rather than getting locked into a single integrated DevOps platform. The best-of-breed approach often makes it simpler to take advantage of new tools as they become available. The downside is the level of integration that DevOps teams need to maintain after they integrate all the tools that make up a DevOps workflow.

    It’s not clear what impact relying on webhooks to share messages in real-time about events as they occur across multiple platforms will have on integrations. However, any effort to streamline DevOps environments without requiring DevOps teams to abandon one tool for another is likely to be well-received. After all, every minute spent managing a DevOps platform means much less time that can be applied to building and deploying applications.

    It will be up to each DevOps team to decide to what degree a best-of-breed approach to DevOps suits their organization. As more organizations realize how dependent they have become on software to, for example, drive digital business transformation initiatives, many IT leaders are making a case to standardize DevOps platforms across their organization. Achieving that goal, however, may be difficult given all the various departments within an enterprise that have already adopted a wide range of tools and platforms.

    In the meantime, the focus on developer and engineering productivity will only increase as organizations look for ways to further accelerate application development and deployment at a time when software updates can now have a profound impact on profits and revenue. In fact, a delay to one software project can have a cascading impact on a wide range of initiatives that can’t go forward because of their dependency on a delayed project. Given the impact all those delays might have on revenue, the tolerance organizations have for delays to software development projects is understandably lessening by the day.

  • CNCF Advances OpenTelemetry Initiative

    CNCF Advances OpenTelemetry Initiative

    The Technical Oversight Committee (TOC) of the Cloud Native Computing Foundation (CNCF) has voted to accept OpenTelemetry as an incubating project as part of an ongoing effort to simplify instrumentation of software using open source agent software.

    OpenTelemetry is a collection of tools, application programming interfaces (APIs) and software development kits (SDKs) that can be employed to collect metrics, logs and traces that make it simpler to determine the root cause of an application issue.

    At its core, OpenTelemetry is based on an OpenTelemetry Protocol (OTLP) specification that describes the encoding, transport and delivery mechanism of telemetry data gathered from sources, intermediate nodes and backend platforms. An OpenTelemetry Collector provides a vendor-agnostic implementation for receiving, processing and exporting telemetry data. APIs and SDKs that make it possible to instrument applications are available for 11 different programming languages.

    Previously a sandbox-level project within the CNCF, the next step would be for OpenTelemetry to become a graduated level project alongside other open source offerings such as Kubernetes and Prometheus, a platform used to monitor IT environments.

    The OpenTelemetry project was created via the merger of the OpenCensus and OpenTracing projects in May 2019. Since then more than 500 developers from 220 companies, including Amazon, Dynatrace, Google, Honeycomb, Lightstep, Microsoft, Splunk and Uber have contributed to the project.

    Most providers of observability platforms have at least signaled their intent to support OpenTelemetry as an alternative to proprietary agent software they had previously created to collect metrics, logs and traces.

    Ben Sigelman, one of the co-creators of both OpenTracing and OpenTelemetry and CEO of Lightstep, said different elements of the OpenTelemetry project are at varying stages of maturity. As such, support for the elements of the project are being added to platforms depending on the degree to which a vendor has confidence in, for example, the ability of OpenTelemetry agent software to collect logs.

    Liz Fong-Jones, principal developer advocate for Honeycomb.io, a provider of an observability platform, said that, in time, most observability platforms will support all the elements of the OpenTelemetry initiative to reduce their own engineering and support costs. In time, those savings will be passed on to organizations that will be able to instrument applications more affordably, she noted.

    It may take a while longer for OpenTelemetry to be widely employed, but the faster organizations transition to microservices-based applications the more pressing the need for open source telemetry software becomes. The dependencies that exist between all the microservices that make up multiple applications will be unmanageable without some form of telemetry. The challenge organizations face is finding an affordable way to instrument all those microservices.

    Of course, organizations have been instrumenting applications for decades. However, the time, effort and cost associated with instrumentation have tended to limit its use to only an organization’s most mission-critical applications. The issue now is that not only do microservices-based applications require instrumentation to manage them but in the age of digital transformation the percentage of applications deemed mission-critical is also steadily rising.

  • How to Achieve AWS Competency Accreditation

    How to Achieve AWS Competency Accreditation

    For businesses in the early stage of their ventures, proving that they have the expertise and technical know-how when it comes to cloud technology can be difficult. An acceleration toward digitization due to COVID-19 and an increasingly crowded cloud market, dominated by the big players, means that simply shouting about your skills in cloud engineering and DevOps isn’t enough to get noticed.

    But what if you can work alongside these giants? Joining forces with expert brands and entering into endorsement partnerships can propel your business forward by supplementing it with seasoned expertise. Accreditation, if you can achieve it, can give you a stamp of approval and reassure prospects that you have the relevant knowledge to support their digital transformation efforts. Collaborating with industry leaders on skills accreditation programs is a great way to demonstrate your capabilities and provides clear goalposts to work toward when it comes to building and learning.

    So, how do businesses achieve this status and what are the long-term benefits of the process?

    Become an AWS Partner

    Achieving accreditation as an AWS DevOps Competency Partner takes a lot of hard work and investment into building the technical skills required to reach such a high standard–logging and evidencing every action your team takes is no mean feat. From beginning to end, the course takes around six months. The process begins with researching and selecting which specific competency your team is interested in achieving. It’s important to understand which competency your business is most suited to, so doing your homework is key. From there, there’s a lot of work to evidence, certificates to achieve and reviews to undergo. Becoming an accredited AWS partner brings a multitude of benefits to your brand, so it is well worth the investment.

    Top Tips for the Accreditation Process:

    Create detailed accounts of your work: To become a certified DevOps partner, businesses need to evidence four case study examples; two public and two private. As such, it can be useful to bring in a tech writer to consolidated your notes and observations throughout the process. We began, for example, with a technical exploration from the team to understand the best case studies to put forward; our work had to be clearly evidenced with strong examples and the documentation to back it up which took a considerable amount of time to pull together. Maintaining clear and detailed accounts of previous work can prevent any delays. This information is firstly analyzed by AWS to check that it met the specific requirements before being passed on to be examined by a team of external auditors, so every detail counts when it comes to showcasing your work.

    Prepare the marketing team: As part of the process, businesses are required to have certain marketing collateral in place across their website to aid with lead generation. For the DevOps competency certification, we needed to evidence three technical blogs–businesses applying for the certification need to ensure that the content best reflects their expertise in the given competency. Once the accreditation is finalized, the partnership should be displayed on the site. Even if a business is tech-agnostic, demonstrating your skills in a particular type of cloud engineering is highly valuable to your brand and AWS is a respected platform to use.

    Advance technical training in the workplace: Having members of the team qualified in cloud engineering is another requirement for accreditation. For DevOps, we needed eight people on the team to have achieved their associate certifications and four to have achieved professional certifications. To fulfill this requirement and ensure we’re on top of our game when it comes to cloud training, we’ve invested in third-party tools, provided access to LinkedIn Learning programs and we give our people additional time off from client-facing work to prepare for their exams. Building and learning in tandem is the fabric of our culture, and a similar approach would benefit any company looking to achieve certification status.

    The Benefits of AWS Accreditation

    Being able to showcase a close association with a major player such as AWS opens up a number of doors for smaller businesses. Operating within this kind of business ecosystem provides the opportunity to demonstrate a business’ DevOps and technical skills to an enormous global audience which can drive new business conversations and drive quality leads. Confirmation that your business is a trusted partner for a major cloud organization will ensure that prospects have confidence in your capabilities and you’re taken seriously as a technical player in the market. Building confidence within your teams and supporting them in their learning is an investment that will pay huge dividends.

    Working alongside AWS as an accredited DevOps partner gives us a springboard to talk about our work and engage with other players in the market. The accreditation is a fantastic talking point and definitely gets you noticed in the cloud community–plus, you will be taken more seriously as a partner.

    Investment in Cloud Tech is Key for the Future

    As businesses everywhere begin to operate in an increasingly digitized environment, technical skills and cloud expertise are a necessary investment for any business. Working toward coveted industry accreditations while prioritizing long-term skills training is imperative. It’s a competitive, buoyant market for cloud engineers and DevOps at the moment; opportunities are at their fingertips and if they aren’t feeling fulfilled by their current work, they’re likely to look elsewhere. Investment in training and nurturing growth in this way breeds employee engagement and loyalty. For a company to thrive technically, it needs to host a wide range of diverse talent and be able to retain it. Learning opportunities should always be the key focus.

    The shortage of this talent means it’s critical to advance the business’s cloud expertise by pursuing recognition from major global players like AWS. Providing advanced training and certification opportunities for your people will ensure long-term talent retention while making the business a leader in cloud engineering. That’s pivotal for long-term success.

  • Popular iOS Apps Request Excessive User Data Permissions

    Popular iOS Apps Request Excessive User Data Permissions

    Most apps require some sort of user data to function properly. Google Maps needs location data to offer routing services. Twitter needs photo library access to upload a photo, and so on. Many users are quick to allow access to personal information without hesitation. However, in recent years, it’s become apparent that many apps are collecting (and leaking) an excessive amount of personal data, resulting in serious privacy issues. This has led Apple to give user privacy a heightened focus in recent iOS updates and marketing campaigns.

    Often, user data is being collected in the background for legitimate use cases, such as tailoring personalized user experiences and refining app performance. But, many apps are also monetizing user data by reselling it to third-party advertising data brokers without explicit user consent. More seriously, recent studies show that commercially available “anonymous” data can be de-anonymized. If location data is exposed, it could put at-risk people in actual danger. These are nuanced privacy violations that U.S. regulators have yet to respond to and address.

    To shed light on the data that apps are collecting, a recent study conducted by Jamf analyzed the iOS permissions requested by nearly 100,000 popular apps on the App Store. The study found the four most-requested mobile app permissions to be photo roll, camera, location data and microphone. The findings also revealed that popular apps are consistently requesting access to data that does not serve the application’s core purpose.

    Users should consciously be aware of the data they allow apps to access. Yet, the onus is also on business leaders and app developers to limit unnecessary data collection and reduce the potential for data overexposure. In the wrong hands, access to device hardware, such as a microphone or camera, could be used to steal sensitive material. And, if leaked, personally identifiable information (PII) exploits could be disastrous to the business.

    Below, I’ll review the key takeaways from the Jamf study and consider if the constant barrage of permissions requests is exacerbating privacy issues during the application development process.

    Four Most-Requested App Permissions

    The Jamf research found the top four types of app permissions were photo library access, camera access, location data and microphone access. In addition, other popular areas included calendar information, contacts and Bluetooth. Less-requested app permissions included voice processing, health data and local network access. The study arranged apps by their App Store categories, such as business, education, entertainment, news, shopping, travel and other areas.

    user data permissions

    1. Photos

    By far, the most requested data access type is photos. At least half of apps across all categories request access to user photo libraries. Naturally, this request is most common among photo and video apps such as YouTube, FaceApp or Splice; 96% of such apps request access to photos. This is followed by 87% of shopping apps (Amazon, Shop, eBay, etc.), and 84% of social networking apps (Facebook, Instagram, Twitter, etc.).

    At issue here is that the level of intended photo library access can be highly variable. For example, granting access to an entire camera roll along with decades of old personal photos just to upload one picture grants a rather invasive privilege. This has led Apple to enhance privacy controls—users can now choose to allow access to select images only or their entire roll.

    With these new constraints in mind, app developers should take steps to respect user privacy. Be aware of potential photo limitations and design apps to only collect necessary photos for the task at hand.

    2. Camera

    The next-most requested permission is for camera access. This is most common among photo and video apps, among which 90% request camera access. This is also common among shopping and social networking apps (83% each) and business apps (75%) like Zoom, Slack or WebEx. Similar to findings about photo library access, enabling camera access is ubiquitous. However, it is also one of the most dangerous permissions to grant indefinitely. If a bad actor obtained account access, this vulnerability could enable them to spy on users.

    Again, developers should consider introducing camera functions only when it’s essential for the task at hand. Obviously, it’s unethical to invoke camera access when users are unaware. Even camera-related bugs can have extreme repercussions. For example, a 2020 lawsuit against Instagram argued the app was spying on users. Due to its threat potential, high-security facilities have disabled camera hardware entirely.

    3. Location

    Location tracking is another permission that can be very useful but is, at times, overprescribed in modern app development. Location data includes cellular, Wi-Fi, GPS and Bluetooth data used to determine a user’s approximate location. The categories that collect location data most often include shopping and food and drink, both at 81%. Social networking and photo and video app categories follow at 72% and 68%, respectively.

    Nowadays, iOS privacy settings offer a check against tracking location data in the background. iOS users can now choose between: ‘Allow While Using App’, ‘Allow Once’ or ‘Don’t Allow’.

    user data

    When an app tracks data in the background, a blue indicator now displays in the status bar to increase user awareness. iOS also reminds users when apps use significant amounts of location data and offers ways to turn location services completely off. Android has followed suit with an ‘Allow only when the app is in use’ function, too.

    user data

    Location tracking feels a bit creepy. In the wrong hands, it could easily be leveraged for nefarious purposes. Out of all the permission-granting options, location data presents much more of a physical safety concern for the end user. From the user’s standpoint, they should only enable access to location tracking while using an app. From a developer’s standpoint, they should be extremely careful when collecting and storing such PII to ensure regulations aren’t broken.

    4. Microphone

    Lastly, access to the user’s microphone is the fourth most commonly requested permission. The survey found 69% of social networking apps, 64% of photo and video apps and 41% of both business and productivity apps (Asana, Google Calendar, TimeTree, etc.) all want to be handed the mic.

    Here, potential privacy concerns are similar to those involving camera access. Bad actors could potentially activate the mic to listen to private conversations, record ambient noise and sell findings to advertising brokers. To increase user awareness, iOS 14 now requires an orange dot to display, indicating when the mic is in use.

    Other User Privacy Concerns

    User behavior tracking and app-to-app information exchange is another ongoing area of privacy concern. Companies often resell such information to data brokers, which is then used to refine highly targeted ad campaigns on third-party sites. This is how your search on Google results in product advertisements popping up in your Instagram feed. Apple is just starting to counter this practice with their App Tracking Transparency feature. In iOS 14.5, app devs must ask whether they can track your activity.

    Recent analysis has confirmed that commercially available consumer data, though labeled as “anonymized,” can successfully be used to piece together an accurate picture of a user by corroborating behavior on other apps. For example, users were able to track and pinpoint behavior that resulted in the outing of a catholic priest on Grindr. Smartphone app data could similarly be used by predators to spy on celebrities or politicians, posing a tangible security threat.

    Another cause for concern is metadata associated with photos. People often enable GPS-embedding into images to help organize their photo memories. If this is inexorably linked to photos, it essentially permits location data sharing to parties with photo library access who might not otherwise have location tracking permissions.

    From Big Data to Big Problem

    Perhaps the most curious finding of this report is that all apps are requesting permissions that seem irrelevant to their core value proposition. For example, why do 87% of shopping apps need access to a user’s entire photo library? Do 46% of food and drink apps really need access to a user’s personal contacts to place a food delivery order? Do navigation apps really need your personal calendar information to calculate a route downtown?

    The answer is not a pretty one. There has been a business imperative, for years, to track and collect large amounts of user data. There is undoubtedly an argument to be made for enhancing the user experience with innovative features and the role of big data in AI. However, this ongoing practice continues to creep into ethical gray areas. The fact is, the more personal data an app collects, the more potential privacy issues and security threat vectors are created in the process.

    Requesting permissions to on-device hardware and data collection is not always necessary. Thus, to stay safe and limit data overexposure, don’t overprescribe data collection. If it’s not fundamental to your app’s core function, don’t collect it, and don’t store it. Think twice before requiring many device permissions, and regularly audit app permissions requests to keep them to a minimum. As iOS and Android introduce new platform guidelines to elevate user privacy, developers should be prepared for users to grant fewer and fewer data collecting permissions. This is their right.

    It would be great if tech companies invested in honest, ethical business models that didn’t rely on harvesting and reselling user data to shady advertisers. And for those that mine user data but say it’s anonymous, I would point to reports that show anonymous data is nothing but pure myth.

  • GitHub Updates CLI to Make Creating Extensions Easier

    GitHub Updates CLI to Make Creating Extensions Easier

    GitHub this week made available an update to its command-line interface (CLI) that enables DevOps teams to add extensions to workflows.

    Billy Griffin, director of engineering for GitHub, said the company’s initial focus was providing DevOps teams with a set of workflows that could be invoked easily via a CLI. Now GitHub is encouraging developers to extend those workflows using version 2.0 of its CLI that hopefully will be shared with the larger community, he added.

    For example, extensions have already been created that order branches by how recently they were created and then display all associated pull requests.

    At the other end of the spectrum, developers are creating simple screensaver extensions for their own amusement.

    The level of interest in extensions varies widely across DevOps teams. Many of them simply want to employ an opinionated workflow that has been created on their behalf. Other DevOps teams want to be able to make modest modifications. Others prefer to customize workflows as much as possible to fit their specific purposes. Version 2.0 of the CLI provided by GitHub enables each DevOps team to decide how much to modify their workflows. However, as a DevOps team gains more experience, increasingly they tend to want to customize workflows.

    The number of workflows being created around the GitHub repository continues to increase steadily as more organizations embrace best GitOps practices. GitHub is the most widely employed repository for managing software artifacts, and the level of sophistication of workflows built using either scripts or GitHub actions is increasing steadily. Much of that effort is focused on streamlining processes that otherwise would get in the way of enabling developers to spend more time writing code.

    Of course, the pressure on developers to write more code faster has never been greater. As organizations embrace digital business transformation, they are discovering how dependent they are on software to create customer experiences that drive revenue. The challenge comes at a time when many developers are working from home to help combat the spread of the COVID-19 pandemic and many existing workflows need to be re-engineered. At the same time, development teams are becoming more distributed as organizations become more comfortable with hiring developers anywhere in the world.

    It’s not clear to what degree the changing application development landscape might drive organizations to re-evaluate what platforms they are employing to drive DevOps processes. Many on-premises development platforms have been replaced by cloud-based alternatives in the past year. However, because necessity is the mother of invention, the workflows relied on prior to the pandemic are not only now forever changed, but they will also likely be altered again as development teams work more intermittently in-office and from home once the pandemic subsides.

    Regardless of how those workflows evolve, the single-valued DevOps attribute is going to be flexibility. The challenge now is determining which platform does the most to enable that flexibility at a time when uncertainty continues to prevail.