staging-devopsy.kinsta.cloud

Tag: Mesosphere

  • Mesos Usage Survey: Big Clusters, Big Workloads

    Mesos Usage Survey: Big Clusters, Big Workloads

    Mesosphere, purveyors of DC/OS, recently released a survey of Mesos users focused on Mesos deployment information.

    There were some interesting results, and a few things I wish they’d dug a little deeper on. If you’ve ever given a survey, then you know that the results inevitably spawn more questions. No doubt Mesosphere has the same new questions I do.

    The first important tidbit I would like to highlight is a cautionary tale. I’ve been around clusters—big data, cloud and, more recently, Mesos-style clustering—for years, and there is one truth this survey shows that is far more global than just Mesos. Clusters always end up bigger than you’d planned. The survey shows that of those using Mesos for more than six months, 25 percent have more than 100 servers backing the cluster.

    mesosservercountsWhen planning a cluster, always assume it will be bigger than you plan and grow faster than you think. Organizations that need this type of technology need it to perform and find themselves using it more than they’d planned, so the clusters always grow. I’ve seen orgs do a 20-server cluster that within six months numbered well over a hundred servers and headed to 200. I’ve heard of faster/larger unexpected growth rates, but not seen it or talked to those responsible. Plan for bigger than you think you need—worst case, you’ll have overplanned, which is easier to handle than being caught with too small a cluster.

    This is one of the areas I think it would be interesting to have a follow-on question delving into how big survey respondents had planned their clusters to be versus how big they were at this point in their deployments.

    The next interesting takeaway from the survey was the usage patterns. While internal/external/hybrid cloud is not a surprising distribution, the fact that so many organizations have moved monolithic workloads to Mesos was a bit of a surprise. The sample size of the survey was relatively small, but still, that’s a lot of “We have Mesos, let’s start running it there” going on.

    mesoswhereappsrun

    I think I’ll work with Mesosphere to talk to one of the organizations running monolithic apps on Mesos and see if we can’t get an interesting interview.

    For the health of open-source projects, I like to see what is happening with those involved—is the audience growing? Are the number of developers growing? Supplemental survey data says yes.

    mesoscontributors

    I double-checked this information, since it should be available publicly, and although the move from GitHub to Apache Git made it more difficult to track down, this Apache post from July does verify the graph above, and offers even more interesting data.

    The thing that still intrigues me about Mesos is that instead of allowing users to pool servers to create virtuals, it pools servers to make a massive parallel system. But running containers on it (as this data indicates most are for resource isolation) carves it up into virtuals, of sorts. Very esoteric.

    I won’t bore you by going over what I think of each and every question. Follow the survey link at the top of the article and you can get the survey response summaries to support your work.

    — Don Macvittie

  • Mesos and Its Impact on the Data Center

    Mesos and Its Impact on the Data Center

    One of the things that many of us have been observing is that in the midst of a sea of other changes, data center architectures also are changing. By a lot.

    RAID array sales are down, even though unstructured data is still going up; networking is getting an application programming interface (API) makeover in some places and pure software-defined networking (SDN) in others; and the topic of this post—the concept of “server” in the data center—is becoming somewhat fluid.

    Until recently, “server” meant either physical hardware or a virtual machine (VM) on VMware. There were other definitions, but no one cared about them. Then came OpenStack, and while the same as VMware in terms of meeting enterprise needs, it brought with it more work and more options than VMware, and started people thinking more about servers and what that word means. Now we have Apache Mesos, and its various incarnations are gaining traction in the data center. This doesn’t even touch on specialized clusters such as those Hadoop requires.

    If you’re busting your tail every day keeping the systems humming and serving the needs of the organization, the differences might be elusive. I’ll take this blog to talk about the differences, and moving forward will focus more on Mesos and its variants because it is an area of DevOps that I think it underserved in the publishing arena.

    While VMWare and OpenStack let you pool servers and then carve them up into virtual servers, Mesos decided to eliminate that “carve up” portion and treat the pool of servers as one big system that can service applications. That is a simplification, but when it comes down to it, think of Mesos as Linux with the resources of many physical systems instead of just one.

    Because making a pool of servers into a single logical server means the server can do a lot more than a normal system, there are a lot of add-ons, bells and whistles. So for process isolation, containers are used to offer a bit of protection from rogues, for example. And since the Linux scheduler was not intended to handle cross-system processes, scheduling is handled by a system that, while not more complex, does require you to know a bit more about, so it seems more complex than just running it and letting the system take care of the rest. When you consider the volume of jobs a cluster of systems can handle compared to a single system, and the work of distributing jobs/apps/programs/whatever-you-call-them across the supporting hardware, it just makes sense that a scheduling and job submission management subsystem is required.

    Because the Mesos scheduler (and more) has an API, there are some definitely interesting DevOps possibilities here: Run all the processes you need and have the ability to hook the scheduler and reporting into your DevOps tool sets? Definitely interesting possibilities that we’ll explore at some point in the future.

    Why would you go with a clustered operating system instead of a cloud platform? That really is a valid question, and one the market is still working out. Early adopters like the fact that an entire layer of abstraction is removed as unnecessary. In VMware or cloud, you have physical hardware -> pool of resources -> virtual Servers -> application.

    Both physical and virtual servers are running copies of an operating system (OS). That’s a lot of overhead.

    With Mesos, it looks much the same, except that instead of VMs, the app is running on the pool of resources (or more correctly, the app is running in a lightweight container on the pool of resources). So there is less to maintain, less CPU/disk/memory overhead, and less install/configuration time.

    Of course, any highly complex system has issues of its own. I am still learning what the real issues that crop up and enterprises have to deal with, so I will promise to report back after I have a handle based upon reality, but they are there—I just need to get a solid enough understanding that I’m not misleading anyone.

    From a DevOps perspective, the ability to assume the configuration of the server the cluster presents itself as, no matter the app being submitted to run, and the management of an entire data center’s worth of applications on a common platform are strengths I intend to dig into.

    If you’ve evaluated or used Mesos (or DC/OS or Mantl or SuperGiant or any of the other tools based on it), don’t hesitate to reach out to me. My experiences are one thing; others might have different issues/concerns/loves.

    We do indeed live in interesting times, with changes in IT coming at a pretty astounding rate. Fun times. I’m actually pretty excited to learn how well Mesos serves the need and enables DevOps. It’s not a simple topic to discuss, there are a lot of moving parts, but we’ll get it covered, and help you figure out if it’s the best solution for your organization.

    — Don Macvittie

  • Xentaurs Announces a Strategic Partnership with Mesosphere to Build Next Generation Big Data Analytics and DevOps Platforms

    Strategic partnership formed to accelerate adoption of container based architectures

    SANTA ANA, CA (PRWEB) April 28, 2016

    Xentaurs, a next generation systems integrator and solutions provider announces a strategic partnership with Mesosphere, creators of the world’s first data center operating system. The partnership combines Xentaurs’ enterprise experience designing and implementing digital architectures with Mesosphere’s leading platform for running microservices, data-driven applications and other modern IT workloads.

    Mesosphere is at the core of the digital platforms we recommend to our customers. DC/OS is the leading distributed operating system out there and provides the modern architecture framework that enterprises need, said Juan Guevara, Chief Strategist for Xentaurs.

    “Enterprise IT understand that to stay relevant and competitive in this new digital world they need to disrupt themselves and adopt new technology platforms that remove the traditional silos and monolithic systems. These digital platforms need to be designed with a common foundation capable of supporting many type of workloads but most importantly, need to be open and enjoy a rich and diverse ecosystem to allow a smooth transition from the traditional systems to new modern architectures,” commented Guevara.

    “Xentaurs is at the forefront of helping businesses recognize unprecedented ROI with modern architectures,” said Mark Musselman, Head of Business Development for Mesosphere. “Our open source DC/OS platform is designed specifically for such environments, and we are honored to partner with a team that shares our dedication of empowering organizations to keep pace with the rapidly changing technology landscape.”

    Together, Mesosphere and Xentaurs will help enterprises looking for big data analytics, devops, and other agile business transformation platforms, with their world-class consulting methodology and professional services teams.

    About Mesosphere

    Mesosphere (Twitter: @Mesosphere) is leading the enterprise transformation toward distributed computing and modern applications with its Datacenter Operating System, and is the principal founding member of the DC/OS open source project. DC/OS is a production-proven datacenter-scale platform for container operations and simple installation of complex distributed systems — including HDFS, Apache Spark, Apache Kafka, Apache Cassandra and more. Mesosphere was founded in 2013 by the architects of hyperscale infrastructures at Airbnb and Twitter, along with the co-creator of Apache Mesos. Backed by A Capital, Andreessen Horowitz, Data Collective, Fuel Capital, Hewlett Packard Enterprise, Khosla Ventures, Kleiner Perkins Caufield & Byers, Triangle Peak Partners and Microsoft, Mesosphere is headquartered in San Francisco with additional offices in New York and Hamburg, Germany.

    About Xentaurs

    Xentaurs (Twitter: @Xentaurs) is a next generation systems integrator and solutions provider specializing on making digital business transformation a reality. Just like the half-horse half-human creature of mythology its name is derived from, it represents the seamless interaction between human and technology at the center of this new digital world. The company’s biggest value is embedded at the core of their engineering-centric culture. From its internal organization structure to their customer consulting methodology, Xentaurs practices a modern and agile operating model to facilitate the new needs of a digital world. A complete culture based on human merit and technology excellence. The company’s expertise and offerings covers advanced analytics, big data, devops, and digital platforms.