{"id":23926,"date":"2018-06-07T14:06:38","date_gmt":"2018-06-07T14:06:38","guid":{"rendered":"http:\/\/www.sickgaming.net\/blog\/2018\/06\/07\/mesos-and-kubernetes-its-not-a-competition\/"},"modified":"2018-06-07T14:06:38","modified_gmt":"2018-06-07T14:06:38","slug":"mesos-and-kubernetes-its-not-a-competition","status":"publish","type":"post","link":"https:\/\/sickgaming.net\/blog\/2018\/06\/07\/mesos-and-kubernetes-its-not-a-competition\/","title":{"rendered":"Mesos and Kubernetes: It’s Not a Competition"},"content":{"rendered":"
<\/div>\n

The roots of Mesos can be traced back to 2009 when Ben Hindman was a PhD student at the University of California, Berkeley working on parallel programming. They were doing massive parallel computations on 128-core chips, trying to solve multiple problems such as making software and libraries run more efficiently on those chips. He started talking with fellow students so see if they could borrow ideas from parallel processing and multiple threads and apply them to cluster management.<\/p>\n

\u201cInitially, our focus was on Big Data,\u201d said Hindman. Back then, Big Data was really hot and Hadoop was one of the hottest technologies. \u00a0\u201cWe recognized that the way people were running things like Hadoop on clusters was similar to the way that people were running multiple threaded applications and parallel applications,\u201d said Hindman. <\/p>\n

However, it was not very efficient, so they started thinking how it could be done better through cluster management and resource management. \u201cWe looked at many different technologies at that time,\u201d Hindman recalled. <\/p>\n

Hindman and his colleagues, however, decided to adopt a novel approach. \u201cWe decided to create a lower level of abstraction for resource management, and run other services on top to that to do scheduling and other things,\u201d said Hindman, \u201cThat\u2019s essentially the essence of Mesos — to separate out the resource management part from the scheduling part.\u201d<\/p>\n

It worked, and Mesos has been going strong ever since.<\/p>\n

The project goes to Apache<\/h3>\n

The project was founded in 2009. In 2010 the team decided to donate the project to the Apache Software Foundation (ASF). It was incubated at Apache and in 2013, it became a Top-Level Project (TLP).<\/p>\n

There were many reasons why the Mesos community chose Apache Software Foundation, such as the permissiveness of Apache licensing, and the fact that they already had a vibrant community of other such projects. \u00a0<\/p>\n

It was also about influence. A lot of people working on Mesos were also involved with Apache, and many people were working on projects like Hadoop. At the same time, many folks from the Mesos community were working on other Big Data projects like Spark. This cross-pollination led all three projects — Hadoop, Mesos, and Spark — to become ASF projects.<\/p>\n

It was also about commerce. Many companies were interested in Mesos, and the developers wanted it to be maintained by a neutral body instead of being a privately owned project.<\/p>\n

Who is using Mesos?<\/h3>\n

A better question would be, who isn\u2019t? Everyone from Apple to Netflix is using Mesos. However, Mesos had its share of challenges that any technology faces in its early days. \u201cInitially, I had to convince people that there was this new technology called \u2018containers\u2019 that could be interesting as there is no need to use virtual machines,\u201d said Hindman. <\/p>\n

The industry has changed a great deal since then, and now every conversation around infrastructure starts with \u2018containers\u2019 — thanks to the work done by Docker. Today convincing is not needed, but even in the early days of Mesos, companies like Apple, Netflix, and PayPal saw the potential. They knew they could take advantage of containerization technologies in lieu of virtual machines. \u201cThese companies understood the value of containers before it became a phenomenon,\u201d said Hindman.<\/p>\n

These companies saw that they could have a bunch of containers, instead of virtual machines. All they needed was something to manage and run these containers, and they embraced Mesos. Some of the early users of Mesos included Apple, Netflix, PayPal, Yelp, OpenTable, and Groupon.<\/p>\n

\u201cMost of these organizations are using Mesos for just running arbitrary services,\u201d said Hindman, \u201cBut there are many that are using it for doing interesting things with data processing, streaming data, analytics workloads and applications.\u201d<\/p>\n

One of the reasons these companies adopted Mesos was the clear separation between the resource management layers. Mesos offers the flexibility that companies need when dealing with containerization.<\/p>\n

\u201cOne of the things we tried to do with Mesos was to create a layering so that people could take advantage of our layer, but also build whatever they wanted to on top,\u201d said Hindman. \u201cI think that’s worked really well for the big organizations like Netflix and Apple.\u201d<\/p>\n

However, not every company is a tech company; not every company has or should have this expertise. To help those organizations, Hindman co-founded Mesosphere to offer services and solutions around Mesos. \u201cWe ultimately decided to build DC\/OS for those organizations which didn\u2019t have the technical expertise or didn’t want to spend their time building something like that on top.\u201d<\/p>\n

Mesos vs. Kubernetes?<\/h3>\n

People often think in terms of x versus y, but it\u2019s not always a question of one technology versus another. Most technologies overlap in some areas, and they can also be complementary. \u201cI don’t tend to see all these things as competition. I think some of them actually can work in complementary ways with one another,\u201d said Hindman.<\/p>\n

\u201cIn fact the name Mesos stands for \u2018middle\u2019; it\u2019s kind of a middle OS,\u201d said Hindman, \u201cWe have the notion of a container scheduler that can be run on top of something like Mesos. When Kubernetes first came out, we actually embraced it in the Mesos ecosystem and saw it as another way of running containers in DC\/OS on top of Mesos.\u201d<\/p>\n

Mesos also resurrected a project called Marathon <\/a>(a container orchestrator for Mesos and DC\/OS), which they have made a first-class citizen in the Mesos ecosystem. However, Marathon does not really compare with Kubernetes. \u201cKubernetes does a lot more than what Marathon does, so you can\u2019t swap them with each other,\u201d said Hindman, \u201cAt the same time, we have done many things in Mesos that are not in Kubernetes. So, these technologies are complementary to each other.\u201d<\/p>\n

Instead of viewing such technologies as adversarial, they should be seen as beneficial to the industry. It\u2019s not duplication of technologies; it\u2019s diversity. According to Hindman, \u201cit could be confusing for the end user in the open source space because it’s hard to know which technologies are suitable for what kind of workload, but that\u2019s the nature of the beast called Open Source.\u201d<\/p>\n

That just means there are more choices, and everybody wins. <\/p>\n","protected":false},"excerpt":{"rendered":"

The roots of Mesos can be traced back to 2009 when Ben Hindman was a PhD student at the University of California, Berkeley working on parallel programming. They were doing massive parallel computations on 128-core chips, trying to solve multiple problems such as making software and libraries run more efficiently on those chips. He started […]<\/p>\n","protected":false},"author":2,"featured_media":23927,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[40],"tags":[],"class_list":["post-23926","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-linux-freebsd-unix"],"_links":{"self":[{"href":"https:\/\/sickgaming.net\/blog\/wp-json\/wp\/v2\/posts\/23926","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/sickgaming.net\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/sickgaming.net\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/sickgaming.net\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/sickgaming.net\/blog\/wp-json\/wp\/v2\/comments?post=23926"}],"version-history":[{"count":0,"href":"https:\/\/sickgaming.net\/blog\/wp-json\/wp\/v2\/posts\/23926\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/sickgaming.net\/blog\/wp-json\/wp\/v2\/media\/23927"}],"wp:attachment":[{"href":"https:\/\/sickgaming.net\/blog\/wp-json\/wp\/v2\/media?parent=23926"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/sickgaming.net\/blog\/wp-json\/wp\/v2\/categories?post=23926"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/sickgaming.net\/blog\/wp-json\/wp\/v2\/tags?post=23926"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}