Apache Myriad 0.1.0 (Incubating) Released
On behalf of the Apache Myriad team, I’m happy to announce the availability of Apache Myriad 0.1.0 (incubating). This is the first community release of Apache Myriad, and is the result of more than 8 months of work by a diverse group of committers and contributors from Mesosphere, MapR, Paypal, the US Government and other organizations.
Many thanks to the following individuals who contributed code/documentation to the Myriad project. This release wouldn’t have been possible without their effort.
- Aashreya Ravi Shankar
- Adam B
- Brandon Gulla
- Darin Johnson
- Jim Klucar
- Kannan Rajah
- Ken Sipe
- Kris Chi
- Larry Edelstein
- Mohit Soni
- Naveen Manivannan
- Paul Read
- Ruth Harris
- Santosh Marella
- Sarjeet Singh
- Shingo Omura
- Suneel Marthi
- Swapnil Daingade
- Yuliya Feldman
- Zhongyue Luo
With Apache Myriad, users can deploy Hadoop YARN applications side-by-side with other enterprise applications, including those in Docker containers, on a single Mesos cluster. This is useful for several use cases, including -
- Elastic resource sharing: scaling up a web server farm during the day when users are active, and scaling up Hadoop workloads like Hive queries during the night when resources are idle.
- Multi-tenancy: provision isolated, logical Hadoop clusters to different users or groups with either dynamic or static resource allocations.
Myriad 0.1.0 contains quite a number of features, including
- Deployment of Hadoop YARN clusters as Mesos frameworks
- Static resource allocation using REST APIs, to expand/contract a Hadoop YARN cluster
- Dynamic resource allocation to expand/contract a Hadoop YARN cluster
- High Availability, with discoverability, restartability, and state recovery
- Ability to distribute Hadoop YARN binaries to Mesos nodes at runtime
- Provisioning of auxiliary Hadoop services like the Job History Server
- Support for constraints based node manager placement
- Management GUI for monitoring and scaling Hadoop YARN clusters
This is the first of many Apache Myriad releases. Below are a few of the things the team is planning to tackle next.
- Simplified provisioning and configuration of multiple YARN clusters
- Docker-based provisioning of both YARN resource managers and node managers
- More auxiliary services, like HiveServer2 and HiveMeta
- Dynamic Reservations
- Revocable Resources
- Data locality
If you’d like to get started we encourage you to start with our documentation wiki. For more details on the 0.1.0 release, please refer to our release notes. The release sources can be downloaded from here. To try out the latest and greatest code, please checkout Myriad’s git repository.
If you have any questions, please feel free to reach us out at firstname.lastname@example.org.