Project Razor: EMC and Puppet Labs

Today EMC and Puppet Labs are releasing via open source community contribution, a key module that serves to ground the last mile of the DevOps process. Project Razor is a tool that I’ve personally been looking for, for years. Razor is a service oriented (RESTful), model based, and policy driven “Operating System provisioning tool” which use their native installers for hardware compatibility.

During the project design meetings, we determined that we wanted a tool that could be:

• a programmable power controller, exploiting BMC/IPMI or an Intelligent powerstrip
• a capability discovery tool, using a downloadable kernel to take and maintain system inventory
• OS delivery processor, that is state machine driven, using native OS installers (ESX, Ubuntu, SLES, RedHat, even Windows), delivering what we might call a targeted “bare minimum” OS (with just enough package to join the MCollective)
• a consistent handoff, of the explicitly provisioned computer, to the DevOps environmentm for configuration control and customization
• and have a pluggable, easily extensible state machine that could act as a framework for downstream work (since no one solution fits IT).

Having architected large scale clouds like Sun Cloud and developed for Cloud Foundry based platform environments, I found myself spending a huge amount of time provisioning bare metal boxes with an ever changing portfolio of hardware and Operating Environments. Then, once I put the box in service, I had another set of scripts (Post Install) that would consistently provision the application stacks. For me, there were really 2 things that were inefficient, one was the fact that the installation models (usually scripts) were brittle because of machine differences, software builds/updates, and configurations, and the second was that I kept cobbling together all the different pieces: ipmi, tftp, httpd, dhcpd, syslog, + a ton of scripts. The second failure was the fact that each OS distributor employed their own provisioning systems isolated to their OS; but my environment needed 3 linux variants, VMware vSphere 5, Windows 8 and even Solaris.

Constantly looking for a better way, we evaluated a variety of DevOps tools to replace the “post OS” scripts, and settled on Puppet from Puppet Labs. For me the ability to finally get back to my programming roots within an traditional “System Administration” context was just the flexibility that we needed. We could build a first class MVC, exploit re-use, modern programming languages and an amazing flexibility of deployment.

In my view, this is an environment designed, not for the traditional system administrator, but for the hyper-productive, administrative developer. Not catering to the lowest common denominator, the flexibility afforded by the PL approach can help to tear down some of the old walls between development and deployment in a valuable and empowering way.

We built Razor to integrate with the DHCP and network boot services, to support the cycling of power, and the taking inventory information.  Interestingly we used Facter here to deliver, dynamically the “inventory taking modules” so that we could keep the inventory kernel small, but support the dynamic delivery of key sensors just in time.  The inventory information is registered with Razor, and the system is put in an wait state until Razor decides what to do with this node.

Razor takes the inventory and provides a set of matching expressions to help one “tag” key features in order to add this machine to the appropriate sets.  Razor then enables the administrator to define a model = what they want done, and a policy that maps the model against the tags, so that when a tag is matched a policy is triggered and a model delivered.  The net result, if the box fits a known profile, the model (OS + configurations) is delivered and Razor then deals with the boot / iPXE delivered OS images, and HTTP delivered customized package sets to that node (including the Puppet Agend), and when all is done (and the second reboot happens), hands this node off to Puppet for subsequent customization.

What’s cool about this solution? Razor with Puppet now support the end-to-end configuration control capability for a brand new “bare metal” node, into a working, deployed, managed, configuration with repeatability, auditability, and flexibility.

Why open source? For EMC, we see that the cloud market place is emerging quickly, and further recognize that the transparency and open-ness of frameworks will be pre-requisite to broad adoption for innovation and safety, respectively.  This just the first of many contributions that EMC technology team will make to the open source communities in support of massive scale cloud computing, scale out architectures, and the improved automation and operational excellence. Our architectural patterns for these environments are strongly influenced by DevOps strategies and SideBand controllers as typified by the OpenStack architecture. In areas like Quantum (Software Defined Networks), Cinder (Storage/Volume Controllers), and Nova (Compute Controllers). To add to this list, I believe that Puppet + Razor become a “Configuration Controller, which, can serve as mature model based policy driven glue-ware for cross controller customized and configuration managed integration.

Not to sell our VMware partners short! We believe that there is an exceptionally strong role for Razor + Puppet = top/bottom DevOps in the ability to stand up both VMware clusters in the vCloud model, but also to scale out the delivery of integrated applications on top of that environment.

We’ll be showing a really neat demonstration of Puppet and Razor at EMC World today, so please do tune in to “Chad’s World“. And, I do know that some now want more detail, but I want to call out the two key EMC developers who made Razor possible, namely, Nick Weaver and Tom McSweeney. Both of them worked tirelessly first on trying to get the other industry configuration controllers (Baracus from SuSE and Crowbar from Dell), and then to architect the pluggable dynamic state machine for Razor detailed here in Nick’s Blog. And of course the Puppet Labs team including Nan, Dan, Teyo, Nigel, Jose, Scott and Luke who integrated with our team seamlessly and made this possible.

Lastly, I need to call out Chuck Hollis, who is constantly looking for the cool factor in the industry and in EMC, who has detailed Razor in his blog.

Share on Facebook

Tweet

Follow us