We are excited to host a webinar this Wednesday with our friends from 10gen and Opscode. Sandeep Parikh from 10gen and Matt Ray from Opscode will be joining Eddie Garcia, our VP of Engineering, live from our downtown Austin office for the webinar and Q&A. This will be a great opportunity to hear about our solutions and ask questions to any of these three companies. We hope you can join us!

Automating a Secure MongoDB Deployment with Opscode Chef and Gazzang zNcrypt
Pop Quiz: What's the quickest way to spin up a secure version of MongoDB? In this webinar, we'll explore the real-life use case of a non-profit education collaborative that wanted to secure sensitive student and teacher data on MongoDB using Gazzang zNcrypt. This kind of information requires a high level of security and confidentiality, but how do you protect this data in a distributed database with hundreds of nodes? We will demonstrate how Chef can help you architect security with Gazzang zNcrypt into your MongoDB environment at the deployment stage.

Date: Wednesday, September 19, 2012
Time: 11am CT
Presenters:
Matt Ray, Senior Technical Evangelist at Opscode
Sandeep Parikh, Product Management at 10gen
Eddie Garcia, Vice President of Development at Gazzang

Register Now!

Pop Quiz: What's the quickest way to spin up a secure version of MongoDB? In this webinar, we'll explore the real-life use case of a non-profit education collaborative that wanted to secure sensitive student and teacher data on MongoDB using Gazzang zNcrypt. This kind of information requires a high level of security and confidentiality, but how do you protect this data in a distributed database with hundreds of nodes? We will demonstrate how Chef can help you architect security with Gazzang zNcrypt into your MongoDB environment at the deployment stage.

Presenters:
Matt Ray, Senior Technical Evangelist at Opscode
Sandeep Parikh, Product Management at 10gen
Eddie Garcia, Vice President of Development at Gazzang

View the Replay Now

I want to show you how to use a Chef cookbook to easily deploy zNcrypt into your cloud infrastructure. Chef is an open-source systems integration framework built specifically for automating the cloud. If you have a multi-node, or certainly a big data environment, you'll want to use a deployment automation tool like Chef. It'll make your
life a whole lot easier.

What follows here is an overview, highlighting key components and sections of code for the zNcrypt Chef Cookbook.   At the end of the video, we’ll tell you where to go download the zNcrypt cookbook in its entirety.

First, let’s look at the files in the zNcyrpt Cookbook…

|-- attributes
|   `-- default.rb
|-- CONTRIBUTING
|-- LICENSE
|-- metadata.rb
|-- README.md
`-- recipes
    |-- activate.rb
    |-- cassandra.rb
    |-- configdirs.rb
    |-- default.rb
    `-- zncrypt.rb

Attributes, default.rb - defines the default location for the storage directories

CONTRIBUTING – how to contribute to this open-source project

LICENSE - contains licensing information

Metadata.rb - a ruby file with metadata about the cookbook

README – this is self explanatory

  Recipes

activate.rb - is the recipe that activates the zncrypt license and
generates a random strong passphrase
cassandrarb - installs cassandra from the DataStax rpm and deb
repositories (optional and can be added to your node if you are
configuring zncrypt with cassandra)
configdirs.rb - configures the storages directories for zNcrypt

            Default.rb – is the default recipe, and for our cookbook,
it is used to call the zncrypt recipe.

            zncrypt.rb – contains all the logic to download and install zNcrypt

Let’s take a look at some of the components of the zncrypt.rb code:

The first section of ruby code is setting up both yum and apt to use the Gazzang repository.

Next we download and import the gazzang gpg key, which we use to verify the authenticity of the gazzang packages in the repository.

After this, we address dkms (Dynamic Kernel Module Support), which zNcrypt requires to dynamically compile the zNcrypt kernel module. Most distributions include the dkms package, however, CentOS may need to have it pre-installed.  The next part of the recipe takes care of this for you.

Now that all the repos are setup, we will assemble the packages and install kernel headers, dkms, ezncryptfs and ezncrypt.

If we move to metadata.rb, you will find some boilerplate attributes, cookbook dependencies are defined and supported operating systems are named.

default.rb, in the zNcrypt Cookbook contains a few lines of simply
pointing to the zncrypt, configdirs and activate recipes to install
zncrypt.

The zNcrypt Cookbook can be found at https://github.com/gazzang/cookbooks.

Contact Gazzang to learn more about our Chef zNcrypt Cookbook and how you can use Gazzang to secure sensitive information in your cloud infrastructure.

Click here to view the video.

All the previous blogs in this Chef cookbook series have been leading up to using zNcrypt to encrypt BigData. In this blog we will see how with Chef we can easily install MongoDB and use zNcrypt to encrypt and protect Mongodb data.

The mongodb recipe is divided into 4 basic steps:

1. Setup the mongodb deb/rpm repositories from 10gen
2. Assemble the mongodb-10gen packages
3. Install the mongo packages
4. Setup zNcrypt with with mongodb using the ezncrypt and ezncrypt-access-control commands

In the first section of the recipe we use the apt cookbook to setup the 10gen apt repository for mongodb.

 # use the apt cookbook
 include_recipe "apt::default"
 # Add the 10gen repo, ubuntu debian
 apt_repository "10gen" do
  uri "http://downloads-distro.mongodb.org/repo/ubuntu-upstart"
  distribution "dist"
  components ["10gen"]
  key "http://docs.mongodb.org/10gen-gpg-key.asc"
  action :add 

In this second section we call apt::default to update apt and setup the list of packages to be installed.

 include_recipe "apt::default"
 %w{mongodb-10gen}

Then to install the package(s) we simple iterate over the list of packages and call the install action

# loop to install packages
mongo_packages.each do |mongo_pack|
  package mongo_pack do
    action :install
  end
end

With mongodb now installed, the last step is to encrypt the mongodb data files, which by default are in /var/lib/mongodb and setup the zNcrypt ACL. As you can see below we re-use the data bag (we created in the prior blog) to retrieve the passphrase which is required to make any ACL updates and passed to the ezncrypt-access-control command.

# Mongo is installed, we proceed to set up the encryption
# the path here is hardcoded, if it does not match yours edit here
acl_rule1="/usr/bin/mongod"
acl_rule2="/bin/mkdir"

# before anything we stop mongodb
# create the ACLs
passphrase=data_bag_item('license_pool', 'license1')['passphrase']
passphrase2=data_bag_item('license_pool', 'license1')['passphrase2']
script "create ACL" do
 interpreter "bash"
 user "root"
 cwd "/tmp"
 code <<-eoh service="" service_name="" stop="" ezncrypt-service="" start="" ezncrypt-access-control="" -a="" allow="" mongodb="" acl_rule1="" -p="" passphrase="" -s="" passphrase2="" acl_rule2="" ezncrypt="" -e="" data_dir="" eoh="" end="" pre="">

Using Chef and the zNcrypt cookbook most of the configuration work is done for you to install MongoDB with transparent data encryption. To run this recipe on your chef client, just download the zNcrypt cookbook and add the zncrypt::default and zncrypt::mongodb recipes to the run list of your chef-client.

July 18, 2012 - Business Wire Press Release

Gazzang today announced the availability of a Chef™ cookbook to automate the deployment and configuration of Gazzang zNcrypt™ for big data and enterprise IT environments. Chef is an open source systems integration framework written by Opscode that makes it easy to deploy servers and scale applications across cloud infrastructures.

Chef cookbooks include “recipes” that describe how part of a server (such as MongoDB, Apache Hadoop™, Apache Cassandra™ or MySQL) should be configured and the desired state for specific resources, including how packages should be installed, which services should be running, and how files should be written.

The Gazzang zNcrypt cookbook provides users a recipe to accelerate the provisioning, configuration, and integration of industry-standard data encryption, robust key management, and process-based access controls in their big data environments.

“Our goal is to help organizations leverage the power of open source by facilitating greater collaboration and sharing of best practices among the DevOps community,” said Jesse Robbins, chief community officer at Opscode. “Security plays a key role in the success of cloud infrastructure deployments, and we’re excited to see Gazzang zNcrypt added to the growing ecosystem of Chef cookbook tools.”

Gazzang zNcrypt transparently encrypts and secures data in real time, whether in the cloud or on premises, ensuring there is minimal performance lag in the encryption or decryption process. Advanced key management and process-based access controls enable organizations to meet compliance regulations and ensure unauthorized parties or malicious actors never gain access to the encrypted data.

“Chef provides a virtual blueprint to automate server deployment and configuration across highly distributed cloud infrastructures,” said Eddie Garcia, vice president of development at Gazzang. “Using a Chef cookbook, DevOps teams can build their data security solution in minutes or hours, versus weeks or months. This is a significant time, cost and resource savings for big data customers who are deploying zNcrypt on multi-hundred node databases in the cloud.”

Additional Resources

• Download the Gazzang zNcrypt Cookbook
• Watch the video on how to install zNcrypt using Chef
• Read the zNcrypt Cookbook blog

About Gazzang
Gazzang provides data security solutions and operational diagnostics that help enterprises protect sensitive information and maintain performance in cloud environments. The company has over 200 customers across multiple industries including SaaS providers, Financial Services, Technology, Healthcare and public sector organizations. Gazzang is backed by Austin Ventures and Silver Creek Ventures. For more information, visit www.gazzang.com.

Chef attributes, much like data bags, are useful to setup configuration variables using ruby for your recipe or cookbook. Let's review the current file structure of our zNcrypt cookbook, the attributes/default.rb file listed below contains the attributes for zNcrypt configuration.

|-- attributes
|   `-- default.rb
|-- CONTRIBUTING
|-- LICENSE
|-- metadata.rb
|-- README.md
`-- recipes
    |-- activate.rb
    |-- configdirs.rb
    |-- default.rb
    `-- zncrypt.rb

Attributes are very easy to setup, in this example we setup default values for the zncrypt_mount and zncrypt_storage attributes, these values can be modified programmatically or edited directly in the file to provide the desired values for your environment.

# setup the mount point for zncrypt
default['zncrypt']['zncrypt_mount'] = '/var/lib/ezncrypt/ezncrypted'
# setup the storage directory for zncrypt
default['zncrypt']['zncrypt_storage'] = '/var/lib/ezncrypt/storage'

Using these attributes is simple, in the code below we pull the attribute into a variable to construct the "ezncrypt-configure-directories" command.

# pull the directory configuration from the data bags
zncrypt_mount = node['zncrypt']['zncrypt_mount']
zncrypt_storage = node['zncrypt']['zncrypt_storage']
config_dirs = "-m #{zncrypt_mount} -s #{zncrypt_storage }"
case node['platform_family']
when "rhel","fedora"
opt = '-l'
when "debian"
opt = '-a'
end

script "config dirs" do
 interpreter "bash"
 user "root"
 code <<-eoh ezncrypt-service="" stop="" ezncrypt-configure-directories="" config_dirs="" opt="" eoh="" end="" pre="">

Now that we have setup our data bags and attributes let's quickly review our default.rb recipe to see how we call the new configdirs and activate recipes:

# installs zncrypt
include_recipe"zncrypt::zncrypt"
# configures the directories using the configuraiton from the databag
include_recipe"zncrypt::configdirs"
# activates the license using the data bag 
 include_recipe"zncrypt::activate"

In Part I of this blog we went over some tips and basic steps to creating a new Chef Cookbook. In Part II of this series we explored in detail the zNcrypt cookbook and recipe to perform a basic installation of zNcrypt. In this edition we will use chef data bags to activate the zNcrypt installation.

Data bags are very useful to pass configuration information to recipes using json. For zNcrypt, we will use a data bag to pass license/passphrase information to the cookbook. There are two basic ways to setup a data bag, you can use the knife command or you can setup the data bag programmatically.

knife data bag create BAG [ITEM] (options)

In our zNcrypt cookbook we will not use knife commands but rather setup the data bag programmatically. Let's review how we do this in the default.rb recipe. We start with a data_bag('license_pool') command to check if the data bag exists, if this call fails the "rescue" section will setup the new data bag.

|# check if the data bag exists, use a begin / rescue to handle the exception
begin
 # check if there is a license pool already and skip creating
 data_bag('license_pool')
rescue

Here in the rescue section of the code, we will use the OpenSSL cookbook to generate a strong password, then setup a license and activation code for each of the servers in our environment. See the openssl cookbook for more information on how to use the secure_password https://github.com/opscode/cookbooks/tree/master/openssl

|# check if the data bag exists, use a begin / rescue to handle the exception
begin
 #include the secure password from openssl recipe
 ::Chef::Recipe.send(:include, Opscode::OpenSSL::Password)

 # create a data bag for licensing pool
 license_pool = Chef::DataBag.new
 license_pool.name('license_pool')
 license_pool.save
 # create json for data bag item for each node
 ubuntu = {
   # use the node name as the id
   "id" => "ubuntu",
   # set your product key provided by Gazzang
   # this license will auto reset every hour, if your first registrationi
   # fails try again in an hour or contact sales@gazzang.com
   "license" => "XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX",
   # set your activation code provided by Gazzang
   "activation_code" => "123412341234",
   # random passphrase
   "passphrase" => secure_password,
   # random passphrase
   "passphrase2" => secure_password,
 }
 databag_item = Chef::DataBagItem.new
 databag_item.data_bag('license_pool')
 databag_item.raw_data = ubuntu
 databag_item.save

Now that we have setup the data bag, let's see how it will be used in our cookbook to activate zNcrypt in the activate.rb recipe. We will use the Chef "node.name" attribute to select the license that matches the server. We can then construct the string to pass as argument to the ezncrypt-activate command.

# check if there is a license pool otherwise skip activation
 data_bag('license_pool')
 license=data_bag_item('license_pool',"#{node.name}")['license']
 activation_code=data_bag_item('license_pool',"#{node.name}")['activation_code']
 # we also need a passhprase and second passphrase, we will generate a random one
 passphrase=data_bag_item('license_pool',"#{node.name}")['passphrase']
 passphrase2=data_bag_item('license_pool',"#{node.name}")['passphrase2']
 # build the arguments to the activate command
 activate_args="--activate --license=#{license} --activation-code=#{activation_code} --passphrase=#{passphrase} --passphrase2=#{passphrase2}"
 script"activate zNcrypt"do
  interpreter"bash"
  user"root"
  code<<-eoh codemkdir="" var="" log="" ezncrypt="" ezncrypt-activate="" activate_args="" eoh="" end="" lt="" pre="">

One problem with this example is that the data bag stores the encryption password in clear text. In future blogs we will see how we can use Chef encrypted data bags to protect the encryption password.

As you can see data bags are a very useful method to pass configuration to cookbooks. Another method to pass configuration information to cookbooks is using Chef Attributes. Please read the next blog to see how we will use Chef Attributes to setup the zNcrypt configuration directories.

See how Gazzang customers can use a Chef Cookbook to automate their data security deployments in big data or multimode environments. To download a copy of the Gazzang zNcrypt Chef Cookbook click here.