In my last blog, I posed a common question that we get from our customers and prospects. “How do I rotate my Master Encryption Key in Gazzang zNcrypt if an admin leaves the company, or because our company key rotation policy?”

The most common way to rotate your Master Encryption Key is by using the “Passphrase Method.” Administration is simply using this technique, and there is no “file” that has to be managed. But as I mentioned in part one, to change the passphrase in zNcrypt, you must first decrypt all your data, then change the passphrase before re-encrypting with the new passphrase. This is not the most desirable method if you have a policy that requires regular key rotation.

So what do you do if you have to change the key in some way, but don’t want to decrypt / re-encrypt? In this blog, I’m going to show you how to use the RSA Key file method for your Master Encryption Key in zNcrypt.

Before setting up zNcrypt with the RSA Key file method, you need to have a key file ready to designate as your Master Encryption Key. Here’s a way to easily generate an RSA Key file:

bitnami@ip-10-110-221-104:~$ ssh-keygen -t rsa -f zncrypt-key.pem
Generating public/private rsa key pair.
Enter passphrase (empty for no passphrase): Enter same passphrase again: 
Your identification has been saved in zncrypt-key.pem.
Your public key has been saved in zncrypt-key.pem.pub.

The key generation command above, highlighted in yellow, will generate a 2048-bit RSA Key pair. zNcrypt only needs the “.pem” file where your identification has been saved.

Note that we don’t use the public key in this scenario, as your zNcrypt client and the Gazzang KSS will manage the authentication process during the times these two systems communicate.

I highlighted (in blue font) the step where you enter an “overlay passphrase” for the RSA Key file. I will explain the advantages of this configuration shortly.

One of the last steps during the zNcrypt configuration and activation process is to create your Master Encryption Key for the server. Choosing option 2) RSA, will generate a prompt similar to what you see below and will ask you to provide the .pem file as your Master Key.

           |  Key options available
           |  1) passphrase
           |  2) RSA
           | option: 2
           | type RSA private key file: /home/bitnami/zncrypt.pem
           | 
           | RECORD AND STORE THE FOLLOWING PASSPHRASE/SALT IN A SAFE PLACE
           | 
           | Passphrase: YazpSU4tyu697KBZYBCFEptJUw1BhFf
           | Salt:       d0C70956k79CF0D4
           | 
           | Press ENTER to continue
       RSA | wrapping encryption/decryption key
           | enter RSA passphrase: 

Let’s run through the process on my server, an instance on the Amazon EC2 Cloud where I am running a single MongoDB node. Currently, my MongoDB data folder is fully encrypted, and I’m using the RSA Key file method for my Master Encryption Key. It is set to the original key file that I set up on this system a month or two ago.

STEP 1 – The first step for the RSA Key file method is different than what we did for the Passphrase method discussed in my previous blog. An advantage of the RSA Key file method is you do not have to decrypt all your data before making a change. The reason this works is you can simply rotate the “overlay passphrase” you created when assigning the RSA Key file as your Master Encryption Key.

Run the ezncrypt-change-key command as seen here. Then follow the prompts to change the overlay password:

bitnami@ip-10-110-221-104:~$ ssh-keygen -t rsa -f zncrypt-key.pem 
Enter old passphrase: Key has comment 'zncrypt.pem' 
Enter new passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved with the new passphrase.

STEP 2 – There aren’t any other steps! You can easily change the RSA overlay passphrase when needed, but you don’t have to change the underlying encryption. In other words, you do NOT need to decrypt/re-encrypt if you only change the passphrase.

If you have any other install or configuration questions, send us an email at info@gazzang.com.

When I’m talking to customers (and future customers) about zNcrypt, a common question I get is, “How do I rotate my Master Encryption Key if an admin leaves the company, or because our company security policy says we must rotate encryption keys every quarter?”

Well, there are two answers to this question, and in this blog, I will give you the first answer. The second answer will follow in another blog very soon, so check back here often!

Most customers use the “Passphrase Method” for the zNcrypt Master Encryption Key. In zNcrypt, our passphrase is a 15 to 32 character password that is typed in during the initial configuration of the zNcrypt client on your server.

Because this passphrase IS the Master Encryption Key (and is what we store on the Gazzang KSS, our separate Key Storage System for your zNcrypt keys), the key change process does require a decryption and re-encryption of the data being protected.

Let’s run through the process on my server, an instance on the Amazon EC2 Cloud where I’m just running a single MongoDB server. Currently, my MongoDB data folder is fully encrypted and my passphrase is set to the original passphrase that I installed with when I built this server a few months ago.

STEP 1 – Decrypt the data before changing the Master Encryption Key (passphrase). Since the data is being used by the mongod process, I will stop MongoDB before decrypting. This is to ensure that no one is connected to my application or database. After stopping MongoDB, it’s time to decrypt the files. Here is what you will see:

bitnami@ip-10-244-51-139:~$sudo ezncrypt --decrypt @mongo /var/lib/mongodb
  ezncrypt | Checking system dependencies
           | getting information about location
           |   > path: /var/lib/ezncrypt/ezncrypted/mongo
  ezncrypt | Checking encryption status
           | done!
       key | Type your encryption/decryption key
           | type passphrase: 
           | type salt: 
           | generating keys	
           | done!
  ezncrypt | decrypting files
           |  > checking disk space
           |  > decrypting /var/lib/mongodb
           | done!
  ezncrypt | congratulations. you have decrypted your Files!!
       log | File: /var/log/ezncrypt/ezncrypt.log

STEP 2 – After decrypting the data, now you can run the ezncrypt-change-key command, which will walk you through the changing of the passphrase. You will be prompted to stop the zNcrypt service. Take the default “YES”.

bitnami@ip-10-244-51-139:~$ sudo ezncrypt-change-key
  ezncrypt | Checking system dependencies
  ezncrypt | Checking ezncrypt status
           | ezncrypt service is already running.
           | ezncrypt needs to be stopped in order to apply changes.
           | Do you want to stop the service? [Y/n]

zNcrypt will stop the service, then as an authentication step, you must type in the “old” passphrase to prove you’re an authorized user for zNcrypt. Enter your old passphrase (and optional salt).

 | Do you want to stop the service? [Y/n] 
           | Stopping ezncrypt service... Ok
  ezncrypt | Enter OLD key
       key | Type your encryption/decryption key
           | type passphrase: 
           | type salt: 
           | decrypting access control list

STEP 3 – Now it’s time to create your new Master Encryption Key. First select either 1) passphrase or 2)RSA. For the sake of this blog, I will stick with my original method, the passphrase. Enter the passphrase, re-type to verify, if you do not want the “Split Security” leave the salt blank , and you have now successfully changed your zNcrypt Master Encryption Key.

ezncrypt | Enter NEW key
       key | Type your encryption/decryption key
           | Key options available
           |  1) passphrase
           |  2) RSA
           | option: 1
           | type passphrase: 
           | re-type passphrase: 
           | type salt: 
  ezncrypt | Applying changes
           | Encrypting access control list... Ok
           | Changing passphrase... Ok
           | Starting ezncrypt service... Ok
  ezncrypt | Finished. Your encryption key has been changed.

All that remains now is to re-encrypt your sensitive data. You will use the exact same encryption command you used originally (this will also be very similar to the decrypt command used above.

SIDE NOTE: You can use the –n option in your encryption command (I did not use it this time). This is for “No Backup”, which means the encryption process will skip backing up the unencrypted files. If there are other means for restoring the data if something were to happen during the encryption process v, having zNcrypt make the backup file is not necessary.

bitnami@ip-10-244-51-139:~$ sudo ezncrypt --encrypt @mongo /var/lib/mongodb 
  ezncrypt | Checking system dependencies
           | Verifying ezncrypt license
           | getting information about location
           |   > path: /var/lib/ezncrypt/ezncrypted/mongo
  ezncrypt | Checking encryption status
           | done!
    keymgr | Retrieving key from KSS
           |  > Encryption password retrieved from KSS
           | generating keys	
           | done!
    backup | backing up data
           | This can take a while. Please be patient
           |  > backing up /var/lib/mongodb
           |  > File: /opt/ezncrypt/backup/2012-07-18/mongodb.tar.gz
           | done!
  ezncrypt | encrypting files
           |  > checking disk space
           |  > encrypting /var/lib/mongodb
           | done!
  ezncrypt | congratulations. you have encrypted your Files!!
       log | File: /var/log/ezncrypt/ezncrypt.log

That is all you have to do to change your “Master Encryption Key” when using the passphrase method.

Watch for my next blog where I will walk through the second method for changing your Master Encryption Key. Next time we’ll use the RSA Key File.

All the previous blogs in this Chef cookbook series have been leading up to using zNcrypt to encrypt BigData. In this blog we will see how with Chef we can easily install MongoDB and use zNcrypt to encrypt and protect Mongodb data.

The mongodb recipe is divided into 4 basic steps:

1. Setup the mongodb deb/rpm repositories from 10gen
2. Assemble the mongodb-10gen packages
3. Install the mongo packages
4. Setup zNcrypt with with mongodb using the ezncrypt and ezncrypt-access-control commands

In the first section of the recipe we use the apt cookbook to setup the 10gen apt repository for mongodb.

 # use the apt cookbook
 include_recipe "apt::default"
 # Add the 10gen repo, ubuntu debian
 apt_repository "10gen" do
  uri "http://downloads-distro.mongodb.org/repo/ubuntu-upstart"
  distribution "dist"
  components ["10gen"]
  key "http://docs.mongodb.org/10gen-gpg-key.asc"
  action :add 

In this second section we call apt::default to update apt and setup the list of packages to be installed.

 include_recipe "apt::default"
 %w{mongodb-10gen}

Then to install the package(s) we simple iterate over the list of packages and call the install action

# loop to install packages
mongo_packages.each do |mongo_pack|
  package mongo_pack do
    action :install
  end
end

With mongodb now installed, the last step is to encrypt the mongodb data files, which by default are in /var/lib/mongodb and setup the zNcrypt ACL. As you can see below we re-use the data bag (we created in the prior blog) to retrieve the passphrase which is required to make any ACL updates and passed to the ezncrypt-access-control command.

# Mongo is installed, we proceed to set up the encryption
# the path here is hardcoded, if it does not match yours edit here
acl_rule1="/usr/bin/mongod"
acl_rule2="/bin/mkdir"

# before anything we stop mongodb
# create the ACLs
passphrase=data_bag_item('license_pool', 'license1')['passphrase']
passphrase2=data_bag_item('license_pool', 'license1')['passphrase2']
script "create ACL" do
 interpreter "bash"
 user "root"
 cwd "/tmp"
 code <<-eoh service="" service_name="" stop="" ezncrypt-service="" start="" ezncrypt-access-control="" -a="" allow="" mongodb="" acl_rule1="" -p="" passphrase="" -s="" passphrase2="" acl_rule2="" ezncrypt="" -e="" data_dir="" eoh="" end="" pre="">

Using Chef and the zNcrypt cookbook most of the configuration work is done for you to install MongoDB with transparent data encryption. To run this recipe on your chef client, just download the zNcrypt cookbook and add the zncrypt::default and zncrypt::mongodb recipes to the run list of your chef-client.

July 18, 2012 - Business Wire Press Release

Gazzang today announced the availability of a Chef™ cookbook to automate the deployment and configuration of Gazzang zNcrypt™ for big data and enterprise IT environments. Chef is an open source systems integration framework written by Opscode that makes it easy to deploy servers and scale applications across cloud infrastructures.

Chef cookbooks include “recipes” that describe how part of a server (such as MongoDB, Apache Hadoop™, Apache Cassandra™ or MySQL) should be configured and the desired state for specific resources, including how packages should be installed, which services should be running, and how files should be written.

The Gazzang zNcrypt cookbook provides users a recipe to accelerate the provisioning, configuration, and integration of industry-standard data encryption, robust key management, and process-based access controls in their big data environments.

“Our goal is to help organizations leverage the power of open source by facilitating greater collaboration and sharing of best practices among the DevOps community,” said Jesse Robbins, chief community officer at Opscode. “Security plays a key role in the success of cloud infrastructure deployments, and we’re excited to see Gazzang zNcrypt added to the growing ecosystem of Chef cookbook tools.”

Gazzang zNcrypt transparently encrypts and secures data in real time, whether in the cloud or on premises, ensuring there is minimal performance lag in the encryption or decryption process. Advanced key management and process-based access controls enable organizations to meet compliance regulations and ensure unauthorized parties or malicious actors never gain access to the encrypted data.

“Chef provides a virtual blueprint to automate server deployment and configuration across highly distributed cloud infrastructures,” said Eddie Garcia, vice president of development at Gazzang. “Using a Chef cookbook, DevOps teams can build their data security solution in minutes or hours, versus weeks or months. This is a significant time, cost and resource savings for big data customers who are deploying zNcrypt on multi-hundred node databases in the cloud.”

Additional Resources

• Download the Gazzang zNcrypt Cookbook
• Watch the video on how to install zNcrypt using Chef
• Read the zNcrypt Cookbook blog

About Gazzang
Gazzang provides data security solutions and operational diagnostics that help enterprises protect sensitive information and maintain performance in cloud environments. The company has over 200 customers across multiple industries including SaaS providers, Financial Services, Technology, Healthcare and public sector organizations. Gazzang is backed by Austin Ventures and Silver Creek Ventures. For more information, visit www.gazzang.com.

Chef attributes, much like data bags, are useful to setup configuration variables using ruby for your recipe or cookbook. Let's review the current file structure of our zNcrypt cookbook, the attributes/default.rb file listed below contains the attributes for zNcrypt configuration.

|-- attributes
|   `-- default.rb
|-- CONTRIBUTING
|-- LICENSE
|-- metadata.rb
|-- README.md
`-- recipes
    |-- activate.rb
    |-- configdirs.rb
    |-- default.rb
    `-- zncrypt.rb

Attributes are very easy to setup, in this example we setup default values for the zncrypt_mount and zncrypt_storage attributes, these values can be modified programmatically or edited directly in the file to provide the desired values for your environment.

# setup the mount point for zncrypt
default['zncrypt']['zncrypt_mount'] = '/var/lib/ezncrypt/ezncrypted'
# setup the storage directory for zncrypt
default['zncrypt']['zncrypt_storage'] = '/var/lib/ezncrypt/storage'

Using these attributes is simple, in the code below we pull the attribute into a variable to construct the "ezncrypt-configure-directories" command.

# pull the directory configuration from the data bags
zncrypt_mount = node['zncrypt']['zncrypt_mount']
zncrypt_storage = node['zncrypt']['zncrypt_storage']
config_dirs = "-m #{zncrypt_mount} -s #{zncrypt_storage }"
case node['platform_family']
when "rhel","fedora"
opt = '-l'
when "debian"
opt = '-a'
end

script "config dirs" do
 interpreter "bash"
 user "root"
 code <<-eoh ezncrypt-service="" stop="" ezncrypt-configure-directories="" config_dirs="" opt="" eoh="" end="" pre="">

Now that we have setup our data bags and attributes let's quickly review our default.rb recipe to see how we call the new configdirs and activate recipes:

# installs zncrypt
include_recipe"zncrypt::zncrypt"
# configures the directories using the configuraiton from the databag
include_recipe"zncrypt::configdirs"
# activates the license using the data bag 
 include_recipe"zncrypt::activate"

In Part I of this blog we went over some tips and basic steps to creating a new Chef Cookbook. In Part II of this series we explored in detail the zNcrypt cookbook and recipe to perform a basic installation of zNcrypt. In this edition we will use chef data bags to activate the zNcrypt installation.

Data bags are very useful to pass configuration information to recipes using json. For zNcrypt, we will use a data bag to pass license/passphrase information to the cookbook. There are two basic ways to setup a data bag, you can use the knife command or you can setup the data bag programmatically.

knife data bag create BAG [ITEM] (options)

In our zNcrypt cookbook we will not use knife commands but rather setup the data bag programmatically. Let's review how we do this in the default.rb recipe. We start with a data_bag('license_pool') command to check if the data bag exists, if this call fails the "rescue" section will setup the new data bag.

|# check if the data bag exists, use a begin / rescue to handle the exception
begin
 # check if there is a license pool already and skip creating
 data_bag('license_pool')
rescue

Here in the rescue section of the code, we will use the OpenSSL cookbook to generate a strong password, then setup a license and activation code for each of the servers in our environment. See the openssl cookbook for more information on how to use the secure_password https://github.com/opscode/cookbooks/tree/master/openssl

|# check if the data bag exists, use a begin / rescue to handle the exception
begin
 #include the secure password from openssl recipe
 ::Chef::Recipe.send(:include, Opscode::OpenSSL::Password)

 # create a data bag for licensing pool
 license_pool = Chef::DataBag.new
 license_pool.name('license_pool')
 license_pool.save
 # create json for data bag item for each node
 ubuntu = {
   # use the node name as the id
   "id" => "ubuntu",
   # set your product key provided by Gazzang
   # this license will auto reset every hour, if your first registrationi
   # fails try again in an hour or contact sales@gazzang.com
   "license" => "XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX",
   # set your activation code provided by Gazzang
   "activation_code" => "123412341234",
   # random passphrase
   "passphrase" => secure_password,
   # random passphrase
   "passphrase2" => secure_password,
 }
 databag_item = Chef::DataBagItem.new
 databag_item.data_bag('license_pool')
 databag_item.raw_data = ubuntu
 databag_item.save

Now that we have setup the data bag, let's see how it will be used in our cookbook to activate zNcrypt in the activate.rb recipe. We will use the Chef "node.name" attribute to select the license that matches the server. We can then construct the string to pass as argument to the ezncrypt-activate command.

# check if there is a license pool otherwise skip activation
 data_bag('license_pool')
 license=data_bag_item('license_pool',"#{node.name}")['license']
 activation_code=data_bag_item('license_pool',"#{node.name}")['activation_code']
 # we also need a passhprase and second passphrase, we will generate a random one
 passphrase=data_bag_item('license_pool',"#{node.name}")['passphrase']
 passphrase2=data_bag_item('license_pool',"#{node.name}")['passphrase2']
 # build the arguments to the activate command
 activate_args="--activate --license=#{license} --activation-code=#{activation_code} --passphrase=#{passphrase} --passphrase2=#{passphrase2}"
 script"activate zNcrypt"do
  interpreter"bash"
  user"root"
  code<<-eoh codemkdir="" var="" log="" ezncrypt="" ezncrypt-activate="" activate_args="" eoh="" end="" lt="" pre="">

One problem with this example is that the data bag stores the encryption password in clear text. In future blogs we will see how we can use Chef encrypted data bags to protect the encryption password.

As you can see data bags are a very useful method to pass configuration to cookbooks. Another method to pass configuration information to cookbooks is using Chef Attributes. Please read the next blog to see how we will use Chef Attributes to setup the zNcrypt configuration directories.

Gazzang has built an interesting business around Transparent Data Encryption, building on top of eCryptfs, adding some mandatory access controls and policy management in a product we call zNcrypt. 

With the addition of zTrustee to the Gazzang product portfolio, we have entered an even more interesting, ambitious, new space in modern Cloud computing -- Opaque Object Storage.

eCryptfs and zNcrypt provide "transparent" data encryption, in that when the encrypted filesystem is mounted, allowed users, applications and processes (possessing the appropriate keys) are allowed to seamlessly read and write data as regular files and directories to the mounted storage filesystem.  No user, application, or process needs to know anything about encryption -- they simply read and write data "transparently" from and to files and directories.  Input/output operations are trapped in the Linux filesystem layer, and eCryptfs handles encrypting and decrypting files as necessary.  Assuming you have safeguarded your keys appropriately, an offline attacker with physical or remote access to the disk would not have access to mounted filesystem and instead only see the cryptographically protected data.

zTrustee was designed from the ground up to store and serve the keys necessary to make eCryptfs and zNcrypt work properly.  But we implemented it in a manner in which we can store and serve keys, certificates, files, directories, and data of any type -- similar to some object storage systems.  However, we added our considerable security expertise to our implementation, and use encryption yet again to our customer's advantage.  Each of these objects stored in zTrustee are actually encrypted and signed with the public GPG keys of the client storing and/or retrieving the data.  This means that even an administrative user with full root access on the zTrustee server will not have introspection into the contents of the data blobs stored as deposits on the zTrustee server.  For this reason, we're calling these deposits "Opaque Objects", and noting that our zTrustee server provides "Opaque Object Storage". 

Moreover, the fine-grained security policies that govern the release of these deposits further differentiate zTrustee from various other object storage products.  Beyond the individual encryption of each zTrustee deposit (object), the policy by which an object is released can:

• limit the  (time to live) TTL
• limit the number of times it can be retrieved (e.g. Mission Impossible message)
• be disabled (and later enabled)
• be purged entirely
• required 0 - N  authenticate and "vote" or "sign off" on the release trustees
• be retrieved by an authenticated, signed/encrypted client, or
• be retrieved anonymously using a nonce URL