Our friends at the Linux Foundation just released their 2013 Enterprise End User Report, “Linux Adoption: Third Annual Survey of World’s Largest Enterprise Linux Users.” The report shows exactly what we are hearing from our customers – for enterprises using the cloud, Linux is by far the dominant platform. Of course as more information moves to the cloud, Linux data security is becoming a hot topic. Encryption and Key Management are the two most effective ways to protect your data against unauthorized access or malicious attack, while also helping to maintain regulatory compliance and reduce risk.

The full report is definitely worth a read. Some of our favorite stats include:

• Linux experienced 12.7 percent year-over-year growth for the quarter while Windows only increased 3.2 percent.
• Linux is the dominant platform for cloud computing with nearly 76 percent using Linux servers for cloud
• 73 percent of organizations use Linux for mission-critical workloads in 2013.
• 80 percent of the world’s largest enterprises plan to increase their use of Linux servers over the next five years

For more on Cloud Security, check out: http://gazzang.com/solutions/cloud-security

Thank you for downloading our Securing Big Data white paper. Please click here to retrieve your download.

There’s a great Wall Street Journal blog by Rachael King on the increase in cyber attacks targeting privileged accounts. A recent survey she sites suggest 64% of IT administrators “believe that the majority of recent security attacks involved the exploitation of so-called privileged accounts.” These are access points built into devices by the manufacturer that make it easer for IT to manage the network. 

I’m a big proponent of multi-layer security that includes security controls not singularly tied to user identity or some proxy user identity method. This is particularly problematic at the OS level but certainly at the database and other levels as well.

In previous roles, I’ve worked with a number of developers and architects who believed OS-level file access controls granted to a user were good enough; that layered security wasn’t always necessary. I realize writing good crypto, key management, and other security code, or finding folks who can, is tough and expensive, but that’s not an excuse for failing to implement multiple levels of security.

There’s a reason your bank locks its doors, and then they lock the safe, and then they lock the safe deposit boxes. Security is about layers, and with virtualization, cloud, big data, and more, building out these layers becomes even more important.

Gazzang is focused on helping organizations secure big data in the cloud.  Our bread and butter is Unified Transparent Encryption through Gazzang zNcrypt. This solution encrypts sensitive data in Linux on the fly as it’s written to disc.  Access to that data is managed by our key storage system and process-based access control lists. We designed the system to limit OS and root privileges, preventing data from being stolen via “privilege escalation.” 

Speaking of layered security, I want to wrap up this blog with some additional thoughts on Gazzang zTrustee, which we announced earlier this week. Our chief architect, Dustin Kirkland, touched on this some yesterday, but fundamentally zTrustee focuses on securing and providing access to those keys, certificates, tokens and other “opaque objects” that act as the gatekeepers to sensitive information about your IT DNA.

This data, if it were to fall into the wrong hands through privileged accounts or other form of attack, could be disastrous to your organization. It’s not unusual to have robust policies governing access to keys, certs, tokens and passphrases, but what about some of the more obscure files or file directories like those containing ACLs or connection strings?

An interesting method of protecting this data is with the concept of a “trustee.” A trustee – often a person or group of persons but could just as well be a service or a combination of the two – control the release of keys and prevent “privilege escalation.”

As we bring zTrustee to market, I’m certain we’ll discover more use cases and continue to innovate on the trustee concept. If you’re interested in learning more, sign up for our free trial.

I have interviewed hundreds of candidates and had the delight of hiring dozens of Linux and open source developers, engineers, and interns over the last 10 years -- at IBM, Canonical, and now Gazzang. The most recent one signed his contract this morning, in fact! It's quite a rush to bring new talent into a small team.

Linux jobs are actually hotter now than ever before! The Wall Street Journal picked this up recently. And while HostGator has been running giant billboards throughout Austin for at least 2 years now, which plainly asks, "Do you know Linux? We're hiring!" -- I was impressed to see that they had the same billboard scaled up to 3-stories in height right in Times Square, New York.
Given that my own well being is so deeply invested in being an open source hacker, I selfishly love seeing the Linux and open source job market expanding so vibrantly.

From the interviewer's chair, however, my poking and prodding of a given candidate's Linux skills have changed a bit over those 10 years. I'm often looking for the candidate's inquisitive nature. I want to know how interested they really are in going down the rabbit hole.

• 10 years ago, you had to know how to deploy and run a LAMP stack, and hack your way around Apache, MySQL,PostgreSQL, PHP, Perl, and Python. You would shriek in horror at bad HTML and CSS and could really make a website sing with a little Javascript.
• 9 years ago, I wanted to see someone who regularly compiled their own upstream kernel, maybe tweaked a few configuration options on or off just for fun. Bonus points for each additional software package you compiled from source. Gentoo users were shoe-ins.
• 8 years ago, I wanted to talk to people who were sending and receiving PGP or GPG signed, encrypted email. I was delighted by those who had at least 1024D keys!
• 7 years ago, I found users who were willing and able to tweak their SELinux policies and AppArmor profiles absolutely intriguing. If you were running SELinux in enforcing mode on a production system, well, damn, you probably got the job!
• 6 years ago, I wanted someone who had built their own Beowulf cluster, for fun, over the weekend. If not Beowulf, then some sort of cluster computing. Maybe Condor, or MPICH.
• 5 years ago, I'd structure some conversation around reinstalling dd-wrt or openwrt firmware on routers. What serious hackers would run stock router firmware?!?
• 4 years ago, I needed you to have experience with open source virtualization, such as KVM, Xen, and QEMU. Oh, and surely you're running MythTV on a few computers around the house, right?
• 3 years ago, it was all about developers who had Launchpad or Github accounts, had written some open source software and packaged it for Ubuntu or Fedora. While your friends update one other over Facebook, you're pushing updates over git and bzr.
• 2 years ago, I needed you to have experience with open source virtualization, such as KVM, Xen, and QEMU.
• A year ago, I was interested in people who had built or deployed their own cloud infrastructure using Eucalyptus or OpenStack and were managing it with Puppet/Chef/Juju.

Nowadays? Well, it's additive, to an extent. Hopefully you have the LAMP stack and kernel compilations in your pocket, can send and receive signed/encrypted email. No real hacker ever runs stock firmware on their router, surely you're using virtual machines and cloud computing on a daily basis, and hopefully you spend as much time on Launchpad/Github as Facebook/Twitter :-)

But you need to be on the cusp of what's next. I'm hoping you've rooted your phone, jacked your bootloader, and installed a CyanogenMod of your choosing -- at least on your phone at least if not your tablet and e-Reader too! Hopefully you've tried out this big data business and threw together a map-reduce Hadoop job or two, just for grins. Clearly you'll have a strong, informed opinions on Unity vs. Gnome3, upstart vs. systemd, and the UEFI secure boot mess.

Oh, and big bonus points if you read my blog. But you knew that already. If you read my blog, you've seen this. And this is what we'll talk about in our interview :-)

:-Dustin

MySQL Cluster usage has certainly continued to spread and recently accelerate well beyond its initial telco vertical roots into Healthcare, Financial Services, SaaS and more. With those additions it certainly becomes desirable for many to provide transparent encryption on the NDB nodes where the data, logs, checkpoints that write to disk. I'll not go into all those reasons in this blog, but certainly there are plenty, visit our whitepapers section for more information, especially if you are running within hosted, managed, or cloud environments platforms.

The solution for ndb in a nutshell was straight forward:

1. Set up Gazzang ezNcrypt Flex Platform
2. Stop the ndb process prior to encrypting the ndb_data directory
3. Encrypt the ndb_data directory
   ezncrypt -e @ndbdata /home/mysql/my_cluster/ndb_data
4. Add a Flex ACL Rules granting ndbd access to the encryption keys.
   ezncrypt-access-control -a "ALLOW @ndbdata * /home/mysql/mysql-cluster-gpl-7.1.18-linux-i686-glibc23/bin/ndbd"
5. Restart ndb

Note: if you setup a single node test environment or if for some reason want to run it for you will also need to add a rule for ndb_mgmd then also add -

ezncrypt-access-control -a "ALLOW @ndbdata * /home/mysql/mysql-cluster-gpl-7.1.18-linux-i686-glibc23/bin/ndb_mgmd”

Certainly there are many more things you can do to protect MySQL Cluster data on Linux - and I will follow through with those details or details on usage in specific environments, but this is a good start and shows how easy this is to accomplish, and Gazzang adds key management, process, access, monitoring, and many other benefits aside from the encryption itself. For more ideas around that see our EMA paper or schedule an overview with us.

With the release of of our 2.2.2 product coming in February of 2012 you will see that we have added ndb to our supported engines list to MySQL. Gazzang's platform is simple and easy to install as you can see here. If you are interested just Try it out.

One of the challenges many face when running a secure encrypted MySQL backup on Linux is using managed or scheduled mysqldumps without exposing them.

1. The data – within the mysqldump backup file
2. The credentials – that are used connect into mysql

So, how might this be accomplished? I’ll show you one option. It starts with the installation of Gazzang’s ezNcrypt. Its not open source, but it is inexpensive and provides you a simple and secure means to protect and encrypt data transparently with the flexibly to map to your environment and applications. Give it a try.

This technique also applies to other backup tools such as xtrabackup.

This can also go along with transparently encrypting your mysql data. I discussed this on oursqlpodcast #55 a few months ago.

Follow these steps to perform a secure mysqldump:

Step 1. Create a mysqldump cnf file. This provides the username, password and secure file destination.

> cat protected.cnf

[client]

user=”<your_user>

password=”<your_password>

result-file=/var/lib/mysqlbackup/<backupfilename>

Note: you can also encrypt this cnf file off the system if needed

Step 2. Encrypt this mysqldump cnf file. If its in plain text it’s not protected.

> sudo ezncrypt –e @mysqlbackup /home/mfrank/protected.cnf

ezncrypt | Checking system dependencies

          | Verifying ezncrypt license

          | getting information about location

          |   > path: /var/lib/ezncrypt/ezncrypted/mysqlbackup

 ezncrypt | Checking encryption status

          | done!

   keymgr | Retrieving key from KSS

          |  > Encryption password retrieved from KSS

          | generating keys

          | done!

 ezncrypt | encrypting files

          |  > checking disk space

          |  > encrypting /home/mfrank/protected.cnf

          | done!

 ezncrypt | congratulations. you have encrypted your Files!!

Step 3. Create a backup directory and set permissions appropriately. This is where the backups will be stored.

> sudo mkdir /var/lib/mysqlbackup

> sudo chown <linux_user> /var/lib/mysqlbackup

Step 4. Encrypt the backup directory with ezNcrypt. All files going to this directory /var/lib/mysqlbackup will be encrypted.

> sudo ezncrypt –encrypt @mysqlbackup /var/lib/mysqlbackup

Step 5. Create the access control rule for mysqldump. This allows access to the key plus permissions to the files in @mysqlbackup.

> sudo ezncrypt-access-control –add “ALLOW @mysqlbackup * /usr/bin/mysqldump”

> passphrase:

> salt:

Step. 6 Run the mysqldump. From either the commandline or cron.

> mysqldump –defaults-extra-file=/home/mfrank/protectedlogin.cnf –all-databases

Note: the backup file and cnf file are actually physically located (for my default installation) in /var/lib/ezncrypt/ezncrypted/mysqlbackup

You can see the links using

> ls –l

Mission accomplished.

Review the benefits of running a mysqldump with ezNcrypt:

1. The OS user can now run the backup without knowing the username and password to mysql.
2. The OS user cannot read the files that were generated by mysqldump.
3. If another process copies the backup file – i.e. a scheduled filesystem backup – the file would be AES-256 encrypted. Without access to the encryption keys the files are protected.
4. For recovery, the file can be easily be decrypted (ezncrypt – decrypt) and restored using mysql.

Note: There are ways to setup a trusted auto restore executable or script such that you can restore with seeing the username/password or mysqldump. data as well. I’ll blog about some other time.

1. The files can be sent to another system and decrypted (if you have the key) and then can be restored.

It’s hard to continually develop your own security solutions with encryption and key management. Transparent encryption solves many problems. For a more in-depth look, you might be interested in this EMA paper “Unifying Data Encryption: Liberating Transparent Encryption for Any Purpose”

In conclusion, I think transparent encryption provides a somewhat novel way to accomplish this task that is both easy and secure. It has become increasingly important to improve security and enforce principles of “need to know” and “separation of duties” across business partners and 3^rd parties, especially in Cloud and PaaS environments. This is one method towards providing that for MySQL backups using mysqldump.