In a previous blog, I wrote about why it’s so important to select the right partner for your data encryption needs. Now, I’d like to cover the five questions every encryption vendor must be able to answer about big data security. And if you don’t like their response, by all means, move on:

Does the solution give you full control over your keys, even as data flows from one system to another?

It’s often said that key management is the hardest part of data encryption. That’s because there’s often a lack of clarity around key management and access. When evaluating encryption vendors, be sure to ask what types of key control policies can be established to prevent unauthorized access, and always be sure the data owner, not the cloud provider or other administrator, has complete control of the encryption keys.

Does the encryption solution allow for separation of duties between authorized personnel and systems administrators?

What good is data encryption if everyone, whether they need it or not, has access to the encrypted data? Proper, policy-controlled key management allows for separation of duties that allows system and cloud administrators to perform their jobs but restricts them from accessing encrypted data. The most important part of key management is ensuring the keys do not reside on the same server as the encrypted data. This is akin to locking your car and leaving the keys in the driver’s side door.

Does the solution work in mixed IT environments where data is stored in public and private clouds as well as in an on-premises data center?

Look for a software-based encryption solution that performs just as well in a data center as it does in the cloud. Remember that regardless of where the data is stored, it’s important that the data owner, not the hosting provider, retain possession and management of the crypto keys. If your encryption solution doesn’t allow you to manage the keys, then look elsewhere.

Has the solution been tested and/or benchmarked on the applications running in your environment?

Most large organizations utilize a variety of database applications from the more traditional like MySQL and PostgreSQL to newer big data apps like Cassandra, MongoDB and HBase. To ensure your encryption utility functions cross-platform and meets your performance standards, ask your provider whether they’ve tested against the databases that are most important to you.

Does the solution use NIST-approved encryption algorithms?

The National Institute of Standards and Technology Computer Security Division publishes security requirements, FIPS 140-2, for cryptographic modules. If your vendor solution uses FIPS-validated crypto modules, you can feel confident in the strength of their cryptographic algorithm.

The bottom line is, you don't need to employ cryptographic experts to secure your data and meet HIPAA, FIPS, FERPA or PCI compliance initiatives. All it takes is a trusted security partner who can help you get where you need to go.

This week at the Gartner Security & Risk Management Summit, analyst Jay Heiser revealed his Top 10 IT Security Myths. While the article doesn’t address big data security specifically, you can easily extrapolate that these myths, if they continue to proliferate, will only get worse as the volume, velocity, variety and sensitive nature of the data grows.

Check the myths out when you get a chance, and I’m sure you’ll nod in approval at some and raise an eyebrow at others. Following up on my encryption myths blog from a few weeks back, I’d like to address Gartner’s IT Security Myth number ten, which covers the same topic:

I agree that encryption is not a magic bullet for keeping sensitive files safe, but it can be your greatest asset if implemented properly. So while the premise for the myth is sound, I want to challenge the supposed “cure” for this myth (how do you cure a myth anyway?).

At Gazzang, we talk to customers each and every day about data encryption and key management. These are IT guys, security experts, engineers, database admins, and architects. All are well incredibly knowledgeable about what they do, and they typically understand some basic concepts of encryption as well. But to expect a customer to have “solid experience in cryptography” is a little much.

A better approach is to partner with a company that has cryptographic and key management expertise AND understands how to implement this security in complex big data environments.

But in order to select the right encryption partner, you must first know what questions to ask. In part two of this blog, we’ll look at five questions that every encryption vendor must be able to answer. Stay tuned.

Want a great analogy for how Gazzang zTrustee key management works?

Pretend Tiago Splitter is an unauthorized user or process, and the rim is a big data Hadoop cluster. 

Not everyone can afford security by LeBron James. For the rest of us, there's Gazzang encryption and key management.

Rarely a day goes by that you don't hear about a data breach. Hospital records stolen. Social media accounts hacked. Education transcripts revealed. Every industry is susceptible and every company is at risk. The result can be embarrassing and expensive at best and absolutely crippling at worst, with potential fines, time-consuming lawsuits, and subsequent loss of customer trust.

The steady pace of breaches reinforces the need for encryption as a last line of defense. Recently however, one of the oldest and most effective security tactics has been largely relegated to an afterthought in today's new cloud and big data environments.

This is the result of some common misperceptions about encryption and key management related to cost, performance and ease of use.

Today we set the record straight, breaking down the nine biggest encryptions myths.

Myth 1: Encryption is only for organizations that have compliance requirements. Certainly any company in a regulated industry that mandates data security and privacy should encrypt. That's a no brainer. But a better way to think about encryption is this: if you've got data about your products, customers, employees or market, that you believe is sensitive/competitive, then you should ALWAYS encrypt it, whether there's a legal obligation or not.

Myth 2: SSL encrypts data everywhere.
SSL only encrypts data in motion; it does not cover data at rest. As data is written to disk, whether it's stored for one minute or several years, it should be encrypted.

Myth 3: Encryption is too complicated and requires too many resources.
Data encryption can be as complicated or as easy as you want to make it. The key is to understand the type of data that needs to be encrypted, where it lives and who should have access to it. There are plenty of readily available, easy to use and affordable encryption tools on the market. If application performance is important, look for a transparent data encryption solution that sits beneath the application layer and does not require modifications to your operating system, application, data or storage.

Myth 4: Encryption will kill database performance.
There are a number of factors that impact database performance, and encryption is just one. Application-level encryption tends to pack the greatest performance hit, while the file-level encryption penalty is much lower. For maximum application performance, run block-level encryption on a system utilizing the Intel AES-NI co-processor.

Myth 5: Encryption doesn't make the cloud more secure.
On the contrary, in many cases storing encrypted data in the cloud is oftentimes more secure than keeping it on premises where insiders may have easier access. To ensure the safekeeping of encrypted data in the cloud, make sure you, not your cloud provider, maintain control of the encryption keys. If your provider requires you to hand over your keys, find another cloud service.

Myth 6: Encrypted data is secure data.
Too many organizations fail to effectively manage their encryption keys, either storing them on the same server as the encrypted data or allowing a cloud provider to manage them. Storing the key on the same server as your data or handing them over to your cloud provider is akin to locking your car and leaving the keys in the door. Good key management, with strong policy enforcement makes all the difference.

Myth 7: Key management requires expensive, cloud-adverse hardware.
While this was once true, today there are effective software-based solutions that enable organizations to deploy key management in the cloud or on premises. These solutions can typically be provisioned far faster than hardware security modules (HSMs), are very cloud friendly and meet most compliance statutes.

Myth 8: If your data is encrypted, it can't be stolen.
There is no security solution that will protect your data 100%. In fact, companies should operate with the mindset that their data can and likely will be compromised at some point in time. Data encryption can make the breach aftermath much more palatable though, since encrypted data cannot be decrypted without the key

Myth 9: Encryption is old school. I need a newer security technology to protect big data.
Data encryption is a proven security technique that works very well in modern NoSQL environments. As big data projects move from pilot to production, sensitive data such as protected health information (PHI), financial records, and other forms of personally identifiable information (PII) will likely be captured, processed, analyzed and stored.  Encryption is just as integral to securing data in NoSQL as it is in traditional relational database systems.

Firewalls and VPNs can provide some protection against data breaches and theft, but there is no substitute for strong encryption and effective key management, especially in big data and cloud environments. Now that the biggest myths have been busted, there's no longer an excuse not to encrypt.

I'd like to address a recent blog post in CloudTweaks titled, "Cloudera Not Cutting It With Big Data Security." The author makes a number of very salient and valid points about Hadoop security… or lack thereof. 

Indeed the Apache Hadoop platform, which includes HDFS and MapReduce and other projects like HBase, Mahout and Hive, was not designed for security. The Hadoop name, for better or worse, is nearly synonymous with big data because it delivers the "three V's" (velocity, variety, volume) at massive scale, enabling organizations to crunch, process, analyze and retain data like never before. 

Clearly there are security and compliance implications to big data. Consider the following: 

• How sensitive is the data being analyzed? Is there a regulatory requirement to secure the data? 
• Who has access to my data? 
• How can I secure data at rest?
• Can I run Hadoop in the cloud? If so, in what zone/country will my data reside?
• Is it even possible to have a HIPAA-compliant Hadoop cluster?

The blog suggests that Cloudera, and I presume other commercial Hadoop vendors, should do more to address the security concerns in Hadoop.

I believe Cloudera absolutely has the right approach to security. 

Cloudera has some of the brightest Hadoop and Apache minds in the world. They're experts in enterprise-class systems management. That's what they do. Addressing the big data needs of customers should always remain the company's primary focus. 

Cloudera has also cultivated one of the most comprehensive partner ecosystems in the big data market. This is important because it enables Cloudera to focus on its core strengths, while leveraging outside expertise in analytics, BI, cloud computing, and of course, security. 

Think about it this way: Would you expect the same company that built your house to install the alarm, mow your lawn and provide Internet service? 

Certainly not, so why then would you demand security from a company that specializes in Hadoop?

The right approach is to look for a company with expertise and experience in securing Hadoop platforms. 

There's a false narrative that traditional relational databases inherently offer cutting-edge security, but the truth is, security was (and remains) a responsibility of the end user. Encryption, authentication, policy enforcement and other security tools, are all available for CDH and Hadoop, even if they're not provided directly by Cloudera. A customer can work with Cloudera to locate the right vendor for their particular challenge and work collaboratively to build an integrated, secure Hadoop platform. 

Data security - particularly Big data security - is quickly becoming a hot topic, as Hadoop-related projects migrate from pilot to production environments. We believe it's incumbent upon big data providers to develop an ecosystem of tightly integrated vendors to complete their security offerings. Cloudera has certainly done that. 

Gazzang is proud to be certified Cloudera partner and will continue to provide enterprise-class data security to Hadoop users. 

At Gazzang, we have a mantra that borders on religious fanaticism.

“Customers First. Always.”

It’s the reason we can claim deep expertise in securing unique, enterprise-scale big data environments. It’s the reason we know cloud encryption better than anyone else. And it’s the reason no one on our customer support team owns a bed.

Customers also have a significant impact on our product development cycles. A perfect example being today’s exciting Gazzang CloudEncrypt™ announcement.

Gazzang CloudEncrypt was designed to meet specific customer use cases for securing sensitive data at every stage of the Amazon EMR process. This is a very different challenge than encrypting data on a persisted cloud platform like Amazon EC2, which can be done with readily available solutions like Gazzang zNcrypt and zTrustee.

CloudEncrypt offers encryption and key management in ephemeral, burstable Amazon EMR processes. The solution, which you can read about in great detail in this white paper, was developed at the request of a handful of Gazzang customers that had two very clear needs in common:

1. That data is always encrypted and protected in Amazon EMR
2. That the encryption keys (and there can be hundreds of thousands) be centrally managed either in a separate cloud instance or on-premises in the customer data center.

More detailed customer use cases are covered in the white paper, but the top three we’ve heard thus far are as follows:

• Enabling compliance initiatives around HIPAA, PCI, FERPA and EU Data Privacy
• Resolving data residency issues and providing the data owner with more leverage in the case of a subpoena concerns
• Preventing unauthorized access to source and MapReduce data outputs

Customer feedback is a part of everything we do at Gazzang. The ability to learn from and innovate in response to what we hear from the companies we serve is a badge of honor that we wear proudly.

As always, we welcome your feedback on Gazzang CloudEncrypt, your solution for securing sensitive datasets and outputs on Amazon EMR.