Howdy all!

It's hard to believe that it's been almost 7 years since many of you helped conceive, design, implement, develop extend, test, document eCryptfs back in the IBM Linux Technology Center's Security team, with many more contributions following from Canonical, Red Hat, Intel, etc.

Over those 7 years, we have helped many of the rest of you reading this secure your private data. Today, there are actually millions of eCryptfs users -- through Ubuntu (and other Linux distros) encrypted home directories, Google's ChromeOS, Synology and Seagate encrypted NAS devices, some Android phones, Gazzang's cloud and big data encryption products, among others.

With the 100th release of the user space eCryptfs utilities (ecryptfs-utils), we'd like to take the opportunity to invite any contributors or users of eCryptfs for a celebration!

Gazzang is providing drinks and snacks on its rooftop (and air-conditioned lobby) in downtown Austin, at 8th and Congress, with great views of the Texas state capitol and all of downtown. Please join us at 804 Congress Avenue, Suite 400, from 5pm-8pm on Thursday, August 2, 2012. This is the day before the Texas Linux Fest in San Antonio, so perhaps we might see a few out of town guests join us... I happen to know at least one guest of honor who will be in from out of town for this event ;-)

Hopefully you'll join some of your colleagues around the Linux, open source, security, encryption, key management and data privacy space for an informal get together here in central Texas!

Cheers,
Dustin Kirkland
eCryptfs co-maintainer

Gazzang has built an interesting business around Transparent Data Encryption, building on top of eCryptfs, adding some mandatory access controls and policy management in a product we call zNcrypt. 

With the addition of zTrustee to the Gazzang product portfolio, we have entered an even more interesting, ambitious, new space in modern Cloud computing -- Opaque Object Storage.

eCryptfs and zNcrypt provide "transparent" data encryption, in that when the encrypted filesystem is mounted, allowed users, applications and processes (possessing the appropriate keys) are allowed to seamlessly read and write data as regular files and directories to the mounted storage filesystem.  No user, application, or process needs to know anything about encryption -- they simply read and write data "transparently" from and to files and directories.  Input/output operations are trapped in the Linux filesystem layer, and eCryptfs handles encrypting and decrypting files as necessary.  Assuming you have safeguarded your keys appropriately, an offline attacker with physical or remote access to the disk would not have access to mounted filesystem and instead only see the cryptographically protected data.

zTrustee was designed from the ground up to store and serve the keys necessary to make eCryptfs and zNcrypt work properly.  But we implemented it in a manner in which we can store and serve keys, certificates, files, directories, and data of any type -- similar to some object storage systems.  However, we added our considerable security expertise to our implementation, and use encryption yet again to our customer's advantage.  Each of these objects stored in zTrustee are actually encrypted and signed with the public GPG keys of the client storing and/or retrieving the data.  This means that even an administrative user with full root access on the zTrustee server will not have introspection into the contents of the data blobs stored as deposits on the zTrustee server.  For this reason, we're calling these deposits "Opaque Objects", and noting that our zTrustee server provides "Opaque Object Storage". 

Moreover, the fine-grained security policies that govern the release of these deposits further differentiate zTrustee from various other object storage products.  Beyond the individual encryption of each zTrustee deposit (object), the policy by which an object is released can:

• limit the  (time to live) TTL
• limit the number of times it can be retrieved (e.g. Mission Impossible message)
• be disabled (and later enabled)
• be purged entirely
• required 0 - N  authenticate and "vote" or "sign off" on the release trustees
• be retrieved by an authenticated, signed/encrypted client, or
• be retrieved anonymously using a nonce URL

I'm out at the GigaOM Structure conference in sunny San Francisco this week, where Gazzang has launched its newest product -- Gazzang zTrustee! My colleagues and I have dedicated the last 6 months to the design, architecture, development and testing of this new product, and I'm thrilled to finally be able to speak freely about it.

Gazzang's original product, zNcrypt is a transparent data encryption solution -- a GPLv2 encrypted filesystem built on top of eCryptfs, adding mandatory access controls and a dynamic policy structure. zNcrypt enables enterprise users to secure data in the cloud, meet compliance regulations, and sleep well at night, ensuring that all information is encrypted before written to the underlying storage.

As of today, Gazzang's newest product, zTrustee is an opaque object storage system, ultimately providing a flexible, secure key management solution for data encryption. Any encryption system, at some point, requires access to keys, and those keys should never be stored on the same system as the encrypted data. While zTrustee was initially designed to store keys, it can actually be used to put and get opaque data objects of any type or size.

Planet Ubuntu readers might recognize a few small-scale ancestors of zTrustee in other projects that I've authored and talked about here in the past... The encrypted pbputs and pbget commands now found in the pastebinit package are similar, in principle, to zTrustee's secure put and get commands. But rather than backing uploads with a pastebin server, we have implemented a powerful, robust, enterprise-ready web service with extensive, flexible policies, redundancy, and fault-tolerance. The zEscrow utility and service are also similar in some other ways to zTrustee, except that zEscrow is intended to share keys with a backup service, while zTrustee blindly and securely stores opaque objects, releasing only to authenticated, allowed clients per policy.

Planet Ubuntu readers may be pleased to hear that our zTrustee servers are currently running Ubuntu 12.04 LTS server, replicated across multiple cloud providers. The RESTful web service is built on top of a suite of high quality open source projects, including: apache2, python wsgi, postgresql, sqlalchemy, postfix, sks, squid, gnupg, and openssl (among others).

The zTrustee client is a lightweight python utility, leveraging libcurl, openssl, and gnupg to send and receive encrypted, signed JSON blobs, to and from one or more zTrustee servers. The client utilizes the zTrustee Python library, which does the hard work, encrypting, decrypting, and processing the messages to and from the zTrustee server. You'll soon be able to interface with zTrustee using either the command line interface, or the Python library directly in your Python scripts.

We've turned our current focus onto Android, while developing a Java interface to zTrustee, so that Java programs and Android applications will soon be able to interface with zTrustee, putting and getting certificates and key material and thereby enabling mobile encryption solutions. Looking a little further out down our road map, we'll also use these Java extensions to support zTrustee clients on iOS, Mac, and Windows.

While I'm big fan and proponent of eCryptfs and zNcrypt, I plainly recognize that there are lots of other ways to encrypt data -- dmcrypt, TrueCrypt, FileVault, BitLocker, HekaFS, among many others. From one perspective, encrypting and decrypting data is now the easy part. Where to store keys, especially in public/private/hybrid cloud environments, is the really hard part. Many people and organizations have punted on that problem. Well as it happens, I like hard problems, and Gazzang likes market opportunities and for that, we're both proud to promote zTrustee as a new solution in this space.

This post is intended as a very basic or brief introduction to the concept, and I'll follow this with a series of examples and tutorials as to how you might use the zTrustee client, library, and mobile interfaces.

Cheers,
:-Dustin

I spent a good half hour on Monday morning with Amber Graner of Linaro. This was my first experience with G+ On Air, a mechanism for conducting video interviews over G+ Hangouts and record them for rebroadcast over YouTube later.

I've known Amber for nearly 4 years now, and she's such a warm, fun, and energetic person. I'm always humbled by her interest and willingness to branch out and learn about new technologies. She's truly an inspiration for us all :-)

In this interview, we talked about Linaro, ARM, Android, Ubuntu, Cloud, Gazzang, Encryption, eCryptfs, and (of course) Byobu :-) Enjoy!

The feedback on eCryptfs' new mascot and logo has been just awesome :-) At the bottom of the last post, we opened a call for name suggestions.

As it turns out, my mom reads my blog from time to time, and with that post, she saw an exercise and opportunity for one of her high school classes. She tasked them with researching eCryptfs and the reasoning behind the new logo. As an extra credit assignment, they were invited to propose names for our tenacious new mascot. These are so much fun, we'll share all of them with you now!

Leading by example, Mrs. Kirkland (aka, my Mom) writes:
I think you should call him...Honey, but play on the "e" ... and the quotes actually look like claws. Hon"e"y. The "e" fits perfectly in the hand. Plus, when he is standing, he really forms an H. I am no artist but I am sure you can see the H in its body. I know you are just looking for a name... but I wanted to show you why I thought the name fit.

Thanks, Mom! The quotes around the "e" does look like claws, and it does refer back to the "e" in eCryptfs.

One of her students, Christopher Bordelon suggests:

I think the name Henry would be the perfect name for the honey badger. The name Henry refers to the noble politician Henry Clay. Henry led a defensive army when it came to the war of 1812. By naming the honey badger Henry it will set the tone of the project to have a well strengthened background. By being a member of the war hawks, Henry was always ready for a battle. He knew he would not be able to be defeated. I feel this is a great name for the honey badger because Henry Clay is a well known political leader in United States history. When people hear this name they will be drawn to it because of the historical accomplishments of Henry Clay. Henry was also known for living a very long time. By Henry living a long time this means that the project will be around for a long time too. He outlived the majority of his fellow leaders. By using his historical context the project will have a face that will never be forgotten. The project will have a mascot with a defensive output that will make customers want to bring their services there.

Thanks, Christopher. We just loved the historical references!

Another of her students, Kristin Seneca, had a different idea:

I really appreciate the new logo; it is a major upgrade to the last. It has more vibrant colors and an all around better design. It isn’t as plain or boring as the key overlapping the pie chart. With a great logo, the project should want a great name to go along with it. This is why Boris the Badger would be a perfect name for your project's new honey badger logo. Boris the Badger would be a great name for eCryptfs' new honey badger logo. This name has significance to the honey badger. A honey badger is fierce and strong, much like what the name means. According to 2000names.com, the name Boris means a battle or fighter. It was derived from the name Bogoris, meaning small. Since the honey badger is one of the smallest and fiercest warrior animals around, I believe that this name definitely suits the eCryptfs honey badger logo. He certainly looks fierce! eCryptfs is a great project and should have a really awesome name to go along with their new logo. Boris the Badger would be the best name for the honey badger because it represents the idea of a warrior or fierce battle. The honey badger is a ferocious warrior animal and will go to great lengths to defend itself, much like the project will go to great lengths to protect files and software.

But our unanimous favorite here at the office was from Mrs. Kirkland's student Blane Palazzo, who wrote:

Reviewing possible names for the new honey badger design, I've decided that "Harvey, the Honey Badger" sounds the best. Not only is this name appropriate because of its beginning with an "H," but the name "Harvey" also means, "battle worthy." When determining which names would be possible for the new logo's design, I kept in mind that defense was a major part of choosing the "fitting" name. Having a defensive name, while at the same time establishing trust, was very important. Not only does the name "Harvey" build trust, it also has a background that allows for an understanding that he "means business." Like a true honey badger, he "takes what's his!" Paul Harvey is famously known for his “Rest of the Story” segment, which was watched by millions until his death at the age of 90. The name Harvey can be related to many things, including the stamina held by Paul Harvey Himself, and the impact of his life felt by millions of Americans. This “Harvey” could be looked to for the “rest of the story” when it comes to protecting software and programs being used. The finality of such a name could be applied to the logo of a project that protects and defends. eCryptfs is software that thrives on protection, and as mentioned in the blog, is a “vibrant and open-sourced” project. Having a fitting name is appropriate when it comes to any new idea or project. In order to be remembered as a project that strives for excellence, the logo has to be focused on and perfected. The name “Harvey” is a name that is not easily forgotten, and provides a significant enough meaning to the new logo.

Well done, Blane. We, here at the Gazzang offices, absolutely love it! And so, I'm pleased to introduce Harvey, the Honey Badger!

Next week is yet another installment of the Ubuntu Developer Week education series. If you been wanting to get involved in Ubuntu or Free Software development, or perhaps just hone your existing skill set, please join us in #ubuntu-classroom on irc.freenode.net Tuesday/Wednesday/Thursday next week. Check the schedule, and hopefully you'll find something that piques your interest.

I'm pleased to note that each member of Gazzang's engineering team will be attending at least two sessions per day! With today's shrinking education budgets, perhaps you can convince your employer to let you attend some excellent, continuing technical education at no additional expense to them. Should be an easy sell ;-)

I will be leading an hour long session on Thursday, February 2nd from 18:30-19:30 UTC -- that's 12:30pm-1:30pm in my local Central Standard Time. My session is on Pair Programming and Code Review in the Cloud.

I've used Pair Programming for years -- ever since I was introduced to the Extreme Programming methodologies in the Tivoli Bootcamp as an intern in 2000. Pair Programming is a relatively simple concept -- two people, one keyboard and screen. It's a great way to teach, learn, and review code. Back then, we were a couple of developers, sitting side by side in the Arboretum in Austin, Texas.

But times have changed! It's highly unlikely that I'm sitting next to the person I need to pair program with. Rather, they're sitting somewhere far across the world.

Welcome to 2012! I'll spend an hour, sharing a screen with a few dozen of you, showing you how some Ubuntu developers work with colleagues across the world, through the Cloud!

I'm going to fire up Amazon's largest instance splurging $2.10 an hour for 60GB of RAM and 16 CPUs. You hardly need this, but I thought it would be fun. If nothing else, drop in and have a look at what this kind of hardware looks like :-) We'll import SSH public keys and users will SSH into a shared Byobu/Tmux session, where I'll demonstrate how to make the most use of our screen resources. We'll split the window horizontally and vertically, look at code side by side, while still tailing log files and conducting builds.

Prerequisites:

• A terminal and an SSH client with Internet access

And to maximize your experience:

• Please run your terminal/SSH client maximized/full-screen
  • I'll open up the classroom IRC channel in there, which you'll be able to read
• Open an account at Launchpad.net and add your public SSH keys
• Print out a copy of the Byobu keyboard shortcuts for quick reference

As a teaser, here's what my terminal currently looks like, and a taste of where we'll get to, in this session. This session can be detached and reattached later, or even by multiple users at the same time.

source code (mount.ecryptfs_private.c and pam_ecryptfs.c). Next, I have a little test window where I'm checking my changes, with a foobar@x220 user logged in, and it's just above a small window where I'm reading some

Hope to Pair Program with you on Thursday!
Cheers,
:-Dustin

The time I have spent around Ubuntu has given me a deep appreciation for the finer points of design, branding, themes, and artwork around software development and user interfaces. From Ubuntu's elegant color schemes and meticulously kerned fonts, to the careful placement and balance of Ubuntu's logos and Canonical's brandmarks, Ubuntu exudes an exquisite level of polish and professionalism, particularly among free software projects.

With this as a backdrop, I have long wanted to refresh and modernize the logos and branding associated with eCryptfs, as an upstream open source project. For over six years, the eCryptfs logo has been an ever-so-trite yellow key overlayed on top of a pie chart.
Yawn :-o I'm pretty sure that "logo" was pulled off of a slide deck for IBM management, when Michael Halcrow and Emily Ratliff originally presented the idea of eCryptfs back in 2004.

So I pitched this idea to my new employer, Gazzang, who, as it turns out, has considerable interest in a healthy eCryptfs community, as it forms the basis for several of our cloud encryption products. Our CEO, Larry, was thrilled by the idea, and gave Heidi (our director of marketing) financial approval to commission the new art from a professional graphic designer.

We felt that eCryptfs, as an active and vibrant open source project, deserved a logo and a mascot that reflects just that. Everyone uses a lock or a key to represent encryption, so we thought we'd do something different. We decided we wanted a stylized animal, in the spirit of Linux's Tux, BSD's Daemon, OpenBSD's Puffy, and of course Ubuntu's every growing zoo!

We settled on the honey badger (Mellivora capensis), inspired by its thick skin and ferocious defensive traits, much as eCryptfs adamantly defends your data against even the most determined attackers (honey badger don't care!) We are, of course, also saluting the running honey badger Internet meme :-)

And the font is modern, crisp, clean, and perhaps a little "techy" even. The "fs" is highlighted, to note the relationship to the filesystem, as well as help demonstrate the pronunciation of the word -- "ecrypt" and then "fs".

Gazzang has contributed all of this artwork to the eCryptfs project under the Creative Commons CC BY-SA 3.0 license. We hope you enjoy it as much as we do :-) Let us know what you think!

Now there is one piece we're still missing. We don't yet have a name for our snarling cryptographic honey badger. So we're putting it out to you... Suggestions? Drop us a comment below!

:-Dustin

Admittedly, using eCryptfs and Ubuntu's Encrypted Home feature in EC2 is a bit circumlocutious. At Gazzang, we're working on making that a bit more seamless, and a lot more secure. But in the meantime, here are some handy instructions on how you can set it up manually for yourself.

But first, why would you want to do this? Good question! Bear in mind that by using EC2 and storing any data there, you're putting a considerable amount of trust in Amazon already. They own the hardware and the hypervisor. They are running a modified Linux/Xen kernel that you cannot even audit, if you wanted to. They haven't released the sourced to that modified Linux kernel, so don't deceive yourself -- their instrumented kernels could be logging your every keystroke. Hopefully not. But you don't know that.

So what can you do? What good is eCryptfs here? Well, if you transparently read and write your data through an eCryptfs encryption/decryption layer, you can add a measurable amount of confidence and security that your data will at least be encrypted when it's at rest, once it lands on a spinning hard disk somewhere in an Amazon data center. In this world of cloud trust, you're explicitly trusting Amazon to "do the right thing" and take reasonable precautions. Amazon is huge, and has a tremendous amount to lose by acting deceptively. But you can't say the same for every single individual between you and your data. In other words, you don't necessarily trust every individual that might brush past your data. Hard disks get stolen and sold on eBay, they're returned to the manufacturer for repair, donated to Goodwill or schools, recycled, repurposed, and reused. So if you could trivially ensure that your bytes are encrypted before being written to disk, would you? Well, as you see below, it's not quite trivial yet, but it is very much possible. Stay tuned here and watch this area of technology evolve. In the meantime, give this a shot...

First, start an Ubuntu VM in EC2. I use the cloud-sandbox command from lp:bikeshed. I'm sure you have your own methods.

Next, SSH into your new VM and install ecryptfs-utils.

sudo apt-get install ecryptfs-utils

Next, you must set a login password for the Ubuntu user. Note that you do not have to enable PasswordAuthentication in /etc/ssh/sshd_config (though you might choose to). As always, make sure you choose a strong passphrase. I recommend at the very least 12 characters, with upper case, lower case, and numbers. You know how to choose a good password. The more important it is that your data stay private, the better the password should be ;-)

sudo passwd ubuntu

Exit byobu, or any other programs you might be running as your ubuntu user, and change out of your $HOME directory, and migrate your home directory. However, if you've encrypted all of your $HOME, you MUST move your .ssh directory out, so that your authorized keys file is not encrypted!!! Make sure you run all of the following commands sequentially, and without terminating your SSH connection, or else you might find yourself locked out of your instance :-)

cd / ; sudo ecryptfs-migrate-home -u ubuntu

sudo ln -s /home/.ecryptfs/ubuntu/.ssh $HOME/

ecryptfs-mount-private
cd $HOME
mv $HOME/.ssh /home/.ecryptfs/ubuntu/

ln -s /home/.ecryptfs/ubuntu/.ssh $HOME/

If that completes successfully, we can clean up our backup of our unencrypted home directory.

sudo rm -rf /home/ubuntu.*

Alternatively, might might choose just to encrypt one private directory, instead of migrating all of your home. To do so, use:

ecryptfs-setup-private

Finally, we will want to be prompted for our login password at every login to automatically mount our home directory, so let's also create a ".profile" in our unencrypted home directory.

ecryptfs-umount-private
echo "ecryptfs-mount-private; . $HOME/.profile; cd" | sudo tee $HOME/.profile
ecryptfs-mount-private

Alright! At this point, we should be able to exit all of our shells and SSH back into our EC2 instance. The SSH public key authentication will get us onto the machine, and then our .profile script should prompt us for our login passphrase and automatically mount our encrypted home directory.

The data that actually gets written to your root ext4 filesystem on /dev/xvda1 are the files that you can find in /home/.ecryptfs/ubuntu/.Private/, which should look something like this:

ubuntu@ip-10-194-246-143:~$ ll /home/.ecryptfs/ubuntu/.Private/
total 68
drwx------ 3 ubuntu ubuntu  4096 Dec 22 18:54 ./
drwxr-xr-x 5 ubuntu ubuntu  4096 Dec 22 18:46 ../
lrwxrwxrwx 1 ubuntu ubuntu   124 Dec 22 18:42 ECRYPTFS_FNEK_ENCRYPTED.FWbSgDSRezlYtETTxmAwbGjiN4WOMkt-2hHi1R07VV4a9quAsP3ATb2JK--- -> ECRYPTFS_FNEK_ENCRYPTED.FYbSgDSRezlYtETTxmAwbGjiN4WOMkt-2hHiSRA-6SgbLQ.LtWP2pwGZY57PtU2wAgzLn-ECMilfrp9dp0YUYlTDNwY6P764.gPo
-rw-r--r-- 1 ubuntu ubuntu 12288 Dec  1 12:50 ECRYPTFS_FNEK_ENCRYPTED.FWbSgDSRezlYtETTxmAwbGjiN4WOMkt-2hHi9KCXyAtK1PsV4KirBxb8fk--
drwx------ 2 ubuntu ubuntu  4096 Dec 22 17:32 ECRYPTFS_FNEK_ENCRYPTED.FWbSgDSRezlYtETTxmAwbGjiN4WOMkt-2hHicFvfubbvnebsd2N8jh9vRU--/
-rw-r--r-- 1 ubuntu ubuntu 12288 Dec  1 12:50 ECRYPTFS_FNEK_ENCRYPTED.FWbSgDSRezlYtETTxmAwbGjiN4WOMkt-2hHifCuJCnlfaXjU4QlrUWfhIU--
-rw-r--r-- 1 ubuntu ubuntu 12288 Dec  1 12:50 ECRYPTFS_FNEK_ENCRYPTED.FWbSgDSRezlYtETTxmAwbGjiN4WOMkt-2hHiNgxmEEQUk9nI3uOlsQkCHk--
lrwxrwxrwx 1 ubuntu ubuntu   104 Dec 22 18:42 ECRYPTFS_FNEK_ENCRYPTED.FWbSgDSRezlYtETTxmAwbGjiN4WOMkt-2hHipvXKHoAMUybcfPOQYgm1WE-- -> ECRYPTFS_FNEK_ENCRYPTED.FXbSgDSRezlYtETTxmAwbGjiN4WOMkt-2hHif.b7-V31EJPzRLnx.vfW9dIwfbnZuIcdSIqqNTvonyo-
lrwxrwxrwx 1 root   root     104 Dec 22 18:54 ECRYPTFS_FNEK_ENCRYPTED.FWbSgDSRezlYtETTxmAwbGjiN4WOMkt-2hHisXvcg5obbXbibbufq7QjyE-- -> ECRYPTFS_FNEK_ENCRYPTED.FXbSgDSRezlYtETTxmAwbGjiN4WOMkt-2hHiGNrAq2Ud8N9P5xVz2YssSWo-.u4wRtBbZLQLIeG-0I2-
-rw-r--r-- 1 ubuntu ubuntu  8192 Dec 22 17:32 ECRYPTFS_FNEK_ENCRYPTED.FXbSgDSRezlYtETTxmAwbGjiN4WOMkt-2hHivZ3-rM86jHHkrHcJAXqMkfoOaMkowIPainVLMFWajCg-

This is what you're hoping your attacker, the unsavory individual who comes into contact with one of those magic cloud hard drives containing your data, sees. These are the encrypted file names, and the file contents are just as unreadable without the necessary keys!

Enjoy,

:-Dustin