OpenSolaris Dedupe for home use
Posted by Sam
I know that dedupe will save a ton of space at work. We have lots of Subversion repos checked out on servers and lots of similar files like swfaddress and jquery that are reused in a lot of our projects. But I've been curious how useful it would be with my personal files. Here's what I've found when I deduped files from my home directory.
- Photos: 5.5GB savings (34GB pre-dedupe/28.5 post-dedupe)
- Music: No savings
- Projects: 1GB savings (2.2GB pre-dedupe/1.22 post-dedupe)
As you can see there are some decent space savings to be had even for home/power users. The savings would be even greater if I had enabled compression as well but I was more curious about dedupe since compression is fairly common these days.
Tags: zfs opensolaris deduplication
Dedupe in the real world
Posted by Sam
For those that haven't heard dedupe has made it's way into OpenSolaris 128a. You can download it at Genunix. To test what kind of savings we might expect to see in the real world I fired a VirtualBox with OpenSolaris 128a and turned on dedupe and compression (gzip level 6). Then I rsync'd 10 gigs worth of files from our staging server. The results? Dedupe dropped that 10 gigs to 6 gigs and compression dropped another 2 gigs which left us at 4 gigs. So dedupe gave us a 40% reduction and compression gave us another 20%. Not too shabby, especially when it takes about 10 seconds to enable it. A little bird told me that they are expecting dedupe in the Sun 7xxx series of storage by the end of the year. The Sun boxes are already the winner for best value and with that they jump even further out front!
Tags: opensolaris solaris deduplication zfs