Dedupe in the real world
Posted by Sam
For those that haven't heard dedupe has made it's way into OpenSolaris 128a. You can download it at Genunix. To test what kind of savings we might expect to see in the real world I fired a VirtualBox with OpenSolaris 128a and turned on dedupe and compression (gzip level 6). Then I rsync'd 10 gigs worth of files from our staging server. The results? Dedupe dropped that 10 gigs to 6 gigs and compression dropped another 2 gigs which left us at 4 gigs. So dedupe gave us a 40% reduction and compression gave us another 20%. Not too shabby, especially when it takes about 10 seconds to enable it. A little bird told me that they are expecting dedupe in the Sun 7xxx series of storage by the end of the year. The Sun boxes are already the winner for best value and with that they jump even further out front!
Tags: opensolaris solaris deduplication zfs
Installing RMagick on Solaris
Posted by Sam
It seems that installing RMagick is a pain on any OS but recently installing on Solaris seemed a bit impossible. Previously we were using ImageMagick 5.2.9 and an older version of RMagick. We've recently switched from Blastwave to OpenCSW to get our Solaris package fix. With that came a new version of ImageMagick and new problems. Turns out ImageMagick from OpenCSW was using compiled using OpenMP but support for that isn't in the current version of GCC that OpenCSW has. Here are the steps I took to install RMagick. There might be a simpler way but this worked for me.
wget http://rubyforge.org/frs/download.php/51093/rmagick-2.9.1.gem
gem unpack rmagick-2.9.1.gem
Change line #139 rmagick-2.9.1/ext/RMagick/extconf.rb to: $CFLAGS = ENV["CFLAGS"].to_s + " -I/opt/csw/include/ImageMagick"
cd rmagick-2.9.1
gem build rmagick.gemspec
gem install rmagick-2.9.1.gem
Installing Phusion Passenger on Solaris
Posted by Sam
I know there are other places on the web where you can piece this together but I thought I'd throw up how I got Phusion Passenger compiled on a Solaris 10 zone with Blastwave packages. Well technically the packages are from opencsw.com but they should behave exactly the same (for now). Also, as of the time of this writing the most recent version (2.0.6) didn't compile. I had to grab the trunk from GitHub.
bash-3.00# crle -l /lib:/usr/lib:/opt/csw/lib
bash-3.00# export PATH=/opt/csw/apache2/bin:/opt/csw/apache2/sbin:$PATH
bash-3.00# export APXS2=/opt/csw/apache2/sbin/apxs
bash-3.00# export APR_CONFIG=/opt/csw/apache2/bin/apr-1-config
bash-3.00# gem install passenger
bash-3.00# ./bin/passenger-install-apache2-module
The first line is not something I use regularly and you definitely want to use it with care. Basically it tells the system where to find libraries. Before I added this line Ruby Enterprise Edition and Phusion couldn't find their runtime libraries unless I set LD_LIBRARY_PATH. Relying on LD_LIBRARY_PATH isn't a good idea and didn't work for daemons so I used crle instead. You should run crle first to make sure you don't have other paths set before overriding it.
Once everything finishes you should have a working Passenger install. Note this has only been tested on Solaris 10 x86 update 6. Also, one machine had the gnu version of ld installed and it was causing a weird error. Once I removed SUNWtoo it compiled fine.
The 5 Minute Guide to Securing Solaris
Posted by Sam
Disclaimer
This is a super quick guide to securing a Solaris server. It's not in depth and I highly recommend taking a deeper look at the technologies that are used.
Install Solaris Security Toolkit (formerly called JASS)
- Download Solaris Security Toolkit
- install and run
# pkgadd -d . SUNWjass
# /opt/SUNWjass/bin
# ./jass-execute secure.driver
A ton of stuff will scroll by. If you need remote access be sure to edit the tcp wrappers allow file otherwise you'll be locked out of the box. Also, if you remotely log in to SSH as root you'll need to allow root access and restart SSH. I always double check that I can remotely log in before closing the current console.
Links for Solaris Security Toolkit
Activate BSM Auditing
Auditing will let you watch as little or as much as you want on your box. It's also zone aware so you can see exactly what's going on in a specific zone. Here's a quick run through on how to turn it on and what I'm monitoring on my boxes. Keep in mind if you have a busy box these settings can produce some very large logs and slow down your system. Take a look at the links to learn exactly what these settings do and then decide how much logging you need.
- Edit /etc/security/audit_control so that it looks like this:
dir:/var/audit
flags:lo,ex,ad,pc,fm,fw,-fc,-fd,-fr
minfree:10
naflags:lo,ex,ad - Edit /etc/security/audit_startup so it looks like this:
/usr/bin/echo "Starting BSM services."
/usr/sbin/auditconfig -setpolicy +cnt
/usr/sbin/auditconfig -setpolicy +argv,arge
/usr/sbin/auditconfig -setpolicy +zonename
/usr/sbin/auditconfig -conf
/usr/sbin/auditconfig -aconf
- Run the bsmconv script
/etc/security/bsmconf - Add the following line to the crontab
0 0 * * * /usr/sbin/audit -n - Reboot and check out your newly created logs in /var/audit
Links for BSM Auditing
Showing Blastwave Packages with Upgrades Available
Posted by Sam
I love love love Blastwave. I couldn't imagine using Solaris without it. One of the annoying things though is if you want to see if newer packages are available you have to wade through every package that they have. And they have a lot. So here is a super simple one liner for showing just the packages that need to be upgraded.
pkg-get -c | grep -v "Not installed" | grep -v SAME | \
mailx -s "Blastwave Updates" yourname@example.com
And since you are probably using Solaris Zones here is a simple script that will look in all non-global zones and email out a report. Currently this sends one email per zone but that could easily be changed to send one email per physical box.
#!/usr/bin/sh
for zone in `/usr/sbin/zoneadm list | /usr/bin/grep -v global`; do
zlogin $zone "/opt/csw/bin/pkg-get -c | \
/usr/bin/grep -v \"Not installed\" | \
/usr/bin/grep -v SAME | \
/usr/bin/mailx -s \"Blastwave Updates for $zone\" yourname@example.com"
done