Installing Solaris 10 update 4 on Dell PowerEdge 2950
Posted by Sam
I flew into Oklahoma City last night to install Solaris 10 on a new Dell PowerEdge 2950. I've been putting Solaris 10 every where I can because it's features like DTrace, ZFS and Zones blow away anything found on Linux. But installing Solaris on non Sun hardware can be a challenge. I'd checked the HCL and it checked out ok as long as you added the appropriate drivers but I knew there were still probably going to be problems. The Perc controller that Dell uses isn't recognized by the Solaris installer so I first tried to install the LSI drivers, but Solaris still couldn't see the drives. So after a little more searching I found drivers from The University of Queensland and they worked without a hitch. After the install I added the Broadcom NetXtreme II drivers to get the networking support going. The Broadcom drivers installed as expected. So far everything is running great. I guess we'll know for sure once this thing goes into production, but as of right now this looks like a successful Solaris installation.
Barracuda Load Balancer 340 Review
Posted by Sam
Last week I tried out the Barracuda 340 Load Balancer. In more informal conversations (is that possible?) I've been referring to the Barracuda as the Barracrappy because that's the impression I was left with. A few weeks ago our Cisco LocalDirector died a sad and lonely death in our NetStandard data center. The LocalDirector had been working flawlessly for better than five years without a single problem and then one day it dies with very little warning. Nothing lasts forever so this wasn't shocking, but it was unexpected. Once the painful recovery process was over I immediately began looking at alternatives. Of course I'd love to go with a BIG-IP, but the cost really isn't reasonable and frankly most of the features are unnecessary for our needs. And so began the search for an inexpensive solution that would be a drop in replacement for the LocalDirector.
After a lot of searching I had basically narrowed down the search to the Barracuda 340 Load Balancer and the Kemp LoadMaster 1500. Both said they would do Layer 4 switching, which I needed so that we could simply drop in the load balancer and have it behave the same as the LocalDirector did. Having to re-ip all the boxes would be a pain and highly disruptive and I really didn't see the benefit. After looking at screenshots and demos I settled on the Barracuda. The price was excellent and as an added bonus it had Intrusion Prevention System which certainly isn't a necessity, but more of a bonus. The other deciding factor was the Barracuda name. I'd never used their products before but I'd heard of them and I like their philosophy on always reaching a human on the phone. So I set out to order the Barracuda 340.
I had a couple problems getting the demo unit from Barracuda. First, I talked to a guy at Barracuda who took my info and was supposed to pass it along to a reseller. I let him know that we were replacing a busted load balancer and I was in a hurry. A full day passed and I didn't hear anything, so I called Barracuda again and talked to somebody else who gave me the information for a reseller nearby and this got the ball rolling. I cut a PO to the reseller who promised to have the unit overnighted. The next day I found out that Barracuda had shipped the load balancer through normal shipping instead of the overnight shipping I'd requested and paid for. So we waited another day for Barracuda to ship another one, overnight this time. Unfortunately, this was only the beginning of the problems. And if this had been the most problematic part of the ordeal it wouldn't have been a big deal.
The demo unit finally arrived, but it was shipped to the wrong person in the company. After I tracked down the unit we immediately went to work to get it configured. My expectation was that we'd simply plug it in, turn it on and after 15-20 minutes of configuration we'd have a load balancer passing traffic. Boy was I wrong! Having configured the LocalDirector through the command line for years I expected the web interface that came with the Barracuda to be a cake walk and it mostly was, except for the fact that it didn't work. We tried everything and I mean everything to get it to work. Finally we called tech support. This really shouldn't have been necessary because like I said it should have been dead simply. After waiting for at least 4 hours for tech support to call me back they finally did, but because I had left for the day I had to have them call me back the next day. Once they called me back I told them what I was trying to do and the guy told me that was impossible in the current setup and when I protested he realized that I had a load balancer and not whatever the heck he thought I had. Somebody else had entered the trouble ticket and they entered it wrong so he was working with faulty information. Grrrr. He transfered me to somebody that can trouble shoot a load balancer. Good call!
Once, I got on the phone with the correct person I was hoping that he'd have some simple solution to our problem. It seems like such a simple thing. The load balancer acts as a bridge, watches the traffic and intercepts and rewrites the traffic it's supposed to handle. I explain our set up to the guy and immediately he grills me on why I'm using Layer 4 load balancing. I explain our setup and that I don't want to re-ip our setup and their instructions on setting up direct server return were less than helpful. And to top it all off they only covered making the changes to Linux. I have happily left the days of Linux behind and we are nearly completely Solaris and Windows. I'd love to leave Windows behind as well but that's currently not an option since our biggest hosting client has to run on Windows. So the direct server return route didn't seem like a viable option and source natting (snat) was definitely out. So I explained most of this and basically told the guy this is how I want it to work, your web site said it will, let's get it working.
The support guy asks all the usual questions and then realizes that our firmware and energize versions are out of date. Energize is the updates for intrusion prevention. Since we have that disabled I'm pretty sure that's not the problem, but the firmware update definitely couldn't hurt. Because we have this on an isolated network (two laptops, two switches and the Barracuda) we have to jump through a bunch of hoops to get this connected to the network. We finally get it setup and update the firmware and the energize versions and try our load balancing setup again. This time it works. Hurray! Unfortunately, that's the last of the good news.
After a couple of minutes of passing traffic through the virtual IP address the load balancer just completely stopped passing traffic for that virtual IP. There was nothing in the logs and no clues to be found. We added a second virtual IP address going to the same two real machines and the same thing happened. It passed traffic for a few minutes and then stopped. We tried a third virtual IP address going to same physical machines and this one worked and seemed to keep on working. I tried deleting the other virtual IP addresses and re-adding the virtual IP addresses and they wouldn't pass traffic to save the world. At this point I was so frustrated with this box and I had zero confidence that I immediately boxed it up and sent it back. When something just doesn't work it's one thing, but when it selectively works that's a very disconcerting thing. Like I said I had zero confidence that this would perform correctly in production and a poorly behaving load balancer is worse than no load balancer.
After packing up the Barracrappy, um I mean Barracuda load balancer I got in touch with my second choice Kemp Technologies' LoadMaster. I'm currently in the process of ordering one and once I've had a chance to play with it I will share my thoughts on it. Hopefully it will be a better experience than the Barracuda load balancer. It certainly couldn't be much worse.
Tags: loadbalancer
Barkley on 20/20
Posted by Sam
Background
Everybody that knows me knows that I work at a pretty cool company. A company that really values it's employees. A company that goes above and beyond. A company that thinks outside the box. Barkley isn't particularly well known, especially outside the ad world, but many of our clients are. But our little secret is about to change in a big way. ABC's 20/20 is going to be documenting Barkley's culture and environment. I thought I'd just run through a few of the interesting things that make Barkley stand out in my mind.
Fun
Every year Barkley has a kidnap. They shutdown the office, load all the employees up on a bunch of busses and take us somewhere fun for the afternoon. Last year was Dave & Buster's and this year was PowerPlay. What PowerPlay's website lacks in design in more than makes up for by having Kansas City's only WhirlyBall court. The great thing about kidnaps, besides the obvious benefit of not being at work, is that you get a chance to connect with people you don't see everyday and of course mingle with people you might not even know.
Barkley also has some fun built into the building. For starters we have a ping pong table, pool table and shuffle board. If it's a nice day you can head out to the deck that the fourth floor walks out onto. And if you get thirsty we have beer and Coke products on tap. And if you get hungry we have a cooler full of Blue Bunny ice cream as well as a popcorn popper. Oh and did I mention this is all free?!
Time Off
Like most companies Barkley has a decent vacation policy, but of course they aren't content with just decent. So on top of the normal vacation the office is closed between Christmas and New Years. That's right. Closed. You don't have to take any vacation. It's above and beyond whatever vacation you have accrued. It doesn't matter if you've been here one week or 100 years. And if that's enough we also get two volunteers days a year. Go volunteer for Habitat for Humanity, Boy Scouts of America or your local church. It doesn't matter. As long as it's a non-profit you can use your two days to do whatever.
Personal and Professional Growth
On top of everything Barkley does to keep work fresh and fun they also care about each employee's growth. Every year that I've been here we've had a Creative Symposium where they bring in guest speakers for a wide variety of talks. In years past we've had Ze Frank, Richard Tait (the co-inventor of Cranium) and numerous other well known and respected individuals whose names I can't remember at the moment.
One Hundred Dollar Bill
I nearly forgot the one hundred dollar bill story! At our last company meeting we had the usual question and answer round at the end of the meeting. Now, the way this normally goes is the big wigs of the company ask if anybody has any questions. We all sit there with dumb looks on our face. They say meeting dismissed. This time, when the question went forth one of the employees spoke up and asked if we could all have $100. I checked with said employee and I was assured that this was completely unrehearsed. She just wanted to know if we could all have an extra $100 for no apparent reason. How one gets the nerve to ask such a question in front of the whole company I'll never know, but hey that's Patty for you. Anyway, the thing is, the heads of the company kind of look at each other for a few seconds and agree to give the whole company a $100 bonus. Given the size of the company that's about thirty thousand dollars! They went to the bank that afternoon and handed everybody a crisp hundred dollar bill that afternoon. I wonder what the bank thought of that? And it was completely on a whim.
So that's the kind of company Barkley is. One that really cares about it's employees. Sure, it's got it's problems like every other company, but if you are having a bad day you can take it out on the ping pong table instead of going postal.
It Airs...
20/20 camera crews are here tomorrow and the show is supposed to air in early November. I'll post more when I know the exact date. Man, this might be the one time in my life when I wish I had a DVR!
Tags: barkley
Join the Barkley Denim Day Team
Posted by Sam
Lee National Denim Day is Friday, October 5th. One week from today. Barkley has set up a team to bring in donations and if you'd like to donate I encourage you to donate to our team. You can find more posts about Denim Day under the Life category.
Tags: life
More Denim Day
Posted by Sam
A few posts ago I put up a Denim Day video from YouTube. I just ran across an older blog from Jeff Risley, one of my partners at Barkley, who talks a little more in depth about what we are doing for this important cause. Denim Day is an important cause and it's coming up soon. I encourage you to join a team and wear your jeans on October 5th to support this great effort.
Tags: life