Lower power usage FreeNAS
For all of us that run our own home labs we know (or should know) the price of what it costs us to run our equipment every month. Previously I spent some time with a Kill-A-Watt meter performing an energy audit on my equipment and found that it uses about 650 watt/hrs of power. At 12 cents per kwh I’m spending $78/month on my lab, over $900/year. 60% of our monthly electric bill goes towards my work hobby.
I’ve tried to slim some servers down by moving them to a laptop running Debian & VMware Server but there are creep-cpu-usage problems involved with that. The laptop running VMware Server wasn’t all the beefy either, a Athlon 2800+ with 512mb of RAM. 1 or 2 guest OSs before things start to slow down. No RAID setup, no firewire, no expandability; a good stop-gap system until I could figure out what I wanted to do.
Through a lot of reading I’ve decided that an VMware ESXi system was the way to go if I wanted to virtualize my servers and have enough room left over to delve into Cisco Call-Managers, IDS/IPS systems, Nessus scanners, Asterisk, Olive’s, and just stand alone systems for trying new things. I also needed a NAS to centralize all the usb/firewire drives that I have laying about. The NAS needed to have RAID capabilities and act as an iSCSI target so I’ve chosen to go with FreeNAS.
In the end I needed everything to be as energy efficient as possible. I’m going to do what ever I can to cut down my hobby’s power bill.
Ticket to FreeNAS
Processor: Celeron 430 Conroe-L 1.8Ghz
Motherboard: Gigabyte GA-EP453T-USB3
RAM: DDR3 2GB Kingston
HDD: Hitachi Deskstar 2TB x6
Ethernet: Rosewill RC-401-EX 1GB x3
Video: ATI HIS Radeon 64mb
The system came together around the 35watt Conroe-L processor. I wanted to find a motherboard with built in video so I wouldn’t have to buy an extra card, but none of the LGA775s had enough fan headers, PCIe 1x slots, SATA ports, etc. I had read an article called “Everything You Need to Know About The Motherboard Voltage Regulator Circuit” that went into depth on how to identify quality parts of the motherboard and what to look for when buying one. I had never put so much thought into the things you don’t ’shop’ for like Ferrite Core Chokes, Lower RDS(on) Mosfets & Solid Capacitors as I did after I read that article. Needless to say the Gigabyte motherboard has it all, AND USB3.0. Something new that I don’t have any of yet, but it’ll be nice for a file server to have this kind of access for when I do.
RAM ended up being what ever was the cheapest DDR3 was at the time. I did a lot of reading when it came which hard drive to buy. Since the evolution of my new system is revolving around energy efficiency buying a Western Digital ‘Green’ drive was a no brainer; or so I thought. The price was right, lower power consumption was good but all the reviews that had to do with using them in a RAID configuration ended in disappointment. A WD article on TLER goes into more depth as to why the ‘Green’ drives don’t play well in a RAIDed setup. The WD RE drives were out of my league for the space that I wanted. I was ready to chalk everybody else’s experience to their lack of it and buy the ‘Green’ drives anyway but when I went to place the order they were no longer on sale. Thanks goodness for that! The next day Hitachi started to run a mail-in rebate for their Deskstar series, so I ended up picking up a few of those.
The HDD were only going to be used for storage so I decided to run FreeNAS from a CF Card by way of an IDE -> CF Card converter. Turns out a lot of people are using the same setup as me, ESX/ESXi & FreeNAS. From here I figured out how to install FreeNAS from my laptop to the CF Card without having to go through the typical burn a CD, find a CDROM, configure the bios, boot, install, eject, reboot, etc. A quick ‘dd’ and FreeNAS was written to the CF Card.
I’ve already got an iStarUSA D300 case that had an older system that would spontaneously restart. Not good. One thing that I’ve begun to accept as fact, is that investing in a few 3u or 4u cases is a good idea. I can’t count the amount of times that I’ve tried to fit a new off the shelf motherboard in Compaq or Dell tower case and have to cut out parts of it to make it fit. Good 3u/4u cases are a necessity. Looking back on things I think I should of gotten 4u cases instead of 3u just for the fact that I would be able to use desktop/tower standard power supplies. These D300 cases offer 2x 5.25 and 1x 3.5 per side. iStarUSA makes a 2×5.25 to 3×3.5 conversion bay that offers trays, hot swap ability, system fan, alarm, etc. Had to get two of these. They ended up being more expensive then when I speced out a system last year. In fact everything seems more expensive this year then last.
Something I didn’t realise is that power supplies have an efficiency factor that plays a huge roll in how much electricity computers guzzle up. 80Plus (http://en.wikipedia.org/wiki/80_PLUS) is initiative that helps promote greater efficiency in power supplies. I needed to buy a PSU that was rated 80Plus or higher. Regular sized power supplies were out since I had 3u cases. I couldn’t find any 3u 80Plus PSU but I did find a 2u, again by iStarUSA. Below is is the data that I collected comparing the standard 3u PSU with the new 2u 80Plus PSU:
CMOS Idling FreeNAS Idling
watts amps watts amps
AMD Athlon 750mhz
72 0.92 67 0.89 3u PSU
58 0.5 56 0.48 2u 80Plus PSU
Intel Conroe-L 1.8ghz
61 0.8 50 0.65 3u PSU
45 0.59 3u PSU w/o video card
49 0.42 39 0.34 2u 80Plus PSU
35 0.31 2u 80Plus PSU w/o video card
40 0.36 2u 80Plus +2x case fans
44 0.37 2u 80Plus +2x case, 3x 1GB eth
103 0.84 2u 80Plus +2x case fans, 3x 1GB eth, 2x HDD chasis with fans, 6x HDD
I’m currently unable to test the watt draw because the new server is doing TB backups via a 100mb connection but I’m pretty sure that without a video card my FreeNAS box will idle just under 100watts of usage which is just incredible! I read a lot about staggered HDD spin up so as not to tax the PSU and maybe blow a fuse but I only registered 1.5amps when they spun up during boot. The server is currently rsyncing data from the older file server that this one will replace and it is only drawing 105 – 112 watts while in full use. For the past 12 hours it has averaged 102.5 watts/hr. All the research I did prior to purchasing everything looked good on paper and I knew I was going to be saving some money but I had no idea how much.
The overall setup and configuring of FreeNAS is really slick. The web interface is very responsive and looks really thought out. There are plenty of articles out there on how to setup FreeNAS, I just needed to pass along the concept of building an energy efficient server. Start with the Conroe-L 1.8ghz processor and an 80Plus PSU and you really can’t go wrong for something that is going to be on 24/7.
Once I can empty my other 3u case I’ll be able to install my VMware server, now that’ll be a blast.