Supermicro X9DRL-IF Motherboard

Supports up to 512 GB of RAM

2x Intel Xeon E5-2670 CPUs. 8 cores each, 2.6 GHz

Unfortunately, I realized after ordering that these are the C1 stepping (SR0H8), which has a bug affecting virtualization, so I have to return these and order CPUs of the C2 stepping (SR0KX). Bah.

2x CPU coolers

4x 8GB DDR3-1600 ECC RAM. 32GB total

2x 120 GB SSD's for the host OS

2x 2.5" hot swap caddies to hold the two OS drives

OS drives mounted in caddies

Supermicro SC-833 3U rackmount chassis

With 2+1 redundant 760W power supply

SATA Backplane (I'm guessing SATA 2, supporting 3 Gbps)

This chassis was actually sold as a complete, functioning server

But the included motherboard, CPU, and RAM were all old and outdated, so out they came.

Supermicro X8SAX, Xeon E5504, and 6 GB RAM

Also includes a 3Ware 9650SE RAID controller with battery-backed cache and ATI RADEON HD5450 video card.

Out with the old

New motherboard installed and wired up

Just as I'm about to install the CPUs, I have a thought -- one of the mounting holes didn't have a standoff under it. That's not a big deal, but it occurred to me that the standoff might be somewhere else instead, touching the bottom of the motherboard.

Sure enough, I pull the board back out and realize several of the RAM socket pins are flattened

I forgot to take a picture of the bent pins. This is after straightening them as much as possible.

The offending standoff

It's permanently installed, can't be unscrewed like some of the others.

So, the natural solution is a Dremel

Much better

Motherboard reinstalled, and CPUs installed

These are still the SR0H8 models

I haven't received the replacements yet, so I installed these for now so I can get started configuring the server.

The next roadblock: These CPU coolers were not designed for a rackmount chassis

When I first ordered my components, I planned to build this in the same tower case that my old server lived in. The plan changed afterwards, and I forgot to reconsider the CPU coolers.

So I submitted another RMA request, and ordered some Noctua 3U coolers instead

Just a reference shot for the amount of thermal paste I used, for future reference

Noctua coolers installed

They fit perfectly, and actually touch each other, but that will just help ensure even heat distribution, or something. Sure.

The replacement CPUs arrived, so I removed the old ones

Looks like the paste didn't quite reach the edges, so I'll use a little more next time.

The SR0KX CPUs are installed

Using just a tiny bit more paste this time

CPUs, RAM, and wiring installed

I was dismayed to discover the 2.5" caddies wouldn't go into the slots properly

Then I realized I had mounted the SSD's upside down, oops. As a bonus, now the label with serial number is visible.

One of the three redundant PSUs is DOA

Even though the PSU as a whole is rated for 760W, each of the three modules is a 380W PSU

I guess it combines the power provided by two of the three units at any given time. Never seen that sort of setup before.

The VM host (ummon) is now happily in its place in the rack alongside the NAS (keats)

The two servers are using 5 power outlets between them, sheesh

2x Chelsio S320E-CX4 10GBe cards

In order to enable 10gb Ethernet between the VM host and the NAS, I ordered a pair of dual-port 10GBe cards and some DAC Twinax cables. However, when they arrive I realize I accidentally ordered the Infiniband version of the cards, which is an older, outdated technology that doesn't work reliably under FreeNAS. Time for yet another RMA!

2x Cisco SFP-H10GB-CU3M 10GBe DAC Twinax cables

Fortunately, the cables are correct.

2x Chelsio S320E-SR 10gbe cards

These should be the right ones to use with the DAC twinax cables I ordered.

Both 10gbe links connected, because why not?

Unfortunately, I haven't had luck getting the 10gbe link working. I don't know what I'm doing regarding 10gbe, but I was told it might be due to the DAC's. I ordered some fiber optic interfaces and cable instead. We'll see.

Chelsio-compatible transcievers from Fiberstore

2x 1m OM3 Fiber