NEW LSI 9300-16i 12Gbps SAS-3 PCIe x8 HBA P16 IT mode ZFS TrueNAS UNRAID (WHL #94)
Here’s a quick and simple one: Cooling advice for 16-port SAS cards.
The LSI 9300-16i cards have gotten surprisingly cheap over the past months. Dirt cheap, in fact. All-16 port SAS cards are traditionally pretty pricey, even though they are “just” two 8-port cards on one PCB. In the past 10, maybe 15 years, there was never a time when buying a 16-port card (used or new) was cheaper than buying two 8s. Sure, sometimes people needed a single-card solution (availability of PCIe slots, PCIe lanes, or space in general), but the usual recommendation for 16-bay chassis like my Supermicro 836 was always to buy 2×8 instead of 1×16.
Well, that changed very recently. I managed to get one 9300-16i for 53.54€ delivered, including all shipping fees and taxes. Not a lucky eBay auction, but a regular fixed-price item. It also said “new”, but I wouldn’t bet on that, despite being absolutely free of any dust or signs of wear. They’re still available in the 60 to 80€ range – and regular -8i cards are about the same. So I upgraded my old 2000 series 8i cards with one of those, and hope for around the same money when selling them. That would be a fantastic upgrade…
Anyway, the issue with the 9300 is that they’re somewhat modern RAID cards. The old LSI 2000 (or even 1000 series) did have heat sinks, but the usual air flow in home servers was enough to keep them at modest temperatures. The 9300 = SAS3008 however is a significant performance bump: PCIe 3.0×8 instead of 2.0 x8 of the 2000 series (except for the 2308 refresh), SAS 12Gb/s instead of 6Gb/s (SATA still 6Gb/s for obvious reasons), 1.2 GHz PowerPC 476 processor (up from an 800MHz 440 that was introduced in 1999), and one million IOPS, up from 200k or 465k of the 2208 chip, depending on their SSD optimization firmware tweaks. SAS2108 was in the order of 80/160k IOPS.
That comes at a price: Heat generation, especially under full load. The 9300-8i is a nominal 13W card, with worst case specced at 19W. The 9300-16i is a nominal 27W card, and the 9300-16e is 28.6W. All of them require 200LFM air flow, which isn’t easy in a home system that doesn’t sound like a starting jet fighter.
But, unlike previous cards that had small heat sinks directly over their CPUs only, the 9300-16i now has an extensive heat sink that covers a significant part of the card. SAS2308-8e (from #P34F1) for reference:
vs. 3008, 16i:
And that heat sink is a) deep enough for screws, b) made with fin spacing that fits M3 screws, and c) also matches the mounting holes of 80mm fans!
No need to use extra fan mounts (people fancy these), or chassis air guides (I had some with my previous packed 8i+8i+8e+10GbE setup), the solution is getting an M3 tap or yankee equivalent, using a fan as drill guide, and bolting down a fan directly onto the card. No disassembly of the heat sink required, just make sure no metal shavings remain on the card and no SMD part on the back gets damaged while tapping and installing the fan.
(I know, my finest out-of-focus photo of the entire week…)
Fins are only scraped but do hold screws properly, especially when all four are installed. Be sure to tap all the way down, but not to strip the threads when hitting the bottom of the heat sink.
Basically perfect spacing for the 71.5mm hole distance of regular 80mm fans. Actually no need to use a full sized 20mm one, fans with 15mm or even 10mm reduced height are suitable as well. A direct stream of air onto the heat sink far surpasses any (reasonable) cooling efforts from the front or rear of the chassis.
YMMV with 20mm fans, but with this slimline one, the card easily fits in two slots. In my case, a shared PCIe x8 slots is blocked by that, so no real loss here, but I could also swap the other cards and block the bottom true x4 slot.
Unfortunately though, since the 9300 cards never were intended to carry a fan, there are no fan headers. And there are no easy spots to tap into 12, 7 or 5 volts from any of the pin headers. It is possible to connect two wires to the PCIe power connector that is required for stable operation – or just connect it to some board header. If you’re running a 16 disk system, chances are the board has a couple unused of those.
Temperatures have dropped from NO TOUCHY within five minutes of operation to less than 50°C. That’s clearly sufficient for 24/7 operation, so my recommendation is a low-RPM slimline 80mm fan for these.
