Skip to content

Hexadecatuple Computing

Sadly, the sixteen boot penguins get cut off because the Nvidia GPU installed doesn’t have kernel mode setting.

Sixteen Cores of sweet, sweet, Opteron

Hexadecatuple core probably won’t enter the vernacular but nothing else runs Ubuntu 9.10 quite like sixteen cores of sweet, sweet, Opteron.

I put this hardware together as a VM platform for running large numbers of KVM systems. It’s 32GB of RAM, four AMD Opteron 8354 CPUs, and not much else. The idea was to pair it with some Intel X25-M SSDs and find out how many VMs one “desktop” computer could run.

Here’s a little benchmark of kernel compiles on it. The number of make processes doubles with each test. I downloaded the 2.6.32_rc4 source and did ‘make -j128; make clean’ before the first posted number. The disk used is an OCZ Solid 30GB SSD. Yes, there is less total disk space than RAM.

make -j1

The baseline of make -j1 – 14 minutes! Memory usage is completely flat, a kernel compile can barely dent 32GB.

make -j64

At make -j64 the number of simultaneous compiles means that the Ubuntu system monitor doesn’t get scheduled often enough to sample correctly. This compresses the graph making the time scale useless.


64 make processes seems to be the optimal number, but the difference in time after 32 processes is so small that it’s hard to say that any particular number is faster.

I ran out of time to actually test KVM. In general use Ubuntu is massively quick on 16-cores. Many applications are single threaded but they interact with services and kernel drivers that run on other cores. If only my laptop could be this fast.

Four sockets

Parts List

In case you wish to play along at home.

  • 1 x TYAN S4980G2NR Thunder n3600QE Motherboard
  • 4 x AMD Opteron 8354 CPUs
  • 4 x Socket AM2/F heatsinks
  • 16 x Kingston 2GB 240-Pin DDR2 SDRAM ECC Registered DDR2 667 (PC2 5300) Server Memory Model KVR667D2D8P5/2G
  • 1 x PC Power & Cooling 750 watt power supply (single rail)
  • 1 x 8-pin ATX extension cable
  • 1 x 24-pin ATX extension cable
  • 1 x COOLER MASTER COSMOS 1000
  • 1 x Tyan M3291 IPMI card (optional but fun!)
  • Your choice of SSDs and hard drives

For workstation or gaming use, add these:

  • 1 x ZOTAC ZT-98GES4P-FDL GeForce 9800 GT 512MB
  • 1 x USB sound card

Build Notes

I tried Unregistered ECC first but it doesn’t work. The system destroyed several budget power supplies (even those rated around 1000 watts) but ran pretty well on both the PC P&C 750W or an Enermax 600W. Go for the 750W if you want to install drives.

Installing Ubuntu 9.10 doesn’t need any fancy tricks. Just boot the installer and click through it.

Post a Comment

Your email is never published nor shared. Required fields are marked *
*
*
Please leave these two fields as-is:

Protected by Invisible Defender. Showed 403 to 655 bad guys.