Personal computing discussed

Moderators: renee, JustAnEngineer

 
demolition
Gerbil First Class
Topic Author
Posts: 123
Joined: Wed Nov 01, 2017 3:27 am

Threadripper build with ECC for VMs

Mon Jan 15, 2018 3:27 am

At work I could use a decent server to host some of VMs for various development/test purposes and have been looking at a Threadripper setup with ECC and ESXi. If that could work, it would provide some serious bang for the bucks.

It seems that Threadripper should work with the latest ESXi update (the previous version needed SMT to be disabled), but I would like to use ECC memory and that limits my choices of motherboards quite substantially. It should not only run with ECC modules in non-ECC mode like most Threadripper boards will do, but should actually use ECC or else it would be pointless. All the motherboards that I have been able to verify (in various forums) to be working with ESXi does not support ECC properly and ECC is something I would really like to have.

Does anyone have any experience with using Threadripper+ECC for virtualization? It does not need to be ESXi if some other solutions works.

If I were to build a similarly specced Xeon or Epyc-based server, I can see that the cost would quickly skyrocket, so that is why I would like to spend some time to first investigate a Threadripper build.
 
ptsant
Gerbil XP
Posts: 397
Joined: Mon Oct 05, 2009 12:45 pm

Re: Threadripper build with ECC for VMs

Mon Jan 15, 2018 6:43 am

I have been following Zen+ECC a bit because I am tempted by the idea of going to 64GB ECC (currently running 32GB non ECC, perfectly stable).

It appears that at least Asrock and ASUS have functional ECC support with ThreadRipper and relevant BIOS options. See this for example: https://www.reddit.com/r/Amd/comments/7 ... ipper_yet/

In the above thread, one user has confirmed ECC on Asrock Taichi and another on Asus Zenith Extreme.

I'm not an expert on virtualization, but the state of affairs in linux seems to be quite good, including exotic stuff like GPU passthrough.
Image
 
demolition
Gerbil First Class
Topic Author
Posts: 123
Joined: Wed Nov 01, 2017 3:27 am

Re: Threadripper build with ECC for VMs

Mon Jan 15, 2018 8:34 am

Thanks for the link. I am not fully convinced by the comment about the ASUS Zenith supporting ECC properly as he doesn't specify how he came to that conclusion, and I have seen many other review saying that it does not work with ECC enabled (although they could have enabled in a later BIOS).

It seems to be working with the Asrock X399 Taichi as you mention, and also the MSI X399 Gaming Pro Carbon AC:
https://www.youtube.com/watch?v=RIGM-ezd7ms

They both seem to have some unresolved driver issues with ESXi though.. Some mention USB/SATA driver issues, and there is also a possible issue with the driver for the Intel I210-AT NIC which is the standard for the X399 platform.
 
ptsant
Gerbil XP
Posts: 397
Joined: Mon Oct 05, 2009 12:45 pm

Re: Threadripper build with ECC for VMs

Mon Jan 15, 2018 2:52 pm

demolition wrote:
Thanks for the link. I am not fully convinced by the comment about the ASUS Zenith supporting ECC properly as he doesn't specify how he came to that conclusion, and I have seen many other review saying that it does not work with ECC enabled (although they could have enabled in a later BIOS).

It seems to be working with the Asrock X399 Taichi as you mention, and also the MSI X399 Gaming Pro Carbon AC:
https://www.youtube.com/watch?v=RIGM-ezd7ms

They both seem to have some unresolved driver issues with ESXi though.. Some mention USB/SATA driver issues, and there is also a possible issue with the driver for the Intel I210-AT NIC which is the standard for the X399 platform.


Aorus also seems to work: https://forum.level1techs.com/t/interes ... /118730/31
And MSI pro carbon again: https://hardforum.com/threads/x399-mb-e ... d.1940840/

People have confirmed ECC support in Ryzen (not Threadripper) MBs from ASUS, especially the Prime (if I remember correctly) by overclocking until the appearance of (corrected) faults. So, I would be surprised if Threadripper ECC were not supported. Still, these are quite expensive so better be sure. Please update if you find something that works for you.
Image

Who is online

Users browsing this forum: No registered users and 1 guest
GZIP: On