TR Forums

Hey guys, I'm trying to do some PC renovation work at my home and I was planning on building an Active Directory (I have a LOT of computers) and wanted to check if this idea would be good - or if I needed to upgrade my hardware before doing this: I want to convert my 1700X machine with 32GBs of RAM into an ESXi server that can support the following:
a) An Active Directory VM (4 GBs of RAM - won't be on all the time, will be hosted within a 1 TB SSD x RAID 1)
b) A CA VM (4 GBs of RAM - won't be on all the time, will be hosted within a 1 TB SSD on RAID 1)
c) A gaming workstation (16 GBs of RAM w/ 2 1070 - SLI on direct pass through - will be on all the time, will have a dedicated 512GB SSD + 4 TBs 5400RPM HDD for storage)
d) Random VMs for testing purposes (within another storage drive that I need to buy or plan for)

I'm debating whether this is a good idea because I'm not sure if I can achieve 60~70 fps when gaming at 1080p (dual monitors, but only one will be gaming) through an ESXi (even though only the gaming machine should have dedicated access to the video cards).

Do you think this is a good idea? or I need to rethink this?

Edit: added storage plans!

AD runs on a potato, but the 1700X isn't really the best gaming chip to support two 1070's even when it's dedicated to that job.

Also, if you're creating an AD domain, you'll want the domain controller to be running 24/7. I think you should build a low-power server for jobs A, C, and D and stick it in a utility cupboard instead, then leave jobs B and maybe D to your gaming-capable workstation.

I also haven't ever run SLI through ESXi. Maybe it'll work but my previous experience and gut tells me that you're not likely to like the results, and getting there will be a pain in the ass.

SLI is still a thing? Other than that, I think it would probably work okay. I'm a fan of dedicated resources where appropriate, and gaming certainly seems to fall down that path.

I really would separate out Gaming Workstation, and Home Server. It's a minor hassle to have AD on ESXi, not because of performance, just the occasional maintenance. I still do it anyway. I usually have to temporarily change DNS servers from AD to router+OpenDNS on ESXi host and gaming workstation to do Maintenance Mode and updates.

ESXi madness

..not sure why you wouldn't use Qemu/KVM and a Linux distro of choice.

If you want a sort of "pre-built" solution like ESXi, then UnRAID (Arch-based) is usually the go-to.


If you don't mind a GUI, then pretty much any distro. with Virt-manager:

https://virt-manager.org/

(..though that will require a cheap video card for the VM server.)

qmacpoint wrote:

a) An Active Directory VM (4 GBs of RAM - won't be on all the time, will be hosted within a 1 TB SSD x RAID 1)

No problems there. Technically, Microsoft doesn't support snapshots with AD prior to Server 2012, but if you only have a single DC then it's not a concern. Why wouldn't you leave it on though? AD is very lightweight.

b) A CA VM (4 GBs of RAM - won't be on all the time, will be hosted within a 1 TB SSD on RAID 1)

No foreseeable problems. You'd usually have an offline root CA and online/issuing intermediate CA, but for home use it doesn't really matter.

c) A gaming workstation (16 GBs of RAM w/ 2 1070 - SLI on direct pass through - will be on all the time, will have a dedicated 512GB SSD + 4 TBs 5400RPM HDD for storage)

Not supported at all. Plus, how are you going to log into the gaming PC? ESXi doesn't have provide local console access to guest VMs.

You may be able to get it working, but I wouldn't count on it. Remember that VMs get a virtual motherboard, so if SLI requires any special sauce then it's probably not going to work.

d) Random VMs for testing purposes (within another storage drive that I need to buy or plan for)

Watch your memory and CPU usage, of course, but this seems reasonable.

Do you think this is a good idea? or I need to rethink this?

I am an ESXi admin at work, and I do not recommend this course of action.

I suggest two alternatives:

1) Gaming PC on physical hardware, and all other VMs on a separate ESXi host. (Recommended for best performance.)

2) Run Hyper-V on your desktop so that the gaming PC has direct access to ALL hardware. All other VMs can be configured as described. (Recommended for lowest cost.)

CScottG wrote:

..not sure why you wouldn't use Qemu/KVM and a Linux distro of choice.

ESXi is better imho, reliability and feature-wise, but...

TheRazorsEdge wrote:

c) A gaming workstation (16 GBs of RAM w/ 2 1070 - SLI on direct pass through - will be on all the time, will have a dedicated 512GB SSD + 4 TBs 5400RPM HDD for storage)

Not supported at all. Plus, how are you going to log into the gaming PC? ESXi doesn't have provide local console access to guest VMs.
You may be able to get it working, but I wouldn't count on it. Remember that VMs get a virtual motherboard, so if SLI requires any special sauce then it's probably not going to work.

...this I haven't heard of any virtual motherboard with SLI support, and I don't think there will be, specially with NVIDIA killing SLI in favor of NVLink.

TheRazorsEdge wrote:

2) Run Hyper-V on your desktop so that the gaming PC has direct access to ALL hardware. All other VMs can be configured as described. (Recommended for lowest cost.)

This whole idea happened because I have have this 1700X that I use purely for gaming - browsing and light gaming even happens on a laptop, and my 1700X is a testing ground of sorts. But the problem is that I do have many computers (between Linux, Windows and Mac OS) and wanted to have an AD to manage the machines the best I could (patches are a completely different thing). Right now I have the setup you mentioned with VMware Workstation, and I tried to make an AD already, but I have problems trying to set up an AD in a VM, where the HOST is a child of the VM AD. While I do agree that a dedicated machine would be a better solution overall, I don't wanna get more hardware in (one-bedroom apartment) so I'll have to make do with what I have. It was good thinking about this though Thanks everyone!

qmacpoint wrote:

I have problems trying to set up an AD in a VM, where the HOST is a child of the VM AD.

The desktop being a domain member shouldn't be a problem. It should boot fine without AD, and you should be able to log into your domain account with cached credentials even if the domain controller is not available. You may have to run "gpupdate /force" manually to get Group Policy if you're using it, as the desktop won't be able to download policies at the normal time (during startup).

I suspect you might run into DNS issues. The desktop needs DNS from the DC to do internal AD stuff, but you need DNS from your router or ISP to access external sites. This can be addressed with manual DNS settings or forwarders.

The only other problem you might have is related to time. The VMware Tools time sync functionality should be disabled in this scenario, or the clocks could drift (it's in the VM advanced properties). Over time, this could break TLS to the outside world.

TheRazorsEdge wrote:

qmacpoint wrote:

I have problems trying to set up an AD in a VM, where the HOST is a child of the VM AD.

The desktop being a domain member shouldn't be a problem. It should boot fine without AD, and you should be able to log into your domain account with cached credentials even if the domain controller is not available. You may have to run "gpupdate /force" manually to get Group Policy if you're using it, as the desktop won't be able to download policies at the normal time (during startup).

I suspect you might run into DNS issues. The desktop needs DNS from the DC to do internal AD stuff, but you need DNS from your router or ISP to access external sites. This can be addressed with manual DNS settings or forwarders.

The only other problem you might have is related to time. The VMware Tools time sync functionality should be disabled in this scenario, or the clocks could drift (it's in the VM advanced properties). Over time, this could break TLS to the outside world.

All good tips. To solve the clock de-synchronization error use an external NTP server (like pool.ntp.org) for all the VMs and the ESXi host. I would also suggest that direct I/O passthrough for SLI may be a challenge, but you can be the guinea pig to let us know how it goes.

qmacpoint wrote:

CScottG wrote:

..not sure why you wouldn't use Qemu/KVM and a Linux distro of choice.

ESXi is better imho, reliability and feature-wise, but...

I originally thought the same, I was wrong. (..maybe ESXI has the advantage (in a few respects) in spun-up VM's per server - where you are talking about a LOT more than a home VM server would use.)

I sympathise with the need to save space, but yeah, I'm afraid it's not really a realistic option.

Hyper-V for the host partition with other VMs running on top works reasonably well (I do that on my gaming PC for the occasional bit of tinkering), but you can't really have AD running "only some of the time" I'm afraid - all kinds of things will get upset, from time sync to DNS. Having the host AD joined also causes some mild irritations.

What I've done at home (not saying it's ideal, just an option) is spun off my "core infrastructure" VMs onto a tiny ITX always-on box. Picked up a Quad core Celeron 3160 board, an old laptop provided some memory and I put an i350 NIC in it. It runs my pfSense router (3160 has AES-NI on the CPU, which is nice) and a 2016 Core DC installation 24/7 for 11w power draw (240v country, ymmv). It's ITX with a DC power input, needs only a single disk (in my case a truly horrendous Crucial V4 SSD that's no good for anything requiring actual disk access), and is sitting in a tiny InWin ITX case with a single fan keeping it all very happy.

If you don't need multiple NICs like me you could always use an old laptop for the same purpose - AD will run on basically anything, especially if you run Server Core instead of full-fat.

The more I think about this, the more a NUC seems reasonable... I do believe dedicated hardware for server/management reasons should be a thing... While I still ponder around this, perhaps I should buy one of those Intel boxes for this purpose.

https://androidpctv.com/review-asrock-j ... j4105-itx/

-both are low power, with iGPU, and have VT-x and VT-d for full virtualization. The biggest limitation is the max memory of 8 gig.