Kinda odd. 8 GPUs to a CPU is pretty much standard, and less ‘wasteful,’ as the CPU ideally shouldn’t do much for ML workloads.
Even wasted CPU aside, you generally want 8 GPUs to a pod for inference, so you can batch a model as much a possible without physically going ‘outside’ the server. It makes me wonder if they just can’t put as much PCIe/NVLink on it as AMD can?
LPCAMM is sick though. So is the sheer compactness of this thing; I bet HPC folks will love it.
Kinda odd. 8 GPUs to a CPU is pretty much standard, and less ‘wasteful,’ as the CPU ideally shouldn’t do much for ML workloads.
Even wasted CPU aside, you generally want 8 GPUs to a pod for inference, so you can batch a model as much a possible without physically going ‘outside’ the server. It makes me wonder if they just can’t put as much PCIe/NVLink on it as AMD can?
LPCAMM is sick though. So is the sheer compactness of this thing; I bet HPC folks will love it.