So, I found a solution. Instead of customizing the VM when deployed, I am now waiting for it to finish then editing the network adapter, then setting the OSCustomizationSpec
New-VM -Name $vm.VMName -ResourcePool $vApp -Location $Location -Template $template -Datastore $datastore -RunAsync:$True
Set-NetworkAdapter (Get-NetworkAdapter $newvm) -MacAddress $vm.MacAddress -Confirm:$False
Set-VM $newvm -OSCustomizationSpec $OSCustomizationSpec
Not sure why I have to do it this way, but this seems to work