Well, removing the IR Receiver and the Bluetooth devices from the OS X VM did not help. I timed how long it took from the OS X VM to go from 95% to 100%. It took 6 min 34 seconds. Then, about 40 seconds later, the ESXi host crashed again before the OS X started responding to pings.
I'm really hoping I did not kill the OS X VM. My next step is to remove the USB controller 00:1d.0 from Direct Path I/O, reboot ESXi, and power up the OS X VM while keeping my fingers crossed. Unfortunately, I haven't backed it up via Time Machine yet, and I moved some files to it that I really do not want to lose. 
Edit 1: After I hard reset the ESXi host again, the 00:1d.0 controller was no longer checked as a device enabled for Direct Path I/O. I did not uncheck it, so somehow ESXi removed this device from Direc Path I/O. However, this still did not resolve my issue. This time I tried to boot OS X VM into EFI, and that part was successful. However, when I reset it from EFI, it attempted to boot into OS X, I saw the Apple logo in the console window but no spinning wheel. After about 40 seconds, my ESXi host crashed again. So, something is horribly wrong with my OS X VM.
I will make the last attempt of enabling 00:1d.0 for Direct Path I/O, rebooting the ESXi host, then disabling 00:1d.0 for Direct Path I/O, then rebooting the ESXi host again, and then trying to boot up the OS X VM. If my ESXi host crashes again, I'm going to try to repair it using the Recovey HD, and if this doesn't work, I would like some input from the community. Otherwise, I would have to roll back to the snapshot I had taken earlier, but that snapshot did not contain the files that I moved to this VM. They would be lost.
Edit 2: After I enabled 00:1d.0 for Direct Path I/O and rebooted ESXi (this time gracefully), I checked if this device was enabled, but it was disabled after the reboot. It appears that ESXi marked this device as bad and refuses to enable it for for Direct Path I/O upon ESXi reboot.