Plus a nice Christmas gifts: 20" LCD monitor and the ever gratifying Costco-sized Peppered beef jerky .

The symptom is that the carp0 interface appears but cannot be communicate, and failover does not happen. Reason being, the vmnet driver is silently dropping the packets on the floor!

To get it working requires a patch (on the host) to vmnet-only/driver.c and recompile.

Here is the patch for vmware-server-1.0.7 for Linux. Save it into /tmp/driver.c.patch

--- vmnet-only/driver.c.orig 2008-10-08 15:37:23.000000000 -0500+++ vmnet-only/driver.c 2008-10-08 15:44:50.000000000 -0500@@ -1284,6 +1284,9 @@     return ((flags & IFF_PROMISC) || MAC_EQ(destAddr, ifAddr) ||      ((flags & IFF_BROADCAST) && MAC_EQ(destAddr, broadcast)) || +  ((destAddr[0] == 0) && (destAddr[1] == 0) &&+   (destAddr[2] == 0x5e) && (destAddr[3] == 0) &&+   (destAddr[4] == 1)) ||     ((destAddr[0] & 0x1) && (flags & IFF_ALLMULTI ||        (flags & IFF_MULTICAST &&         VNetMulticastFilter(destAddr, ladrf)))));

The file is a little hard to find, being hidden in a vmnet.tar below /usr/lib/vmware/modules/source/ ...

cd /usr/lib/vmware/modules/sourcetar xvf vmnet.tarpatch < /tmp/driver.c.patchtar cvf vmnet.tar vmnet-onlyvmware-config.pl

Then, choose the option to recompile the kernel drivers, specifically vmnet.

The one other addl. need (I'm not 100% sure on this) is that it may be necessary to allow promiscous on the ethernet device, in the .vmx file...

ethernet0.nopromisc = "false"

References...

• VRRP & CARP with VMware Server
  
• CARP problem with VMWare Server
  

The secret sauce lies both in knowing what values are understood by each product inside the VMDK file and making sure you pass the right options to vcbExport.

Going from ESX to Server, you want to export your disk like so:

export VMNAME=YOURVMNAME (for bash) setenv VMNAME YOURVMNAME (for csh, tcsh) /usr/sbin/vcbExport -M 0 -F 0 \ -d /path/to/exported/$VMNAME/$VMNAME.vmdk \ -s /vmfs/volumes/datastore1/$VMNAME/$VMNAME.vmdk

Going back to ESX from Server can be accomplished with:

/usr/sbin/vcbExport -M 1 -F 1 \ -d /vmfs/volumes/datastore1/$VMNAME/$VMNAME.vmdk -s /path/to/exported/$VMNAME/$VMNAME.vmdk

Afterwards, you need to "tweek" the resulting VMDK file...
Change
createType="monolithicFlat"
to
createType="vmfs"

and change
# Extent description
RW 80668672 FLAT "YOURVMNAME-flat.vmdk" 0
to
RW 80668672 VMFS "YOURVMNAME-flat.vmdk" 0

This is great!

However it is also a bittersweet circumstance, in that the version they choose to support is on a very short leash. Support for 4.11-RELEASE by the FreeBSD team is due to expire in just a few months (Jan '07). Hopefully VMware will recognize the situation for what it is and get the 6.x series into a supported state. 6.x is the horse they should be riding at this point.

References:
http://www.vmware.com/support/esx25/doc/releasenotes_esx254.html
http://lists.freebsd.org/pipermail/freebsd-security/2006-October/004045.html

If you must know, Xen beat ESX hands down in performance. I have the data to prove it.

See Xen's official performance analysis for a legally-safe alternative. They were able to publish only because they used an older version of VMware (Workstation 3.2). The quote that tipped me off was "the latest version of any VMware product which allows the publication of comparative benchmarks".

The VMware ESX license agreement can be found here. The license states "you may not disclose the results of any benchmark test of the Software to any third party without VMware's prior written approval".

Sounds like they don't want the public to know their product might be inferior - at least in terms of performance.