So once I built my ZFS Home File Server box using OpenSolaris and the hardware I have listed, and I put VirtualBox on it. I did this mostly because I wanted to run Ubuntu but I also have a personal web server that I run as well as a personal SQL Server I do a lot of test coding on.
I created my ZFS pool's, shared them using CIFS with some articles I had read on the internet (attached at the bottom of this article). This worked great, seamless even. Then I installed VirtualBox and copied my Ubuntu and Windows machines from my old Ubuntu box which this OpenSolaris server was replacing. I then proceeded to use the VirtualBox feature that allows you to create shares that are visible to the guest machine through the VBox drivers.
I did this so I can run Mediatomb and pyTivo on my Ubuntu Virtual Machine because they don't yet run on OpenSolaris without a lot of compiling, tweaking, and other things I really don't want to do (although have tried, and failed.. OpenSolaris is a bear to get things to compile). You can find plenty of articles that show you how to do these things, but I decided not to until I have an easy way to pull down some software from a repo that's ready to go.
Everything works pretty damn good up until this point, and in a few hours of putting the hardware together my network is back the way it was before with no more physical Ubuntu box, only the OpenSolaris box running Ubuntu in a virtual machine. So I rebuild my old box and it is now my 3 year old sons gaming machine for Disney stuff. :)
Then things started acting weird. Whenever I would shut down / reboot the Ubuntu box (for whatever reason, patching, or I was editing / adding shares, etc.) it would literally freeze the OpenSolaris box and then about 15 seconds later the box would reboot. Also I noticed if I copied large files over the network it would cause the Virtual Machines to fall off the face of the earth (my network).
This is when I found there is an issue with ZFS, CIFS and VirtualBox somehow inter related to each other. After a few days of trouble shooting to find out what was causing it, and rebuilding the OpenSolaris box (thinking I did something wrong) I figured out that if I didn't turn on CIFS everything worked fine... or so I thought. This only solved the problem of rebooting the Ubuntu virtual machine causing my entire system to crash and burn with a reboot.
I installed a fresh OpenSolaris, did NOT configure CIFS, and configured the virtual box with the internal shares to the Ubuntu virtual machine. To share the files on my network this time I put Samba on the Ubuntu as it was just easy for me to do that instead of put it on OpenSolaris at the time, I am much more familiar with Ubuntu. Then every few days the computer would freeze up (but stopped the automatic reboot! :-/) also if I started to copy large files over the network, it brought the Solaris machine to it's knees, making it freeze up again. Although this time there was no reboot, it just froze completely.
The way I solved it was to not use the VirtualBox shares, not use CIFS but configure SAMBA on the OpenSolaris box and then mount those SAMBA shares inside the Ubuntu virtual machine. Now everything is running happily and no more issues. At least haven't had any issues for a few weeks now. :)
The link I promised above for Simon's Comprehensive ZFS Setup Blog: