Okay I finally gave up on OpenSolaris and switched back to Ubuntu. I am kind of bummed, back in 1994 I started my career working on SunOs boxes (now Solaris) on a Sparc 10 working at an ISP. I eventually left as the office / systems manager, and really enjoyed working with the OS then. It was one of the most secure and stable OS's at the time for running a shop, although I am sure everyone has their own critique on that and which is the best (BSD, Slackware, et. all). But I had high hopes for OpenSolaris. It's just not "there" yet, hopefully in the coming year or two they will shape up and I can give them a shot again.
Now that I am back on Ubuntu I am so happy, it just works. Everything I wanted to do, I can do again, and without problem. It seems also to run quite a bit faster than OpenSolaris on the X86 hardware I have, I have more headroom it seems to run a few more Virtual boxes, and not only that but I am also able to run 5 desktops, Firefox and Google Chrome Beta for Ubuntu.
Mediatomb just works, Handbrake just works, pyTivo just works. I am once again happy about the OS but unhappy about the software RAID. It's soo slow on copying things compared to ZFS, and very slow when adding drives or building the raid itself. Once I lost a drive, it took nearly 18 hours to rebuild the array thrashing the hard drives the entire time.
I notice they have ZFS for Ubuntu now located here: https://wiki.ubuntu.com/ZFS and a good little how to do various things here: https://wiki.ubuntu.com/ZFS/ZPool.
Hopefully this weekend I will finally get my OPML tutorial up and running and on the forums, I haven't been to the playonplugins site in a while, and I have to say I am really happy lately with the latest version of PlayOn as it seems to be working a LOT better with Netflix than in the past for me by far.
Thursday, October 8, 2009
Thursday, August 20, 2009
Perhaps I spoke too soon...
To lay the groundwork, because of the issues I had been having I no longer open Firefox or any other terminal windows (as I usually had done in the past) since I wanted to try to narrow down any problems and make sure nothing else could be taking up resources or RAM or causing instability in the system. I have even been considering running the box headless and just a command prompt to rule out Gnome.
As of the last entry I thought I had completely fixed the issues I seemed to have had with VirtualBox and Solaris. Wrong! The other day I had a performance issue with one of my virtual machines, and I noticed the box was using up all it's RAM. I put 1 gig to each of 2 virtual machines and 4 gigs to a 3rd. That should be 6 gigs, and I have 8 gigs in the machine and should have been enough. But whenever logging onto the host, or into one of the VM's the hosts hard drives would thrash and upon looking at the host OS stats (Solaris) I noticed 100% of ram taken up and quite a bit of disk cache as well (don't remember exactly number, but in the many megabytes).
I did the natural thing and that was to shut down the three guests (virtual machines) so that I could reboot and set some of the ram requirements per virtual machine slightly lower. Upon shutting down some of the VMs... OpenSolaris froze, it did not auto reboot as it had done when I was using the VBox shares, but.. I had to manually reboot. Doh.
I think my final solution might be to run VMWare and see if it's actually VirtualBox causing these issues or not. Or at least give it a go to see if it's not just OpenSolaris that's a little flakey. I don't seem to get really good performance either in my VM boxes with 2 CPU's assigned to one of them using VirtualBox. I tend to think the VirtualBox multi-core technology just isn't "there" yet and VMWare has been around a while so I am hoping it will be more performant with VMWare too. Anyone know of some good sites that give you a walk through of installing VMWare on OpenSolaris? Leave a comment if you do. Also if I find a few before next post I will be sure to post them with the update.
As of the last entry I thought I had completely fixed the issues I seemed to have had with VirtualBox and Solaris. Wrong! The other day I had a performance issue with one of my virtual machines, and I noticed the box was using up all it's RAM. I put 1 gig to each of 2 virtual machines and 4 gigs to a 3rd. That should be 6 gigs, and I have 8 gigs in the machine and should have been enough. But whenever logging onto the host, or into one of the VM's the hosts hard drives would thrash and upon looking at the host OS stats (Solaris) I noticed 100% of ram taken up and quite a bit of disk cache as well (don't remember exactly number, but in the many megabytes).
I did the natural thing and that was to shut down the three guests (virtual machines) so that I could reboot and set some of the ram requirements per virtual machine slightly lower. Upon shutting down some of the VMs... OpenSolaris froze, it did not auto reboot as it had done when I was using the VBox shares, but.. I had to manually reboot. Doh.
I think my final solution might be to run VMWare and see if it's actually VirtualBox causing these issues or not. Or at least give it a go to see if it's not just OpenSolaris that's a little flakey. I don't seem to get really good performance either in my VM boxes with 2 CPU's assigned to one of them using VirtualBox. I tend to think the VirtualBox multi-core technology just isn't "there" yet and VMWare has been around a while so I am hoping it will be more performant with VMWare too. Anyone know of some good sites that give you a walk through of installing VMWare on OpenSolaris? Leave a comment if you do. Also if I find a few before next post I will be sure to post them with the update.
Tuesday, August 18, 2009
ZFS and VirtualBox issues
So once I built my ZFS Home File Server box using OpenSolaris and the hardware I have listed, and I put VirtualBox on it. I did this mostly because I wanted to run Ubuntu but I also have a personal web server that I run as well as a personal SQL Server I do a lot of test coding on.
I created my ZFS pool's, shared them using CIFS with some articles I had read on the internet (attached at the bottom of this article). This worked great, seamless even. Then I installed VirtualBox and copied my Ubuntu and Windows machines from my old Ubuntu box which this OpenSolaris server was replacing. I then proceeded to use the VirtualBox feature that allows you to create shares that are visible to the guest machine through the VBox drivers.
I did this so I can run Mediatomb and pyTivo on my Ubuntu Virtual Machine because they don't yet run on OpenSolaris without a lot of compiling, tweaking, and other things I really don't want to do (although have tried, and failed.. OpenSolaris is a bear to get things to compile). You can find plenty of articles that show you how to do these things, but I decided not to until I have an easy way to pull down some software from a repo that's ready to go.
Everything works pretty damn good up until this point, and in a few hours of putting the hardware together my network is back the way it was before with no more physical Ubuntu box, only the OpenSolaris box running Ubuntu in a virtual machine. So I rebuild my old box and it is now my 3 year old sons gaming machine for Disney stuff. :)
Then things started acting weird. Whenever I would shut down / reboot the Ubuntu box (for whatever reason, patching, or I was editing / adding shares, etc.) it would literally freeze the OpenSolaris box and then about 15 seconds later the box would reboot. Also I noticed if I copied large files over the network it would cause the Virtual Machines to fall off the face of the earth (my network).
This is when I found there is an issue with ZFS, CIFS and VirtualBox somehow inter related to each other. After a few days of trouble shooting to find out what was causing it, and rebuilding the OpenSolaris box (thinking I did something wrong) I figured out that if I didn't turn on CIFS everything worked fine... or so I thought. This only solved the problem of rebooting the Ubuntu virtual machine causing my entire system to crash and burn with a reboot.
I installed a fresh OpenSolaris, did NOT configure CIFS, and configured the virtual box with the internal shares to the Ubuntu virtual machine. To share the files on my network this time I put Samba on the Ubuntu as it was just easy for me to do that instead of put it on OpenSolaris at the time, I am much more familiar with Ubuntu. Then every few days the computer would freeze up (but stopped the automatic reboot! :-/) also if I started to copy large files over the network, it brought the Solaris machine to it's knees, making it freeze up again. Although this time there was no reboot, it just froze completely.
The way I solved it was to not use the VirtualBox shares, not use CIFS but configure SAMBA on the OpenSolaris box and then mount those SAMBA shares inside the Ubuntu virtual machine. Now everything is running happily and no more issues. At least haven't had any issues for a few weeks now. :)
The link I promised above for Simon's Comprehensive ZFS Setup Blog:
http://breden.org.uk/2008/03/02/a-home-fileserver-using-zfs/
I created my ZFS pool's, shared them using CIFS with some articles I had read on the internet (attached at the bottom of this article). This worked great, seamless even. Then I installed VirtualBox and copied my Ubuntu and Windows machines from my old Ubuntu box which this OpenSolaris server was replacing. I then proceeded to use the VirtualBox feature that allows you to create shares that are visible to the guest machine through the VBox drivers.
I did this so I can run Mediatomb and pyTivo on my Ubuntu Virtual Machine because they don't yet run on OpenSolaris without a lot of compiling, tweaking, and other things I really don't want to do (although have tried, and failed.. OpenSolaris is a bear to get things to compile). You can find plenty of articles that show you how to do these things, but I decided not to until I have an easy way to pull down some software from a repo that's ready to go.
Everything works pretty damn good up until this point, and in a few hours of putting the hardware together my network is back the way it was before with no more physical Ubuntu box, only the OpenSolaris box running Ubuntu in a virtual machine. So I rebuild my old box and it is now my 3 year old sons gaming machine for Disney stuff. :)
Then things started acting weird. Whenever I would shut down / reboot the Ubuntu box (for whatever reason, patching, or I was editing / adding shares, etc.) it would literally freeze the OpenSolaris box and then about 15 seconds later the box would reboot. Also I noticed if I copied large files over the network it would cause the Virtual Machines to fall off the face of the earth (my network).
This is when I found there is an issue with ZFS, CIFS and VirtualBox somehow inter related to each other. After a few days of trouble shooting to find out what was causing it, and rebuilding the OpenSolaris box (thinking I did something wrong) I figured out that if I didn't turn on CIFS everything worked fine... or so I thought. This only solved the problem of rebooting the Ubuntu virtual machine causing my entire system to crash and burn with a reboot.
I installed a fresh OpenSolaris, did NOT configure CIFS, and configured the virtual box with the internal shares to the Ubuntu virtual machine. To share the files on my network this time I put Samba on the Ubuntu as it was just easy for me to do that instead of put it on OpenSolaris at the time, I am much more familiar with Ubuntu. Then every few days the computer would freeze up (but stopped the automatic reboot! :-/) also if I started to copy large files over the network, it brought the Solaris machine to it's knees, making it freeze up again. Although this time there was no reboot, it just froze completely.
The way I solved it was to not use the VirtualBox shares, not use CIFS but configure SAMBA on the OpenSolaris box and then mount those SAMBA shares inside the Ubuntu virtual machine. Now everything is running happily and no more issues. At least haven't had any issues for a few weeks now. :)
The link I promised above for Simon's Comprehensive ZFS Setup Blog:
http://breden.org.uk/2008/03/02/a-home-fileserver-using-zfs/
PlayOn Plugins - Podcast (OPML) Tutorial
I produced another tutorial video on using the Podcast Plugin on http://www.playonplugins.com and how to add additional OPML files into the plugin to get the most content customized to your wants. Check it out below.
Thursday, July 30, 2009
New ZFS Home Fileserver Hardware for OpenSolaris
In posts coming up I will talk about my setup, software, etc. This post I am just dedicating to my hardware and main OpenSolaris setup.
So I finally put together my ZFS OpenSolaris box. The specifications for the main pieces of hardware I have used are below (from Newegg with links):
The box also has four 1 terabyte drives in it giving it a full 3 terabytes to it's storage capacity, using Raid-Z (the ZFS raid) with single parity (although dual parity is supported without effort, as well as other configurations). So one drive is the parity drive and effectively I could lose one without any data loss. I have all my media now on this server, and I am putting more of my son's DVD's on the server and putting them in a box and storing them. I love this, as my children now will no longer scratch DVD's beyond use and every TV in the house has "video on demand" if you will with the setup I have.
I use the built on video card at the moment, so nothing fancy there. But this is a server that mostly works headless once I have it setup, so I don't need anything really in this case.
So I finally put together my ZFS OpenSolaris box. The specifications for the main pieces of hardware I have used are below (from Newegg with links):
- AMD Phenom II X4 810 2.6GHz Socket AM3 95W Quad-Core Processor Model HDX810WFGIBOX - Retail
- Four Western Digital Caviar Green WD10EADS 1TB SATA 3.0Gb/s 3.5" Internal Hard Drive - OEM
- Two G.SKILL 4GB (2 x 2GB) 240-Pin DDR2 SDRAM DDR2 1066 (PC2 8500) Dual Channel Kit Desktop Memory Model F2-8500CL5D-4GBPK - Retail
- GIGABYTE GA-MA770T-UD3P AM3 AMD 770 ATX AMD Motherboard - Retail
- Antec TPQ-850 850W Continuous Power ATX12V / EPS12V SLI Certified CrossFire Ready 80 PLUS BRONZE Certified Modular Active ... - Retail
- SUPERMICRO CSE-M35T-1B Black 5 Bay Hot-Swapable SATA HDD Enclosure - Retail
- Antec Nine Hundred Black Steel ATX Mid Tower Computer Case - Retail
The box also has four 1 terabyte drives in it giving it a full 3 terabytes to it's storage capacity, using Raid-Z (the ZFS raid) with single parity (although dual parity is supported without effort, as well as other configurations). So one drive is the parity drive and effectively I could lose one without any data loss. I have all my media now on this server, and I am putting more of my son's DVD's on the server and putting them in a box and storing them. I love this, as my children now will no longer scratch DVD's beyond use and every TV in the house has "video on demand" if you will with the setup I have.
I use the built on video card at the moment, so nothing fancy there. But this is a server that mostly works headless once I have it setup, so I don't need anything really in this case.
Friday, May 22, 2009
PlayOn Plugin Installation Tutorial
I produced a PlayOn plugin installation tutorial video in April and have it on YouTube, and had not had a chance to post it here yet. You can see it below:
If you like the video, please do comment!
If you like the video, please do comment!
ZFS the Media Storage Panacea!
Okay as I have stated in my blog before, I have an Ubuntu box at the moment running software raid with two 1 terbayte hard drives as the place I store my media. I also copy over data from all my other PC's (son's desktop, my ubuntu laptop, my desktop) to my Ubuntu server onto this raid set so that I ensure my data is safe. There are some drawbacks to the software raid (or even hardware for that matter). The biggest being if I lose a hard drive, it takes me literally all day long to rebuild the raid set. I cannot touch the drive during this time.
Enter a friend of mine from work who is a Sun maniac. He tells me about ZFS and I do some research and seriously, this stuff is WAY cool. If I had lost a drive, he told me basically within a few minutes my drives would have been back in synch and ready to go. It's completely different than RAID but accomlpishes the same functionality as having a RAID pair. It's not just RAID but you can tack on additional functionality you can do like taking snapshots, going back to a point in time, creating a clone of your data, lightweight filesystem creation, adding a new drive into the set almost seemlessly, and so much more.
Enter a friend of mine from work who is a Sun maniac. He tells me about ZFS and I do some research and seriously, this stuff is WAY cool. If I had lost a drive, he told me basically within a few minutes my drives would have been back in synch and ready to go. It's completely different than RAID but accomlpishes the same functionality as having a RAID pair. It's not just RAID but you can tack on additional functionality you can do like taking snapshots, going back to a point in time, creating a clone of your data, lightweight filesystem creation, adding a new drive into the set almost seemlessly, and so much more.
I think I am going to install OpenSorlaris on a new box, and then replace my Ubuntu eventually with that running some kind of hot swappable SATA2 enclosure of some kind. Here are some links below in case you're interested in doing ZFS for yourself:
The wiki page on ZFS:
Video How To's from Sun:
A Demo of the Time Slider (How to go back in time):
I will document my findings and how it goes as I move forward with this project, as well as what hardware I use for the new box. :)
Subscribe to:
Posts (Atom)