OpenVZ seems to be the hot open source container based virtualization tool these days. Instead of tools like VMWare and Xen which virtualize the hardware and allow each guest operating system to run their own kernel, OpenVZ uses operating system level virtualization. While less flexible and less "secure" in some instances, this allows for better performance of the guests due to lower overhead.
I've been tinkering with using OpenVZ for a project to provide rapidly deployable emergency copies of infrastructure for situations where the primary and secondary hardware go down (DNS servers, LDAP servers, etc). OpenVZ meets the need here because it has command line management tools, is low overhead, and these kinds of services don't depend on a specific kernel or hardware stack as much as some others might. The tricky part for me is that some of these services live on separate VLANs.
In this setup, each machine (including the OpenVZ host) has two Gigabit Ethernet interfaces bonded together to two ports on separate switches that are stacked together. This provides higher throughput and prevents interruption of service if a switch, cable, or interface fails. The hosts typically don't know about VLANs and the interfaces on the switches are in access mode which automatically tags all traffic to the proper vlan. However, the OpenVZ host will need access to multiple VLANs so that it's guest machines can get to the right places on the network, so some things need to change. It will need it's own VLAN as well as the VLAN for each guest machine.
Firstly, the switchports are configured to trunk the right VLANs to both of the ports that the OpenVZ host is plugged into. Note that if you do this, you'll loose access to the machine so make sure you're connected out-of-band to the console! On the switch in a config shell (Cisco IOS example):
# interface Gi1/0/1 # switchport trunk encapsulation dot1q # switchport trunk allowed vlan 10,20-30 # switchport mode trunk # interface Gi2/0/1 # switchport trunk encapsulation dot1q # switchport trunk allowed vlan 10,20-30 # switchport mode trunk
Then the OpenVZ machine is configured to support VLANs by adding kernel modules and creating a new interface. Note that these instructions are for RHEL5/CentOS5:
- Add "modprobe 8021q" and "modprobe vzethdev" to /etc/rc.modules
- chmod +x /etc/rc.modules
- Manually run /etc/rc.modules. It will be automatically run when the system boots
- reconfigure the /etc/sysconfig/network-scripts/ifcfg-bond0 to have no IP or BOOTPROTO information, "ONBOOT=yes" and "MODE=trunk"
- create /etc/sysconfig/network-scripts/ifcfg-bond0.10 like the following:
DEVICE=bond0.10 IPADDR=10.0.10.2 NETMASK=255.255.255.0 GATEWAY=10.0.10.1 NETWORK=10.0.10.0 BROADCAST=10.0.10.255 ONBOOT=yes BOOTPROTO=none USERCTL=no VLAN=yes PHYSDEV=bond0
- and load the interface with "ifcfg bond0.10 && ifup bond0.10"
- make sure that proxy_arp and forwarding are enabled for bond0.10 in /proc/sys/net/ipv4/conf/bond0.10/. If not, you should reconfigure your system to set these by default. Consult your operating system documentation for instructions on this.
Once this is done, you should be able to use this host on the network (on VLAN 10) like nothing changed! If not, make sure routes are set up right, ifconfig looks right, etc. Assuming it works, you're halfway there! Up next is creating an interface for each vlan you want mapped. Here's an example for /etc/sysconfig/network-scripts/ifcfg-bond0.20 on VLAN 20:
DEVICE=bond0.20 ONBOOT=yes BOOTPROTO=none USERCTL=no VLAN=yes PHYSDEV=bond0
Note that it doesn't have any IP information. We'll specify this inside of the OpenVZ instance. Next, we actually create a blank OpenVZ instance (you can use an existing one, but this is provided for completeness sake) and give it an eth0 interface. I'm using 20 as the ID here because this instance will be on VLAN 20, but this is not a requirement.
vzctl create 20 --ostemplate centos-5-x86_64-default-5.2-20081013 --config vps.basic vzctl set 20 --onboot no --save vzctl set 20 --hostname vlan20host.local --save vzctl set 20 --numothersock 120 --save vzctl set 20 --nameserver 10.0.10.1 --save vzctl start 20 vzctl set 20 --netif_add eth0 --save
On each host, use "eth0" as the name of the interface. OpenVZ will automatically create the eth0 interface in the guest and an interface like "veth20.0" on the host where "20" in the name represents the guest ID and the .0 indicates that this is the default interface for the guest. You could add an eth0.21 interface to the guest with vzctl if you wanted VLAN 21 also piped into the guest, which would create a eth0.21 on the guest and veth20.21 on the host.
Now that it has the interface, enter the instance with "vzctl enter 20" and set up it's networking by creating /etc/sysconfig/network-scripts/ifcfg-eth0:
DEVICE=eth0 IPADDR=10.0.20.1 NETMASK=255.255.255.0 GATEWAY=10.0.20.1 NETWORK=10.0.20.0 BROADCAST=10.0.20.255 ONBOOT=yes BOOTPROTO=none USERCTL=no
Then "ifcfg eth0 && ifup eth0". You won't be able to send traffic yet, but configuration inside of the guest here is done. Head back to the OpenVZ host and set up a bridge to connect things. Here, we name the bridge so that it's recognizable as this VLAN and add the VLAN interface and OpenVZ host interface to it. Making one bridge per VLAN is probably the right thing to do, and if multiple guests are on the same VLAN, just add their host interfaces to the same bridge.
brctl addbr vzbr20 brctl addif vzbr20 bond0.20 brctl addif vzbr20 veth20.0 ifup vzbr20 0
Again, make sure that forwarding and arp_proxy are enabled for the bridge and the veth20.0 interface (created by adding eth0 to the guest). And that's it! You should be able to ping the guest's gateway from the guest. If you can't, run a ping from the guest and run tcpdump from the host to see where packets stop, looking at interfaces in this order:
Whichever one it stops on, make sure you have forwarding and proxy_arp enabled, and make sure that the bridge has the two things it should as members with "brctl show"
Each time the vm is rebooted, you'll need to add it's host interface to the bridge again and may need to re-enable forwarding and proxy_arp depending on how your OS is configured. If you are using a newer version of OpenVZ (>3.0.22) there is a easy workaround for this on the Veth wiki page. There is a more complex workaround for older OpenVZ versions there as well, but I should be using the newest version of OpenVZ when it's time to deploy this thing so I'm waiting!
Hopefully you find this useful, please let me know with a comment on the original article at ckdake.com or via an email if you have any suggestions, comments, or corrections!
It seems I'm not the only one thats notices that bike companies just seem to do a better job of customer service than anyone else that I've dealt with:
- Bike people are better on Bicycles and Icicles
- New bicycle adventures on San's blog. She doesn't go into depth about how awesome Atlanta Cycling was to her, but they went above and beyond to get her the bike she wanted.
- Bikes and customer service here
- Customer Service is Alive and Well, the last two paragraphs
All of those stories are very representative of my experiences. Most recently, a screw fell out of one of my Crank Brothers Quattro SL pedals. They're just decorative and don't affect the functionality of the pedals, but I sent them an email (during Interbike when everyone in the cycling world shouldn't be checking their email) and they quickly responded that they'd get a replacement out ASAP. A few days later, an envelope showed up with new screws and a few other parts for those pedals "just in case", as well as another card from "The Pedal Spa", a handful of stickers, and a note from "Drew" suggesting that I should put Blue Loctite on those screws to keep it from happening again. Awesome.
It's definitely been a busy couple of weeks. We've had a few weekend events at work as we slowly are moving things to a setup that will prevent us from needing to do anything on weekends, and there's been a whole lot of bike riding. But first, my most recent tinkering with HDR photography:
Thats three exposures of one of the trees in my backyard, taken at 11mm on the new tripod with a remote shutter. On to what I've been up to the past few weeks that you might be interested in:
- Tomorrow morning I'll hit 2000 miles of road riding this year, with 1750 of them on the newest road bike. That's bringing me pretty close to 4000 miles so far since February 1st, so hopefully I'll round off the year with 5000+?
- I got new wheels on my track bike and will be training on that this winter for next years season (assuming I don't break anything between now and then!). I've been mountain biking a couple of times and am getting back into the habit of 30mile+ mountain rides, and I'm slowly working out a training plan so that I can maybe win some races next year.
- After 20 or so years of never having a real camera tripod and taking pictures, I finally purchased one. Here's a crummy picture of the Acratech Ultimate ballhead with Leveler mounted on the Gitzo Explorer Basalt legs. It's sturdy, light, easy to use, and I'm looking forward to using it a lot. Hopefully it will last me another 20 years or so.
- I was able to get a Wii Fit for list price! Daniel couldn't have said it better: the internet rules.
- FM.24.08 happened again and was an even bigger success than last year. The tracking system worked great, but I unfortunately didn't take as many pictures as at FM.24.07 last year. Here is this year's good set.
- The final Dick Lane Velodrome Festival of Speed of the year went down and I was able to take plenty of pictures there since almost nobody else from Faster Mustache showed up to spectate! Friday Night Sprint pics and pictures from Saturday. After sprints on Friday, I rode around town and took some awesome night shots from the top of a building in downtown Atlanta, and before FoS started on Saturday San and I went by Connolly Nature Preserve and I got some shots of the champion trees there. (Read more about the trees here).
- Lastly, I'm slowly whipping my mini-web hosting company into shape. If you're looking for hosting, head to ckdake.com/hosting and keep your eyes peeled on www.ithought.org for some exciting news sometime in the next month or so.
This past weekend, Ben and I drove North for some mountain biking. We stayed at my Grandparent's lakehouse on Lake Rabun, and drove up to Tsali in North Carolina on Saturday morning. Unfortunately we didn't plan super well and the two trails there I wanted to ride (the left and right loops) are Horse-only on Saturdays, but this turned out not to be much of a mistake. We rode Mouse Branch first including the optional scenic overlook portion, and it was pretty nice. The trail was a bit crowded and some people going at a leisurely pace didn't want to let anyone pass them, but we managed to finish the 9.7 miles in under an hour which is pretty fast for us. (The woman that won the gold in Olympic mountain biking this year averaged 12mph if I remember right.) After PBJs, we did a lap on Thompson Loop which was awesome. The first part is very fast singletrack, then a lot of climbing, and then one of the longest and fastest downhill sections of singletrack I've ever ridden (elevation graph). 25mph on a mountain bike for a couple of minutes feels pretty fast! So of course, we had to ride that loop again for a total of ~25 miles for the day and as many hours on the trail as it took us to drive there.
On Sunday, we drove the 15 minutes from the lakehouse to the Stonewall Falls Loop in Clayton, GA. From reading around online, I didn't know quite what to expect. There was only one car in the gravel parking lot and no real signage, but we hopped on our bikes and set out. The first 7 miles or so seemed like they were all uphill, and for the first time in a while I actually had to walk up a few sections, but we were rewarded with the second 7 miles that were mostly downhill. I don't need to ride this again, but probably will if I'm at the lake for more than a day. Ben really liked it, so you might! It was "real" mountain biking with almost every kind of terrain: rocks and dirt, slowly recovering forest fire areas, flat trail following a stream with several stream crossings, super sketchy downhill sections on powerline cuts, banked turns, sand, narrow cuts through heavy undergrowth, steep uphills, etc. I like trails that are a little more consistent so that I can get in the appropriate "zone" for the trail, and this one kept me on my toes. My arms are a bit scratched up from all the bushes, and my shoes are still drying out, but if you're comfortable on a mountain bike and in the area this trail is worth trying. Our loop ended up being a little over 13 miles and took us about 2 hours.
These were the first mountain bike rides I've done of the length I was used to before the whole collarbone thing, and while it's nice to be back on that bike again, I still have some catching up to do in the mountains and on the track! Since Feb 1st this year, 1593 miles on road rides, 230 on casual ones, 822 on a trainer, 125 on the mountain bike, and 108 at the track.
So for no real reason, I'm pretty interested in disasters: finding out details, figuring out what went wrong, etc. I'm finishing up a great book: "Normal Accidents" that is highly recommended to anyone else interested in these sorts of things. (Though you should probably read Silent Spring first.)
As part of this interest, every now and then I'll spend a few hours poking around the Internet and Wikipedia to learn more. If you're not aware of any of the huge human-caused catastrophes, here are some jumping off points on Wikipedia to get you started:
- Environmental disasters in the US - Love Canal in particular
- Medical Disasters
- Civilian radiation accidents (not involving fission, fissile matter or a reactor)
- Civilian nuclear accidents (involving fission, a reactor or fissile material from one)
- Military nuclear accidents
So after spending who knows how many hours reading around, here are a few things I've found that are worth sharing:
- The Rocky Flats Plant just outside of Denver, CO was used for nuclear weapons production. As detailed in the Wikipedia article, it had a horrendous safety record and during it's operation, none of this was known to the public. See this brief article in Time magazine for an example. The facility has been shut down, and it's pretty interesting to see the transformation in satellite photos. Check out this map on maps.live.com and zoom in and out, note what disappears! The same map on maps.google.com is a mix of older photography and newer ones, so only the left half of the plant is there. Note the highway that goes in a circle around Denver and how it's missing a section that would have gone through this area. According to wikipedia this work was completed in October 2005 and the site is on it's way to being a wildlife refuge.
- While the Chernobyl disaster was huge, most people don't know about all of the other awful nuclear/environmental things going on in the Soviet Union at the time. Disasters at Mayak between 1945 and the late 1960s have killed many and exposed over half a million people to dangerous amounts of radiation. You can see what remains of the biggest failed reactor here, and from what I understand it's essentially just being left there because it's too expensive to clean up. Lake Karachay, the lake they used _for dumping their nuclear waste into_ has been called "the most polluted spot on earth". You should take a look at it here: make sure to zoom out and pan around to get a sense of scale.
- And then there are the bioweapons. Things like a 1971 smallpox outbreak caused by Soviet aerosol weapons tests, an anthrax leak in Sverdlovsk, Soviet Union in 1979, etc. These have all kinds of potential to go wrong and it's worth reading Biohazard by Ken Alibek if you want to know more about these sorts of things. (Or atleast follow some links about Biopreparat.)
I hope that was interesting, and I bet I'll post something like this every now and then. Let me know what you think!
It's that time of year again: In a few weeks, we at Faster Mustache are putting on our 4th annual 24 hour urban relay! Head on over to 24.fastermustache.org to pre-register to guarantee your spot, as this is the cycling event of the year in the south east, and the only event of it's kind in the world.
September 6th and 7th at Johnny's Pizza on North Highland. Take it easy on a 6 person team and just have a good time doing at least one lap a person (~10 miles or so), or go for the solo win which will take 300+ miles of riding! Trust me, the prizes are worth it!