Partitioning Lots of Drives in a Linux RAID

Written by Mike Young on February 21, 2007 at 8:07 pm

So, I had a really fun question and I’ll open this up for others to volunteer other solutions. But I ran into one of those tedious little jobs while prototyping some Linux-based storage solutions. Underneath it all is a RAID (Redundant Array of Independent Disks). But I wanted to mess around with some different tuning parameters to make the solution as fast possible while also making use of the most disk space. Make sense?Well, to do my little mad scientist tricks, I had to configure the partitions on 16 disk drives. And I had various iterations where I needed to tweak the sizes of each partition. Now, if you’ve ever used fdisk or parted for disk partitioning, you know this is a pain in the butt. Can you imagine trying to do all of this in a GUI? I’ve done that too. Forget it. It just takes way too long.

As common a task as this appears to me, I couldn’t find one solution for automating this across a number of drives. Most of the tools simply partition one drive at a time, and one partition at a time. Oh yeah, I loved how creating partitions within parted would typically result in partitions not ending on a cylinder boundary. I got all sorts of system complaints with that one, especially within fdisk.

Anyway, I digress. I didn’t have the time or patience to write this in a more comfortable language like perl or python, or C# which is my current favorite language. Rather I stuck with simple Bash (Bourne Again Shell) for the task.

So, after much digging, the best thing I found was that I could use redirection to feed a script to fdisk. Basically, it looks like this:

fdisk /dev/sda < myfdisk.script

Pretty simple, huh? The key here is to get the myfdisk.script right. Here’s a sample:

n
p
1
1
4
n
p
2
5
460
n
p
3
461

w

Now then, I’m hoping you’ll notice a pattern of n, p and then 1, 2, 3. This sets up new, primary partitions 1, 2 and 3. Okay?

The key is the size of the partitions. I plan to RAID together a couple of 30MB partitions for my /boot RAID1. That’s covered by blocks 1 through 4. Wondering how I came up with that? Well, (4 cylinders*16065 sectors/cylinder*512 bytes/sector)/(1024*1024). That’s how. Just in case you were wondering. ;) BTW, the blank space between the 461 and the w tells fdisk to accept the very last cylinder boundary as the ending one for partition 3.

But that only comes up with my partitioning scheme. There’s 16 drives, remember. I suppose I could do this 16 times. It would sure be a lot faster than manually going into fdisk to do this. It’s be a lot faster than parted too. But it’s still more work than I’d like. So, I need a simple loop. I did this in bash as well. This is what I came up with:

#!/bin/bash

DISKS = “/dev/sd[a-p]“

for disk in $DISKS; do
fdisk $disk < myfdisk.script
done

exit 0

That’s it. This creates a very simple list of 16 drives and ripples them through the fdisk routine. Simple and crude, perhaps. But it gets the job done without having to write any code.

If you’ve got another way of doing this more simply, I’d love to hear from you.

Update: 2/27/2007 => I went ahead and made a simple change to the script to make things a bit easier. Rather than typing out all 16 disk paths, I repaced it all with brackets, sd[a-p]. Easier, huh?

Comments (2)

Category: Techy Stuff

Wildblue.net Connection Reset Issue

Written by Mike Young on February 4, 2007 at 8:26 pm

Back in November, I did a post about getting high-speed Internet access restored via Wildblue. The access equipment we were using was fairly dated and I was looking at a pretty hefty fee to get things back up to speed (pun intended) via HughesNet. Well, the service has had its ups and downs over the last couple of months. With all of the snow storms we’ve been hit by, it’s somewhat understandable to have some interruptions, although I’m not quite sure why these temporary outages haven’t affected our television programming.

Those are the niceties. Now I have some gripes. Over the last couple of months I have encountered a growing number of sites that I can no longer view. Some of you know who you are. Well, the list has grown again today. And despite three complaints into Wildblue, the problem not only persists, but is growing.

Today, I have informed them of my intent to go to you, the readers, fellow site owners, businesses and ISPs that are affected by this issue. You see, the folks at Wildblue, rather than reset their routers, have been content to blame you, your ISPs, and even the equipment manufacturers. Obviously the problem must be on our end.

I’d really like to hear from some of you, especially anyone else using Wildblue service. If things aren’t resolved by the end of this week, I will be coming to all of you for your support. It’s a real shame when you have to go to Starbucks to be able to access a site you either work on or depend on because your ISP doesn’t see it as his problem. Also, please note that the tech support person I spoke to confirmed that she couldn’t access the site list I had provided from her end. We immediately ruled out the problem being on my end. And by going to other networks, I’ve ruled out the problem on the side of the content providers.

Well, I don’t want to stand for this any more. And again, I hope you’re with me on this. The best thing about this forum is that we can finally have a voice against piss-poor service. Let’s make it a loud voice.

If your site or company has been affected by this issue, please email or leave a comment. The comments are moderated. So if you don’t want your comment to be public, just say so. At this point I’m more interested in compiling a list.

Thanks,

Mike

Technorati Tags: , , , ,

Comments (3)

Category: Misc. Rants

Performancing for Making Posts

Written by Mike Young on February 3, 2007 at 11:38 am

Okay, found another useful tool. And the primary reason it’s useful is because it’s making up for a shortcoming in Wordpress 2.1. Since making the recent upgrade, I’ve noticed some strange behavior with the WYSIWYG editor. At first, it had very limited button functionality. In fact, I could bold, italicize or strike out words. Sure, I could do this in HTML, but sometimes it’s nice to use the cheats. Today, however, all I have is code view. Hmm… Not really sure what’s happening.

No worries, though. I’ve discovered Performancing for Firefox. It’s a plug-in for Firefox and it will allow you to easily create or edit posts in pretty much all of the major blogging platforms. This is what the interface looks like as I’m creating this post:

The nice part is that I can pretty much operate in the code view page and all of the buttons are still available. So I can write my posts very quickly and easily and make simple tweaks without having to change views. This is even easier than using Dreamweaver. I’m liking it a lot. And if you happen to have multiple blogs, you can edit them all via this one editor by simply hitting F8 and selecting the blog you wish to post to. This assumes you’ve already added the blog to your account settings.

I hope you like it. I certainly am.

Technorati Tags: , ,

Comments (4)

Category: Techy Stuff

Get Control of Your Comments

Written by Mike Young on February 2, 2007 at 7:54 am

I can’t tell you how many times I’ve heard people complain about how blogging can be so time consuming. It’s one thing to write posts. I actually think that’s the easier part. For me, the fun in blogging is in the exchanging of comments. Sometimes it’s like a game of dueling banjos. Ever do rapid fire comment exchange? I have. It’s not as optimal as using IM. But still fun.

Lately, I’ve been writing about those little things that make my life a bit easier. For tracking the various blogs I read, I use Bloglines. This site aggregates the RSS feeds from all the sites I monitor, and highlights a link when a new post is made.  But it doesn’t do well with comments. Sure I can track comments as RSS feeds too. But this can get way too much to manage. Think about a new Bloglines entry for each post, not blog, but post that is made on all the sites you happen to follow.

To make things a bit easier, I use coComment.  Now, I’ve used this for a little while, but recently found a way to make life even easier. Originally, I used the booklet feature. This places a link on your bookmark bar so that when you leave a comment on a site that you wish to track, you need to click the link to add the comments page to your list of tracked conversations. Needless to say it’s convenient, but very manual.

Well, I finally decided to use the plug-in feature for Firefox. Holy Cow! Comment tracking is actually automatic now. And it’s much quicker than the other method. I highly recommend you install the plug-in and start using this tool. It should save a lot of time tracking comments as opposed to having to go to each site to see if there have been any updates.

Enjoy!

Comments (5)

Category: Techy Stuff


wildernessVoice is totally for my own purposes; to vent, rant, elaborate and to generally discuss my thoughts and views. Sometimes I write about theological items, political perspectives and even technical tidbits.

It's my sincerest desire that during the course of my writings, I somehow find a way to either amuse, enlighten or otherwise enrich you. Please stop by for a while and enjoy.