Stupid git tricks

I had a directory in CVS that I used for a catch-all for random scripts. (For example, this is where cpanspec lived before moving it to Sourceforge CVS.) Now that I'm using github, I'm trying to split these scripts up into separate git repos. This is the procedure I've come up with...

First I use git cvsimport to pull in the whole CVS tree:

git cvsimport -d :ext:user@host:/cvsroot -C myscript cvs_module

This will create a directory named myscript. Next, go into that directory and use git filter-branch to remove everything but the file(s) we care about (in this case, myscript again).

git filter-branch --prune-empty --tree-filter 'find -maxdepth 1 -type f \! -name myscript -delete' HEAD

This ends up leaving some stale objects that can be cleaned up by removing everything other than master in .git/refs/heads/, the entire directory .git/refs/original/, and any unrelated tags in .git/refs/tags/ (at least in my example with no branches and such), then cleaning up with a few git commands:

git gc --aggressive
git prune
git repack -a -d

The total number of objects listed by git gc and git repack should be much smaller than the original number git cvsimport reported. (I also confirmed that git fsck --unreachable doesn't find anything.)

[Update] Apparently I had found this answer to my problem a while back and forgot about it. Oops.

Where do we go next with RAID?

So a friend of mine sent me a link to this blog post.  A couple of things jumped out at me...

When a drive fails in a 7 drive, 2 TB SATA disk RAID 5, you’ll have 6 remaining 2 TB drives. As the RAID controller is reconstructing the data it is very likely it will see an URE. At that point the RAID reconstruction stops.

And later...

RAID proponents assumed that disk failures are independent events, but long experience has shown this is not the case: 1 drive failure means another is much more likely.

That sounds an awful lot like what I've been saying for 8 or 9 years now...  (Well, not specifically about 2TB drives, but you know what I mean...  :-)

So the myth that I've been hearing for the last 15 years or so is that you get speed and data security with RAID 5.  The fact is that the speed of an intact array is terrible, and to use the word "speed" in regards

to a degraded array would be an oxymoron.  Add that to the odds of a failure of one of your "good" drives during a rebuild, and you get one big pile of fail.

The advantage of RAID 5 is capacity.  Period.  Any other RAID solution costs more in terms of raw storage capacity.  RAID 6 gives you one less drive of capacity in exchage for improving your odds of a successful

rebuild, but as you all know, I still don't trust it for anything that we don't have a mirror of somewhere.

We've been doing a lot of RAID 1 and RAID 1+0, which is fine, but ultimately you have the same problem there with likely failures while trying to rebuild an array, but you have the added bonus problem that

errors may go undetected.  They may kill performance, but the checksums on RAID 5 and 6 do give you an added safety net since you can detect corrupted data.

For some of our largest arrays, we've been doing mirrored (or rsync'd) RAID 5 or 6, which, while extraordinarily wasteful in terms of storage space, gives us very good odds of recovery from catastrophic hardware failure.

I have to wonder if the real answer here might ultimately be to add parity to a stripe/mirror set, so that any combination of drive failures in an array of n drives that leaves you with at least (n-2)/2 working

drives is easily recoverable...  (Maybe doing RAID 6 over pairs of mirrored drives would be sufficient.  I have to think on that a bit...)

Open-Source Point of Sale?

Dear Lazyweb,

I need an open-source POS solution for a client. They have a small cafeteria-type restaurant + gift shop. Currently they are using a craptastic closed-source commercial solution that offers no support despite requiring a huge service contract.

Lots of bonus points for something web-based, since the POS terminals they have are rather low-end.

FWIW, we've tried the following:

• Adempiere + Posterita
• OpenBravo POS
• L’âne (because the developer is local-ish)
• OFBiz (both the web POS and the java client)

Posterita comes the closest to being what we want (web-based, AJAX-y, etc.), but development has gone closed-source apparently. OpenBravo POS is probably the most functional, but it's difficult to figure out how to do much of anything with it. OFBiz has a nice, simple POS app, but it's horribly buggy (and rather slow too).

Given an infinite amount of free time, I'd probably hack on the Adempiere + Posterita (last open-source release) combo, but, well, time is not on my side here...

Why developers suck as admins

So Fedora 12 allows regular users to install packages as long as a) the user is logged in on the console, and b) they are signed and from a trusted repository.

I think it's great that we can enable this functionality, and I'd even argue that it should be on by default on the Live spins (and installs from the Live spins), but in the general case this is a horrible idea.

I've seen a few arguments why this doesn't matter. For example, anyone with local console access already owns the box, right? Well, console access != physical access. Think, well, anything in a server room (systems in locked cabinets attached to a KVM, or virtual machines). While this gives more ammo to us old-timers for not putting X on our servers (I'm looking at you, skvidal), the reality is that it's not realistic to expect all servers to run without a GUI.

I don't want to re-hash the whole (long) thread linked to above, but I think it is important to point out some of the solid reasons why this change is a bad idea.

• The installation of one package shouldn't change the behavior of the system. (This one package changes the behavior of the system, plus allows for other packages to be installed that could do the same.) If you take into account that unintended dependencies tend to pull in random stuff during upgrades, this becomes especially important.
• Can we really guarantee that there are no signed packages available that are exploitable, all the time?
• This is a major change in behavior from Fedora 11 that did not go through the Feature Process, unless I'm missing something.
• Possibly even worse, if this "feature" makes it into RHEL 6, you run the risk of a lot of semi- and non-technical sysadmins having yet another security decision made for them, probably without their knowledge. (How many are aware of ctrl-alt-del, console users being able to shutdown/reboot, grub allowing kernel options ("single", for example) unless you set a password, etc.? Of course I've never understood the logic of all of that being open, but magic sysrq being off by default.)
• At the very least, this is a DoS attack vector, although more likely due to somebody screwing up and installing a bunch of packages rather than somebody intentionally trying to fill /, /usr, or /var.

Oh, and about the title... Developers of software get stuck into a mindset of "make my software work, no matter what", and, on a related note, tend to have tunnel vision about the use cases for their software. One of the things I love about Fedora is that we have a lot of sysadmins who happen to be coders, so we tend to find a good balance between "usability" (AKA letting the developers go nuts) and maintenance/security. This one slipped by us, but I hope the decision will be made to push an update with a more sane default. [Update: It was.]

Ohio LinuxFest 2009

So I'm trying to prepare for the 7-hour commute to OLF on Thursday and Friday's class. It should be fun, as always...

If anyone reading this is within driving distance (which, in my mind, includes most people in the eastern half of the US), I highly recommend making the trip. And they extended the registration deadline to midnight tonight (9/22), if I'm reading the site correctly...

Note for amavisd-new users

I just pushed amavisd-new 2.6.4 to rawhide. The major new change in this version is the introduction of a script called amavisd-snmp-subagent. To quote the release notes...

- newly supplied with the package is a program amavisd-snmp-subagent,
 acting as a SNMP AgentX, exporting amavisd statistical counters database
 (snmp.db) as well as a child process status database (nanny.db) to a
 SNMP daemon supporting the AgentX protocol (RFC 2741), such a NET-SNMP.

 It is similar to combined existing utility programs amavisd-agent
 and amavisd-nanny, but instead of writing results as text to stdout,
 it exports data to a SNMP server running on a host (same or remote),
 making them available to SNMP clients (such a Cacti or mrtg) for
 monitoring or alerting purposes.

See the release notes for more details.

Since this script introduces new dependencies, I made a sub-package for it, which includes an init script. All of this is completely untested, so if anyone would like to try it out, please grab it from rawhide (or koji until it shows up there) and let me know how it goes.

Hey, spot!

After reading spot's post, I wanted to try out chromium. Shortly after installing it, I realized that name resolution stopped working.

The problem? Installing chromium.i586 pulled in nss-mdns.i586 on my x86_64 system. I installed nss-mdns.x86_64 and the problem went away.

Whee!

Want to use chromium on x86_64? Do this:

cat > /etc/yum.repos.d/chromium.repo <<'END'
[chromium]
name=Chromium Test Packages
baseurl=http://spot.fedorapeople.org/chromium/F$releasever/
enabled=0
gpgcheck=0
END

yum --enablerepo=chromium install chromium nss-mdns.x86_64