Shallow Thoughts : : Dec

The Joys of Shell Pipelines ...

A friend called me for help with a sysadmin problem they were having at work. The problem: find all files bigger than one gigabyte, print all the filenames, add up all the sizes and print the total. And for some reason (not explained to me) they needed to do this all in one command line.

This is Unix, so of course it's possible somehow! The obvious place to start is with the find command, and man find showed how to find all the 1G+ files:

find / -size +1G

(Turns out that's a GNU find syntax, and BSD find, on OS X, doesn't support it. I left it to my friend to check man find for the OS X equivalent of -size _1G.)

But for a problem like this, it's pretty clear we'd need to get find to execute a program that prints both the filename and the size. Initially I used ls -ls, but Saz (who was helping on IRC) pointed out that du on a file also does that, and looks a bit cleaner. With find's unfortunate syntax, that becomes:

find / -size +1G -exec du "{}" \;

But now we needed awk, to collect and add up all the sizes while printing just the filenames. A little googling (since I don't use awk very often) and experimenting led to the final solution:

find / -size +1G -exec du "{}" \; | awk '{print $2; total += $1} END { print "Total is", total}'

Ah, the joys of Unix shell pipelines!

Update: Ed Davies suggested an easier way to do the same thing. turns out du will handle it all by itself: du -hc `find . -size +1G` Thanks, Ed!

Tags: linux, CLI, shell, backups, pipelines
[ 17:53 Dec 29, 2006    More linux | permalink to this entry | ]

On the Twelfth Day of Christmas, My True Love Gave to Me ...

At dinner last night, amid the ubiquitous miasma of egregious Christmas music which is inescapable in public places starting in mid November, during "The Twelve Days of Christmas" Dave got a faraway expression in his eyes. My mother asked why, and he explained that he was thinking about the mathematics of the song: how many items of each type have been given by the end, and which items are more numerous?

There are two ways to interpret the song.

On the second day of Christmas, my true love gave to me
Two turtle doves
And a partridge in a pear tree.

So by the second day, you have two turtle doves, and you have the original partridge -- but do you also have a second partridge, as a literal interpretation of the song implies? Or is the song simply repeating all the previous gifts, not implying that they're given again?

Most people seem to assume the latter, but let's take the song literally and assume that on the third day, you get three french hens, plus two more turtle doves (that makes four) and one new partridge (for a total of three).

My first thought was that at time step T, you double what you had in step T-1 (you're getting all the same stuff yet again) and add T for the new gifts. But that's not right: you get a new load of each item (one partridge, two doves, three hens, and so forth) but you don't double all the accumulated extras who are now crowding your back yard. Time to start writing down the sums.

At each time T, the quantity you have of the Jth item is:

	T
NJ,T =	Σ J
	i=J

That's easy: it's just N_J,T = J*T- J*(J-1) (pretend you've given J of the Jth object at each time step; but since you didn't give it before timestep J, subtract all the ones up to timestep J-1).

N_J,T = J * (T - J + 1)

If all you want to know is how many of each item you have at the end (on the 12th day), plug in T-12:

N_J,12 = J * (13 - J)

A quick sanity check: that means you'll have 12 of item 1 (partridges in pear trees), because you've gotten one new one each time, and 12 of item 12 (drummers drumming), which you got in one big noisy box on the last day. Likewise, you'll have 22 each of items 2 (turtle doves, of which you got two every day except the first day) and 11 (pipers piping), which you got on day 11 and again on day 12.

So the curve which interested Dave is an inverted parabola; you get the least number of the first and last gifts, and the largest quantity of the two middle gifts: six geese a'laying and seven swans a'swimming. How many geese and swans do you get in the end? Here's the surprising answer:

N_6,12 = N_7,12 = 6 * 7 = 42

Douglas Adams fans will immediate recognize this as the solution to the ultimate question of Life, the Universe, and Everything. Now you know what the question was!

One last question: how many items, total, of all types will you have by the end of the twelfth day?

Since you already know how many of each item you have, just add them all up:

	12		12		12		12
Ntot =	Σ j * (13-j)	=	Σ (j * 13 - j2)	= 13 *	Σ j	-	Σ j2
	j=1		j=1		j=1		j=1

Fortunately, we know that

A
Σ i	= A (A + 1) / 2
i=1

and

A
Σ i2	= A (A + 1) (2A + 1) / 6
i=1

so we can use those identities to figure out how many total items we'll have:

Ntot	= { 13 * (12 * 13) / 2 } - { 12 * 13 * 25 / 6 }
	= 364

So it turns out that true love packs a present for just about every day of the year into those twelve days!

(And I found an excuse to play with using HTML tables to display equations.)

Tags: science
[ 12:59 Dec 21, 2006    More science | permalink to this entry | ]

Getting a Wacom Tablet Working under Edgy

Another person popped into #gimp today trying to get a Wacom tablet working (this happens every few weeks). But this time it was someone using Ubuntu's new release, "Edgy Eft", and I just happened to have a shiny new Edgy install on my laptop (as well as a Wacom Graphire 2 gathering dust in the closet because I can never get it working under Linux), so I dug out the Graphire and did some experimenting.

And got it working! It sees pressure changes and everything. It actually wasn't that hard, but it did require some changes. Here's what I had to do:

1. Install wacom-tools and xinput
2. Edit /etc/X11/xorg.conf and comment out those ForceDevice lines that say "Tablet PC ONLY".
3. Reconcile the difference between udev creating /dev/input/wacom and xorg.conf using /dev/wacom: you can either change xorg.conf, change /etc/udev/rules.d/65-wacom.rules, or symlink /dev/input/wacom to /dev/wacom (that's what I did for testing, but it won't survive a reboot, so I'll have to pick a device name and make udev and X consistent).

A useful tool for testing is /usr/X11R6/bin/xinput list (part of the xinput package). That's a lot faster than going through GIMP's input device preference panel every time.

I added some comments to Ubuntu's bug 73160, where people had already described some of the errors but nobody had posted the steps to work around the problems.

While I was fiddling with GIMP on the laptop, I decided to install the packages I needed to build the latest CVS GIMP there. It needed a few small tweaks from the list I used on Dapper. I've updated the package list on my GIMP Building page accordingly.

Tags: linux, X11, imaging, gimp, ubuntu
[ 16:12 Dec 09, 2006    More linux | permalink to this entry | ]