cultural exception

Too much good leads to bad (part 2)

Tue, 28 Nov 2006 16:46:47 +0000

I wrote why too much tricks could leads to unexpected behaviors here. Facing some corrupted tar archive, some requirements are needed in order to repair them successfully:

programming skills
knowledge on how the corruption occured
basic informations on tar format
times
perseverance

Given my affinity for perl the choice was obvious. In fact perl fits pretty well in strings processing (scoop).
The relevant part on tar format can be found in the manual pages archives(4)

#define TBLOCK 512 /* length of tar header and data blocks */

For the record I 'll put my code in a simplify version. Through trial&error it is necessary to expand the regular expression section to match all intruders.

#!/usr/local/bin/perl

my $file=$ARGV[0];
my $new_file=$ARGV[1];
my $akt_pos=0;
my $record="";
my $size_buf=1024;
my $eof=0;

open (FH,"<$file") or die "Couldn't open file $file\n";
open (FH_NEW,">$new_file") or die "Couldn't open file $new_file\n";

until ( eof(FH)){
if (length($record) < $size_buf) {
$akt_pos = tell (FH);
if (read (FH,$txt, $size_buf-length($record)) == $size_buf-length($record)){
$record.=$txt;
} else {
seek(FH,$akt_pos,0);
while ((read(FH,$txt,1) == 1)){
$record .= $txt;
$eof=1;
}
}
} else {
$c=0;
}
$txt = $record;
$record =~ s/a \.\/somepath\/.* \d+K\n//mg;
$record =~ s/a .* symbolic link to .*\n//mg;
$record =~ s/a \.\/somepath\/.* excluded\n//mg;
if ( ! $eof ) {
if ( $record eq $txt ) {
if (length($record) == $size_buf) {
printf (FH_NEW "%.512s",$record);
$record=substr($record,512);
}
}
} else {
printf (FH_NEW "%s", $record);
}
}
close(FH);
close(FH_NEW);

On a last note, I would recommend using star as an improved version over many tar implementation including GNU tar and stock Solaris version.
bbr

Too much good leads to bad

Tue, 28 Nov 2006 16:05:31 +0000

Each cook knows it for sure, don't follow the "too much" path if you want to succeed. This rule can although be applied to computer. Take UNIX, it is a real cool OS. You can do art on the command line, piping tools together, using complex regular expression,... As a technogeek working on leading edge projects it might be ok but as sysadmin for a bank it is probably plain wrong. Just keep things simple.

What's wrong with tar, gzip and nohup

tar is used to create archive. It is a very usefull utility, well known and in UNIX for years. If you want to create an archive you provide a filename or a device where it will be create. After that you can compress it with gzip to save some space. Supposed you want to archive a lot of files, depending on I/O performance it can take hours. It is then a good practice to use nohup to stay imune to HUP signal. Put this 2 lines (tar and gzip) in a script, start it with nohup and you are fine...
unless you want too much good. Of course (and it is not new) you can use dash instead of filename within tar. It will then write your archive on STDOUT which you can redirect through pipe to gzip. Put nohup in front of it. You have now a much more elegant one liner doing exactly the same.
Not quite.
The man page for nohup state
"If nohup.out is not writable in the current directory, out-
put is redirected to $HOME/nohup.out. If a file is created,
the file will have read and write permission (600, see
chmod(1)). If the standard error is a terminal, it is
redirected to the standard output, otherwise it is not
redirected. The priority of the process run by nohup is not
altered."
OUTCH
Unfortunatly tar writes progress messages on STDERR. This will under certain circonstances corrupts the tarfile, making it completely needless!
If you have this kind of story I'll be glad to hear from it. Next article will deal with the recovery process from a 80 GB corrupted tar archive.
bbr

Oops I did it again

Tue, 28 Nov 2006 16:00:00 +0000

As an aficionados of numbers, statistics and movement I worked with gnucash a lot.
Back to 2001 I 've started on Mandrake which was my distro of choice at that time. Gnucash was included, everything was fine. But then the dependencies hell drove me mad. I still can't accept one have to update the kernel and other major libraries just because you want the last version of "ethereal". After some tests with redhat and debian (no improvements) I decided to give Solaris x86 a try. It was a natural move because Solaris was (and still is) my main OS at work.
No gnucash binaries for Solaris exist on the internet and I decided to compile this stuff by my own ...

2 months later

Today I laugh about it but at that time it costed me a lot of gray hairs. Gnucash is one (if not the one) of the hardest software to compile. Internet attests it in a lot of places. I remember how funny some emails are, full of frustration and anger. It was a painfull process but I managed to compile gnucash in the version 1.8.6 and since I used it almost everyday.

today

Things aren't better from a software point of view. You still need slib, guile, a ton of old gnome 1.x libraries and packages. The gnucash developers didn't managed to create a gnome 2.x version due to short ressources (if you can hack C please consider getting involved). But there is some improvment from other fronts:

Solaris 10 come otb with gcc
DTrace is a great help to debug things
blastwave provides a lot of needed libraries

DTrace

The best DTrace recipe I've saw so far and for me the most usefull is the one that reads every bytes landing in /dev/null. I urge the creator of this little gem to release it.

Blastwave

Really cool site for Solaris software. If you use it please make a donation. I've contacted them to maintain gnucash in the CSW tree and hope that you will soon (less than 2 months) find gnucash in the list of packages. Eventually I'll report on progress.
bbr

certified

Tue, 28 Nov 2006 15:58:04 +0000

The result arrived over the weekend and guess what I
passed. Call me SCSA
After 15+ years experience with Solaris, everyone should pass the test "mit links" or "les doigts dans le nez" but indeed those tests didn't say much. You've well prepared the exam, you 've enough money and time to take the trainings. Companies should't make their mind and shouldn't choose employes or consultants based on certification.
But reality is quiet different as nowadays employe-time is very expensive! Only
few companies are willing to spend money for recruiting and managing human ressources. Certification is then some kind of insurance.
Anyway now that I'm on my own, I've to play the game.

bbr

L'angoisse de la page blanche

Tue, 28 Nov 2006 13:59:20 +0000

Georges Brassens says in one of his song one will always remember the first time you've hugged someone. Not sure if this apply to blogs. However, now that I have more control over my time (being no longer at Sun) I'll try to write down some of my thoughts. I choose a handful subject to ordonate my chaos but bloglines, for now, doesn't support classification. The last decade was spent in the Frankfurt area dealing with security issues at customer sites. It comes without surprise that my first blog reflect that. Something really clever when it comes to IT security is to use parallel to make your point. IT security is for most obscure. A good argumentation using common sense on well known subject will help on much case. Cars, castles, banks, towns architecture, evolution,...no limit to your fantasy.
Look at highways, in most of the countries in europe it is quiet a safe place to drive. At least the statistic in France says so here. Better at 200 kmh beetween Munich and Frankfurt than in Paris on a bicycle. Want some insigth how to drive in germany? Look here . In belgium, highways are like broadway, at least for my kinds. All this lights are for them amazing. That's why we have to drive by night when we visit our french family. I'm not sure how this affects security but what they are doing very well is communication. Every 20 km you will see beneath the road a big ads on addressing one valid point about security. At the moment it deals with belts and it is for kids, the next drivers. It's not a big deal but you can't miss it, and so your kids (remember the lights?).
During 10 years in diverse companies, I never saw any kind of communication on IT security. A lot have to be said. No idea? You should look into your company security policy! No one. Starts with "don't give your password away", "don't open unknow attachement", "lock your display while you're away",...Communication is a huge need and can help a lot. Think escalation process, information flow, CERT, ....
To be honest at Sun we did recieved some emails about it security!bbr