Monday, August 20, 2007

Perl script to remove duplicate files created by itunes

The following scripts recursively runs and scans directories. If it finds a directory that has files: song.mp3 and song 1.mp3 or song 2.mp3 then it keeps song.mp3 and removes the others. It does a size check before removing just to make sure they are duplicates (it doesnt do content checking or bit by bit comparision).

By the way this is a windows specific script. it needs a bit of customization for unix, etc.
i am using fc to do a binary file compare on vista.

Begin Perl Script: removedupmusicfiles.pl



# This script scans the itunes music folder and removes duplicated files created by itunes.
# run this as: /perl removedupmusicfiles.pl

sub traverse($)
{
my($dir) = @_;


# print "\nTraverse: $dir";
removeDuplicates($dir);


local *DIR;
opendir (DIR, $dir) or die "Could not open directory $dir: $!";

while (defined(my $f = readdir (DIR)))
{
# Only recurse on directories, which do not start with '.', and skip symbolic links
if (-d "$dir/$f" &&
!(-l "$dir/$f") &&
($f !~ /^\.{1,2}$/)
)
{
traverse ("$dir/$f");
}
}

}

sub removeDuplicates($)
{
my($dir) = @_;
my %fnamehash;


# print "\nremoveDuplicates: $dir";

local *DIR;
opendir (DIR, $dir) or die "Could not open directory $dir: $!";

while (defined(my $f = readdir (DIR)))
{
#my $fsize = -s $f;
$fnamehash{$f} = $f;
}


foreach $f ( keys %fnamehash ) {

# if you find files like "foo 1.mp3"
if($f =~ /(.+)\s\d\.mp3/) {

#print "\n$dir: $f could possibly be duplicate of $orig";

$orig = "$dir/$1.mp3";


if(-e $orig) {
my $fsize = -s "$dir/$f";
my $osize = -s "$orig";

#print "\n$f Size: $fsize could be duplicate of $orig Size: $osize ";

if($osize == $fsize) {

my $cmd = "\"$orig\" \"$dir/$f\"";
$cmd =~ s/\//\\/g;
my $output = `fc /B $cmd`;
if ($output =~ /no differences encountered/){
my $name = "$dir/$f";
print "\n$dir/$f ( $fsize bytes) is a duplicate of $orig ( $osize bytes ) ";
print "\nRemoving: $name";

unlink($name) || die "Cannont unlink $name: $!";
}


}
}
}

}
}


$dir = shift;
traverse($dir);








End Perl Script: removedupmusicfiles.pl

itunes management

itunes presents several problems for an advanced user to work around. If you have multiple computers, keeping the libraries in the different computers synchronized is a difficult and non-trivial task even for an advanced user.





I finally copied all my music into the new vista PC with a 300G hard drive. However, due to the paranoia about losing any content, i ended up with multiple copies of my music files in different harddrives and computers. Beyond Compare and Check Identical Files are the 2 software that i used to minimize file duplication. But a strange thing happens when you move files and then get itunes to rescan the folders. itunes creates duplicate copies of files itself (and i have not kept track of the exact conditions that trigger this). For example, in your itunes folder: c:\users\joe\itunesmusic\myalbum\song.mp3 existed before itunes would create: c:\users\joe\itunesmusic\myalbum\song 1.mp3





This effectively duplicated my entire collection. i had to create a perl script to clean this mess (see my other post today on this)


I then used: ITLU (itunes library updater) utility to cleanup itunes database. After removing the files, running itlu can scan and identify orphan entries and it will clean them up.

Wednesday, August 08, 2007

Firefox and thunderbird woes

Firefox 100% CPU hog and freeze symptoms

Running: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.6) Gecko/20070725 Firefox/2.0.0.6

Firefox is not getting better in stability in some corner cases. Some of the webpages (probably content rich pages) seem to quickly hang the laptop. I found this link that recommends disabling flash thru flashblock plugin and java:
http://blog.taragana.com/index.php/archive/how-to-solve-100-cpu-hogging-by-firefox-in-2-steps/

im going to try that.

But my Thunderbird woes are worse than Firefox. I would love to upgrade to TB 2.0 but the
remove duplicate messages plugin:
https://addons.mozilla.org/en-US/thunderbird/addon/956

is not ready for TB 2.0

i like the tagging feature in tb2 and would like to use it. i am moving away from using folders.
I have bought into the notion of tag+search is much better than ontology and classification.
I keep all of my mail in a handful of folders these days and i search them using gmail like expressions f:santa s:christmas using the gmailui plugin. But TB is not quite elegant in handling folders with large number of messages.
Case in point: http://kb.mozillazine.org/Duplicate_messages_received
and
http://kb.mozillazine.org/Thunderbird_:_Tips_:_Compacting_Folders

With TB's instabilities and without remove duplicate messages plugin, my mail handling will not be productive.

I am fondly looking forward to the day when my TB (or any free competitor) would work reliably without choking, freezing, duplicating, etc.

Tuesday, August 07, 2007

Using free LAME encoder with MediaMonkey

I have finally figured this out. It was actually pretty simple. MediaMonkey says this in it's help search for LAME:

>>Unlimited MP3 encoding - Allows you to encode MP3s in an unlimited fashion (the free version of MediaMonkey includes a time-limited LAME MP3 encoder. If you're so inclined, you can manually replace it with the free LAME encoder, or you can just upgrade to MediaMonkey Gold).

You dont have to upgrade just to get the free LAME encoder. Although i did not find this through multiple searches on this topic, here is whatneeds to be done:
o google LAME and it will get you to LAME sourceforge site
o click on version (i did 3.97). It takes you to the software downloads page. click on "LAME related links". Click on LAME binaries for Windows link. It shows Rarewares.org site reference.
o Download lame_enc.dll from this site.
o goto c:\program files\mediamonkey directory. Rename the LAME_ENC.dll file to somethingelse. Copy the file you grabbed.
o Happy rippin...
o I use the Standard Very High Quality+ format, which is VBR/192kbps. I have read that this is one of the best tradeoff between filesize and bitrate/quality.

Monday, August 06, 2007

Lots of posts around digital audio

stumbled across this while researching how to replace the encoder in mediamonkey to the free lame encoder. Im yet to find an answer to that....