List a directory with millions files on linux

I needed to list all the files in a directory which had a lot of files, actually 4060276 to be exact. Usually I do this with ls or find, but with ls it used over 600 MB memory, and a lot of time. So I set out to find a better way of doing this. I found a post “You can list a directory containing 8 million files! But not with ls..” which explained the problem. But I wasn’t happy with compiling a C program to do this. That’s not very easy to do on all servers, so, as the Perl guy I am, I resorted to Perl to find an easier way. And as usually, Perl ftw 🙂

Turns out the readdir function can do this directly:

#!/usr/bin/perl

my $dir = $ARGV[0];
opendir(my $dh, $dir) || die;
while(my $file = readdir $dh) {
   print "$dir/$file\n";
}
closedir $dh;

The nice part about doing it this way is you get the files right away. With ls you have to wait until it has read all the files before it starts to dump the content to disk.