]>
gitweb.pimeys.fr Git - scripts-20-100.git/blob - mails/mailbox_to_maildir.pl
5 # Trouvé sur le web pour transformer un fichier mailbox en
6 # un dossier au format maildir
9 # $Id: mb2md.pl,v 1.26 2004/03/28 00:09:46 juri Exp $
11 # mb2md-3.20.pl Converts Mbox mailboxes to Maildir format.
15 # currently maintained by:
16 # Juri Haberland <juri@koschikode.com>
20 # This script's web abode is http://batleth.sapienti-sat.org/projects/mb2md/ .
21 # For a changelog see http://batleth.sapienti-sat.org/projects/mb2md/changelog.txt
23 # The Mbox -> Maildir inner loop is based on qmail's script mbox2maildir, which
24 # was kludged by Ivan Kohler in 1997 from convertandcreate (public domain)
25 # by Russel Nelson. Both these convert a single mailspool file.
27 # The qmail distribution has a maildir2mbox.c program.
32 # Reads a directory full of Mbox format mailboxes and creates a set of
33 # Maildir format mailboxes. Some details of this are to suit Courier
34 # IMAP's naming conventions for Maildir mailboxes.
36 # http://www.inter7.com/courierimap/
38 # This is intended to automate the conversion of the old
39 # /var/spool/mail/blah file - with one call of this script - and to
40 # convert one or more mailboxes in a specifed directory with separate
41 # calls with other command line arguments.
43 # Run this as the user - in these examples "blah".
45 # This version supports conversion of:
47 # Date The date-time in the "From " line of the message in the
48 # Mbox format is the date when the message was *received*.
49 # This is transformed into the date-time of the file which
50 # contains the message in the Maildir mailbox.
52 # This relies on the Date::Parse perl module and the utime
55 # The script tries to cope with errant forms of the
56 # Mbox "From " line which it may encounter, but if
57 # there is something really screwy in a From line,
58 # then perhaps the script will fail when "touch"
59 # is given an invalid date. Please report the
60 # exact nature of any such "From " line!
68 # In the Mbox message, flags for these are found in the
69 # "Status: N" or "X-Status: N" headers, where "N" is 0
70 # or more of the following characters in the left column.
72 # They are converted to characters in the right column,
73 # which become the last characters of the file name,
74 # following the ":2," which indicates IMAP message status.
80 # D -> T Tagged for Deletion (Trash)
82 # This is based on the work of Philip Mak who wrote a
83 # completely separate Mbox -> Maildir converter called
84 # perfect_maildir and posted it to the Mutt-users mailing
85 # list on 25 December 2001:
87 # http://www.mail-archive.com/mutt-users@mutt.org/msg21872.html
89 # Michael Best originally integrated those changes into mb2md.
92 # In addition, the names of the message files in the Maildir are of a
93 # regular length and are of the form:
95 # 7654321.000123.mbox:2,xxx
97 # Where "7654321" is the Unix time in seconds when the script was
98 # run and "000123" is the six zeroes padded message number as
99 # messages are converted from the Mbox file. "xxx" represents zero or
100 # more of the above flags F, R, S or T.
103 # ---------------------------------------------------------------------
109 # Run this as the user of the mailboxes, not as root.
113 # mb2md [-c] -m [-d destdir]
114 # mb2md [-c] -s sourcefile [-d destdir]
115 # mb2md [-c] -s sourcedir [-l wu-mailboxlist] [-R|-f somefolder] [-d destdir] [-r strip_extension]
117 # -c use the Content-Length: headers (if present) to find the
118 # beginning of the next message
119 # Use with caution! Results may be unreliable. I recommend to do
120 # a run without "-c" first and only use it if you are certain,
121 # that the mbox in question really needs the "-c" option
123 # -m If this is used then the source will
124 # be the single mailbox at /var/spool/mail/blah for
125 # user blah and the destination mailbox will be the
126 # "destdir" mailbox itself.
129 # -s source Directory or file relative to the user's home directory,
130 # which is where the the "somefolders" directories are located.
131 # Or if starting with a "/" it is taken as a
132 # absolute path, e.g. /mnt/oldmail/user
136 # A single mbox file which will be converted to
139 # -R If defined, do not skip directories found in a mailbox
140 # directory, but runs recursively into each of them,
141 # creating all wanted folders in Maildir.
142 # Incompatible with '-f'
144 # -f somefolder Directories, relative to "sourcedir" where the Mbox files
145 # are. All mailboxes in the "sourcedir"
146 # directory will be converted and placed in the
147 # "destdir" directory. (Typically the Inbox directory
148 # which in this instance is also functioning as a
149 # folder for other mailboxes.)
151 # The "somefolder" directory
152 # name will be encoded into the new mailboxes' names.
153 # See the examples below.
155 # This does not save an UW IMAP dummy message file
156 # at the start of the Mbox file. Small changes
157 # in the code could adapt it for looking for
158 # other distinctive patterns of dummy messages too.
160 # Don't let the source directory you give as "somefolders"
161 # contain any "."s in its name, unless you want to
162 # create subfolders from the IMAP user's point of
163 # view. See the example below.
165 # Incompatible with '-f'
168 # -d destdir Directory where the Maildir format directories will be created.
169 # If not given, then the destination will be ~/Maildir .
170 # Typically, this is what the IMAP server sees as the
171 # Inbox and the folder for all user mailboxes.
172 # If this begins with a '/' the path is considered to be
173 # absolute, otherwise it is relative to the users
176 # -r strip_ext If defined this extension will be stripped from
177 # the original mailbox file name before creating
178 # the corresponding maildir. The extension must be
179 # given without the leading dot ("."). See the example below.
181 # -l WU-file File containing the list of subscribed folders. If
182 # migrating from WU-IMAP the list of subscribed folders will
183 # be found in the file called .mailboxlist in the users
184 # home directory. This will convert all subscribed folders
186 # /bin/mb2md -s mail -l .mailboxlist -R -d Maildir
187 # and for all users in a directory as root you can do the
189 # for i in *; do echo $i;su - $i -c "/bin/mb2md -s mail -l .mailboxlist -R -d Maildir";done
195 # We have a bunch of directories of Mbox mailboxes located at
196 # /home/blah/oldmail/
198 # /home/blah/oldmail/fffff
199 # /home/blah/oldmail/ggggg
200 # /home/blah/oldmail/xxx/aaaa
201 # /home/blah/oldmail/xxx/bbbb
202 # /home/blah/oldmail/xxx/cccc
203 # /home/blah/oldmail/xxx/dddd
204 # /home/blah/oldmail/yyyy/huey
205 # /home/blah/oldmail/yyyy/duey
206 # /home/blah/oldmail/yyyy/louie
208 # With the UW IMAP server, fffff and ggggg would have appeared in the root
209 # of this mail server, along with the Inbox. aaaa, bbbb etc, would have
210 # appeared in a folder called xxx from that root, and xxx was just a folder
211 # not a mailbox for storing messages.
213 # We also have the mailspool Inbox at:
215 # /var/spool/mail/blah
218 # To convert these, as user blah, we give the first command:
222 # The main Maildir directory will be created if it does not exist.
223 # (This is true of any argument options, not just "-m".)
225 # /home/blah/Maildir/
227 # It has the following subdirectories:
229 # /home/blah/Maildir/tmp/
230 # /home/blah/Maildir/new/
231 # /home/blah/Maildir/cur/
233 # Then /var/spool/blah file is read, split into individual files and
234 # written into /home/blah/Maildir/cur/ .
236 # Now we give the second command:
238 # mb2md -s oldmail -R
240 # This reads recursively all Mbox mailboxes and creates:
242 # /home/blah/Maildir/.fffff/
243 # /home/blah/Maildir/.ggggg/
244 # /home/blah/Maildir/.xxx/
245 # /home/blah/Maildir/.xxx.aaaa/
246 # /home/blah/Maildir/.xxx.bbbb/
247 # /home/blah/Maildir/.xxx.cccc/
248 # /home/blah/Maildir/.xxx.aaaa/
249 # /home/blah/Maildir/.yyyy/
250 # /home/blah/Maildir/.yyyy.huey/
251 # /home/blah/Maildir/.yyyy.duey/
252 # /home/blah/Maildir/.yyyy.louie/
254 # The result, from the IMAP client's point of view is:
256 # Inbox -----------------
258 # | fffff -----------
259 # | ggggg -----------
261 # - xxx -------------
267 # - yyyy ------------
272 # Note that although ~/Maildir/.xxx/ and ~/Maildir/.yyyy may appear
273 # as folders to the IMAP client the above commands to not generate
274 # any Maildir folders of these names. These are simply elements
275 # of the names of other Maildir directories. (if you used '-R', they
276 # whill be able to act as normal folders, containing messages AND folders)
278 # With a separate run of this script, using just the "-s" option
279 # without "-f" nor "-R", it would be possible to create mailboxes which
280 # appear at the same location as far as the IMAP client is
281 # concerned. By having Mbox mailboxes in some directory:
282 # ~/oldmail/nnn/ of the form:
284 # /home/blah/oldmail/nn/xxxx
285 # /home/blah/oldmail/nn/yyyyy
289 # mb2md -s oldmail/nn
291 # will create two new Maildirs:
293 # /home/blah/Maildir/.xxx/
294 # /home/blah/Maildir/.yyyy/
296 # Then what used to be the xxx and yyyy folders now function as
297 # mailboxes too. Netscape 4.77 needed to be put to sleep and given ECT
298 # to recognise this - deleting the contents of (Win2k example):
300 # C:\Program Files\Netscape\Users\uu\ImapMail\aaa.bbb.ccc\
302 # where "uu" is the user and "aaa.bbb.ccc" is the IMAP server
304 # I often find that deleting all this directory's contents, except
305 # "rules.dat", forces Netscape back to reality after its IMAP innards
306 # have become twisted. Then maybe use File > Subscribe - but this
307 # seems incapable of subscribing to folders.
309 # For Outlook Express, select the mail server, then click the
310 # "IMAP Folders" button and use "Reset list". In the "All"
311 # window, select the mailboxes you want to see in normal
315 # This script did not recurse subdirectories or delete old mailboxes, before addition of the '-R' parameter :)
317 # Be sure not to be accessing the Mbox mailboxes while running this
318 # script. It does not attempt to lock them. Likewise, don't run two
319 # copies of this script either.
322 # Trickier usage . . .
323 # ====================
325 # If you have a bunch of mailboxes in a directory ~/oldmail/doors/
326 # and you want them to appear in folders such as:
328 # ~/Maildir/.music.bands.doors.Jim
329 # ~/Maildir/.music.bands.doors.John
331 # etc. so they appear in an IMAP folder:
333 # Inbox -----------------
342 # Then you could rename the source directory to:
344 # ~/oldmail/music.bands.doors/
348 # mb2md -s oldmail -f music.bands.doors
351 # Or simply use '-R' switch with:
352 # mb2md -s oldmail -R
355 # Stripping mailbox extensions:
356 # =============================
358 # If you want to convert mailboxes that came for example from
359 # a Windows box than you might want to strip the extension of
360 # the mailbox name so that it won't create a subfolder in your
364 # You have several mailboxes named Trash.mbx, Sent.mbx, Drafts.mbx
365 # If you don't strip the extension "mbx" you will get the following
379 # This is more than ugly!
381 # mb2md -s oldmail -r mbx
383 # Note: don't specify the dot! It will be stripped off
386 #------------------------------------------------------------------------------
395 # print the usage message
399 print " mb2md [-c] -m [-d destdir]\n";
400 print " mb2md [-c] -s sourcefile [-d destdir]\n";
401 die " mb2md [-c] -s sourcedir [-l wu-mailboxlist] [-R|-f somefolder] [-d destdir] [-r strip_extension]\n";
405 getopts
('d:f:chms:r:l:R', \
%opts) || usage
();
406 usage
() if ( defined($opts{h
})
407 || (!defined($opts{m
}) && !defined($opts{s
})) );
409 # Get uid, username and home dir
410 my ($name, $passwd, $uid, $gid, $quota, $comment, $gcos, $homedir, $shell) = getpwuid($<);
412 # Get arguments and determine source
413 # and target directories.
414 my $mbroot = undef; # this is the base directory for the mboxes
415 my $mbdir = undef; # this is an mbox dir relative to the $mbroot
416 my $mbfile = undef; # this is an mbox file
418 my $strip_ext = undef;
419 my $use_cl = undef; # defines whether we use the Content-Length: header if present
421 # if option "-c" is given, we use the Content-Length: header if present
422 # dangerous! may be unreliable, as the whole CL stuff is a bad idea
423 if (defined($opts{c
}))
430 # first, if the user has gone the -m option
431 # we simply convert their mailfile
432 if (defined($opts{m
}))
434 if (defined($ENV{'MAIL'})) {
435 $mbfile = $ENV{'MAIL'};
436 } elsif ( -f
"/var/spool/mail/$name" ) {
437 $mbfile = "/var/spool/mail/$name"
438 } elsif ( -f
"/var/mail/$name" ) {
439 $mbfile = "/var/mail/$name"
441 die("I searched \$MAIL, /var/spool/mail/$name and /var/mail/$name, ".
442 "but I couldn't find your mail spool file - ");
445 # see if the user has specified a source directory
446 elsif (defined($opts{s
}))
448 # if opts{s} doesn't start with a "/" then
449 # it is a subdir of the users $home
450 # if it does start with a "/" then
451 # let's take $mbroot as a absolut path
452 $opts{s
} = "$homedir/$opts{s}" if ($opts{s
} !~ /^\
//);
454 # check if the given source is a mbox file
460 # otherwise check if it is a directory
464 # get rid of trailing /'s
467 # check if we have a specified sub directory,
468 # otherwise the sub directory is '.'
469 if (defined($opts{f
}))
472 # get rid of trailing /'s
477 # otherwise we have an error
480 die("Fatal: Source is not an mbox file or a directory!\n");
486 defined($opts{d
}) && ($dest = $opts{d
}) || ($dest = "Maildir");
487 # see if we have anything to strip
488 defined($opts{r
}) && ($strip_ext = $opts{r
});
490 if((defined($opts{R
}))&&(defined($opts{f
}))) { die "No recursion with \"-f\"";}
491 # Get list of folders
493 if(defined($opts{l
}))
495 open (LIST
,$opts{l
}) or die "Could not open mailbox list $opts{l}: $!";
500 # if the destination is relative to the home dir,
501 # check that the home dir exists
502 die("Fatal: home dir $homedir doesn't exist.\n") if ($dest !~ /^\// && ! -e
$homedir);
505 # form the destination value
506 # slap the home dir on the front of the dest if the dest does not begin
508 $dest = "$homedir/$dest" if ($dest !~ /^\
//);
509 # get rid of trailing /'s
513 # Count the number of mailboxes, or
514 # at least files, we found.
515 my $mailboxcount = 0;
517 # Since we'll be making sub directories of the main
518 # Maildir, we need to make sure that the main maildir
522 # Now we do different things depending on whether we convert one mbox
523 # file or a directory of mbox files
524 if (defined($mbfile))
526 if (!isamailboxfile
($mbfile))
528 print "Skipping $mbfile: not a mbox file\n";
532 print "Converting $mbfile to maildir: $dest\n";
533 # this is easy, we just run the convert function
534 &convert
($mbfile, $dest);
537 # if '-f' was used ...
538 elsif (defined($mbdir))
540 print "Converting mboxdir/mbdir: $mbroot/$mbdir to maildir: $dest/\n";
542 # Now set our source directory
543 my $sourcedir = "$mbroot/$mbdir";
545 # check that the directory we are supposed to be finding mbox
546 # files in, exists and is a directory
547 -e
$sourcedir or die("Fatal: MBDIR directory $sourcedir/ does not exist.\n");
548 -d
$sourcedir or die("Fatal: MBDIR $sourcedir is not a directory.\n");
551 &convertit
($mbdir,"");
553 # Else, let's work in $mbroot
556 opendir(SDIR
, $mbroot)
557 or die("Fatal: Cannot open source directory $mbroot/ \n");
560 while (my $sourcefile = readdir(SDIR
))
562 if (-d
"$mbroot/$sourcefile") {
563 # Recurse only if requested (to be changed ?)
564 if (defined($opts{R
})) {
565 print "convertit($sourcefile,\"\")\n";
566 &convertit
($sourcefile,"");
568 print("$sourcefile is a directory, but '-R' was not used... skipping\n");
571 elsif (!-f
"$mbroot/$sourcefile")
573 print "Skipping $mbroot/$sourcefile : not a file nor a dir\n";
576 elsif (!isamailboxfile
("$mbroot/$sourcefile"))
578 print "Skipping $mbroot/$sourcefile : not a mbox file\n";
583 &convertit
($sourcefile,"");
585 } # end of "while ($sfile = readdir(SDIR))" loop.
587 printf("$mailboxcount files processed.\n");
593 # My debbugging placeholder I can put somewhere to show how far the script ran.
594 # die("So far so good.\n\n");
596 # The isamailboxfile function
597 # ----------------------
599 # Here we check if the file is a mailbox file, not an address-book or
601 # If file is empty, we say it is a mbox, to create it empty.
603 # Returns 1 if file is said mbox, 0 else.
606 return 1 if(-z
$mbxfile);
607 sysopen(MBXFILE
, "$mbxfile", O_RDONLY
) or die "Could not open $mbxfile ! \n";
620 # The convertit function
621 # -----------------------
623 # This function creates all subdirs in maildir, and calls convert()
624 # for each mbox file.
625 # Yes, it becomes the 'main loop' :)
628 # Get subdir as argument
629 my ($dir,$oldpath) = @_;
631 $oldpath =~ s/\/\///;
633 # Skip files beginning with '.' since they are
634 # not normally mbox files nor dirs (includes '.' and '..')
637 print "Skipping $dir : name begins with a '.'\n";
640 my $destinationdir = $dir;
641 my $temppath = $oldpath;
643 # We don't want to have .'s in the $targetfile file
644 # name because they will become directories in the
645 # Maildir. Therefore we convert them to _'s
646 $temppath =~ s/\./\_/g;
647 $destinationdir =~ s/\./\_/g;
649 # Appending $oldpath => path is only missing $dest
650 $destinationdir = "$temppath.$destinationdir";
652 # Converting '/' to '.' in $destinationdir
653 $destinationdir =~s/\/+/\
./g
;
656 my $srcdir="$mbroot/$oldpath/$dir";
658 printf("convertit(): Converting $dir in $mbroot/$oldpath to $dest/$destinationdir\n");
659 &maildirmake
("$dest/$destinationdir");
660 print("destination = $destinationdir\n");
662 opendir(SUBDIR
, "$srcdir") or die "can't open $srcdir !\n";
663 my @subdirlist=readdir(SUBDIR
);
665 foreach (@subdirlist) {
668 print("convertit($_,\"$oldpath/$dir\")\n");
669 &convertit
($_,"$oldpath/$dir");
672 # Source file verifs ....
674 return if(defined($opts{l
}) && !inlist
("$oldpath/$dir",@flist));
676 if (!isamailboxfile
("$mbroot/$oldpath/$dir"))
678 print "Skipping $dir (is not mbox)\n";
682 # target file verifs...
684 # if $strip_extension is defined,
685 # strip it off the $targetfile
686 defined($strip_ext) && ($destinationdir =~ s/\.$strip_ext$//);
687 &convert
("$mbroot/$oldpath/$dir","$dest/$destinationdir");
691 # The maildirmake function
692 # ------------------------
694 # It does the same thing that the maildirmake binary that
695 # comes with courier-imap distribution
700 -d
$_ or mkdir $_,0700 or die("Fatal: Directory $_ doesn't exist and can't be created.\n");
702 -d
"$_/tmp" or mkdir("$_/tmp",0700) or die("Fatal: Unable to make $_/tmp/ subdirectory.\n");
703 -d
"$_/new" or mkdir("$_/new",0700) or die("Fatal: Unable to make $_/new/ subdirectory.\n");
704 -d
"$_/cur" or mkdir("$_/cur",0700) or die("Fatal: Unable to make $_/cur/ subdirectory.\n");
708 # The inlist function
709 # ------------------------
711 # It checks that the folder to be converted is in the list of subscribed
716 my ($file,@flist) = @_;
718 # Get rid of the first / if any
720 foreach my $folder (@flist) {
722 if ($file eq $folder) {
728 print "$file is not in list\n";
731 print "$file is in list\n";
739 # The convert function
740 # ---------------------
742 # This function does the down and dirty work of
743 # actually converting the mbox to a maildir
747 # get the source and destination as arguments
748 my ($mbox, $maildir) = @_;
750 printf("Source Mbox is $mbox\n");
751 printf("Target Maildir is $maildir \n") ;
753 # create the directories for the new maildir
755 # if it is the root maildir (ie. converting the inbox)
756 # these already exist but thats not a big issue
758 &maildirmake
($maildir);
760 # Change to the target mailbox directory.
764 # Converts a Mbox to multiple files
766 # This is adapted from mbox2maildir.
768 # Open the Mbox mailbox file.
771 if (sysopen(MBOX
, "$mbox", O_RDONLY
))
773 #printf("Converting Mbox $mbox . . . \n");
777 die("Fatal: unable to open input mailbox file: $mbox ! \n");
780 # This loop scans the input mailbox for
781 # a line starting with "From ". The
782 # "^" before it is pattern-matching
783 # lingo for it being at the start of a
786 # Each email in Mbox mailbox starts
787 # with such a line, which is why any
788 # such line in the body of the email
789 # has to have a ">" put in front of it.
791 # This is not required in a Maildir
792 # mailbox, and some majik below
793 # finds any such quoted "> From"s and
794 # gets rid of the "> " quote.
796 # Each email is put in a file
797 # in the cur/ subdirectory with a
800 # nnnnnnnnn.cccc.mbox:2,XXXX
803 # "nnnnnnnnn" is the Unix time since
804 # 1970 when this script started
805 # running, incremented by 1 for
806 # every email. This is to ensure
807 # unique names for each message
810 # ".cccc" is the message count of
811 # messages from this mbox.
813 # ".mbox" is just to indicate that
814 # this message was converted from
817 # ":2," is the start of potentially
818 # multiple IMAP flag characters
819 # "XXXX", but may be followed by
822 # This is sort-of compliant with
823 # the Maildir naming conventions
826 # http://www.qmail.org/man/man5/maildir.html
828 # This approach does not involve the
829 # process ID or the hostname, but it is
830 # probably good enough.
832 # When the IMAP server looks at this
833 # mailbox, it will move the files to
834 # the cur/ directory and change their
835 # names as it pleases. In the case
836 # of Courier IMAP, the names will
839 # 995096541.25351.mbox:2,S
841 # with 25351 being Courier IMAP's
842 # process ID. The :2, is the start
843 # of the flags, and the "S" means
844 # that this one has been seen by
845 # the user. (But is this the same
846 # meaning as the user actually
847 # having opened the message to see
848 # its contents, rather than just the
849 # IMAP server having been asked to
850 # list the message's Subject etc.
851 # so the client could list it in the
854 # This contrasts with a message
855 # created by Courier IMAP, say with
856 # a message copy, which is like:
858 # 995096541.25351.zair,S=14285:2,S
860 # where ",S=14285" is the size of the
863 # Courier Maildrop's names are similar
864 # but lack the ":2,XXXX" flags . . .
865 # except for my modified Maildrop
866 # which can deliver them with a
867 # ":2,T" - flagged for deletion.
869 # I have extended the logic of the
870 # per-message inner loop to stop
871 # saving a file for a message with:
873 # Subject: DON'T DELETE THIS MESSAGE -- FOLDER INTERNAL DATA
875 # This is the dummy message, always
876 # at the start of an Mbox format
877 # mailbox file - and is put there
878 # by UW IMAPD. Since quite a few
879 # people will use this for
880 # converting from a UW system,
881 # I figure it is worth it.
883 # I will not save any such message
884 # file for the dummy message.
889 # We want to read the entire Mbox file, whilst
890 # going through a loop for each message we find.
892 # We want to read all the headers of the message,
893 # starting with the "From " line. For that "From "
894 # line we want to get a date.
896 # For all other header lines, we want to store them
897 # in $headers whilst parsing them to find:
899 # 1 - Any flags in the "Status: " or "X-Status: " or
900 # "X-Mozilla-Status: " lines.
902 # 2 - A subject line indicating this is the dummy message
903 # at the start (typically, but not necessarily) of
906 # Once we reach the end of the headers, we will crunch any
907 # flags we found to create a file name. Then, unless this is
908 # the dummy message we create that file and write all the
911 # Then we continue reading the Mbox, converting ">From " to
912 # "From " and writing it to the file, until we reach one of:
914 # 1 - Another "From " line (indicating the start of another
919 # 2 - The end of the Mbox.
921 # In the former case, which we detect at the start of the loop
922 # we need to close the file and touch it to alter its date-time.
924 # In the later case, we also need to close the file and touch
925 # it to alter its date-time - but this is beyond the end of the
932 my $messagecount = 0;
934 # For generating unique filenames for
935 # each message. Initialise it here with
936 # numeric time in seconds since 1970.
939 # Name of message file to delete if we found that
940 # it was created by reading the Mbox dummy message.
942 my $deletedummy = '';
944 # To store the complete "From (address) (date-time)
945 # which delineates the start of each message
950 # Set to 1 when we are reading the header lines,
951 # including the "From " line.
953 # 0 means we are reading the message body and looking
954 # for another "From " line.
958 # Variable to hold all headers (apart from
959 # the first line "From ...." which is not
960 # part of the message itself.
963 # Variable to hold the accumulated characters
964 # we find in header lines of the type:
972 # To build the file name for the message in.
976 # The date string from the "From " line of each
977 # message will be written here - and used by
978 # touch to alter the date-time of each message
979 # file. Put non-date text here to make it
980 # spit the dummy if my code fails to find a
981 # date to write into this.
983 my $receivedate = 'Bogus';
985 # The subject of the message
988 my $previous_line_was_empty = 1;
990 # We record the message start line here, for error
994 # If defined, we use this as the number of bytes in the
995 # message body rather than looking for a /^From / line.
998 # A From lines can either occur as the first
999 # line of a file, or after an empty line.
1000 # Most mail systems will quote all From lines
1001 # appearing in the message, but some will only
1002 # do it when necessary.
1003 # Since we initialise the variable to true,
1004 # we don't need to check for beginning of file.
1008 # exchange possible Windows EOL (CRLF) with Unix EOL (LF)
1012 && $previous_line_was_empty
1013 && (!defined $contentlength)
1016 # We are reading the "From " line which has an
1017 # email address followed by a receive date.
1018 # Turn on the $inheaders flag until we reach
1019 # the end of the headers.
1023 # record the message start line
1027 # If this is not the first run through the loop
1028 # then this means we have already been working
1031 if ($messagecount > 0)
1033 # If so, then close that message file and then
1034 # use utime to change its date-time.
1036 # Note this code should be duplicated to do
1037 # the same thing at the end of the while loop
1038 # since we must close and touch the final message
1039 # file we were writing when we hit the end of the
1043 if ($messagefn ne '') {
1044 my $t = str2time
($receivedate);
1045 utime $t, $t, $messagefn;
1049 # Because we opened the Mbox file without any
1050 # variable, I think this means that we have its
1051 # current line in Perl's default variable "$_".
1052 # So all sorts of pattern matching magic works
1055 # We are currently reading the first line starting with
1056 # "From " which contains the date we want.
1058 # This will be of the form:
1060 # From dduck@test.org Wed Nov 24 11:05:35 1999
1062 # at least with UW-IMAP.
1064 # However, I did find a nasty exception to this in my
1065 # tests, of the form:
1067 # "bounce-MusicNewsletter 5-rw=test.org"@announce2.mp3.com
1069 # This makes it trickier to get rid of the email address,
1070 # but I did find a way. I can't rule out that there would
1071 # be some address like this with an "@" in the quoted
1074 # Unfortunately, testing with an old Inbox Mbox file,
1075 # I also found an instance where the email address
1076 # had no @ sign at all. It was just an email
1077 # account name, with no host.
1079 # I could search for the day of the week. If I skipped
1080 # at least one word of non-whitespace (1 or more contiguous
1081 # non-whitespace characters) then searched for a day of
1082 # the week, then I should be able to avoid almost
1083 # every instance of a day of the week appearing in
1084 # the email address.
1086 # Do I need a failsafe arrangement to provide some
1087 # other date to touch if I don't get what seems like
1088 # a date in my resulting string? For now, no.
1090 # I will take one approach if there is an @ in the
1091 # "From " line and another (just skip the first word
1092 # after "From ") if there is no @ in the line.
1094 # If I knew more about Perl I would probably do it in
1095 # a more elegant way.
1097 # Copy the current line into $fromline.
1101 # Now get rid of the "From ". " =~ s" means substitute.
1102 # Find the word "From " at the start of the line and
1103 # replace it with nothing. The nothing is what is
1104 # between the second and third slash.
1106 $fromline =~ s/^From // ;
1109 # Likewise get rid of the email address.
1110 # This first section is if we determine there is one
1111 # (or more . . . ) "@" characters in the line, which
1112 # would normally be the case.
1114 if ($fromline =~ m/@/)
1116 # The line has at least one "@" in it, so we assume
1117 # this is in the middle of an email address.
1119 # If the email address had no spaces, then we could
1120 # get rid of the whole thing by searching for any number
1121 # of non-whitespace characters (\S) contiguously, and
1122 # then I think a space. Subsitute nothing for this.
1124 # $fromline =~ s/(\S)+ // ;
1126 # But we need something to match any number of non-@
1127 # characters, then the "@" and then all the non-whitespace
1128 # characters from there (which takes us to the end of
1129 # "test.org") and then the space following that.
1131 # A tutorial on regular expressions is:
1133 # http://www.perldoc.com/perl5.6.1/pod/perlretut.html
1135 # Get rid of all non-@ characters up to the first "@":
1137 $fromline =~ s/[^@]+//;
1140 # Get rid of the "@".
1144 # If there was an "@" in the line, then we have now
1145 # removed the first one (lets hope there aren't more!)
1146 # and everything which preceded it.
1148 # we now remove either something like
1149 # '(foo bar)'. eg. '(no mail address)',
1150 # or everything after the '@' up to the trailing
1153 # FIXME: all those regexp should be combined to just one single one
1155 $fromline =~ s/(\((\S*| )+\)|\S+) *//;
1159 # Stash the date-time for later use. We will use it
1160 # to touch the file after we have closed it.
1162 $receivedate = $fromline;
1166 # print "$receivedate is the receivedate of message $messagecount.\n";
1167 # $receivedate = "Wed Nov 24 11:05:35 1999";
1169 # To look at the exact date-time of files:
1171 # ls -lFa --full-time
1173 # End of handling the "From " line.
1177 # Now process header lines which are not the "From " line.
1179 if ( ($inheaders eq 1)
1183 # Now we are reading the header lines after the "From " line.
1184 # Keep looking for the blank line which indicates the end of the
1188 # ".=" means append the current line to the $headers
1191 # For some reason, I was getting two blank lines
1192 # at the end of the headers, rather than one,
1193 # so I decided not to read in the blank line
1194 # which terminates the headers.
1196 # Delete the "unless ($_ eq "\n")" to get rid
1199 $headers .= $_ unless ($_ eq "\n");
1201 # Now scan the line for various status flags
1202 # and to fine the Subject line.
1204 $flags .= $1 if /^Status: ([A-Z]+)/;
1205 $flags .= $1 if /^X-Status: ([A-Z]+)/;
1206 if (/^X-Mozilla-Status: ([0-9a-f]{4})/i)
1208 $flags .= 'R' if (hex($1) & 0x0001);
1209 $flags .= 'A' if (hex($1) & 0x0002);
1210 $flags .= 'D' if (hex($1) & 0x0008);
1212 if(/^X\-Evolution:\s+\w{8}\-(\w{4})/oi)
1214 $b = pack("H4", $1); #pack it as 4 digit hex (0x0000)
1215 $b = unpack("B32", $b); #unpack into bit string
1217 # "usually" only the right most six bits are used
1218 # however, I have come across a seventh bit in
1219 # about 15 (out of 10,000) messages with this bit
1221 # I have not found any documentation in the source.
1222 # If you find out what it does, please let me know.
1225 # Evolution 1.4 does mark forwarded messages.
1226 # The sixth bit is to denote an attachment
1228 $flags .= 'A' if($b =~ /[01]{15}1/); #replied
1229 $flags .= 'D' if($b =~ /[01]{14}1[01]{1}/); #deleted
1230 $flags .= 'T' if($b =~ /[01]{13}1[01]{2}/); #draft
1231 $flags .= 'F' if($b =~ /[01]{12}1[01]{3}/); #flagged
1232 $flags .= 'R' if($b =~ /[01]{11}1[01]{4}/); #seen/read
1234 $subject = $1 if /^Subject: (.*)$/;
1237 $contentlength = $1 if /^Content-Length: (\d+)$/;
1240 # Now look out for the end of the headers - a blank
1241 # line. When we find it, create the file name and
1242 # analyse the Subject line.
1246 # We are at the end of the headers. Set the
1247 # $inheaders flag back to 0.
1251 # Include the current newline in the content length
1253 ++$contentlength if defined $contentlength;
1255 # Create the file name for the current message.
1257 # A simple version of this would be:
1259 # $messagefn = "cur/$unique.$messagecount.mbox:2,";
1261 # This would create names with $messagecount values of
1262 # 1, 2, etc. But for neatness when looking at a
1263 # directory of such messages, sorted by filename,
1264 # I want to have leading zeroes on message count, so
1265 # that they would be 000001 etc. This makes them
1266 # appear in message order rather than 1 being after
1267 # 19 etc. So this is good for up to 999,999 messages
1268 # in a mailbox. It is a cosmetic matter for a person
1269 # looking into the Maildir directory manually.
1270 # To do this, use sprintf instead with "%06d" for
1271 # 6 characters of zero-padding:
1273 $messagefn = sprintf ("cur/%d.%06d.mbox:2,", $unique, $messagecount) ;
1276 # Append flag characters to the end of the
1277 # filename, according to flag characters
1278 # collected from the message headers
1280 $messagefn .= 'F' if $flags =~ /F/; # Flagged.
1281 $messagefn .= 'R' if $flags =~ /A/; # Replied to.
1282 $messagefn .= 'S' if $flags =~ /R/; # Seen or Read.
1283 $messagefn .= 'T' if $flags =~ /D/; # Tagged for deletion.
1286 # Opens filename $messagefn for output (>) with filehandle OUT.
1288 open(OUT
, ">$messagefn") or die("Fatal: unable to create new message $messagefn");
1290 # Count the messages.
1294 # Only for the first message,
1295 # check to see if it is a dummy.
1296 # Delete the message file we
1297 # just created if it was for the
1298 # dummy message at the start
1301 # Add search terms as required.
1302 # The last 2 lines are for rent.
1304 # "m" means match the regular expression,
1305 # but we can do without it.
1307 # Do I need to escape the ' in "DON'T"?
1308 # I didn't in the original version.
1310 if ( (($messagecount == 1) && defined($subject))
1311 && ($subject =~ m/^DON'T DELETE THIS MESSAGE -- FOLDER INTERNAL DATA/)
1314 # Stash the file name of the dummy message so we
1315 # can delete it later.
1317 $deletedummy = "$messagefn";
1320 # Print the collected headers to the message file.
1322 print OUT
"$headers";
1325 # Clear $headers and $flags ready for the next message.
1330 # End of processing the headers once we found the
1331 # blank line which terminated them
1334 # End of dealing with the headers.
1338 if ( $inheaders eq 0)
1341 # We are now processing the message body.
1343 # Now we have passed the headers to the
1344 # output file, we scan until the while
1345 # loop finds another "From " line.
1347 # Decrement our content length if we're
1348 # using it to find the end of the message
1351 if (defined $contentlength) {
1353 # Decrement our $contentlength variable
1355 $contentlength -= length($_);
1357 # The proper end for a message with Content-Length
1358 # specified is the $contentlength variable should
1359 # be exactly -1 and we should be on a bare
1360 # newline. Note that the bare newline is not
1361 # printed to the end of the current message as
1362 # it's actually a message separator in the mbox
1363 # format rather than part of the message. The
1364 # next line _should_ be a From_ line, but just in
1365 # case the Content-Length header is incorrect
1366 # (e.g. a corrupt mailbox), we just continue
1367 # putting lines into the current message until we
1368 # see the next From_ line.
1370 if ($contentlength < 0) {
1371 if ($contentlength == -1 && $_ eq "\n") {
1372 $contentlength = undef;
1375 $contentlength = undef;
1380 # We want to copy every part of the message
1381 # body to the output file, except for the
1382 # quoted ">From " lines, which was the
1383 # way the IMAP server encoded body lines
1384 # starting with "From ".
1386 # Pattern matching Perl majik to
1387 # get rid of an Mbox quoted From.
1389 # This works on the default variable "$_" which
1390 # contains the text from the Mbox mailbox - I
1391 # guess this is the case because of our
1392 # (open(MBOX ....) line above, which did not
1393 # assign this to anything else, so it would go
1394 # to the default variable. This enables
1395 # inscrutably terse Perlisms to follow.
1397 # "s" means "Subsitute" and it looks for any
1398 # occurrence of ">From" starting at the start
1399 # of the line. When it finds this, it replaces
1402 # So this finds all instances in the Mbox message
1403 # where the original line started with the word
1404 # "From" but was converted to ">From" in order to
1405 # not be mistaken for the "From ..." line which
1406 # is used to demark each message in the Mbox.
1407 # This was was a destructive conversion because
1408 # any message which originally had ">From" at the
1409 # start of the line, before being put into the
1410 # Mbox, will now have that line without the ">".
1414 # Glorious tersness here. Thanks Simon for
1417 # "print OUT" means print the default variable to
1418 # the file of file handle OUT. This is where
1419 # the bulk of the message text is written to
1422 print OUT
or die("Fatal: unable to write to new message to $messagefn");
1425 # End of the if statement dealing with message body.
1428 $previous_line_was_empty = ( $_ eq "\n" );
1430 # End of while (MBOX) loop.
1432 # Close the input file.
1436 # Close the output file, and duplicate the code
1437 # from the start of the while loop which touches
1438 # the date-time of the most recent message file.
1441 if ($messagefn ne '') {
1442 my $t = str2time
($receivedate);
1443 utime $t, $t, $messagefn;
1446 # After all the messages have been
1447 # converted, check to see if the
1448 # first one was a dummy.
1449 # if so, delete it and make
1450 # the message count one less.
1452 if ($deletedummy ne "")
1454 printf("Dummy mail system first message detected and not saved.\n");
1455 unlink $deletedummy;
1461 printf("$messagecount messages.\n\n");