Search the FAQ Archives

3 - A - B - C - D - E - F - G - H - I - J - K - L - M
N - O - P - Q - R - S - T - U - V - W - X - Y - Z
faqs.org - Internet FAQ Archives

MH Frequently Asked Questions (FAQ) with Answers
Section - Removing duplicate messages (Bourne)

( Single Page )
[ Usenet FAQs | Web FAQs | Documents | RFC Index | Airports ]


Top Document: MH Frequently Asked Questions (FAQ) with Answers
Previous Document: HP-UX 10.20 config file
See reader questions & answers on this topic! - Help others by sharing your knowledge
Date: 20 Nov 1995 18:51:24 GMT

  Here's a simple-minded Bourne shell version.  It uses
  "scan" to get the message number and message-id of each message.  If
  a message has the same message-id as the previous message, the
  script adds its message number to the "remove" shell variable.


	#!/bin/sh
	lastmsgid=hahahaha
	remove=
	scan -width 300 -format '%(msg) %{message-id}' |
	while read msg msgid; do
	    if [ "$msgid" = "$lastmsgid" ]; then
		remove="$remove $msg"
	    else
		lastmsgid="$msgid"
	    fi
	done
	rmm $remove

  That's pretty simple-minded.  For example, if the $remove variable
  gets too big, your system may complain.  And I'm sure there are some
  more-efficient ways to find the list of duplicate message-ids.  But
  that's the idea.
  
Subject: Removing duplicate messages (Perl)
From: rtor at ansa.co.uk (Owen Rees)
Date: 20 Nov 1995 12:39:47 GMT

  I wrote a perl script to do this some time ago. All the usual dire
  warnings about destructive technology apply - take a backup, do it on
  a copy, try it on a small test case first etc. Don't use this script
  unless you are prepared to accept the consequences.

#!/usr/local/bin/perl

$version = "rmmdup 1";

if (@ARGV == 0) { $folder = ""; }
elsif (@ARGV == 1) { $folder = $ARGV[0];
                     unless ( $folder =~ /^\+.+$/ )
                      { die "usage $0 [+folder]\n"; };
                   }
else { die "usage $0 [+folder]\n"; };

$rmmlist = "";

open (scan, "scan $folder -format '%(msg) %{message-id}'|");
while (<scan>)
 { if ( ($msg,$msgid) = /^(\d+) (<.*>)$/)
    { if ($msgs{$msgid})
       { print "$msg duplicates $msgs{$msgid}\n";
         $rmmlist .= " $msg";
       }
      else { $msgs{$msgid} = $msg; };
    };
 };
if ( $rmmlist ) { exec "rmm $folder $rmmlist"; };
exit;

Subject: Removing duplicate messages (Perl)
From: Bill Wohler <wohler at newt.com>
Date: Sun, 17 Oct 2004 13:00:20 -0700

#!/usr/bin/perl -w
#
# Id: mhfinddup 6593 2004-09-02 16:34:24Z wohler 

=head1 NAME

mhfinddup - find duplicate messages

=head1 SYNOPSIS

mhfinddup [options] [folder ...]

=head1 DESCRIPTION

B<mhfinddup> finds and removes duplicate MH messages in the folders listed on
the command line (default: current folder). By default, you deal with
duplicate messages interactively. You can either remove the duplicate, not
remove the duplicate, or view the original and duplicate message before
deciding.

If you use the B<-msgid> option to B<send>, then you probably don't want to
list any F<+outbox> folders if you are using the B<--no-same-folder> option
and you want to preserve your sent messages as well as your messages to
mailing lists.

Note that if you specify one or more folders, or if you use the B<--all>
option, B<mhfinddup> recursively descends the given folders.

=head1 CONTEXT

Context is per B<flist>(1). That is, if F<+folder> is given, it will become
the current folder. If multiple folders are given, the last one specified will
become the current folder.

=head1 OPTIONS

=over 4

=item --all

Look for duplicates in all folders. If any folders are specified, this option
is ignored.

=item --debug

Turn on debugging messages.

=item --help

Display the usage of this command.

=item --list

List duplicated messages.

=item --no-same-folder

Since it is common to use C<refile -link> to file a message in multiple
folders, this script doesn't consider messages in different folders to be
duplicates. Specify this option to list or remove duplicates across folders.

=item --rmm

Remove messages non-interactively. Use with care! For safety, the B<--list>
option takes precedence if specified and is a good option to use before using
B<--rmm>.

=item --version

Display program version.

=back

=head1 RETURN VALUE

Returns 0 if all is well; non-zero otherwise.

=head1 EXAMPLES

=over 0

=item mhfinddup

Interactively remove duplicates from the current folder.

=item mhfinddup --all --list --no-same-folder

List all duplicates regardless if they are in different folders or not.

=item mhfinddup --rmm +lists

Remove all duplicates in F<+lists>, recursively.

=back

=head1 SEE ALSO

B<rmm>(1), B<mhl>(1), B<scan>(1)

=head1 VERSION

Revision: 6593 

=head1 AUTHOR

Bill Wohler <wohler at newt.com>

Copyright (c) 2003 Newt Software. All rights reserved.

This program is free software; you can redistribute it and/or
modify it under the terms of the GNU General Public License
as published by the Free Software Foundation; either version 2
of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

You should have received a copy of the GNU General Public License
along with this program; if not, you can find it at
http://www.gnu.org/copyleft/gpl.html or write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA  02111-1307, USA.

=head1 METHODS

=cut

# Packages and pragmas.
use Getopt::Long;

use strict;

# Constants.
my $cmd;                                # name by which command called
($cmd = $0) =~ s|^\./||;                # ...minus the leading ./
my $ver = '6593';			# program version with CVS noise

# Variables (may be overridden by arguments).
my $all = 0;				# look in all folders
my $debug = 0;				# verbose mode
my $help = 0;				# display usage
my $version = 0;			# display version
my $list = 0;				# list duplicates
my $no_same_folder = 0;			# consider duplicates across folders
my $rmm = 0;				# remove duplicates without asking

# Constants.
my $mhl = "/usr/lib/mh/mhl";
my $tmp = "/tmp/mhfinddup$$";

# Parse command line.
# The use of the posix_default option is to ensure that folders like +a are
# not confused with --all. I'd really prefer to set prefix_pattern to "(--|-)"
# so that abbreviations of options can be used without being confused with
# folders, but I couldn't make it so.
my %opts;
Getopt::Long::Configure("pass_through", "posix_default");
GetOptions('all'		=> \$all,
	   'debug'		=> \$debug,
	   'help'		=> \$help,
	   'list'		=> \$list,
	   'no-same-folder'	=> \$no_same_folder,
	   'rmm'		=> \$rmm,
	   'version'		=> \$version,
	  ) or usage();

show_version() if ($version);
usage() if ($help || int(@ARGV) != int(map(/^\+/, @ARGV)));

my @folders = expand_folders(@ARGV);
print("Expanded " . join(" ", @ARGV) . " into\n" . join("\n", @folders) . "\n")
    if ($debug);

print("Scanning for duplicate messages...\n");
my %msgs;
foreach my $folder (sort @folders) {
    print("Scanning $folder...\n") if ($debug);
    open (SCAN,
	  "MHCONTEXT=$tmp scan +$folder -format '%(msg) %{message-id}'|");
    while (<SCAN>) {
	if (my ($msg, $msgid) = /^(\d+) (<.*>)$/) {
	    if ($msgs{$msgid}) {
		$msgs{$msgid} =~ m|^\+(.*)/(\d+)$|;
		my($f, $m) = ($1, $2);
		if ($folder eq $f || $no_same_folder) {
		    handle_dup($f, $m, $folder, $msg);
		}
	    } else {
		$msgs{$msgid} = "+$folder/$msg";
	    }
	}
    }
    close(SCAN);
}

unlink("$tmp");

sub expand_folders {
    my @folders = @_;

    print("Getting list of folders...");
    open(FOLDERS,
	 "flist -recurse "
	  . (($all == 1 && @folders == 0) ? "-all" : join(" ", @folders))
	  . "|")
	or die("Could not determine folders\n");
    @folders = ();
    chomp(my $current_folder = `mhparam Current-Folder`);
    $current_folder = quotemeta($current_folder);
    while (<FOLDERS>) {
	chomp;
	my ($folder, $a, $b, $c, $d, $e, $f, $g, $count) = split;
	if ($folder =~ /^$current_folder\+$/) {
	    $folder =~ s/\+$//; # remove current folder indication
	}
	next if ($count == 0);
	push(@folders, $folder);
    }
    close(FOLDERS);
    print("done\n");

    return(@folders);
}

sub handle_dup {
    my($f1, $m1, $f2, $m2) = @_;

    my $ans;

 repeat:
    print("+$f2/$m2 duplicate of +$f1/$m1");

    if ($list) {
	print("\n");
    } else {
	if ($rmm) {
	    $ans = "y";
	    print("\n");
	} else {
	    print(", remove? [Yns?] ");
	    chomp($ans = <STDIN>);
	}

	if ($ans eq "y" || $ans eq "") {
	    system("rmm +$f2 $m2");
	} elsif ($ans eq "s") {
	    system("$mhl `mhpath +$f1 $m1` `mhpath +$f2 $m2`");
	    goto repeat;
	} elsif ($ans eq "?") {
	    print("y, remove message (default)\n" .
		  "n, don't remove message\n" .
		  "s, show messages\n" .
		  "?, show this message\n");
	    goto repeat;
	}
    }
}


=head2 usage

Display usage information and exit.

=cut

sub usage {
    print <<EOF;
Usage: $cmd [options] [folder ...]
--all			remove duplicates in all folders
--debug			print actions that program takes
--help			display this message
--list			list duplicates only
--no-same-folder	consider duplicates even if in different folders
--rmm			remove duplicates without asking
--version		display program version
EOF
    exit(1);
}

=head2 show_version

Display version information and exit.

=cut

sub show_version {
    print("$cmd version $ver\n".
          "Copyright (c) 2003 Bill Wohler <wohler at newt.com>\n\n".
          "$cmd comes with ABSOLUTELY NO WARRANTY.\n\n".
          "This is free software, and you are welcome\n".
          "to redistribute it under certain conditions.\n\n".
          "See `http://www.gnu.org/copyleft/gpl.html' for details.\n");
    exit(0);
}


Local Variables:
mode: outline
outline-regexp: "^Subject:"
fill-prefix: "  "
End:

User Contributions:

1
Report this comment as inappropriate
Sep 18, 2019 @ 9:09 am
Where to invest $ 3000 once and receive every month from $ 55000: https://hec.su/nLfW?TGtEESPDx
2
Report this comment as inappropriate
Sep 18, 2019 @ 7:19 pm
Earn Free Bitcoin 0.2 BTC Per day: https://hec.su/nIrt?jzNjFuyIjDi
3
Report this comment as inappropriate
Sep 18, 2019 @ 11:23 pm
Invest in cryptocurrency and get passive income of $ 5000 per week: https://hec.su/nJPO?Orq4u
4
Report this comment as inappropriate
Sep 19, 2019 @ 7:07 am
Buy Essay and Get Time for Pleasures of Life: https://vk.cc/9OTGak?id=buyessayonline111zgu
5
Report this comment as inappropriate
Sep 19, 2019 @ 7:07 am
Buy Essay Online At Writing Service from Canada: https://vk.cc/9OTGak?id=buyessayonline112e4y
6
Report this comment as inappropriate
Sep 19, 2019 @ 7:07 am
Buy Essay: 100% Original. Low Price. In 3 Hours. Safe & Legit: https://vk.cc/9OTGak?id=buyessayonline113x86
7
Report this comment as inappropriate
Sep 19, 2019 @ 10:10 am
Buy Essay Online, Essay Writing Service, Write My Essay: https://vk.cc/9OTGak?id=buyessayonline1138za
8
Report this comment as inappropriate
Sep 19, 2019 @ 11:11 am
Buy Essay Online At Writing Service from Australia: https://vk.cc/9OTGak?id=buyessayonline112he2
9
Report this comment as inappropriate
Sep 20, 2019 @ 3:15 pm
Buy Essay Online, Essay Writing Service, Write My Essay: https://vk.cc/9OTGak?id=buyessayonline11312t
10
Report this comment as inappropriate
Sep 20, 2019 @ 6:18 pm
Buy Essays Online in Australia: https://vk.cc/9OTGak?id=buyessayonline113zbs
The Best Dating Apps for 2019: https://huit.re/bestsexygirls360002?u=kopl
Beautiful girls are looking for sex in your city: https://onlineuniversalwork.com/bestsexygirls262569?u=to
13
Report this comment as inappropriate
Sep 26, 2019 @ 9:09 am
Beautiful girls are looking for sex in your city: https://is.gd/sexygirlsinyourcity23298?u=7u
14
Report this comment as inappropriate
Sep 26, 2019 @ 11:11 am
10 Best CA Dating Sites 2019: https://is.gd/sexygirlsinyourcity98201?u=wt
15
Report this comment as inappropriate
Sep 26, 2019 @ 10:22 pm
5 Best Casual Dating Apps: https://onlineuniversalwork.com/bestsexygirls352268?u=460
5 Best Casual Dating Apps: https://frama.link/bestsexygirls149180?u=1546
How to find a woman for casual sex: https://ecuadortenisclub.com/bestsexygirls148355?u=ye
Top 5 AU Casual Dating Sites 2019: https://is.gd/sexygirlsinyourcity38534?u=mde
Meet sexy girls in your city: https://is.gd/sexygirlsinyourcity63345?u=jj
Dating sites for sex - 545 women want to meet for sex in your city: https://bestsexygirlsinyourcity.blogspot.no?r=59
Girls are looking for sex in your city (518 women want to meet for sex in your city): https://siteadultdatingforsex.blogspot.ca?a=10
Sex App For Adult Dating, Sex Now Websites - 575 women want to meet for sex in your city: https://the-best-adult-dating-sites.blogspot.nl?e=91
Sexy girls in your city are looking for dating (762 girls want to meet for sex in your city): https://siteadultdatingforsex.blogspot.ie?p=04
Meet for Sex - Find Sex Tonight (167 beautiful women want sex in your city right now): https://vae.me/lYSI?y=2062
Adult Dating Site, Meet and Fuck - 234 women want to meet for sex in your city: https://bestadultforsex.blogspot.co.uk?u=98
7+ Dating Sex Sites, Adult Dating Hook Up Sites - 267 beautiful girls want sex in your city right now: https://bestadultdatingsite999.blogspot.ae?g=38
Sexy girls in your city are looking for dating (629 women want to meet for sex in your city): https://darknesstr.com/adultdatingsex542120?m=8844
Nur wie man 7027 $ SCHNELL, schnelles Geld verdient, der beschäftigte Wellensittich: https://earn7btcperweek.blogspot.nl?v=82
Bezahlte Studien: Gewinnen Sie 6065 USD oder mehr pro Woche: https://make3btcperweek.blogspot.kr?t=58
Forex + Kryptowährung = $ 6465 pro Woche: https://earn1btcperweek.blogspot.co.uk?v=98
So erhalten Sie 6735 USD pro Woche: https://make-1-btc-per-week.blogspot.com.es?i=97
Just how would you use $51493 to make more money: https://3-btc-per-day.blogspot.mx?y=613
Invest $ 2864 and get $ 84448 every month: https://7-btc-per-day.blogspot.hk?j=664
How to Make $7621 FAST, Quick Loan, The Busy Budgeter: https://earn1btcperweek.blogspot.fi?h=25
Paid Surveys: Earn $8512 Or More Each week: https://7-btc-per-day.blogspot.co.nz?j=285
Paid Surveys: Earn $6549 Or Even more Each week: https://earn7btcperweek.blogspot.be?u=61
How to earn 0,883 BTC per week: https://make-1-btc-per-week.blogspot.fi?m=64
Invest $ 4934 and get $ 12365 every month: https://make1btcperweek.blogspot.ro?c=85
How to Make $8329 FAST, Quick Cash, The Busy Budgeter: https://get3btcperweek.blogspot.com.au?i=82
Invest $ 2882 and get $ 33114 every month: https://earn1btcperweek.blogspot.mx?p=53
Invest $ 4236 and get $ 78427 every month: https://make-2-btc-perday.blogspot.com.es?e=27
How to make $ 8619 per day: https://hideuri.com/qJzrZw?&gszku=Z7mLcZFn1Gk4Zb
Invest $ 75155 in Bitcoin once and get $ 941179 passive income per month: https://hideuri.com/aBDYO3?ZOAru
Just how to Make $5628 FAST, Fast Money, The Busy Budgeter: https://get-2-btc-perday.blogspot.tw?u=93
How to invest in Bitcoin and receive from $ 3411 per day: https://make-3-btc-perday.blogspot.gr?p=12
Exactly how to Make $6761 FAST, Rapid Money, The Busy Budgeter: https://hideuri.com/a51yyo?&kkhcw=M73V7r
Paid Surveys: Gain $9135 Or Even more Weekly: https://vae.me/aqz2?l=7194
Just how would you use $76676 to make more cash: https://hideuri.com/a7gGk5?&jfjjs=9gTFWgEG
What's the easiest means to earn $95329 a month: https://fla.kr/earnonebtc963598?h=7988
50
Report this comment as inappropriate
Oct 17, 2019 @ 7:19 pm
How to invest in Cryptocurrency $ 37886 - get a return of up to 3812%: https://hideuri.com/K0EP8W?Wp24o422
51
Report this comment as inappropriate
Oct 19, 2019 @ 2:14 pm
Invest $ 7275 and get $ 97378 every month: https://soo.gd/1vVIY?&ptshn=4sGaN
How to invest in Bitcoin and receive from $ 8383 per day: https://cutt.us/H7KgvRc?a5kQdKDU5W
How to Make $6226 FAST, Quick Cash, The Busy Budgeter: https://hideuri.com/a7o81M?&ppmne=Yu1yhHoF1
Just how to Make $7457 FAST, Rapid Cash, The Busy Budgeter: https://hideuri.com/Kjn8ZA?r4nEosWMTJLgNE
How to get 0,988 Bitcoin per week: https://soo.gd/inar?5fp7mb
52
Report this comment as inappropriate
Oct 19, 2019 @ 2:14 pm
Invest $ 6538 and get $ 32856 every month: https://v.ht/afMoj?otspvux0z8d
Just how would certainly you utilize $92431 to make more money: https://cutt.us/PXn2tI5gS?X3U0km
What's the simplest means to earn $76177 a month: https://hideuri.com/qJodXr?z022EajPURJW
Forex + Bitcoin = $ 6282 per week: https://soo.gd/3di0p?&tdpud=PQQ23hz
Exactly how to Make $6711 FAST, Rapid Money, The Busy Budgeter: https://v.ht/mlPaUOv?ck5z2dNXkBNk9
Invest $ 56448 in Bitcoin once and get $ 947918 passive income per month: https://soo.gd/D6SV?bsJVKfIhmby
How to Make $5411 FAST, Fast Cash, The Busy Budgeter: https://hideuri.com/KAoRr3?&pgbpk=tUxb1AiQEE9
55
Report this comment as inappropriate
Oct 19, 2019 @ 6:18 pm
Invest $ 6953 and get $ 32997 every month: https://v.ht/zYRz5DP?&tgojk=CzNWu4hx
Forex + Bitcoin = $ 4171 per week: https://cutt.us/DuuNMCrKx?&bddar=51Ep8
Invest $ 2218 and get $ 63872 every month: https://v.ht/Cud6E4m?ywlfI5R43BGfMX
Invest $ 84453 in Cryptocurrency once and get $ 338351 passive income per month: https://tiny.pl/tl1q5?&rbrti=i3Um2CK
How to get 0,931 BTC per week: https://v.ht/G2WkL?L3YWcDi
What's the simplest method to earn $53213 a month: https://v.ht/m1JYbb?KcmFczSqTL
How to earn on investments in Cryptocurrency from $ 2871 per day: https://soo.gd/MeaoG?&okhsw=TcBchaLl89
Forex + Bitcoin = $ 7181 per week: https://v.ht/hjGCX?aqjmGXBBM
Exactly how would you use $53745 to make more money: https://v.ht/OuYm5Z?&sgbto=NRo2Esxo
Which wallet should you transfer to $914326,98: https://is.gd/WfHUIx

Comment about this article, ask questions, or add new information about this topic:




Top Document: MH Frequently Asked Questions (FAQ) with Answers
Previous Document: HP-UX 10.20 config file

Single Page

[ Usenet FAQs | Web FAQs | Documents | RFC Index ]

Send corrections/additions to the FAQ Maintainer:
Bill Wohler <wohler@newt.com>





Last Update March 27 2014 @ 02:11 PM