<?xml version="1.0" encoding="utf-8" ?><rss version="2.0" xml:base="https://www.webmaster-forums.net/crss/node/1012066" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title></title>
    <link>https://www.webmaster-forums.net/crss/node/1012066</link>
    <description></description>
    <language>en</language>
          <item>
    <title>Thanks!</title>
    <link>https://www.webmaster-forums.net/serverside-scripting/removing-duplicate-lines#comment-1069661</link>
    <description> &lt;p&gt;Thanks, richjb! Your script worked great!!&lt;/p&gt;
&lt;p&gt;--Edge&lt;/p&gt;
 </description>
     <pubDate>Thu, 17 Aug 2000 16:12:11 +0000</pubDate>
 <dc:creator>Edge</dc:creator>
 <guid isPermaLink="false">comment 1069661 at https://www.webmaster-forums.net</guid>
  </item>
  <item>
    <title></title>
    <link>https://www.webmaster-forums.net/serverside-scripting/removing-duplicate-lines#comment-1069640</link>
    <description> &lt;p&gt;Sorry guys,&lt;/p&gt;
&lt;p&gt;but I&#039;ll have to do this.&lt;br /&gt;
I assume you all run on unix hosts....&lt;br /&gt;
&lt;div class=&quot;codeblock&quot;&gt;&lt;code&gt;cat your.dat|sort &amp;gt;sort.dat;&lt;br /&gt; cat sort.dat|uniq &amp;gt; clean.dat;&lt;br /&gt; diff --minimal clean.dat sort.dat |grep &amp;quot;^&amp;gt;&amp;quot; |sed &amp;quot;s/^&amp;gt;//&amp;quot; &amp;gt; extra.dat&lt;/code&gt;&lt;/div&gt;&#039;&lt;br /&gt;
this isn&#039;t optimal, but it took me only 1 minute to write it  down &lt;img src=&quot;https://www.webmaster-forums.net/misc/smileys/wink.png&quot; title=&quot;Wink&quot; alt=&quot;Wink&quot; class=&quot;smiley-content&quot; /&gt;&lt;/p&gt;
&lt;p&gt;btw:&lt;br /&gt;
do you lower case all emails when you enter them to the file ?&lt;br /&gt;
Otherwise &lt;a href=&quot;mailto:XXX@yyy.com&quot;&gt;XXX@yyy.com&lt;/a&gt; and &lt;a href=&quot;mailto:xxx@yyy.com&quot;&gt;xxx@yyy.com&lt;/a&gt; will stay in the list.&lt;/p&gt;
&lt;p&gt;anti&lt;/p&gt;
&lt;p&gt;[Edited by anti on 08-17-2000 at 05:35 AM]&lt;/p&gt;
 </description>
     <pubDate>Thu, 17 Aug 2000 09:31:59 +0000</pubDate>
 <dc:creator>anti</dc:creator>
 <guid isPermaLink="false">comment 1069640 at https://www.webmaster-forums.net</guid>
  </item>
  <item>
    <title></title>
    <link>https://www.webmaster-forums.net/serverside-scripting/removing-duplicate-lines#comment-1069636</link>
    <description> &lt;p&gt;Here&#039;s your code with comments on errors in bold:&lt;/p&gt;
&lt;p&gt;&lt;div class=&quot;codeblock&quot;&gt;&lt;code&gt;#!/usr/bin/perl&lt;br /&gt;&lt;br /&gt;$file = &amp;quot;your.dat&amp;quot;;&lt;br /&gt;&lt;br /&gt;open(FILE, &amp;quot;&amp;lt;$file&amp;quot;) || die &amp;quot;Can&amp;#039;t open $file: $!&amp;quot;;&lt;br /&gt;@File = &amp;lt;FILE&amp;gt;;&lt;br /&gt;close(FILE);&lt;br /&gt;&lt;br /&gt;foreach $emial (@File) { # &amp;lt;strong&amp;gt;Typo&amp;lt;/strong&amp;gt;&lt;br /&gt;$Revised = join(&amp;quot; &amp;quot;, @Revised);&lt;br /&gt;chomp($emial); # &amp;lt;strong&amp;gt;Typo&amp;lt;/strong&amp;gt;&lt;br /&gt;unless ($Revised =~ $email) {&lt;br /&gt;push(@Revised, $email);&lt;br /&gt;}&lt;br /&gt;}&lt;br /&gt;&lt;br /&gt;open(FILE, &amp;quot;&amp;gt;$file&amp;quot;) || die &amp;quot;Can&amp;#039;t open $file: $!&amp;quot;;&lt;br /&gt;foreach $email (@Revised) {&lt;br /&gt;print FILE &amp;quot;$email\n&amp;quot;;&lt;br /&gt;close(FILE);&lt;br /&gt;# &amp;lt;strong&amp;gt; No ending } No content type!&amp;lt;/strong&amp;gt;&lt;br /&gt;print &amp;quot;&amp;#039;Twas a success!&amp;quot;;&lt;/code&gt;&lt;/div&gt;&#039;&lt;/p&gt;
&lt;p&gt;Those are the only mistakes I see. &lt;img src=&quot;https://www.webmaster-forums.net/misc/smileys/smile.png&quot; title=&quot;Smiling&quot; alt=&quot;Smiling&quot; class=&quot;smiley-content&quot; /&gt;&lt;/p&gt;
&lt;p&gt;Richard&lt;br /&gt;
richjb::410&lt;/p&gt;
 </description>
     <pubDate>Thu, 17 Aug 2000 08:20:41 +0000</pubDate>
 <dc:creator>richjb</dc:creator>
 <guid isPermaLink="false">comment 1069636 at https://www.webmaster-forums.net</guid>
  </item>
  <item>
    <title>Oopsi!   :eek:</title>
    <link>https://www.webmaster-forums.net/serverside-scripting/removing-duplicate-lines#comment-1069633</link>
    <description> &lt;p&gt;Sorry... should be&lt;br /&gt;
chomp($email);&lt;/p&gt;
&lt;p&gt;Should it work now?&lt;/p&gt;
 </description>
     <pubDate>Thu, 17 Aug 2000 07:48:46 +0000</pubDate>
 <dc:creator>Mark Hensler</dc:creator>
 <guid isPermaLink="false">comment 1069633 at https://www.webmaster-forums.net</guid>
  </item>
  <item>
    <title></title>
    <link>https://www.webmaster-forums.net/serverside-scripting/removing-duplicate-lines#comment-1069632</link>
    <description> &lt;p&gt;Albert, welcome to TWF. &lt;img src=&quot;https://www.webmaster-forums.net/misc/smileys/smile.png&quot; title=&quot;Smiling&quot; alt=&quot;Smiling&quot; class=&quot;smiley-content&quot; /&gt;&lt;/p&gt;
&lt;p&gt;Also, you&#039;re code won&#039;t work as you typoed &quot;emial?&quot; &lt;img src=&quot;https://www.webmaster-forums.net/misc/smileys/wink.png&quot; title=&quot;Wink&quot; alt=&quot;Wink&quot; class=&quot;smiley-content&quot; /&gt;&lt;/p&gt;
&lt;p&gt;Richard&lt;br /&gt;
richjb::409&lt;/p&gt;
 </description>
     <pubDate>Thu, 17 Aug 2000 07:29:06 +0000</pubDate>
 <dc:creator>richjb</dc:creator>
 <guid isPermaLink="false">comment 1069632 at https://www.webmaster-forums.net</guid>
  </item>
  <item>
    <title></title>
    <link>https://www.webmaster-forums.net/serverside-scripting/removing-duplicate-lines#comment-1069629</link>
    <description> &lt;p&gt;Haven&#039;t tested this, just a quicky thing.&lt;/p&gt;
&lt;p&gt;&lt;div class=&quot;codeblock&quot;&gt;&lt;code&gt;#!/usr/bin/perl&lt;br /&gt;&lt;br /&gt;$file = &amp;quot;your.dat&amp;quot;;&lt;br /&gt;&lt;br /&gt;open(FILE, &amp;quot;&amp;lt;$file&amp;quot;) || die &amp;quot;Can&amp;#039;t open $file: $!&amp;quot;;&lt;br /&gt;@File = &amp;lt;FILE&amp;gt;;&lt;br /&gt;close(FILE);&lt;br /&gt;&lt;br /&gt;foreach $emial (@File) {&lt;br /&gt;$Revised = join(&amp;quot; &amp;quot;, @Revised);&lt;br /&gt;chomp($emial);&lt;br /&gt;unless ($Revised =~ $email) {&lt;br /&gt;push(@Revised, $email);&lt;br /&gt;}&lt;br /&gt;}&lt;br /&gt;&lt;br /&gt;open(FILE, &amp;quot;&amp;gt;$file&amp;quot;) || die &amp;quot;Can&amp;#039;t open $file: $!&amp;quot;;&lt;br /&gt;foreach $email (@Revised) {&lt;br /&gt;print FILE &amp;quot;$email\n&amp;quot;;&lt;br /&gt;close(FILE);&lt;br /&gt;&lt;br /&gt;print &amp;quot;&amp;#039;Twas a success!&amp;quot;;&lt;/code&gt;&lt;/div&gt;&#039;&lt;/p&gt;
 </description>
     <pubDate>Thu, 17 Aug 2000 06:07:10 +0000</pubDate>
 <dc:creator>Mark Hensler</dc:creator>
 <guid isPermaLink="false">comment 1069629 at https://www.webmaster-forums.net</guid>
  </item>
  <item>
    <title></title>
    <link>https://www.webmaster-forums.net/serverside-scripting/removing-duplicate-lines#comment-1069608</link>
    <description> &lt;p&gt;The following script will go through a file and put any unique entries into the $new_file and if it finds an entry that already exists, then it will add it to $extra_file.&lt;/p&gt;
&lt;p&gt;So, if all you want to do is filter duplicate subscriptions, this script will do it:&lt;/p&gt;
&lt;p&gt;&lt;div class=&quot;codeblock&quot;&gt;&lt;code&gt;#!/usr/bin/perl&lt;br /&gt;&lt;br /&gt;BEGIN {&lt;br /&gt; use CGI::Carp qw(fatalsToBrowser);&lt;br /&gt;}&lt;br /&gt;&lt;br /&gt;use CGI qw(param header);&lt;br /&gt;print header;&lt;br /&gt;&lt;br /&gt;$old_file = &amp;quot;/home/brevig/www/remove_extra/old.dat&amp;quot;;&lt;br /&gt;$new_file = &amp;quot;/home/brevig/www/remove_extra/new.dat&amp;quot;;&lt;br /&gt;$extra_file = &amp;quot;/home/brevig/www/remove_extra/extra.dat&amp;quot;;&lt;br /&gt;&lt;br /&gt;open(OLD, &amp;quot;&amp;lt;$old_file&amp;quot;) || die &amp;quot;Can&amp;#039;t open $old_file: $!&amp;quot;;&lt;br /&gt;@OLD = &amp;lt;OLD&amp;gt;;&lt;br /&gt;close(OLD);&lt;br /&gt;&lt;br /&gt;@list = (&amp;quot;first_element&amp;quot;);&lt;br /&gt;&lt;br /&gt;open(NEW, &amp;quot;&amp;gt;&amp;gt;$new_file&amp;quot;) || die &amp;quot;Can&amp;#039;t open $new_file: $!&amp;quot;;&lt;br /&gt;&lt;br /&gt;open(EXTRA, &amp;quot;&amp;gt;&amp;gt;$extra_file&amp;quot;) || die &amp;quot;Can&amp;#039;t open $new_file: $!&amp;quot;;&lt;br /&gt;&lt;br /&gt;foreach $old_check (@OLD) {&lt;br /&gt; $now = &amp;quot;no&amp;quot;;&lt;br /&gt; chomp($old_check);&lt;br /&gt; foreach (@list) {&lt;br /&gt;&amp;nbsp; if($old_check eq $_) {&lt;br /&gt;&amp;nbsp;&amp;nbsp; $now = &amp;quot;yes&amp;quot;;&lt;br /&gt;&amp;nbsp; }&lt;br /&gt; }&lt;br /&gt; if ($now eq &amp;quot;no&amp;quot;) {&lt;br /&gt;&amp;nbsp; push(@list, $old_check);&lt;br /&gt;&amp;nbsp; print &amp;quot;new: $old_check&amp;lt;br&amp;gt;&amp;quot;;&lt;br /&gt;&amp;nbsp; print NEW &amp;quot;$old_check\n&amp;quot;;&lt;br /&gt; } else {&lt;br /&gt;&amp;nbsp; print &amp;quot;&amp;lt;b&amp;gt;extra&amp;lt;/b&amp;gt;: $old_check&amp;lt;br&amp;gt;&amp;quot;;&lt;br /&gt;&amp;nbsp; print EXTRA &amp;quot;$old_check\n&amp;quot;;&lt;br /&gt; }&lt;br /&gt;}&lt;br /&gt;&lt;br /&gt;close(NEW);&lt;br /&gt;close(EXTRA);&lt;br /&gt;&lt;br /&gt;print &amp;quot;&amp;lt;BR&amp;gt;&amp;lt;BR&amp;gt;&amp;lt;BR&amp;gt;done&amp;quot;;&lt;/code&gt;&lt;/div&gt;&#039;&lt;/p&gt;
&lt;p&gt;Tell me if you have any problems.&lt;/p&gt;
&lt;p&gt;Richard&lt;br /&gt;
richjb::399&lt;/p&gt;
 </description>
     <pubDate>Wed, 16 Aug 2000 20:26:04 +0000</pubDate>
 <dc:creator>richjb</dc:creator>
 <guid isPermaLink="false">comment 1069608 at https://www.webmaster-forums.net</guid>
  </item>
  <item>
    <title></title>
    <link>https://www.webmaster-forums.net/serverside-scripting/removing-duplicate-lines#comment-1069605</link>
    <description> &lt;p&gt;I also need to know how to delete emails if they have emails if they have been submitted twice but i dont want to delte them off my list, just ignore the second one.&lt;/p&gt;
 </description>
     <pubDate>Wed, 16 Aug 2000 19:11:11 +0000</pubDate>
 <dc:creator>minton</dc:creator>
 <guid isPermaLink="false">comment 1069605 at https://www.webmaster-forums.net</guid>
  </item>
  <item>
    <title></title>
    <link>https://www.webmaster-forums.net/serverside-scripting/removing-duplicate-lines#comment-1069603</link>
    <description> &lt;p&gt;As in duplicate lines?  Or duplicate entries (subscribes twice? Or like blank lines?&lt;/p&gt;
&lt;p&gt;Richard&lt;br /&gt;
richjb::397&lt;/p&gt;
 </description>
     <pubDate>Wed, 16 Aug 2000 18:58:08 +0000</pubDate>
 <dc:creator>richjb</dc:creator>
 <guid isPermaLink="false">comment 1069603 at https://www.webmaster-forums.net</guid>
  </item>
  </channel>
</rss>
