Solved

Read last unique records from a file

Posted on 2007-03-30
11
186 Views
Last Modified: 2013-12-25
A little while ago Adam314 gave me this excellent snippet. It does exactly what I needed.
I was wondering if there's a way to get the LAST occurrence of NAMES rather than the first.
Since the newest data is at the bottom of the file, getting the last records with a unique NAME would be the same as getting the latest unique name.
I don't suppose there's a way of reading a file from the bottom up.
I thought it might be worth a try...

Thanks
#############################
while(<IN>){
    next if /^\s+$/;     #Skip blank lines
    my ($mtopic,$cdsid,$date,$name,$prior,$comment)=split /\t/,$_,6;  #Limit to 6, in case of tabs in comment
    next if exists($Names{$name});   #Skip duplicates
    $Names{$name}=1;
    next if $mtopic ne $topic;
    print OUT "$cdsid\t$mtopic\t$name\n";
}
0
Comment
Question by:tmccar10
  • 5
  • 5
11 Comments
 
LVL 39

Expert Comment

by:Adam314
ID: 18824584
while(<DATA>){
    next if /^\s+$/;     #Skip blank lines
    my ($mtopic,$cdsid,$date,$name,$prior,$comment)=split /\t/,$_,6;  #Limit to 6, in case of tabs in comment
    next if $mtopic ne $topic;
    $Names{$name}="$cdsid\t$mtopic\t$name\n";
}

print $Names{$_} foreach (keys %Names);
0
 
LVL 84

Expert Comment

by:ozo
ID: 18826295
There is a way of reading a file from the bottom up, but it may be simpler in this case to save each ccurrence of NAMES as you find it, so that you end up with tle last one
Adam314's suggestion does this, but it may be different from reading a file from the bottom up in the case that the last occurance of names has $mtopic ne $topic
the original code would not print anything for the $name in that case, whereas Adam314's method will print the last n$ame that does have $mtopic eq $topic
Also, it will not print the names in the same order, if that matters.
0
 

Author Comment

by:tmccar10
ID: 18835823
Sorry ozo-

I've been trying to figure out how to do what you suggest.
I'm not supposed to make another file to read, am I?
I don't know how to save and what to do when I have saved it.

Incidentally, I'm actually trying to get each last occurence of each $mtopic. I referenced the $names since that's what Adam helped me with.

I can get the code below to get one occurence per $mtopic, but it's alway the first one.

Thanks
________________________________
my %Topics;
while(<TXTFILE>){
    next if /^\s+$/;                                      
    my ($mtopic,$cdsid,$date,$name,$prior,$comment)=split /\t/,$_,6;
    next if exists($Topics{$mtopic});   #Skip duplicates
    $Topics{$mtopic}=1;
     next if $mtopic eq $topic;
0
 
LVL 39

Expert Comment

by:Adam314
ID: 18838135
Can you provide some sample data of your input and what you want for output.  
0
 

Author Comment

by:tmccar10
ID: 18838230
Thanks-

Sure can. (tab delimited, 6 fields)

I hope this way of sending it over is OK

##################################################
brand new topic      tmccar10       03/01/07       ghdgh sdg d fg      000      adsfg aer g aer yay53y 5y aeh

brand new topic      tmccar10       03/02/07       ghdgh sdg d fg      111      adsfg aer g aer yay53y 5y aeh

new new new      tmccar10       03/01/07       new new      NEW      123       new new new new new new new new new new new new new new new

new topic for 3-15      tmccar10       03/01/07       Thom McCarthy      911      This is the NEW comment for this topic--I'll now try to send an updated one

another new one      tmccar10       03/01/07       thom mccarthy      999      another new one another new one another new one another new one another new one

another new one      tmccar10       03/02/07       1111111111111      111      This is the NEW 'Another new topic'. The next one (111) will be path21

new topic for 3-15      tmccar10       03/02/07       The Answerer      001      see if this works this time

todays try at a topic      tmccar10       03/01/07       Thom      000      This is the NEW 'Today's try at a topic'. The next one (111) will be path2

todays try at a topic      tmccar10       03/02/07       thom--again      111      second comment for todays try at a topic
todays try at a topic      tmccar10       03/03/07       requestor      222      Third comment for todays try at a topic

todays try at a topic      tmccar10       03/04/07       me me m e      333      fourth comment for todays try at a topic
0
How to run any project with ease

Manage projects of all sizes how you want. Great for personal to-do lists, project milestones, team priorities and launch plans.
- Combine task lists, docs, spreadsheets, and chat in one
- View and edit from mobile/offline
- Cut down on emails

 
LVL 39

Expert Comment

by:Adam314
ID: 18838906
And what output do you want for this?
To get the each last $mtopic, use this code:

my %Topics;
while(<DATA>){
    next if /^\s+$/;     #Skip blank lines
    my ($mtopic,$cdsid,$date,$name,$prior,$comment)=split /\t/,$_,6;  #Limit to 6, in case of tabs in comment
    $Topics{$mtopic} = "$cdsid\t$mtopic\t$name\n";
}

print $Topics{$_} foreach (keys %Topics);


If this isn't what you want, please clarify.
0
 

Author Comment

by:tmccar10
ID: 18842175
Sorry Adam-

Looks as if I didn't describe what I needed very clearly.
I'm trying to get a list of the last occurence of EACH $mtopic.
I output that and the date to an HTML table.

Thanks
0
 
LVL 39

Expert Comment

by:Adam314
ID: 18843574
I guess I'm still not understanding.  The code I gave should display the last occurance of each $mtopic (not in any particular order).  
So, for the sample data you gave, you get the last occurance of these $mtopic:
    brand new topic
    new new new
    new topic for 3-15
    another new one
    todays try at a topic

Given the above sample data, what output would you like?  And does the order matter?  If so, what order would you like?
0
 

Author Comment

by:tmccar10
ID: 18844179
I'm the one who's sorry for continuing to confuse, Adam

The reason I wanted the last one of each is that I could also print a kind of "Last Updated" date from the flat file too. The order of the $mtopics isn't important.

Maybe the html I'm using is bad, although it works fine when displaying the first occurence of each $mtopics.

I'm putting the other part of the code.
Thanks
#### ###### #### ###### ###### ######
...
my %Topics;
while(<TXTFILE>){
    next if /^\s+$/;     #Skip blank lines
    my ($mtopic,$cdsid,$date,$name,$prior,$comment)=split /\t/,$_,6;  #Limit to 6, in case of tabs in comment
    $Topics{$mtopic} = "$cdsid\t$mtopic\t$name\n";
## Print each column as a column in the Table
print qq{
<TD bgcolor="wheat"valign=top  colspan=2><A HREF="../cgi-bin/pickpath2.cgi?category=$category&topic=$mtopic"><font size=2>$mtopic</a></font></TD>
<TD bgcolor="wheat"valign=top><font size=2>$date</font></TD>
</tr>
};
</table>
}
</center>
0
 
LVL 39

Accepted Solution

by:
Adam314 earned 125 total points
ID: 18844325
To display the last data, you won't be able to do it in the first loop.  The first time through the loop, the program just finds the data.  You then need another loop that prints the data.  In the code I gave, the second loop was this line:
    print $Topics{$_} foreach (keys %Topics);

Here is some sample code that should help you get what you need:

my %Topics;
##### This loop reads the file, and save the last occurance.  It doesn't display anything
while(<TXTFILE>){
      next if /^\s+$/;     #Skip blank lines
      my ($mtopic,$cdsid,$date,$name,$prior,$comment)=split /\t/,$_,6;  #Limit to 6, in case of tabs in comment
      $Topics{$mtopic} = [$cdsid, $mtopic, $name, $date];
      ## Print each column as a column in the Table
      #print qq{
      #            <TD bgcolor="wheat"valign=top  colspan=2><A HREF="../cgi-bin/pickpath2.cgi?category=$category&topic=$mtopic"><font size=2>$mtopic</a></font></TD>
      #            <TD bgcolor="wheat"valign=top><font size=2>$date</font></TD>
      #      </tr>
      #};
      #printf "%-25s %s\n", $mtopic, $date;
}

print "<table border=1><tr> <td>cdsid</td> <td>mtopic</td> <td>name</td> <td>last updated date</td> </tr>\n";
##### This loop display the data saved in the previous loop
foreach my $entry (keys %Topics) {
      print "<tr>";
      print " <td>$Topics{$entry}->[0]</td>";
      print " <td>$Topics{$entry}->[1]</td>";
      print " <td>$Topics{$entry}->[2]</td>";
      print " <td>$Topics{$entry}->[3]</td>";
      print "</tr>\n";
}
print "</table>\n";
0
 

Author Comment

by:tmccar10
ID: 18845439
Thanks Adam-

I knew it had to be something like that. I could see doing it with other "lanuages" but I couldn't figure out how to get there.
I still need to look this over, but it tested out fine.
Once I get this worked in, I will have learned a good techique.

Thanks again
0

Featured Post

6 Surprising Benefits of Threat Intelligence

All sorts of threat intelligence is available on the web. Intelligence you can learn from, and use to anticipate and prepare for future attacks.

Join & Write a Comment

I hope you'll find this tutorial useful and interesting. So let's try to extend Tcl with a new package.  For anyone more deeply interested please check out the book "Practical Programming in Tcl and Tk". It's really one of the best written books abo…
Active Directory replication delay is the cause to many problems.  Here is a super easy script to force Active Directory replication to all sites with by using an elevated PowerShell command prompt, and a tool to verify your changes.
Learn the basics of strings in Python: declaration, operations, indices, and slicing. Strings are declared with quotations; for example: s = "string": Strings are immutable.: Strings may be concatenated or multiplied using the addition and multiplic…
The viewer will learn how to create and use a small PHP class to apply a watermark to an image. This video shows the viewer the setup for the PHP watermark as well as important coding language. Continue to Part 2 to learn the core code used in creat…

705 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

21 Experts available now in Live!

Get 1:1 Help Now