Solved

Fast array item search and fast file write

Posted on 2004-04-25
2
533 Views
Last Modified: 2007-12-19
Hey everyone,

    I do have a couple of questions and I'm ready to loose some points.
    1 problem: I need a -fast- code to look for a substring in an array of strings. The strings are website names, so the strings may contain [ .,-,a-zA-Z, / ] symbols. The script should check if a  given word is a part (sub string or equal) of any string in array. I wrote a perl script to do it, but I'm afraid it'll be way too slow for large number of searches.

     2 problem: similar. fast way to flush results into a file. Should I keep one file open all the time the script is running, so I can write to it anytime, or is there a better way to do this?

    Any advice greatly appreciated.
0
Comment
Question by:intoxicated
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
2 Comments
 
LVL 18

Accepted Solution

by:
kandura earned 175 total points
ID: 10912406
1) There's a new article on perl.com which might be of interest to you: http://www.perl.com/pub/a/2004/04/08/bloom_filters.html
If you're only interested in knowing whether there is a match, then this is a very efficient solution.
But if you also need to know which items in your array matched, then I don't think there is any other way than to iterate over the array and checking for a match.

2) It depends on your requirements. Of course it isn't very efficient to open and close a file very often. But it also not very efficient to do lots of little writes to a file. The general answer would be to collect as much of your output as possible, and write it out in one large chunk, but you'd have to balance that with the memory usage. Note that this assumes that by "fast" you mean "don't spend a lot of time with the file".
If you mean "fast" as in "make new data available at the earliest possible moment", then your description would seem to fit best.

0
 

Author Comment

by:intoxicated
ID: 10912589
Hm, very interesting.
Sorry about my english, it's not my first language.
Thanks alot for your help.
0

Featured Post

Technology Partners: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
Edit Odbc.ini using perl module 2 96
Perl Untar File 1 76
Perl Snippet to Parse String 1 57
Use Perl to Remove Large Quantity of Files Quickly 8 26
I've just discovered very important differences between Windows an Unix formats in Perl,at least 5.xx.. MOST IMPORTANT: Use Unix file format while saving Your script. otherwise it will have ^M s or smth likely weird in the EOL, Then DO NOT use m…
A year or so back I was asked to have a play with MongoDB; within half an hour I had downloaded (http://www.mongodb.org/downloads),  installed and started the daemon, and had a console window open. After an hour or two of playing at the command …
Explain concepts important to validation of email addresses with regular expressions. Applies to most languages/tools that uses regular expressions. Consider email address RFCs: Look at HTML5 form input element (with type=email) regex pattern: T…

738 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question