Solved

Fast array item search and fast file write

Posted on 2004-04-25
2
519 Views
Last Modified: 2007-12-19
Hey everyone,

    I do have a couple of questions and I'm ready to loose some points.
    1 problem: I need a -fast- code to look for a substring in an array of strings. The strings are website names, so the strings may contain [ .,-,a-zA-Z, / ] symbols. The script should check if a  given word is a part (sub string or equal) of any string in array. I wrote a perl script to do it, but I'm afraid it'll be way too slow for large number of searches.

     2 problem: similar. fast way to flush results into a file. Should I keep one file open all the time the script is running, so I can write to it anytime, or is there a better way to do this?

    Any advice greatly appreciated.
0
Comment
Question by:intoxicated
2 Comments
 
LVL 18

Accepted Solution

by:
kandura earned 175 total points
ID: 10912406
1) There's a new article on perl.com which might be of interest to you: http://www.perl.com/pub/a/2004/04/08/bloom_filters.html
If you're only interested in knowing whether there is a match, then this is a very efficient solution.
But if you also need to know which items in your array matched, then I don't think there is any other way than to iterate over the array and checking for a match.

2) It depends on your requirements. Of course it isn't very efficient to open and close a file very often. But it also not very efficient to do lots of little writes to a file. The general answer would be to collect as much of your output as possible, and write it out in one large chunk, but you'd have to balance that with the memory usage. Note that this assumes that by "fast" you mean "don't spend a lot of time with the file".
If you mean "fast" as in "make new data available at the earliest possible moment", then your description would seem to fit best.

0
 

Author Comment

by:intoxicated
ID: 10912589
Hm, very interesting.
Sorry about my english, it's not my first language.
Thanks alot for your help.
0

Featured Post

How to run any project with ease

Manage projects of all sizes how you want. Great for personal to-do lists, project milestones, team priorities and launch plans.
- Combine task lists, docs, spreadsheets, and chat in one
- View and edit from mobile/offline
- Cut down on emails

Join & Write a Comment

Many time we need to work with multiple files all together. If its windows system then we can use some GUI based editor to accomplish our task. But what if you are on putty or have only CLI(Command Line Interface) as an option to  edit your files. I…
I have been pestered over the years to produce and distribute regular data extracts, and often the request have explicitly requested the data be emailed as an Excel attachement; specifically Excel, as it appears: CSV files confuse (no Red or Green h…
Explain concepts important to validation of email addresses with regular expressions. Applies to most languages/tools that uses regular expressions. Consider email address RFCs: Look at HTML5 form input element (with type=email) regex pattern: T…
In this seventh video of the Xpdf series, we discuss and demonstrate the PDFfonts utility, which lists all the fonts used in a PDF file. It does this via a command line interface, making it suitable for use in programs, scripts, batch files — any pl…

744 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

11 Experts available now in Live!

Get 1:1 Help Now