sapbucket
asked on
Implementing a timer thread
Hello,
I'm a newb when it comes to threading. I have a very specific thing I would like to try out.
I want to download html from a list of url's I have. There are approximately 300 url's. I can usually get through about 40-50 url's before the downloader I am using hangs up. I want to use a timer thread to call a method after 10 seconds (I define a "hangup" as any download that is taking longer than 10 seconds: the average url takes about 1 sec to download, so this is reasonable). The timer is reset after each successful download. If a "hangup" occurs I want to stop the current thread (the one that is hung up) and start a new thread using the same url.
So something like this:
1.) start process (main)
2.) load url list from file.
3.) loop through each url, processing each one in the following manner:
a.) start thread timer.
b.) start download of url. <---(**note 1**)
c.) if download successful, stop timer and repeat loop using next url.
d.) if thread timer expires (download is unsuccesful), stop timer and repeat loop using same url. increment 'downloadAttempts' by 1.
e.) if 'downloadAttempts' > 6, stop timer and repeat loop using next url.
However, when i implement the above timer thread algorithm I get all kinds of unexpected errors. I believe they stem from the fact that when the "hang up" occurs (and I start a new thread using the timer) the OLD thread (the one that is "hung up") has not been disposed of and is using resources I need.
So a couple questions:
1.) is the general layout of my timer thread algorithm ok? Am I approaching this problem correctly?
2.) how to dispose of the thread that is hung up? can I save the id of a thread, and if it gets hung up, dispose of it from a new thread (using the timer thread for example)? (see **note 1**)
**note 1**
// it seems like at this point instead of using the main process thread I could launch a new one:
b.) start download of url in its own thread:
1.) start thread (save id of it) and use it to perform download
2.) if timer thread expires ("hang up" occurs) delete thread (we saved the id, so this seems possible)
3.) repeat loop using same url
Many thanks for your expert advice!
I'm a newb when it comes to threading. I have a very specific thing I would like to try out.
I want to download html from a list of url's I have. There are approximately 300 url's. I can usually get through about 40-50 url's before the downloader I am using hangs up. I want to use a timer thread to call a method after 10 seconds (I define a "hangup" as any download that is taking longer than 10 seconds: the average url takes about 1 sec to download, so this is reasonable). The timer is reset after each successful download. If a "hangup" occurs I want to stop the current thread (the one that is hung up) and start a new thread using the same url.
So something like this:
1.) start process (main)
2.) load url list from file.
3.) loop through each url, processing each one in the following manner:
a.) start thread timer.
b.) start download of url. <---(**note 1**)
c.) if download successful, stop timer and repeat loop using next url.
d.) if thread timer expires (download is unsuccesful), stop timer and repeat loop using same url. increment 'downloadAttempts' by 1.
e.) if 'downloadAttempts' > 6, stop timer and repeat loop using next url.
However, when i implement the above timer thread algorithm I get all kinds of unexpected errors. I believe they stem from the fact that when the "hang up" occurs (and I start a new thread using the timer) the OLD thread (the one that is "hung up") has not been disposed of and is using resources I need.
So a couple questions:
1.) is the general layout of my timer thread algorithm ok? Am I approaching this problem correctly?
2.) how to dispose of the thread that is hung up? can I save the id of a thread, and if it gets hung up, dispose of it from a new thread (using the timer thread for example)? (see **note 1**)
**note 1**
// it seems like at this point instead of using the main process thread I could launch a new one:
b.) start download of url in its own thread:
1.) start thread (save id of it) and use it to perform download
2.) if timer thread expires ("hang up" occurs) delete thread (we saved the id, so this seems possible)
3.) repeat loop using same url
Many thanks for your expert advice!
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
Thanks Idle_Mind!
I had to invert my thinking a bit: I was putting the timer thread in the calling class, and not in the encaspulated class.
So when the timer goes off, I simply dispose of the encapsulated class.
Also, I WAS using a singleton for my downloader. Its a little slower, but now I instance a downloader for each url. These get cleaned up by the timer thread (after 8 seconds). I also added an event class to synchronize some properties that I need for each download (imagine that the downloads are firing asynchronously - I need to have the filename passed in with the event args - otherwise i don't know which download I am responsing too).
Result: download is a little slower (by a second or two per url) but I don't get any hangups. I get a 99/100 success rate. The 1 that slips by saves to files with 0 kb size. So, to make it 100%, when the process is done, I search the directory for any files that are > 1kb and download them again.
Thanks again!
I had to invert my thinking a bit: I was putting the timer thread in the calling class, and not in the encaspulated class.
So when the timer goes off, I simply dispose of the encapsulated class.
Also, I WAS using a singleton for my downloader. Its a little slower, but now I instance a downloader for each url. These get cleaned up by the timer thread (after 8 seconds). I also added an event class to synchronize some properties that I need for each download (imagine that the downloads are firing asynchronously - I need to have the filename passed in with the event args - otherwise i don't know which download I am responsing too).
Result: download is a little slower (by a second or two per url) but I don't get any hangups. I get a 99/100 success rate. The 1 that slips by saves to files with 0 kb size. So, to make it 100%, when the process is done, I search the directory for any files that are > 1kb and download them again.
Thanks again!
2) If it don't hang all your pc..yes