Solved

Way to remember which files to download?

Posted on 2011-09-26
8
182 Views
Last Modified: 2013-12-17
Hi,

I worked out the web site downloader (web crawler) but this doesn't remember where it left off if it crashes.
So I want to implement something very simple using a text file or something which files haven't download yet if I restart the app.

1) if app crashes and restarts, it needs to remember where it was lastly and pick up from there.

So I thought about creating somekind of 2 containers.
1) grab all file listing first  and insert them into this container,filetodownloadContainer
2) once it is downloaded, insert it into the dowloadedContainer
if the app goes downdown, restart and grab all the file listing again and see if it is in the downloadedContainer.

Anyway, can you point me to the right direction please?


0
Comment
Question by:dkim18
8 Comments
 
LVL 2

Expert Comment

by:yogsoft
ID: 36707823
I think you don't need to have two container for this solution. Only downloadedFileContainer would suffice your purpose. Using this container only, you can resume the state of website after application restart (i.e. Checking which file was downloaded before application restart).
0
 
LVL 40
ID: 36709650
He needs both. If he has only the downloadedContainer, how can he know which file to download next when he restarts.

The solution presented in the question makes sense.

However, I would replace the downloadedContraire by a simple countOfDownloadedFiles, with a simple counter that you increment each time that a download completes. When the application restart, retrieve that count and start at count+1 file in the filetodownloadContainer.
0
 
LVL 44

Expert Comment

by:AndyAinscow
ID: 36710012
To be honest I'd fix the problem in the app so it doesn't crash rather than trying to decide which is the best sticking plaster.
0
How our DevOps Teams Maximize Uptime

Our Dev teams are like yours. They’re continually cranking out code for new features/bugs fixes, testing, deploying, responding to production monitoring events and more. It’s complex. So, we thought you’d like to see what’s working for us. Read the use case whitepaper.

 
LVL 40
ID: 36711323
Andy, you have a point, but there are environments out there that are not stable as the application and the application could crash anyway.

If the solution was as simple as you say, there would be no need for transactions and rollbacks.

And in the first phases of deployment for a new application, unexpected problems could arise. That is part of life for a programmer. An unfinished job could cause problems for the user.

You drive well, so you do not have to but your seat belt on, don't you?

Any good applications has such contingencies. This is good design. Specially when the way to treat it is so easy to implement.
0
 
LVL 44

Accepted Solution

by:
AndyAinscow earned 250 total points
ID: 36711426
>>there would be no need for transactions and rollbacks.
They have numbers of useful functions that are nothing to do with app stability.

Typically a crash is a mistake by the programmer.  (poor code or poor logic)
0
 
LVL 40

Assisted Solution

by:Jacques Bourgeois (James Burger)
Jacques Bourgeois (James Burger) earned 250 total points
ID: 36711736
A crash could also be from a defective driver, a power failure, name it.

I had a problem last year in one of the classrooms where I teach. Visual Studio 2010 kept crashing. I think that the guys who wrote that stuff are knowing what they do. And am I glad that they built an autorecovery feature in the software. We pinpointed the problem to the video drivers used in that classroom, where the computers were replaced at the beginning of the session.

In an industrial environment, you would be surprised at the number of power drops they can have sometimes during a day. And you might be surprised to learn that they do not install a UPC on all the secretaries computers. The poor programmer who makes no mistake in his program (which is almost impossible by the way, unless you have a very simple program) has no control about crashes.

Having a recovery procedure in an application is just good practice. Even if my environment is very stable, I would never use software that I need to earn my paycheck that does not have a recovery feature in case of unexpected problem.

0
 
LVL 44

Expert Comment

by:AndyAinscow
ID: 36711904
I agree that there can be hardware failures.
In my experience they are rare (and individual) occurances, interesting to hear about your experiences with them.
0
 

Author Comment

by:dkim18
ID: 36814154
can someone show me code example of this or something similar to what i am trying to do?
0

Featured Post

Free Tool: Postgres Monitoring System

A PHP and Perl based system to collect and display usage statistics from PostgreSQL databases.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Entity Framework is a powerful tool to help you interact with the DataBase but still doesn't help much when we have a Stored Procedure that returns more than one resultset. The solution takes some of out-of-the-box thinking; read on!
Exception Handling is in the core of any application that is able to dignify its name. In this article, I'll guide you through the process of writing a DRY (Don't Repeat Yourself) Exception Handling mechanism, using Aspect Oriented Programming.
Two types of users will appreciate AOMEI Backupper Pro: 1 - Those with PCIe drives (and haven't found cloning software that works on them). 2 - Those who want a fast clone of their boot drive (no re-boots needed) and it can clone your drive wh…

856 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question