[Okta Webinar] Learn how to a build a cloud-first strategyRegister Now

x
  • Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 194
  • Last Modified:

Way to remember which files to download?

Hi,

I worked out the web site downloader (web crawler) but this doesn't remember where it left off if it crashes.
So I want to implement something very simple using a text file or something which files haven't download yet if I restart the app.

1) if app crashes and restarts, it needs to remember where it was lastly and pick up from there.

So I thought about creating somekind of 2 containers.
1) grab all file listing first  and insert them into this container,filetodownloadContainer
2) once it is downloaded, insert it into the dowloadedContainer
if the app goes downdown, restart and grab all the file listing again and see if it is in the downloadedContainer.

Anyway, can you point me to the right direction please?


0
dkim18
Asked:
dkim18
2 Solutions
 
yogsoftCommented:
I think you don't need to have two container for this solution. Only downloadedFileContainer would suffice your purpose. Using this container only, you can resume the state of website after application restart (i.e. Checking which file was downloaded before application restart).
0
 
Jacques Bourgeois (James Burger)Commented:
He needs both. If he has only the downloadedContainer, how can he know which file to download next when he restarts.

The solution presented in the question makes sense.

However, I would replace the downloadedContraire by a simple countOfDownloadedFiles, with a simple counter that you increment each time that a download completes. When the application restart, retrieve that count and start at count+1 file in the filetodownloadContainer.
0
 
AndyAinscowCommented:
To be honest I'd fix the problem in the app so it doesn't crash rather than trying to decide which is the best sticking plaster.
0
Independent Software Vendors: We Want Your Opinion

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 
Jacques Bourgeois (James Burger)Commented:
Andy, you have a point, but there are environments out there that are not stable as the application and the application could crash anyway.

If the solution was as simple as you say, there would be no need for transactions and rollbacks.

And in the first phases of deployment for a new application, unexpected problems could arise. That is part of life for a programmer. An unfinished job could cause problems for the user.

You drive well, so you do not have to but your seat belt on, don't you?

Any good applications has such contingencies. This is good design. Specially when the way to treat it is so easy to implement.
0
 
AndyAinscowCommented:
>>there would be no need for transactions and rollbacks.
They have numbers of useful functions that are nothing to do with app stability.

Typically a crash is a mistake by the programmer.  (poor code or poor logic)
0
 
Jacques Bourgeois (James Burger)Commented:
A crash could also be from a defective driver, a power failure, name it.

I had a problem last year in one of the classrooms where I teach. Visual Studio 2010 kept crashing. I think that the guys who wrote that stuff are knowing what they do. And am I glad that they built an autorecovery feature in the software. We pinpointed the problem to the video drivers used in that classroom, where the computers were replaced at the beginning of the session.

In an industrial environment, you would be surprised at the number of power drops they can have sometimes during a day. And you might be surprised to learn that they do not install a UPC on all the secretaries computers. The poor programmer who makes no mistake in his program (which is almost impossible by the way, unless you have a very simple program) has no control about crashes.

Having a recovery procedure in an application is just good practice. Even if my environment is very stable, I would never use software that I need to earn my paycheck that does not have a recovery feature in case of unexpected problem.

0
 
AndyAinscowCommented:
I agree that there can be hardware failures.
In my experience they are rare (and individual) occurances, interesting to hear about your experiences with them.
0
 
dkim18Author Commented:
can someone show me code example of this or something similar to what i am trying to do?
0

Featured Post

Independent Software Vendors: We Want Your Opinion

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

Tackle projects and never again get stuck behind a technical roadblock.
Join Now