Solved

Way to remember which files to download?

Posted on 2011-09-26
8
180 Views
Last Modified: 2013-12-17
Hi,

I worked out the web site downloader (web crawler) but this doesn't remember where it left off if it crashes.
So I want to implement something very simple using a text file or something which files haven't download yet if I restart the app.

1) if app crashes and restarts, it needs to remember where it was lastly and pick up from there.

So I thought about creating somekind of 2 containers.
1) grab all file listing first  and insert them into this container,filetodownloadContainer
2) once it is downloaded, insert it into the dowloadedContainer
if the app goes downdown, restart and grab all the file listing again and see if it is in the downloadedContainer.

Anyway, can you point me to the right direction please?


0
Comment
Question by:dkim18
8 Comments
 
LVL 2

Expert Comment

by:yogsoft
ID: 36707823
I think you don't need to have two container for this solution. Only downloadedFileContainer would suffice your purpose. Using this container only, you can resume the state of website after application restart (i.e. Checking which file was downloaded before application restart).
0
 
LVL 40
ID: 36709650
He needs both. If he has only the downloadedContainer, how can he know which file to download next when he restarts.

The solution presented in the question makes sense.

However, I would replace the downloadedContraire by a simple countOfDownloadedFiles, with a simple counter that you increment each time that a download completes. When the application restart, retrieve that count and start at count+1 file in the filetodownloadContainer.
0
 
LVL 44

Expert Comment

by:AndyAinscow
ID: 36710012
To be honest I'd fix the problem in the app so it doesn't crash rather than trying to decide which is the best sticking plaster.
0
 
LVL 40
ID: 36711323
Andy, you have a point, but there are environments out there that are not stable as the application and the application could crash anyway.

If the solution was as simple as you say, there would be no need for transactions and rollbacks.

And in the first phases of deployment for a new application, unexpected problems could arise. That is part of life for a programmer. An unfinished job could cause problems for the user.

You drive well, so you do not have to but your seat belt on, don't you?

Any good applications has such contingencies. This is good design. Specially when the way to treat it is so easy to implement.
0
3 Use Cases for Connected Systems

Our Dev teams are like yours. They’re continually cranking out code for new features/bugs fixes, testing, deploying, testing some more, responding to production monitoring events and more. It’s complex. So, we thought you’d like to see what’s working for us.

 
LVL 44

Accepted Solution

by:
AndyAinscow earned 250 total points
ID: 36711426
>>there would be no need for transactions and rollbacks.
They have numbers of useful functions that are nothing to do with app stability.

Typically a crash is a mistake by the programmer.  (poor code or poor logic)
0
 
LVL 40

Assisted Solution

by:Jacques Bourgeois (James Burger)
Jacques Bourgeois (James Burger) earned 250 total points
ID: 36711736
A crash could also be from a defective driver, a power failure, name it.

I had a problem last year in one of the classrooms where I teach. Visual Studio 2010 kept crashing. I think that the guys who wrote that stuff are knowing what they do. And am I glad that they built an autorecovery feature in the software. We pinpointed the problem to the video drivers used in that classroom, where the computers were replaced at the beginning of the session.

In an industrial environment, you would be surprised at the number of power drops they can have sometimes during a day. And you might be surprised to learn that they do not install a UPC on all the secretaries computers. The poor programmer who makes no mistake in his program (which is almost impossible by the way, unless you have a very simple program) has no control about crashes.

Having a recovery procedure in an application is just good practice. Even if my environment is very stable, I would never use software that I need to earn my paycheck that does not have a recovery feature in case of unexpected problem.

0
 
LVL 44

Expert Comment

by:AndyAinscow
ID: 36711904
I agree that there can be hardware failures.
In my experience they are rare (and individual) occurances, interesting to hear about your experiences with them.
0
 

Author Comment

by:dkim18
ID: 36814154
can someone show me code example of this or something similar to what i am trying to do?
0

Featured Post

Is Your Active Directory as Secure as You Think?

More than 75% of all records are compromised because of the loss or theft of a privileged credential. Experts have been exploring Active Directory infrastructure to identify key threats and establish best practices for keeping data safe. Attend this month’s webinar to learn more.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Many of us here at EE write code. Many of us write exceptional code; just as many of us write exception-prone code. As we all should know, exceptions are a mechanism for handling errors which are typically out of our control. From database errors, t…
More often than not, we developers are confronted with a need: a need to make some kind of magic happen via code. Whether it is for a client, for the boss, or for our own personal projects, the need must be satisfied. Most of the time, the Framework…
Windows 10 is mostly good. However the one thing that annoys me is how many clicks you have to do to dial a VPN connection. You have to go to settings from the start menu, (2 clicks), Network and Internet (1 click), Click VPN (another click) then fi…
Learn how to create flexible layouts using relative units in CSS.  New relative units added in CSS3 include vw(viewports width), vh(viewports height), vmin(minimum of viewports height and width), and vmax (maximum of viewports height and width).

867 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

22 Experts available now in Live!

Get 1:1 Help Now