Solved

Way to remember which files to download?

Posted on 2011-09-26
8
179 Views
Last Modified: 2013-12-17
Hi,

I worked out the web site downloader (web crawler) but this doesn't remember where it left off if it crashes.
So I want to implement something very simple using a text file or something which files haven't download yet if I restart the app.

1) if app crashes and restarts, it needs to remember where it was lastly and pick up from there.

So I thought about creating somekind of 2 containers.
1) grab all file listing first  and insert them into this container,filetodownloadContainer
2) once it is downloaded, insert it into the dowloadedContainer
if the app goes downdown, restart and grab all the file listing again and see if it is in the downloadedContainer.

Anyway, can you point me to the right direction please?


0
Comment
Question by:dkim18
8 Comments
 
LVL 2

Expert Comment

by:yogsoft
ID: 36707823
I think you don't need to have two container for this solution. Only downloadedFileContainer would suffice your purpose. Using this container only, you can resume the state of website after application restart (i.e. Checking which file was downloaded before application restart).
0
 
LVL 40
ID: 36709650
He needs both. If he has only the downloadedContainer, how can he know which file to download next when he restarts.

The solution presented in the question makes sense.

However, I would replace the downloadedContraire by a simple countOfDownloadedFiles, with a simple counter that you increment each time that a download completes. When the application restart, retrieve that count and start at count+1 file in the filetodownloadContainer.
0
 
LVL 44

Expert Comment

by:AndyAinscow
ID: 36710012
To be honest I'd fix the problem in the app so it doesn't crash rather than trying to decide which is the best sticking plaster.
0
 
LVL 40
ID: 36711323
Andy, you have a point, but there are environments out there that are not stable as the application and the application could crash anyway.

If the solution was as simple as you say, there would be no need for transactions and rollbacks.

And in the first phases of deployment for a new application, unexpected problems could arise. That is part of life for a programmer. An unfinished job could cause problems for the user.

You drive well, so you do not have to but your seat belt on, don't you?

Any good applications has such contingencies. This is good design. Specially when the way to treat it is so easy to implement.
0
Enabling OSINT in Activity Based Intelligence

Activity based intelligence (ABI) requires access to all available sources of data. Recorded Future allows analysts to observe structured data on the open, deep, and dark web.

 
LVL 44

Accepted Solution

by:
AndyAinscow earned 250 total points
ID: 36711426
>>there would be no need for transactions and rollbacks.
They have numbers of useful functions that are nothing to do with app stability.

Typically a crash is a mistake by the programmer.  (poor code or poor logic)
0
 
LVL 40

Assisted Solution

by:Jacques Bourgeois (James Burger)
Jacques Bourgeois (James Burger) earned 250 total points
ID: 36711736
A crash could also be from a defective driver, a power failure, name it.

I had a problem last year in one of the classrooms where I teach. Visual Studio 2010 kept crashing. I think that the guys who wrote that stuff are knowing what they do. And am I glad that they built an autorecovery feature in the software. We pinpointed the problem to the video drivers used in that classroom, where the computers were replaced at the beginning of the session.

In an industrial environment, you would be surprised at the number of power drops they can have sometimes during a day. And you might be surprised to learn that they do not install a UPC on all the secretaries computers. The poor programmer who makes no mistake in his program (which is almost impossible by the way, unless you have a very simple program) has no control about crashes.

Having a recovery procedure in an application is just good practice. Even if my environment is very stable, I would never use software that I need to earn my paycheck that does not have a recovery feature in case of unexpected problem.

0
 
LVL 44

Expert Comment

by:AndyAinscow
ID: 36711904
I agree that there can be hardware failures.
In my experience they are rare (and individual) occurances, interesting to hear about your experiences with them.
0
 

Author Comment

by:dkim18
ID: 36814154
can someone show me code example of this or something similar to what i am trying to do?
0

Featured Post

How to run any project with ease

Manage projects of all sizes how you want. Great for personal to-do lists, project milestones, team priorities and launch plans.
- Combine task lists, docs, spreadsheets, and chat in one
- View and edit from mobile/offline
- Cut down on emails

Join & Write a Comment

This document covers how to connect to SQL Server and browse its contents.  It is meant for those new to Visual Studio and/or working with Microsoft SQL Server.  It is not a guide to building SQL Server database connections in your code.  This is mo…
Real-time is more about the business, not the technology. In day-to-day life, to make real-time decisions like buying or investing, business needs the latest information(e.g. Gold Rate/Stock Rate). Unlike traditional days, you need not wait for a fe…
Polish reports in Access so they look terrific. Take yourself to another level. Equations, Back Color, Alternate Back Color. Write easy VBA Code. Tighten space to use less pages. Launch report from a menu, considering criteria only when it is filled…
This video explains how to create simple products associated to Magento configurable product and offers fast way of their generation with Store Manager for Magento tool.

708 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

17 Experts available now in Live!

Get 1:1 Help Now