Want to win a PS4? Go Premium and enter to win our High-Tech Treats giveaway. Enter to Win

x
?
Solved

Way to remember which files to download?

Posted on 2011-09-26
8
Medium Priority
?
191 Views
Last Modified: 2013-12-17
Hi,

I worked out the web site downloader (web crawler) but this doesn't remember where it left off if it crashes.
So I want to implement something very simple using a text file or something which files haven't download yet if I restart the app.

1) if app crashes and restarts, it needs to remember where it was lastly and pick up from there.

So I thought about creating somekind of 2 containers.
1) grab all file listing first  and insert them into this container,filetodownloadContainer
2) once it is downloaded, insert it into the dowloadedContainer
if the app goes downdown, restart and grab all the file listing again and see if it is in the downloadedContainer.

Anyway, can you point me to the right direction please?


0
Comment
Question by:dkim18
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
8 Comments
 
LVL 2

Expert Comment

by:yogsoft
ID: 36707823
I think you don't need to have two container for this solution. Only downloadedFileContainer would suffice your purpose. Using this container only, you can resume the state of website after application restart (i.e. Checking which file was downloaded before application restart).
0
 
LVL 40
ID: 36709650
He needs both. If he has only the downloadedContainer, how can he know which file to download next when he restarts.

The solution presented in the question makes sense.

However, I would replace the downloadedContraire by a simple countOfDownloadedFiles, with a simple counter that you increment each time that a download completes. When the application restart, retrieve that count and start at count+1 file in the filetodownloadContainer.
0
 
LVL 45

Expert Comment

by:AndyAinscow
ID: 36710012
To be honest I'd fix the problem in the app so it doesn't crash rather than trying to decide which is the best sticking plaster.
0
Efficient way to get backups off site to Azure

This user guide provides instructions on how to deploy and configure both a StoneFly Scale Out NAS Enterprise Cloud Drive virtual machine and Veeam Cloud Connect in the Microsoft Azure Cloud.

 
LVL 40
ID: 36711323
Andy, you have a point, but there are environments out there that are not stable as the application and the application could crash anyway.

If the solution was as simple as you say, there would be no need for transactions and rollbacks.

And in the first phases of deployment for a new application, unexpected problems could arise. That is part of life for a programmer. An unfinished job could cause problems for the user.

You drive well, so you do not have to but your seat belt on, don't you?

Any good applications has such contingencies. This is good design. Specially when the way to treat it is so easy to implement.
0
 
LVL 45

Accepted Solution

by:
AndyAinscow earned 1000 total points
ID: 36711426
>>there would be no need for transactions and rollbacks.
They have numbers of useful functions that are nothing to do with app stability.

Typically a crash is a mistake by the programmer.  (poor code or poor logic)
0
 
LVL 40

Assisted Solution

by:Jacques Bourgeois (James Burger)
Jacques Bourgeois (James Burger) earned 1000 total points
ID: 36711736
A crash could also be from a defective driver, a power failure, name it.

I had a problem last year in one of the classrooms where I teach. Visual Studio 2010 kept crashing. I think that the guys who wrote that stuff are knowing what they do. And am I glad that they built an autorecovery feature in the software. We pinpointed the problem to the video drivers used in that classroom, where the computers were replaced at the beginning of the session.

In an industrial environment, you would be surprised at the number of power drops they can have sometimes during a day. And you might be surprised to learn that they do not install a UPC on all the secretaries computers. The poor programmer who makes no mistake in his program (which is almost impossible by the way, unless you have a very simple program) has no control about crashes.

Having a recovery procedure in an application is just good practice. Even if my environment is very stable, I would never use software that I need to earn my paycheck that does not have a recovery feature in case of unexpected problem.

0
 
LVL 45

Expert Comment

by:AndyAinscow
ID: 36711904
I agree that there can be hardware failures.
In my experience they are rare (and individual) occurances, interesting to hear about your experiences with them.
0
 

Author Comment

by:dkim18
ID: 36814154
can someone show me code example of this or something similar to what i am trying to do?
0

Featured Post

Free Backup Tool for VMware and Hyper-V

Restore full virtual machine or individual guest files from 19 common file systems directly from the backup file. Schedule VM backups with PowerShell scripts. Set desired time, lean back and let the script to notify you via email upon completion.  

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Calculating holidays and working days is a function that is often needed yet it is not one found within the Framework. This article presents one approach to building a working-day calculator for use in .NET.
It was really hard time for me to get the understanding of Delegates in C#. I went through many websites and articles but I found them very clumsy. After going through those sites, I noted down the points in a easy way so here I am sharing that unde…
Sometimes it takes a new vantage point, apart from our everyday security practices, to truly see our Active Directory (AD) vulnerabilities. We get used to implementing the same techniques and checking the same areas for a breach. This pattern can re…
How to fix incompatible JVM issue while installing Eclipse While installing Eclipse in windows, got one error like above and unable to proceed with the installation. This video describes how to successfully install Eclipse. How to solve incompa…

618 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question