Go Premium for a chance to win a PS4. Enter to Win

  • Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 295
  • Last Modified:

.Net: Parsing websites and storing in SQL Server

Hi, I'm new to windows programming and I need to do a program running on server, downloading some pages from web, parsing and storing in a SQL-server. The program should be performing it's tasks on scheduled times.

Now, what I'd like to ask is some general guidance... What kind of program do you suggest I do?

 I have most of the code ready in C# .Net application (windows forms) which could be run on selected intervals by Windows scheduler (or I could do a timer in the app itself?), but is there some kind of server application or similar (without the UI?)? I'm looking for a way that doesn't eat too much performance....

Thanks in Advance,
  • 2
1 Solution
Here are two suggestions to meet your needs:

First, you could create a Windows Service (this website has a lot of information about this:
http://www.c-sharpcorner.com/WindowsServices.asp). To do this, you would have to use a timer inside of the service itself because the service will be running continuously in the background and, thus, can't be scheduled using the Windows Scheduler.

Alternately, you could create a simple Console application that is scheduled using the Windows Scheduler. This is far easier to implement than the service, but if you REALLY need to make sure the application fires at every specified interval, the Windows Scheduler may not be reliable enough for you, particularly if the machine is used heavily. Believe it or not, Windows Scheduler does not guarantee that a particular task will be executed at a given time - instead, it just guarantees that the task won't be executed BEFORE the specified time. It will TRY to run it as soon after the specified time as possible, but that doesn't mean that it WILL run it. If you want to go this route, there are some cron utilities out there for Windows that are far more reliable than the Windows Scheduler that you could use the schedule the application.

The question of which of the above solutions is superior comes down to a question of personal taste. The service will use slightly more CPU cycles than the console application, but if you follow my advice and install a 3rd party cron utility for scheduling it will run as a service and so will increase the overall CPU footprint of the console app such that it will meet or exceed that of the service. At any rate, the performance hit we're looking at is so minimal that it probably isn't worth worrying about it.

Let me know if this helps!


I work for a company by the name of Crocus Plains Software Ltd, located in Calgary, AB, Canada.   We have been working on a project that does exactly what you are looking for, and are looking for more Beta testers.   The scheduler runs across a network, and as the parsing of these websites can be extremely CPU intensive, the scheduler manages as many application servers that you need, which run the actual parsing.  

If you are interested in becoming a Beta tester, please contact us.
(note: currently the website for this project does not have information)


Brett Swift
Thanks for the points! Let me know if you have any more questions.

mmiikaAuthor Commented:
Sure, I did it as console app... might test doing a service later, just afraid that there's more problem getting it work.

Thanks, Miika

Featured Post

[Webinar] Cloud and Mobile-First Strategy

Maybe you’ve fully adopted the cloud since the beginning. Or maybe you started with on-prem resources but are pursuing a “cloud and mobile first” strategy. Getting to that end state has its challenges. Discover how to build out a 100% cloud and mobile IT strategy in this webinar.

  • 2
Tackle projects and never again get stuck behind a technical roadblock.
Join Now