• Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 222
  • Last Modified:

MSSQL CE Keep track of URLs

Alright I have written a C# program that spiders the internet and I am having an issue where the spider is visiting the website more than once.

So I devised a plan to store the data in a flat text file which resulted in a horrible idea because of the volume of entries. I believe the next step is using a database.

I have decided to use MSSQL Compact Edition as opposed to MySQL or SQLite.

Now being a complete beginner with databases I am having trouble wrapping my head around how to make this work:

I have a table called "Projects" and it has two columns as of now, ID (Unique auto increment Key) and Name which is a user defined name. I would like to store ALL the websites the spider visits per unique project.

Example:

Project Key 1 name Blue Widgets this project spiders for the term blue widget and I would like to place all the sites it visits in the database and associate the two.

Any ideas?

-Jace
0
snow4dayz
Asked:
snow4dayz
  • 4
1 Solution
 
snow4dayzAuthor Commented:
Furthermore I would like to create a hash with the url using CRC8 for quicker lookups. So save the URL to DB associate it with the unique project and create a hash that goes with the URL.
0
 
nmarunCommented:
Here are some leads for you that talk about connecting to a sql database from asp.net:

http://www.beansoftware.com/ASP.NET-Tutorials/ASP.NET-SQL-Server.aspx
http://www.tutorialized.com/view/tutorial/Connecting-to-a-SQL-database-from-ASP-.NET-II/5745

Go ahead and start with something. Once you get stuck, post the actual issue and you'll get many people to help you rather than ask for 'how to do an entire project'.

Thanks,
Arun
0
 
snow4dayzAuthor Commented:
I don't think I explained the question properly:

Stuck on:
How would I make a column accept hundreds of websites and URL CRCs? Such as:
abc.com/test BJBGBVJKBNK
bca.com/yo HJGFHJBGVFHJF

Again completely not understanding how to setup the tables, I know how to connect and read from them.
0
The new generation of project management tools

With monday.com’s project management tool, you can see what everyone on your team is working in a single glance. Its intuitive dashboards are customizable, so you can create systems that work for you.

 
KrtyknmCommented:
Hello, Just make the datatype of that column to NVARCHAR(MAX). This datatype will accept unicode values, so you can insert the URL.
0
 
snow4dayzAuthor Commented:
After some intense study into MSSQL I found what I was missing there is a way to make tables relational in the VS2010 IDE, just was over looking it. Thanks for the input.
0
 
snow4dayzAuthor Commented:
Had to answer my own question, next time around I will better phrase my question. Thanks.
0

Featured Post

The new generation of project management tools

With monday.com’s project management tool, you can see what everyone on your team is working in a single glance. Its intuitive dashboards are customizable, so you can create systems that work for you.

  • 4
Tackle projects and never again get stuck behind a technical roadblock.
Join Now