converting (string) urls to 4 byte integer numbers

you could create a LUT in the database, with the ID, URL as columns (e.g. In oracle, use a SEQUENCE to generate the ID), and use the ID as foreign key everywhere.

Other than that, you could generate a CRC function, or some type of hashcode function, on the URL to convert the URL to a code. This would not be guaranteed to be unique though...

That sounds like the best idea (creating an autonumber as an ID).  The other problem with a hash, besides the fact that with a 4-byte hash you will have a reasonable probability of collisions, is that it is one way.  You cannot back-figure the URL from the hash, so you would need to store the URL anyway unless you are only looking up based on a URL as input.

CRC generation is a proven technology. And it works on strings of unlimited length. Any function you code yourself, may or may not gurantee the uniqueness of a the key associated with a URL.

Another solution, and a little easier, can be to generate and use GUIDs. Its standrad on Windows platform, and can be relaibly used to store keys. Its a virtual gurantee that a GUID will be unique.

lets say you have a 50 character url ...

there is no way it could possibly be uniquely hashed into a 4 byte integer (simple math on the number of combinations)

the idea of the uniqueid is good ...

another idea would just be to use a hash function ...

    unsigned long
    hash(unsigned char *str)
    {
        unsigned long hash = 5381;
        int c;

        while (c = *str++)
            hash = ((hash << 5) + hash) + c; /* hash * 33 + c */

        return hash;
    }


is a decent one.

If you are using MS SQL

create table myURLS
(
    recid int IDENTITY(1,1) NOT NULL,
    URL varchar(256) PRIMARY KEY NOT NULL
)

Then

insert into myURLS ( URL ) value ('www.1.com' )
insert into myURLS ( URL ) value ( 'www.2.com' )
...

Good for 2^32 URLS which is slightly more URLS than are currently out there.

hi,
 you can specify the IP address in integer format (called in windows uint32 - unsigned long.).
 you can get it by inet_addr (const char * ipaddress ). ipaddress is in dotted notation format.
 you can get host name/ url using  gethostbyaddr method.
Syntax:
 struct hostent FAR * gethostbyaddr ( const char FAR * addr,  int len, int type )
 addr - dotted notation IP address.
 len - length of address.
 type - type of address

hostent structure:

struct hostent {
    char FAR *       h_name;
    char FAR * FAR * h_aliases;
    short            h_addrtype;
    short            h_length;
    char FAR * FAR * h_addr_list;
};

try that.
all the best

aravindtj, at best this would work for domain names, not full URLs, and the overhead of doing the DNS lookup (etc) to get the ID would probably be over the top...

Also there is the fact that IP's have a one to many relationship with domain names. A server with a single IP can host more than 1 domain.

Thanks everone who posted a comment  and thanks to cjjclifford for some relevant suggestions.  In the end (in case anyone is interested) I decided to use java's GZIP to compress the string url.  Cheers,
          everton690.