?
Solved

C Language question

Posted on 1997-06-05
4
Medium Priority
?
246 Views
Last Modified: 2010-04-15
I am in need of source code (or fragments thereof) for writting a small 'webcrawler' style utility.

The program needs to open a session with a (specified) Webserver on port 80, send a URL and then write the resultant page to a local flat file.

My C skills are largely self taught but once I have the basic functions I should be able to work out the rest.


Thanks in advance.

Marty
0
Comment
Question by:mravell
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 3
4 Comments
 
LVL 5

Accepted Solution

by:
julio011597 earned 0 total points
ID: 1250948
I'll assume you are on Unix box.

Doing the job is quite simple, just involves a bit of socket programming, since you are actually coding a simple client application.

Following is a sample code for the get_url function:
please check function return types and include files on your OS, since there may be slight differences.

--//--
/*
#include <stdlib.h>
#include <stdio.h>
#include <string.h>     /* bzero() and bcopy() */
#include <netdb.h>      /* gethostbyname() */
#include <sys/socket.h> /* socket() and connect() */
#include <arpa/inet.h>  /* htons() */

/* get_url()
 * answer: file to write to (must be already open for writing)
 * host: e.g. "www.experts-exchange.com"
 * url : e.g. "/topics/comp/lang/c"
 * err_str: to eventually store an error message
 *
 * returns 0 if succesful, errno otherwise
 */

int get_url(FILE *answer, const char *host, const char *url, char err_str[])
{
  int sock;
  struct hostent *hep;
  struct sockaddr_in sai;
  FILE *infile, *outfile;
  char buff[BUFSIZ];

  if(!(hep = gethostbyname(host))) {
    strcpy(err_str, "gethostbyname error");
    return errno;
  }
  if((sock = socket(AF_INET, SOCK_STREAM, 0)) == -1) {
    strcpy(err_str, "socket error");
    return errno;
  }

  bzero((char *)&sai, sizeof(struct sockaddr_in));
  sai.sin_family = AF_INET;
  sai.sin_port = htons((unsigned short)80);
  bcopy(hep->h_addr, (char *)&(sai.sin_addr.s_addr), hep->h_length); /* note: could just be 'sai.sin_addr' */

  if(connect(sock, (struct sockaddr *)&sai, sizeof(sai)) == -1) {
    strcpy(err_str, "connect error");
    return errno;
  }

  /* fine: now we are connected */
  /* let's work with stdio */
  if(!(infile = fdopen(sock, "r"))) {
    strcpy(err_str, "fdopen(in) error");
    return errno;
  }
  if(!(outfile = fdopen(sock, "w"))) {
    strcpy(err_str, "fdopen(out) error");
    return errno;
  }

  /* now, the query: i'll leave error checking now on! */
  fprintf(outfile, "GET %s\n", url);
  fflush(outfile); /* this is mandatory: outfile would buffer */

  while(fgets(buff, BUFSIZ, infile)) fputs(buff, answer);
  fflush(answer); /* this is not mandatory:) */

  /* cleanup */
  /* note: the two fclose's are also mandatory: */
  /* the file tables are not automagically deallocated */
  /* if you just close the socket (at least on my system), */
  /* and you could go out of resources if you loop */
  /* to connect to more than one site */
  fclose(infile); fclose(outfile);
  close(sock);

  return 0;
}
--//--

Ok, hope i didn't make too many mistakes in this small window...

Let me know, julio
0
 
LVL 5

Expert Comment

by:julio011597
ID: 1250949
Hello, i tried it and works! <g>

Joke apart, i had to add:
#include <sys/errno.h>
for errno to be defined.

Also, forget the '/*' just after the first cut-here.

Cheers, julio
0
 
LVL 5

Expert Comment

by:julio011597
ID: 1250950
Anybody in??
-julio
0
 

Author Comment

by:mravell
ID: 1250951
Thanks Julio, this is exactly what I was after!

Cheers
Marty
0

Featured Post

VIDEO: THE CONCERTO CLOUD FOR HEALTHCARE

Modern healthcare requires a modern cloud. View this brief video to understand how the Concerto Cloud for Healthcare can help your organization.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Windows programmers of the C/C++ variety, how many of you realise that since Window 9x Microsoft has been lying to you about what constitutes Unicode (http://en.wikipedia.org/wiki/Unicode)? They will have you believe that Unicode requires you to use…
This is a short and sweet, but (hopefully) to the point article. There seems to be some fundamental misunderstanding about the function prototype for the "main" function in C and C++, more specifically what type this function should return. I see so…
Video by: Grant
The goal of this video is to provide viewers with basic examples to understand and use for-loops in the C programming language.
The goal of this video is to provide viewers with basic examples to understand and use switch statements in the C programming language.

719 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question