Getting information from the Web

Posted on 1997-05-17
Last Modified: 2013-12-25
Is there a way I can get information from the Web without CGI.
I need to get information from the Web, record it into a file (text file) and thats all. I have shell account and I know more or less UNIX.
NOTE: If it is so I gonna increse the puntuation

Question by:cacost
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 4
  • 3

Accepted Solution

Fordream earned 100 total points
ID: 1828030
That is possible but why don't you use CGI??
 However, if you have unix shell account, you can make a program running in background that makes the result HTML file. Or you can also use at or cron.
 I guess the 'information' means variable information like weather information. Am I wrong?
 For example, yes, you are to make weather information page without CGI. You can make main page(index.html) that refers weather.html and program that updates weather.html everyday.
 Client-requesting method is impossible without CGI or very special way because program should be able to get query from web. And ... you may not mean 'constant information', you do?


Author Comment

ID: 1828031
1.- I don't know much about CGI and have not cgi access.
2.- What is this:   at or cron.
3.- Yes you are right information are variables. Can I record it into a file.

What I need is to get just 1 line (8 letters) from the Web and put it into a file. For example I have I file like this:

and I want get the word Charles from the web and update my file to:

and so over and over again. If I can please tell me how


Expert Comment

ID: 1828032
1. CGI is not hard. It's easy when you know UNIX. It's just like UNIX programming. What you have to do for CGI is change some input/output format from general unix program(whatever it is shell script or perl or C program..).
 You said you do not have CGI access but I don't think so. in the most case, you can make CGI program when you have shell prompt. Check your HTTP demon again.

2. AT is a utility that runs a program on time you want. For example, you can make it to mail file list of your diretory to someone on 5/21 10:00.AM., and you can whatever you want on reserved time.
 CRON is a utility like AT. But AT runs programs once. CRON runs program periodically(everymonth, everyday, or everyhour). For example you can make it to check system and mail report every monday...
 These utilities doesn't mean important to do what you want. They can be used to make periodically updated information.

3. To do what you want, you have to use CGI because it needs query data("just 1 line" is it.) You can do what you said as example with simple work. But it is impossible or very hard to get information from Web without CGI.

Containers and Docker for Everyone

Containers are an incredibly powerful technology that can provide you and/or your engineering team with huge productivity gains. Using containers, you can deploy, back up, replicate, and move apps and their dependencies quickly and easily.


Author Comment

ID: 1828033
Hello, again.
1.- O.K. How do I combine my Unix script in order to receive one line from the web?
2.- I know how to use crontab (I have a daemon running into my account with crontab).

Thanks again. You are very pacient.


Author Comment

ID: 1828034
Check the value again Fordream

Expert Comment

ID: 1828035
Thanks cacost. and sorry for late answer... I didn't have time to come Experts-Exchange.

1. First, you make a form in html file to input one line like following:

<form action="submit.cgi" method=get>
Please input what you want to input:
<input type=text name=data size=9 maxlength=8>
Then please submit the data!
<input type=submit name=Submit>

 This will make to get a line from web and submit the data to submit.cgi with GET method. GET method? GET method is one of CGI data-posting method. there are two methods: GET and POST. GET sends data to CGI as program arguments(argv[] variable in C, and $1,$2... in Shell script or Perl), and POST sends data to CGI as STDIN (Standard Input Stream)... GET method is easier to use but it can't accept large mount of data.

 Then, let's make the CGI. I recommend several CGI libraries because it is so annoying to make routine for CGI program(query data phrasing, character converting, etc...). And careless in these may occur security problem. You can get many CGI libraries for C or Perl from Web(Try to enter "CGI" on yahoo query!).

 Query data is sent as this form:


 In above case, inputed line will be sent like this:


 The program must separate each input data (in this case it is not necessary because input data is only one.) and name and value in each data.
 In using GET method, your CGI will get data as program arguments. Pharse the data using CGI-Library and process(Add data to file, mail him something or whatever), and output the result, then it's done.
 In CGI output, you should put CGI header in head of output.
 CGI header is usually this string:

Content-Type: text/html
(a blank line)

 and CGI library may have a routine to output header.

2. crontab is not necessary in this case ;). forget it.

 Oh it is very difficult to explain how to do. moreover I can't speak English well... I hope you to get a good CGI library and read the manual and example file. It is not too hard if you know Unix. If you have any trouble in making, mail me to ( I'll try to help you as I can. Or please explain me exactly what operation you want then I could make you the HTML and CGI as well.

Author Comment

ID: 1828036
I'll try all what you said, however if it doesn't work at all I have learned a lot.

Thanks, my e-mail is

Featured Post

Free Tool: Site Down Detector

Helpful to verify reports of your own downtime, or to double check a downed website you are trying to access.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Recently I have been answering a lot of questions like this in IT forums that I frequent. The question posed is usually something along the lines of "We have software X installed and need to uninstall it for reason Y" or some other variant of the sa…
In threads here at EE, each comment has a unique Identifier (ID). It is easy to get the full path for an ID via the right-click context menu. However, we often want to post a short link within a thread rather than the full link. This article shows a…
The viewer will learn the basics of jQuery, including how to invoke it on a web page. Reference your jQuery libraries: (CODE) Include your new external js/jQuery file: (CODE) Write your first lines of code to setup your site for jQuery.: (CODE)
Video by: Mark
This lesson goes over how to construct ordered and unordered lists and how to create hyperlinks.

724 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question