We help IT Professionals succeed at work.

Getting information from the Web

cacost
cacost asked
on
Medium Priority
298 Views
Last Modified: 2013-12-25
Hello.
Is there a way I can get information from the Web without CGI.
I need to get information from the Web, record it into a file (text file) and thats all. I have shell account and I know more or less UNIX.
NOTE: If it is so I gonna increse the puntuation

Thanks.
Comment
Watch Question

Commented:
That is possible but why don't you use CGI??
 However, if you have unix shell account, you can make a program running in background that makes the result HTML file. Or you can also use at or cron.
 I guess the 'information' means variable information like weather information. Am I wrong?
 For example, yes, you are to make weather information page without CGI. You can make main page(index.html) that refers weather.html and program that updates weather.html everyday.
 Client-requesting method is impossible without CGI or very special way because program should be able to get query from web. And ... you may not mean 'constant information', you do?
 Bye

Not the solution you were looking for? Getting a personalized solution is easy.

Ask the Experts

Author

Commented:
Thanks
1.- I don't know much about CGI and have not cgi access.
2.- What is this:   at or cron.
3.- Yes you are right information are variables. Can I record it into a file.

What I need is to get just 1 line (8 letters) from the Web and put it into a file. For example I have I file like this:
Smith
John
David

and I want get the word Charles from the web and update my file to:
Smith
John
David
Charles

and so over and over again. If I can please tell me how



Commented:
1. CGI is not hard. It's easy when you know UNIX. It's just like UNIX programming. What you have to do for CGI is change some input/output format from general unix program(whatever it is shell script or perl or C program..).
 You said you do not have CGI access but I don't think so. in the most case, you can make CGI program when you have shell prompt. Check your HTTP demon again.

2. AT is a utility that runs a program on time you want. For example, you can make it to mail file list of your diretory to someone on 5/21 10:00.AM., and you can whatever you want on reserved time.
 CRON is a utility like AT. But AT runs programs once. CRON runs program periodically(everymonth, everyday, or everyhour). For example you can make it to check system and mail report every monday...
 These utilities doesn't mean important to do what you want. They can be used to make periodically updated information.

3. To do what you want, you have to use CGI because it needs query data("just 1 line" is it.) You can do what you said as example with simple work. But it is impossible or very hard to get information from Web without CGI.

Author

Commented:
Hello, again.
Thanks.
1.- O.K. How do I combine my Unix script in order to receive one line from the web?
2.- I know how to use crontab (I have a daemon running into my account with crontab).

Thanks again. You are very pacient.


Author

Commented:
Check the value again Fordream

Commented:
Thanks cacost. and sorry for late answer... I didn't have time to come Experts-Exchange.

1. First, you make a form in html file to input one line like following:

<form action="submit.cgi" method=get>
Please input what you want to input:
<input type=text name=data size=9 maxlength=8>
<p>
Then please submit the data!
<input type=submit name=Submit>
</form>

 This will make to get a line from web and submit the data to submit.cgi with GET method. GET method? GET method is one of CGI data-posting method. there are two methods: GET and POST. GET sends data to CGI as program arguments(argv[] variable in C, and $1,$2... in Shell script or Perl), and POST sends data to CGI as STDIN (Standard Input Stream)... GET method is easier to use but it can't accept large mount of data.

 Then, let's make the CGI. I recommend several CGI libraries because it is so annoying to make routine for CGI program(query data phrasing, character converting, etc...). And careless in these may occur security problem. You can get many CGI libraries for C or Perl from Web(Try to enter "CGI" on yahoo query!).

 Query data is sent as this form:

name=value;name=value;name=value...

 In above case, inputed line will be sent like this:

data=Smith
data=Jones
.
.

 The program must separate each input data (in this case it is not necessary because input data is only one.) and name and value in each data.
 In using GET method, your CGI will get data as program arguments. Pharse the data using CGI-Library and process(Add data to file, mail him something or whatever), and output the result, then it's done.
 In CGI output, you should put CGI header in head of output.
 CGI header is usually this string:

Content-Type: text/html
(a blank line)

 and CGI library may have a routine to output header.

2. crontab is not necessary in this case ;). forget it.

 Oh it is very difficult to explain how to do. moreover I can't speak English well... I hope you to get a good CGI library and read the manual and example file. It is not too hard if you know Unix. If you have any trouble in making, mail me to (ddongper@aminet.co.kr). I'll try to help you as I can. Or please explain me exactly what operation you want then I could make you the HTML and CGI as well.

Author

Commented:
Thanks!!.
I'll try all what you said, however if it doesn't work at all I have learned a lot.

Thanks, my e-mail is aacosta@planetall.com
Access more of Experts Exchange with a free account
Thanks for using Experts Exchange.

Create a free account to continue.

Limited access with a free account allows you to:

  • View three pieces of content (articles, solutions, posts, and videos)
  • Ask the experts questions (counted toward content limit)
  • Customize your dashboard and profile

*This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

OR

Please enter a first name

Please enter a last name

8+ characters (letters, numbers, and a symbol)

By clicking, you agree to the Terms of Use and Privacy Policy.