?
Solved

getting URL contants on file

Posted on 2003-03-05
13
Medium Priority
?
219 Views
Last Modified: 2010-04-21
I have one URL which is showing some logging- So when ever some activity happned -- this URL will show on the browse lower frame and after 10 min. http session will gone. If we refresh this URL what ever previos logs will clear and new session will connect.

So I would like one small script which will keep on fetching logs from URL and append to the HTML file as I want to save URL logging to my machine for future ref..

Anybody can help..
0
Comment
Question by:arvind
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 5
  • 4
  • 2
  • +2
13 Comments
 
LVL 2

Expert Comment

by:jimbb
ID: 8086351
Hi, I'm not sure I understand exactly what you mean.  By "URL" do you mean the actual link ('http:/blabla/') or the page itself?

Would you be able to use the 'wget' command to direct the page to a file?

wget -O - www.whatever.org >> logfile

Please clarify if that's not what you mean.
0
 
LVL 15

Expert Comment

by:samri
ID: 8087431
or you can you lynx (http://lynx.browser.org/), and use

lynx -source http://www.anysite.com > output.txt

and you can examine the output in output.txt (the file would be the HTML source).

cheers
0
 
LVL 5

Author Comment

by:arvind
ID: 8090986
yeah that is Perfect HTTP URL and this is kind of network logger. so for gettting that contecting --I'm keep connceted that URL on browser -- when ever some activity will happen -- browser will show the log. After 10 min(or depand on net connectivity). it will show "Logger session closed due to inactivity." ...

So Id tried lynx option which is submited by samri -- lynx will collect data and quit.. But I would to connect as session and when ever we'll get "Logger session closed due to inactivity." we need to reconnect the session...

I think this will help
0
Technology Partners: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 
LVL 15

Expert Comment

by:samri
ID: 8100987
arvind,

is this link goes to a standard HTML document, and not PHP, CGI, SHTML, etc. etc.

I would suspect that the page you are visiting is somehow have some persistent connection stuff being implemented.

Check the html documents for any Java/Javascript code for any clue.

cheers.
0
 
LVL 5

Author Comment

by:arvind
ID: 8101557
Samri,,


This is JSP page -- so same like CGI -- dynamic content.
0
 
LVL 15

Expert Comment

by:samri
ID: 8102282
hi Arvind,

I would suspect that the server-side is keeping track of session information, and expire it once you are idle (or after a preset time).

One possibility that I can think of (at the moment) would be running some kind of proxy server (locally) which logs all session.  Most proxy would allow this (even the free one).   I personally use Deerfield Interquick (long gone - but search thru the web may yield some download location.  I found an old Deerfield mirror - http://ftp.k2.net/mirrors/deerfield/pub/current/iqsetup.exe), or Squid (Unix and NT).  Anyway, whatever 3rd party product, we are interested to keep track of the session loggin -- just as you wanted it.

give the option a try, and let us know.
0
 
LVL 5

Author Comment

by:arvind
ID: 8102324
Samri,

My basic question is how we trap the URL logging to one file-- during this period whatever is going on like session closed or new session.

I would like the script which will stay connected to URL and when ever session got disconnect --script will connect new session ..

Some Suggestion -- need to track sessionid or cookies
0
 
LVL 15

Expert Comment

by:samri
ID: 8108995
hi arvind,

if that is the case, you could use lynx (http://www.lynxbrowser.org/).  Lynx has several feature where you could save session cookie, trace session, etc.

0
 
LVL 5

Author Comment

by:arvind
ID: 8109830
Samri -- that means I need to create script to myself -- OK what ever you people like. I'll create script
0
 
LVL 1

Expert Comment

by:veeruns
ID: 8182476
you could use
perl::LWP arvnd..it could help you
there is openbook on perl LWP o'reilly
u could use that
hope it help
veeruns
0
 
LVL 5

Author Comment

by:arvind
ID: 8372129
Tried LWP but its not session based. Am still working to solve this problem
0
 

Expert Comment

by:modulo
ID: 8383529
Dear expert(s),

A request has been made to close this Q in CS:
http://www.experts-exchange.com/Community_Support/Q_20592203.html

Without a response in 72 hrs, a moderator will finalize this question by:

 - Saving this Q as a PAQ and refunding the points to the questionner

When you agree or disagree, please add a comment here.

Thank you.

modulo

Community Support Moderator
Experts Exchange
0
 

Accepted Solution

by:
modulo earned 0 total points
ID: 8400217
Saving this Q as a PAQ and refunding the points to the questionner

modulo

Community Support Moderator
Experts Exchange
0

Featured Post

Understanding Linux Permissions

Linux for beginners: How to view the permissions associated with files and directories and also how you can change them.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

The purpose of this article is to fix the unknown display problem in Linux Mint operating system. After installing the OS if you see Display monitor is not recognized then we can install "MESA" utilities to fix this problem or we can install additio…
The purpose of this article is to demonstrate how we can upgrade Python from version 2.7.6 to Python 2.7.10 on the Linux Mint operating system. I am using an Oracle Virtual Box where I have installed Linux Mint operating system version 17.2. Once yo…
In this video we outline the Physical Segments view of NetCrunch network monitor. By following this brief how-to video, you will be able to learn how NetCrunch visualizes your network, how granular is the information collected, as well as where to f…
Have you created a query with information for a calendar? ... and then, abra-cadabra, the calendar is done?! I am going to show you how to make that happen. Visualize your data!  ... really see it To use the code to create a calendar from a q…
Suggested Courses
Course of the Month9 days, 6 hours left to enroll

764 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question