Expiring Today—Celebrate National IT Professionals Day with 3 months of free Premium Membership. Use Code ITDAY17

x
?
Solved

Wget dowload web site log files and delete file

Posted on 2006-11-10
15
Medium Priority
?
360 Views
Last Modified: 2008-02-01
Hi

The log files are delivered to a folder on web server /purpleski/www.purpleski.com/logs.  Can the following be done with wget and if so can anybody give me the necessary instructions to do it!?

When run the wget command fetches any files in the directory /purpleski/www.purpleski.com/logs downloads these to my local computer to E:\Websites\purpleski\logs and then deletes any files in /purpleski/www.purpleski.com/logs.

This means that each day when it is run there will only be one file to download.

If the above won't work does anybody know of another solution to achieve the above?

Thanks.

purpleperson
0
Comment
Question by:purpleski
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 8
  • 7
15 Comments
 
LVL 18

Expert Comment

by:Morcalavin
ID: 17914349
I don't think wget would be able to delete the log on the server.  wget simply retrives files via an httpclient session.  Does this website have ftp access?  If so, this could be done with ftp fairly easily.
0
 

Author Comment

by:purpleski
ID: 17914785
Yes I think it has ftp access. How could I tell?

Thanks
0
 
LVL 18

Expert Comment

by:Morcalavin
ID: 17914810
Attempt to ftp to the site:
From a dos prompt:
c:\ftp myhostname.com
0
Concerto Cloud for Software Providers & ISVs

Can Concerto Cloud Services help you focus on evolving your application offerings, while delivering the best cloud experience to your customers? From DevOps to revenue models and customer support, the answer is yes!

Learn how Concerto can help you.

 

Author Comment

by:purpleski
ID: 17915261
I get

Connected to purpleski.com.
220 pop3.gradwell.net NcFTPd Se
User (purpleski.com:(none)):

so yes I think!
0
 
LVL 18

Expert Comment

by:Morcalavin
ID: 17915528
Assuming you know your username and password, you could create a batch job that would ftp for you.  I recommend you test it on some uselss files first.
Copy and paste EVERYTHING below this line into a file called get.bat:
@echo off
>script.ftp echo open ftp.somesite.com
>>script.ftp echo user
>>script.ftp echo pass
>>script.ftp cd /your/path/to/log/files
>>script.ftp echo prompt
>>script.ftp echo mget *
>>script.ftp echo del *
>>script.ftp echo quit
ftp -s:script.ftp
del script.ftp
0
 

Author Comment

by:purpleski
ID: 17916143
Thanks

tried the following (obviously changed password for this post!)

Microsoft Windows XP [Version 5.1.2600]
(C) Copyright 1985-2001 Microsoft Corp.

C:\Documents and Settings\Michael>cd\

C:\>get
The system cannot find the path specified.
ftp> open ftp.hathor.xcalibre.co.uk
Unknown host ftp.hathor.xcalibre.co.uk.
ftp> purpleski
Invalid command.
ftp> PASSWORD
Invalid command.
ftp> prompt
Interactive mode Off .
ftp> mget *
Not connected.
ftp> del ex060809.log
Not connected.
ftp> quit

What am I doing wrong
C:\>
0
 
LVL 18

Expert Comment

by:Morcalavin
ID: 17916196
ftp> open ftp.hathor.xcalibre.co.uk
Unknown host ftp.hathor.xcalibre.co.uk.

Your host doesn't exist or doesn't have an ftp  client running.  Are you sure that is where you need to ftp to?
0
 

Author Comment

by:purpleski
ID: 17916508
When I use CuteFTP to upload/download files I enter

Host Address
hathor.xcalibre.co.uk

Username
purpleski

password
************

any advice?

purpleperson
0
 
LVL 18

Expert Comment

by:Morcalavin
ID: 17916535
ftp.hathor.xcalibre.co.uk isn't the same as hathor.xcalibre.co.uk
@echo off
>script.ftp echo open hathor.xcalibre.co.uk
>>script.ftp echo user
>>script.ftp echo pass
>>script.ftp cd /your/path/to/log/files
>>script.ftp echo prompt
>>script.ftp echo mget *
>>script.ftp echo del *
>>script.ftp echo quit
ftp -s:script.ftp
del script.ftp
0
 

Author Comment

by:purpleski
ID: 17916941
Getting there. I now get

Microsoft Windows XP [Version 5.1.2600]
(C) Copyright 1985-2001 Microsoft Corp.

C:\Documents and Settings\Michael>cd\

C:\>get
The system cannot find the path specified.
ftp> open hathor.xcalibre.co.uk
Connected to hathor.xcalibre.co.uk.
220 Microsoft FTP Service
User (hathor.xcalibre.co.uk:(none)):
331 Password required for purpleski.

230 User purpleski logged in.
ftp> prompt
Interactive mode Off .
ftp> mget xxxx.log
xxxx.log: The system cannot find the file specified.
200 Type set to A.
ftp> del xxxx.log
550 xxxx.log: The system cannot find the file specified.
ftp> quit
221

C:\>

Looks like

>>script.ftp cd/home/11550/purpleski/www.purpleski.com/logs

I have been told that  Physical filename on the Windows Server is

E:\home\11550\purpleski\www.purpleski.com


What am I doing wrong any thoughts?

Thanks for all your help on this one
0
 
LVL 18

Expert Comment

by:Morcalavin
ID: 17917098
For one thing, there should be a space:
cd /home/11550/purpleski/www.purpleski.com/logs
For another, it depends on how the ftp server is set up.  Try log in yourself and type the command:
cd /home/11550/purpleski/www.purpleski.com/logs
and see where it takes you.

0
 

Author Comment

by:purpleski
ID: 17921258
Ok we are nearly there hope you can still help me

I can log on manually at a command prompt and do it all manually

Microsoft Windows XP [Version 5.1.2600]
(C) Copyright 1985-2001 Microsoft Corp.

C:\Documents and Settings\Michael>cd\

C:\>ftp
ftp> open hathor.xcalibre.co.uk
Connected to hathor.xcalibre.co.uk.
220 Microsoft FTP Service
User (hathor.xcalibre.co.uk:(none)): purpleski
331 Password required for purpleski.
Password:
230 User purpleski logged in.
ftp> cd www.purpleski.com/logs
250 CWD command successful.
ftp> prompt
Interactive mode Off .
ftp> mget xxxx.log
200 Type set to A.
200 PORT command successful.
150 Opening ASCII mode data connection for xxxx.log(3766362 bytes).
226 Transfer complete.
ftp: 3766362 bytes received in 16.70Seconds 225.49Kbytes/sec.
ftp> del xxxx.log
250 DELE command successful.
ftp> quit
221

C:\>

but if I run the script you gave me:

@echo off
>script.ftp echo open hathor.xcalibre.co.uk
>>script.ftp echo purpleski
>>script.ftp echo mRgyzUpF
>>script.ftp cd www.purpleski.com/logs
>>script.ftp echo prompt
>>script.ftp echo mget xxxx.log
>>script.ftp echo del xxxx.log
>>script.ftp echo quit
ftp -s:script.ftp
del script.ftp

The output is


C:\>get
The system cannot find the path specified.
ftp> open hathor.xcalibre.co.uk
Connected to hathor.xcalibre.co.uk.
220 Microsoft FTP Service
User (hathor.xcalibre.co.uk:(none)):
331 Password required for purpleski.

230 User purpleski logged in.
ftp> prompt
Interactive mode Off .
ftp> mget xxxx.log
xxxx.log: The system cannot find the file specified.
200 Type set to A.
ftp> del xxxx.log
550 xxxx.log: The system cannot find the file specified.
ftp> quit
221

C:\>

It looks like the cd instruction is being ignored.

Upped the points as this is obviously more difficult than I thought!!

purpleperson

0
 

Author Comment

by:purpleski
ID: 17921553
Hi

Did some googling which led me to change the script to

open hathor.xcalibre.co.uk
USERNAME
PASSWORD
cd www.purpleski.com/logs
prompt
mget *
del *
quit
ftp -s:script.ftp
del script.ftp

then at the command line I type

ftp -s:SCRIPT.TXT

This works.

Any thoughts?
0
 
LVL 18

Accepted Solution

by:
Morcalavin earned 750 total points
ID: 17922680
Ahh.  I see the problem now:
>>script.ftp cd www.purpleski.com/logs

This should be :
>>script.ftp echo cd www.purpleski.com/logs
0
 

Author Comment

by:purpleski
ID: 17928908
Thanks for all that it works perfectly as follows:

@echo off
>script.ftp echo open hathor.xcalibre.co.uk
>>script.ftp echo purpleski
>>script.ftp echo mRgyzUpF
>>script.ftp echo cd www.purpleski.com/logs
>>script.ftp echo prompt
>>script.ftp echo mget *
>>script.ftp echo del *
>>script.ftp echo quit
ftp -s:script.ftp
del script.ftp

Going to use it properley as a scheduled task tonight at 00.01 GMT let us hope it works!

Points are yours
0

Featured Post

Concerto's Cloud Advisory Services

Want to avoid the missteps to gaining all the benefits of the cloud? Learn more about the different assessment options from our Cloud Advisory team.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

When it comes to write a Context Sensitive Help (an online help that is obtained from a specific point in state of software to provide help with that state) ,  first we need to make the file that contains all topics, which are given exclusive IDs. …
When the s#!t hits the fan, you don’t have time to look up who’s on call, draft emails, call collaborators, or send text messages. An instant chat window is definitely the way to go, especially one like HipChat. HipChat is a true business app. An…
Explain concepts important to validation of email addresses with regular expressions. Applies to most languages/tools that uses regular expressions. Consider email address RFCs: Look at HTML5 form input element (with type=email) regex pattern: T…
This tutorial will teach you the core code needed to finalize the addition of a watermark to your image. The viewer will use a small PHP class to learn and create a watermark.
Suggested Courses

718 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question