Solved

How to write a Batch script to go to a webpage and save that page in .csv

Posted on 2010-11-10
17
103 Views
Last Modified: 2014-11-24
Hi
I am looking for script  which can save  a webpage in .csv.
This  particular page has text content.
What now i do is go to that intranet site   click save as name.csv...and convert it into a text file then send that to ETL team.
Is there a way i can automate this process untill converting it into txt file using a batch script?
any help is appreciated.
0
Comment
Question by:sakytech
  • 6
  • 5
  • 4
17 Comments
 
LVL 33

Expert Comment

by:knightEknight
ID: 34107098
google and download a tool called  httpget.exe

then in your script you can do this:

httpget.exe  "http://server.com/webpage.ext"  >  result.csv
0
 

Author Comment

by:sakytech
ID: 34107192
Thanx for reply Knight,
But i can't download 3rd part tool on sys,
i tried wget.. but that doesn't work,it errors out.    I tried this ..[wget  -l 1 c:\test.csv http:// <URL>]... this does prety what i want to ..but it won't work for
my intranet url, it throws error 404. any insights?

any  suggestions??

Thanks,
0
 
LVL 33

Expert Comment

by:knightEknight
ID: 34107202
hmm, wget should work just as well.  Try putting quotes around the URL ... other than that, I can't offer much more, sorry.
0
 

Author Comment

by:sakytech
ID: 34107253
@knight ..well i tried that..it won't work, batch script does nothing. so i tried from command prompt
which gives the error. does it have anything to do with webpage, this site is accesable  only within company's network.

anyway thanks for trying ,
0
 
LVL 52

Expert Comment

by:Bill Prew
ID: 34108641
Are you running this command on a computer inside the company, or outside the company?

~bp
0
 

Author Comment

by:sakytech
ID: 34108999
Thnx bill for replyin

inside the company..just trying to automate the process
0
 
LVL 52

Expert Comment

by:Bill Prew
ID: 34111482
Do you have any special characters in the URL string you are trying to invoke, like >, <, &, ^, or | perhaps.  Those can cause problems since they are special characters to the command shell.

~bp
0
3 Use Cases for Connected Systems

Our Dev teams are like yours. They’re continually cranking out code for new features/bugs fixes, testing, deploying, testing some more, responding to production monitoring events and more. It’s complex. So, we thought you’d like to see what’s working for us.

 
LVL 33

Expert Comment

by:knightEknight
ID: 34111580
I was thinking the same thing Bill ... I thought maybe quoting the url would resolve that.
0
 
LVL 52

Expert Comment

by:Bill Prew
ID: 34111630
Agreed knight, often quoting works in general, I haven't done a lot with wget so wasn't sure.  And without seeing the specific URL it's hard to propose a lot of thoughts on what might be causing a problem.

sakytech is it possible for you to share the full command line exactly as you are executing it?  Since the site is internal to your network I don't think sharing the URL would be a security risk, but that's best for you to decide.

Also, if you just do the following, does that open the page in your default browser?

START "http:\\yoururl.html"

~bp
0
 

Author Comment

by:sakytech
ID: 34114539
thanks for the response,
this is the link looks like this http://icwebprod.wv.xyz.com/scripts/saky/

yea it works just fine when  i do start "http://icwebprod.wv.xyz.com/scripts/saky ".
as you mentioned placing the quotes worked in command prompt, but not in  batch script
[@echo off
cls
wget  "http://icwebprod.wv.xyz.com/scripts/saky" ]  am i missing something??
and also..conversion to .csv and to .txt! i have no clue how to do that ..in script!
any one know how to?


Thanx, bill.knight





yeaa bill,  when i  do start "http://icwebprod.wv.xyz.com/scripts/saky/
0
 
LVL 33

Expert Comment

by:knightEknight
ID: 34114835
wget / httpget will create the file in the source format ... so if you wget a .jpg file it will create a .jpg file, if you wget a text file it will create a text file (by any name or extension you give it, e.g.  .txt or .csv or whatever).   In other words, wget will not "convert" from one format to another for you -- it all depends on what is returned from the specified URL.
0
 
LVL 52

Expert Comment

by:Bill Prew
ID: 34116263
Have you tried adding the -d option to the command line, which should provide some additional debugging info from wget.

~bp
0
 
LVL 52

Accepted Solution

by:
Bill Prew earned 500 total points
ID: 34116329
Have you tried adding the -d option to the command line, which should provide some additional debugging info from wget.

I also just noticed one other thing.  You stated that this was your command line:

wget  -l 1 c:\test.csv http://icwebprod.wv.xyz.com/scripts/saky/

but that isn't correct syntax.  It needs to be:

wget  -l 1 -O c:\test.csv http://icwebprod.wv.xyz.com/scripts/saky/

I tried this from both the command line and in a BAT file and both worked fine here.

~bp
0
 

Author Comment

by:sakytech
ID: 34138720

Yea bill, sorry was caught up with different work.. it works just fine.. i missed that syntax..!!

Thanks for the help! It is solved!
0
 
LVL 52

Expert Comment

by:Bill Prew
ID: 34539938
==> sakytech

If this is solved can you accept and close?

~bp
0

Featured Post

DevOps Toolchain Recommendations

Read this Gartner Research Note and discover how your IT organization can automate and optimize DevOps processes using a toolchain architecture.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

You may have already been in the need to update a whole folder stucture using a script. Robocopy does it well and even provides a list of non-updated files in a log (if asked to). Generally those files that were locked by a user or a process by the …
Use this article to create a batch file to backup a Microsoft SQL Server database to a Windows folder.  The folder can be on the local hard drive or on a network share.  This batch file will query the SQL server to get the current date & time and wi…
In this video I am going to show you how to back up and restore Office 365 mailboxes using CodeTwo Backup for Office 365. Learn more about the tool used in this video here: http://www.codetwo.com/backup-for-office-365/ (http://www.codetwo.com/ba…
Learn how to create flexible layouts using relative units in CSS.  New relative units added in CSS3 include vw(viewports width), vh(viewports height), vmin(minimum of viewports height and width), and vmax (maximum of viewports height and width).

867 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

17 Experts available now in Live!

Get 1:1 Help Now