• Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 492
  • Last Modified:

using filename url method

Hi,
 
I sometime use sas to query web sites multiple times by running a macro that changes the values of the input parameters and then save the results to a sas dataset where I can parse out the information I need. The problem is that I have trouble with the return webpage getting truncated either by width and/or length.
When I run the code below, I get a 282 records, lots of which are empty and not the complete webpage. How can set record length so that I get all the data and preferably on one line?

I would be willing to get a perl or python answer, but the respondant is going to have to spoon feed me.

Thanks,

Bruce


filename foo url
"http://maps.google.com/maps?q=45.3906+-75.6881&hl=en&sll=37.0625,-95.677068&sspn=34.313287,66.533203&vpsrc=0&t=m&z=16&output=html"
lrecl=5000;


data a ;
infile foo length=len;
   input record $varying5000. len;
 run;

0
Diaphanosoma
Asked:
Diaphanosoma
  • 3
  • 3
2 Solutions
 
Dave BaldwinFixer of ProblemsCommented:
That particular page contains a lot of javascript, not data and not HTML.  The actual data for the map is retrieved by AJAX from the server after the page is loaded in a browser.  What are you expecting to get?
0
 
DiaphanosomaAuthor Commented:
The 'output=html' parameter is supposed to output html. When I open the web page with view source, I see the street name and address name that I am hoping to parse out. However, that "text" doesn't appear in the dataset downloaded by sas.
0
 
Dave BaldwinFixer of ProblemsCommented:
Then you need to look at how SAS is creating the data sets.  There are very few line breaks or returns in the text for that page.  If '5000' is the length of an individual record, it might be too small.  Since you can compare the View Source to the data you have, maybe you can find out where it is being left off.
0
What does it mean to be "Always On"?

Is your cloud always on? With an Always On cloud you won't have to worry about downtime for maintenance or software application code updates, ensuring that your bottom line isn't affected.

 
Dave BaldwinFixer of ProblemsCommented:
Unless SAS is emulating a browser and accepts cookies, you might not be seeing exactly the same thing as your View Source in a browser.
0
 
d507201Commented:
I've never worked with the URL engine, but I'd set the length to 32000.  The end of line at 4096 might not be there when you use SAS to read the file.
0
 
DiaphanosomaAuthor Commented:
I've been playing around with the varying length. The max is 32767 which manages to catch the text I'm interested in. Not sure what one would do if the text was past that number.

I'll keep the question open till late Monday, in case someone has something else to add.

Bruce
0
 
DiaphanosomaAuthor Commented:
Thanks for the help. I'll be able to get it going now.
0

Featured Post

Free Tool: SSL Checker

Scans your site and returns information about your SSL implementation and certificate. Helpful for debugging and validating your SSL configuration.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

  • 3
  • 3
Tackle projects and never again get stuck behind a technical roadblock.
Join Now