Solved

using filename url method

Posted on 2011-09-09
7
448 Views
Last Modified: 2013-11-16
Hi,
 
I sometime use sas to query web sites multiple times by running a macro that changes the values of the input parameters and then save the results to a sas dataset where I can parse out the information I need. The problem is that I have trouble with the return webpage getting truncated either by width and/or length.
When I run the code below, I get a 282 records, lots of which are empty and not the complete webpage. How can set record length so that I get all the data and preferably on one line?

I would be willing to get a perl or python answer, but the respondant is going to have to spoon feed me.

Thanks,

Bruce


filename foo url
"http://maps.google.com/maps?q=45.3906+-75.6881&hl=en&sll=37.0625,-95.677068&sspn=34.313287,66.533203&vpsrc=0&t=m&z=16&output=html"
lrecl=5000;


data a ;
infile foo length=len;
   input record $varying5000. len;
 run;

0
Comment
Question by:Diaphanosoma
  • 3
  • 3
7 Comments
 
LVL 82

Expert Comment

by:Dave Baldwin
Comment Utility
That particular page contains a lot of javascript, not data and not HTML.  The actual data for the map is retrieved by AJAX from the server after the page is loaded in a browser.  What are you expecting to get?
0
 
LVL 1

Author Comment

by:Diaphanosoma
Comment Utility
The 'output=html' parameter is supposed to output html. When I open the web page with view source, I see the street name and address name that I am hoping to parse out. However, that "text" doesn't appear in the dataset downloaded by sas.
0
 
LVL 82

Expert Comment

by:Dave Baldwin
Comment Utility
Then you need to look at how SAS is creating the data sets.  There are very few line breaks or returns in the text for that page.  If '5000' is the length of an individual record, it might be too small.  Since you can compare the View Source to the data you have, maybe you can find out where it is being left off.
0
What Is Threat Intelligence?

Threat intelligence is often discussed, but rarely understood. Starting with a precise definition, along with clear business goals, is essential.

 
LVL 82

Accepted Solution

by:
Dave Baldwin earned 250 total points
Comment Utility
Unless SAS is emulating a browser and accepts cookies, you might not be seeing exactly the same thing as your View Source in a browser.
0
 
LVL 7

Assisted Solution

by:d507201
d507201 earned 250 total points
Comment Utility
I've never worked with the URL engine, but I'd set the length to 32000.  The end of line at 4096 might not be there when you use SAS to read the file.
0
 
LVL 1

Author Comment

by:Diaphanosoma
Comment Utility
I've been playing around with the varying length. The max is 32767 which manages to catch the text I'm interested in. Not sure what one would do if the text was past that number.

I'll keep the question open till late Monday, in case someone has something else to add.

Bruce
0
 
LVL 1

Author Closing Comment

by:Diaphanosoma
Comment Utility
Thanks for the help. I'll be able to get it going now.
0

Featured Post

IT, Stop Being Called Into Every Meeting

Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

Join & Write a Comment

Introduction: Often, when running a query with joins, the results show up "duplicates", and often, those duplicates can be "eliminated" in the results using DISTINCT, for example. Using DISTINCT is simple: just add it after the SELECT keyword, an…
I annotated my article on ransomware somewhat extensively, but I keep adding new references and wanted to put a link to the reference library.  Despite all the reference tools I have on hand, it was not easy to find a way to do this easily. I finall…
Video by: Steve
Using examples as well as descriptions, step through each of the common simple join types, explaining differences in syntax, differences in expected outputs and showing how the queries run along with the actual outputs based upon a simple set of dem…
Polish reports in Access so they look terrific. Take yourself to another level. Equations, Back Color, Alternate Back Color. Write easy VBA Code. Tighten space to use less pages. Launch report from a menu, considering criteria only when it is filled…

744 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

14 Experts available now in Live!

Get 1:1 Help Now