eriklee
asked on
CFHTTP - password protected page
i have a password protected page i needed to scape data..
How can i use CFHTTP with chttpparam to get passed to login page and then onto the page with information i can scrape.
Doing CFHTTP with the correct userid/password is no problem --but is it POSSIBLE to continue afterthereon to a page which is authenticated based on a session variable which is set when a webuser is authenticated from normal browser access.
Is that at all possible?
How can i use CFHTTP with chttpparam to get passed to login page and then onto the page with information i can scrape.
Doing CFHTTP with the correct userid/password is no problem --but is it POSSIBLE to continue afterthereon to a page which is authenticated based on a session variable which is set when a webuser is authenticated from normal browser access.
Is that at all possible?
ASKER
it won't work. the page is protected with a test on a session.variable value. you can only CFHTTP to the authentication page with the above template. . .
I played around with cfhttp for a long time trying to get it to work on a screen scaper prog for an ssl'ed & pwd protected site and never did get something useable. I suppose a better coder might have been able.
Your best bet is to use CFX_HTTP5 available for trial at http://www.adiabata.com/Docs/Products.cfm it's 45 bucks but it is fantastic for screen scraping and maintain sessions.
With this custom tag and cf_reExtract http://www.contentbox.com/claude/customtags/REextract/testREextract.cfm I built a fully functional app which logs into a secured site, fills out something like 140 form fields and parses out the response info for storage in the local db. ...in about 2 weeks.
Your best bet is to use CFX_HTTP5 available for trial at http://www.adiabata.com/Docs/Products.cfm it's 45 bucks but it is fantastic for screen scraping and maintain sessions.
With this custom tag and cf_reExtract http://www.contentbox.com/claude/customtags/REextract/testREextract.cfm I built a fully functional app which logs into a secured site, fills out something like 140 form fields and parses out the response info for storage in the local db. ...in about 2 weeks.
I would imagine... Since a session variable is created by the foreign server...
If you can figure out where in the process the session variable is being set, you might be able to cfhttp to that page first, obtain the session setting for your connection and then cfhttp to the authentication page using the format mentioned by js_vaughan above.
This is an untested 'guess'... hope it helps.
William
If you can figure out where in the process the session variable is being set, you might be able to cfhttp to that page first, obtain the session setting for your connection and then cfhttp to the authentication page using the format mentioned by js_vaughan above.
This is an untested 'guess'... hope it helps.
William
ASKER
sidfishes
i assume you have used cfhttp5 extensively...
i ve downloaded the trial version .. i experimented with some calls to localhost sites (some external sites) but i kept getting the error:
Error: 12006. Message: Could not connect to the server. [SYSTEM] Loc: 1. The URL does not use a recognized protocol
//////////////////////////
<cfset crlf = chr(13) & chr(10)>
<CFSET HEADERS="Content-Type: application/x-www-form-url
<CFSET BODY="proclogin=enter&user
<CFSET URL="http://localhost/bbs2/admin/index.cfm">
<CFX_HTTP5 METHOD="GET" URL=#URL# HEADERS=#HEADERS# BODY=#BODY# OUT="RESult">
<CFIF STATUS NEQ "OK">
<CFOUTPUT>Error: #ERRN#. Message: #MSG#</CFOUTPUT>
<CFABORT>
</CFIF>
i assume you have used cfhttp5 extensively...
i ve downloaded the trial version .. i experimented with some calls to localhost sites (some external sites) but i kept getting the error:
Error: 12006. Message: Could not connect to the server. [SYSTEM] Loc: 1. The URL does not use a recognized protocol
//////////////////////////
<cfset crlf = chr(13) & chr(10)>
<CFSET HEADERS="Content-Type: application/x-www-form-url
<CFSET BODY="proclogin=enter&user
<CFSET URL="http://localhost/bbs2/admin/index.cfm">
<CFX_HTTP5 METHOD="GET" URL=#URL# HEADERS=#HEADERS# BODY=#BODY# OUT="RESult">
<CFIF STATUS NEQ "OK">
<CFOUTPUT>Error: #ERRN#. Message: #MSG#</CFOUTPUT>
<CFABORT>
</CFIF>
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
<cfhttp url="http://www.website.com" method="post" resolveurl="Yes" throwOnError="Yes">
<cfhttpparam name="username" value="theuser" type="formfield">
<cfhttpparam name="userpassword" value="thepass" type="formfield">
</cfhttp>
<cfoutput>#CFHTTP.filecont