Solved

CFHTTP - password protected page

Posted on 2006-11-16
6
697 Views
Last Modified: 2013-12-24
i have a password protected page i needed to scape data..

How can i use CFHTTP with chttpparam to get passed to login page and then onto the page with information i can scrape.
Doing CFHTTP with the correct userid/password is no problem --but is it POSSIBLE to continue afterthereon to a page which is authenticated based on a session variable which is set when a webuser is authenticated from normal browser access.

Is that at all possible?
0
Comment
Question by:eriklee
6 Comments
 
LVL 10

Expert Comment

by:js_vaughan
ID: 17963403
You would need to enter the URL of the action page, not the login page itself.

<cfhttp url="http://www.website.com" method="post" resolveurl="Yes" throwOnError="Yes">
  <cfhttpparam name="username" value="theuser" type="formfield">
  <cfhttpparam name="userpassword" value="thepass" type="formfield">
</cfhttp>

<cfoutput>#CFHTTP.filecontent#</cfoutput>
0
 

Author Comment

by:eriklee
ID: 17965238
it won't work. the page is protected with a test on a session.variable value. you can only CFHTTP to the authentication page with the above template. . .
0
 
LVL 36

Expert Comment

by:SidFishes
ID: 17965391
I played around with cfhttp for a long time trying to get it to work on a screen scaper prog for an ssl'ed & pwd protected site and never did get something useable. I suppose a better coder might have been able.

Your best bet is to use CFX_HTTP5 available for trial at http://www.adiabata.com/Docs/Products.cfm it's 45 bucks but it is fantastic for screen scraping and maintain sessions.

With this custom tag and cf_reExtract http://www.contentbox.com/claude/customtags/REextract/testREextract.cfm I built a fully functional app which logs into a secured site, fills out something like 140 form fields  and parses out the response info for storage in the local db. ...in about 2 weeks.
0
How your wiki can always stay up-to-date

Quip doubles as a “living” wiki and a project management tool that evolves with your organization. As you finish projects in Quip, the work remains, easily accessible to all team members, new and old.
- Increase transparency
- Onboard new hires faster
- Access from mobile/offline

 
LVL 1

Expert Comment

by:coderWil
ID: 17967045
I would imagine...     Since a session variable is created by the foreign server...

If you can figure out where in the process the session variable is being set, you might be able to cfhttp to that page first, obtain the session setting for your connection and then cfhttp to the authentication page using the format mentioned by js_vaughan above.  

This is an untested 'guess'...  hope it helps.

William
0
 

Author Comment

by:eriklee
ID: 17970620
sidfishes

i assume you have used cfhttp5 extensively...

i ve downloaded the trial version .. i experimented with some calls to localhost sites (some external sites) but i kept getting the error:
Error: 12006. Message: Could not connect to the server. [SYSTEM] Loc: 1. The URL does not use a recognized protocol

//////////////////////////////////// sample code
<cfset crlf = chr(13) & chr(10)>
<CFSET HEADERS="Content-Type: application/x-www-form-urlencoded#CRLF#">
<CFSET BODY="proclogin=enter&userid=erlee&password=charlie123&goto=list.cfm">
<CFSET URL="http://localhost/bbs2/admin/index.cfm">

<CFX_HTTP5 METHOD="GET" URL=#URL# HEADERS=#HEADERS# BODY=#BODY# OUT="RESult">
<CFIF STATUS NEQ "OK">
   <CFOUTPUT>Error: #ERRN#. Message: #MSG#</CFOUTPUT>
   <CFABORT>
</CFIF>

0
 
LVL 36

Accepted Solution

by:
SidFishes earned 125 total points
ID: 17982597
not seen any issues like that...But i've never tried to use it on localhost...

Can you use it to return something from a public site?

 <cfx_HTTP5 URL="http://www.google.com"
                OUT="oResult"
                  METHOD="get"
                  TIMEOUT="200000">

<cfoutput>#oresult#</cfoutput>

might be you need to set the method to get or post?
0

Featured Post

Complete Microsoft Windows PC® & Mac Backup

Backup and recovery solutions to protect all your PCs & Mac– on-premises or in remote locations. Acronis backs up entire PC or Mac with patented reliable disk imaging technology and you will be able to restore workstations to a new, dissimilar hardware in minutes.

Join & Write a Comment

When it comes to showing a 404 error page to your visitors, you do not want that generic page to show, and you especially do not want your hosting provider’s ad error page to show either. In this article, I will show you how to enable the custom 40…
Meet the world's only “Transparent Cloud™” from Superb Internet Corporation. Now, you can experience firsthand a cloud platform that consistently outperforms Amazon Web Services (AWS), IBM’s Softlayer, and Microsoft’s Azure when it comes to CPU and …
This demo shows you how to set up the containerized NetScaler CPX with NetScaler Management and Analytics System in a non-routable Mesos/Marathon environment for use with Micro-Services applications.
You have products, that come in variants and want to set different prices for them? Watch this micro tutorial that describes how to configure prices for Magento super attributes. Assigning simple products to configurable: We assigned simple products…

709 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

13 Experts available now in Live!

Get 1:1 Help Now