Solved

CFHTTP not returning all content from certain websites

Posted on 2003-11-03
5
320 Views
Last Modified: 2013-12-24
I'm trying to get the source for the search results on dice.com using CFHTTP using the following code:

---------------------------------------
<cfhttp method = "GET" url = "http://seeker.dice.com/jobsearch/index.jsp" resolveurl="yes">
</cfhttp>

<CFOUTPUT>
content = #cfhttp.filecontent# <BR>
mimetype = #cfhttp.mimeType# <BR>
header = #cfhttp.header# <BR>
</cfoutput>
---------------------------------------

but it's only returning partial content. I think this is because the server is using "transfer encoding: chunked" so only 1 chunk is being sent to me.

Is there a way to get all of the content?

Your prompt response is greatly appreciated.

Johnny G.
0
Comment
Question by:javanigus
  • 3
  • 2
5 Comments
 
LVL 4

Expert Comment

by:procept
ID: 9669659
Hi,

I just tested your code and it returned a complete page, up to the closing </html> tag. SO maybe you just had a bad connection when you tried it. ;-))

You can add a timeout value to the <cfhttp> tag:

<cfhttp ...... timeout="timeout_in_seconds">

HTH,

Chris
0
 

Author Comment

by:javanigus
ID: 9669821
Oops, I gave you the wrong code snippet. The code that doesn't give the complete page is

--------------------------------------
<cfhttp method = "POST" url = "http://seeker.dice.com/jobsearch/servlet/JobSearch" resolveurl="yes">
      <cfhttpparam name = "PREV_DOC" type = "FormField" value = "0">
      <cfhttpparam name = "NEXT_DOC" type = "FormField" value = "31">
      <cfhttpparam name = "op" type = "FormField" value = "1001">
      <cfhttpparam name = "FILTER_TEXT" type = "FormField" value = "">
</cfhttp>

<CFOUTPUT>
content = #cfhttp.filecontent# <BR>
mimetype = #cfhttp.mimeType# <BR>
header = #cfhttp.header# <BR>
</cfoutput>

-------------------------------------------

If you go to

http://askabboudi.com/misc/httppost_dice.cfm

you'll see the code in action not showing the whole page.

Any ideas?

Thanks,

Johnny G.
0
 
LVL 4

Accepted Solution

by:
procept earned 250 total points
ID: 9669903
Hi Johnny,

I took your code, played around with it, looked at the dice-site, played some more, and found... you need to change the value of the paramter "op" to "1000" and it will work. Don't know why, but, it works. :-))

HTH,

Chris
0
 

Author Comment

by:javanigus
ID: 9669913
Dude, thanks alot.

Johnny G.
0
 
LVL 4

Expert Comment

by:procept
ID: 9669971
You're welcome. :-))

Chris
0

Featured Post

Complete VMware vSphere® ESX(i) & Hyper-V Backup

Capture your entire system, including the host, with patented disk imaging integrated with VMware VADP / Microsoft VSS and RCT. RTOs is as low as 15 seconds with Acronis Active Restore™. You can enjoy unlimited P2V/V2V migrations from any source (even from a different hypervisor)

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Article by: kevp75
Hey folks, 'bout time for me to come around with a little tip. Thanks to IIS 7.5 Extensions and Microsoft (well... really Windows 8, and IIS 8 I guess...), we can now prime our Application Pools, when IIS starts. Now, though it would be nice t…
Periodically we have to update or add SSL certificates for customers. Depending upon your hosting plan you may be responsible for the installation and/or key generation. In the wake of Heartbleed many sites were forced to re-key. We will concen…
Internet Business Fax to Email Made Easy - With  eFax Corporate (http://www.enterprise.efax.com), you'll receive a dedicated online fax number, which is used the same way as a typical analog fax number. You'll receive secure faxes in your email, f…
With the power of JIRA, there's an unlimited number of ways you can customize it, use it and benefit from it. With that in mind, there's bound to be things that I wasn't able to cover in this course. With this summary we'll look at some places to go…

863 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

26 Experts available now in Live!

Get 1:1 Help Now