[2 days left] What’s wrong with your cloud strategy? Learn why multicloud solutions matter with Nimble Storage.Register Now

x
?
Solved

CFHTTP not returning all content from certain websites

Posted on 2003-11-03
5
Medium Priority
?
328 Views
Last Modified: 2013-12-24
I'm trying to get the source for the search results on dice.com using CFHTTP using the following code:

---------------------------------------
<cfhttp method = "GET" url = "http://seeker.dice.com/jobsearch/index.jsp" resolveurl="yes">
</cfhttp>

<CFOUTPUT>
content = #cfhttp.filecontent# <BR>
mimetype = #cfhttp.mimeType# <BR>
header = #cfhttp.header# <BR>
</cfoutput>
---------------------------------------

but it's only returning partial content. I think this is because the server is using "transfer encoding: chunked" so only 1 chunk is being sent to me.

Is there a way to get all of the content?

Your prompt response is greatly appreciated.

Johnny G.
0
Comment
Question by:javanigus
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 3
  • 2
5 Comments
 
LVL 4

Expert Comment

by:procept
ID: 9669659
Hi,

I just tested your code and it returned a complete page, up to the closing </html> tag. SO maybe you just had a bad connection when you tried it. ;-))

You can add a timeout value to the <cfhttp> tag:

<cfhttp ...... timeout="timeout_in_seconds">

HTH,

Chris
0
 

Author Comment

by:javanigus
ID: 9669821
Oops, I gave you the wrong code snippet. The code that doesn't give the complete page is

--------------------------------------
<cfhttp method = "POST" url = "http://seeker.dice.com/jobsearch/servlet/JobSearch" resolveurl="yes">
      <cfhttpparam name = "PREV_DOC" type = "FormField" value = "0">
      <cfhttpparam name = "NEXT_DOC" type = "FormField" value = "31">
      <cfhttpparam name = "op" type = "FormField" value = "1001">
      <cfhttpparam name = "FILTER_TEXT" type = "FormField" value = "">
</cfhttp>

<CFOUTPUT>
content = #cfhttp.filecontent# <BR>
mimetype = #cfhttp.mimeType# <BR>
header = #cfhttp.header# <BR>
</cfoutput>

-------------------------------------------

If you go to

http://askabboudi.com/misc/httppost_dice.cfm

you'll see the code in action not showing the whole page.

Any ideas?

Thanks,

Johnny G.
0
 
LVL 4

Accepted Solution

by:
procept earned 1000 total points
ID: 9669903
Hi Johnny,

I took your code, played around with it, looked at the dice-site, played some more, and found... you need to change the value of the paramter "op" to "1000" and it will work. Don't know why, but, it works. :-))

HTH,

Chris
0
 

Author Comment

by:javanigus
ID: 9669913
Dude, thanks alot.

Johnny G.
0
 
LVL 4

Expert Comment

by:procept
ID: 9669971
You're welcome. :-))

Chris
0

Featured Post

Free Tool: Path Explorer

An intuitive utility to help find the CSS path to UI elements on a webpage. These paths are used frequently in a variety of front-end development and QA automation tasks.

One of a set of tools we're offering as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

One of the typical problems I have experienced is when you have to move a web server from one hosting site to another. You normally prepare all on the new host, transfer the site, change DNS and cross your fingers hoping all will be ok on new server…
When it comes to security, close monitoring is a must. According to WhiteHat Security annual report, a substantial number of all web applications are vulnerable always. Monitis offers a new product - fully-featured Website security monitoring and pr…
In this video, Percona Solutions Engineer Barrett Chambers discusses some of the basic syntax differences between MySQL and MongoDB. To learn more check out our webinar on MongoDB administration for MySQL DBA: https://www.percona.com/resources/we…
Want to learn how to record your desktop screen without having to use an outside camera. Click on this video and learn how to use the cool google extension called "Screencastify"! Step 1: Open a new google tab Step 2: Go to the left hand upper corn…

649 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question