• Status: Solved
  • Priority: Low
  • Security: Public
  • Views: 195
  • Last Modified:

wp_remote_request/cURL response timeout unless ran from command line

Problem: wp_remote_get/wp_remote_request of a specific https target fails to return a response unless ran via WP_CLI function.

$wp_remote_args => array(
    'timeout' => 5,
    'sslverify' => true,
    'blocking' => true,
    'user-agent' => 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:57.0) Gecko/20100101 Firefox/57.0',
    'headers' => array(
        'Accept' => 'application/json, text/plain, */*',
        'Accept-Encoding' => 'gzip, deflate, br',
        'Accept-Language' => 'en-US,en;q=0.5',
        'Authorization' => '...',
        'Connection' => 'keep-alive',
        'Content-Type' => 'application/json; charset=UTF-8; text/plain',
        'Host' => '...',
        'Origin' => '...',
        'Referer' => '...',
    ),
)
wp_remote_request($url, $wp_remote_args)

Open in new window


I can only assume that something is perceived differently from the remote server when the GET request is received from standard PHP processing vs WP_CLI. Other wp_remote_request/cURL URLs are sent and received without issue.
0
Tim Bunch
Asked:
Tim Bunch
  • 5
  • 4
1 Solution
 
gr8gonzoConsultantCommented:
When it fails, is there an error message? Or does it just timeout?
0
 
Tim BunchAuthor Commented:
It simply times out, no response from the server. I should say that increasing the timeout length has no effect on the request.
0
 
gr8gonzoConsultantCommented:
Does it actually take the full 5 seconds to run the attempt, or does the request end early? So for example, if you increase the timeout to 30, does it sit there for 30 seconds, or does it simply indicate there's no response after a few seconds?

I'd hesitate to say that it has anything to do with a timeout unless the remote server / request is REALLY slow. If it finishes the function in LESS time than the timeout, then it might be failing for some other reason (but appearing incorrectly as a timeout).

I notice you don't capture the result of wp_remote_request($url, $wp_remote_args). What's the output if you var_dump it:

var_dump(wp_remote_request($url, $wp_remote_args));

You -SHOULD- get some kind of response from wp_remote_request, whether it's successful data or an instance of the WP_Error class (for failures). The error object should be a little more explicit in the problem details.

My gut says it's either an SSL verification problem or an extra header that's throwing things off, or a DNS / URL issue.
0
Keep up with what's happening at Experts Exchange!

Sign up to receive Decoded, a new monthly digest with product updates, feature release info, continuing education opportunities, and more.

 
Tim BunchAuthor Commented:
It does take the full 5 seconds, or 30 seconds to make the attempt.

{"error":"No response from https://..."}

Open in new window


The odd thing here is that the function runs perfectly fine if initiated via command line. My thought was that it was headers or a ssl verification issue. I set ssl to not verify, and the headers are identical to the successful command line job.

The remote server must be differentiating between the two methods somehow to not return a response.
0
 
gr8gonzoConsultantCommented:
It's pretty unlikely that a server would timeout one request while allowing another, especially if they're coming from the same server and using SSL. It's not impossible, but very unlikely. Usually if a request is bad or malformed in some way, a server will simply reject it, but it will reject it using some HTTP error code or an error response of some kind, rather than timing out.

The timeout suggests that either:
1. The request isn't making it to the remote server.
2. The remote server is taking a really long time to process the request.
3. The response isn't making it BACK into your server.
4. The remote server has a firewall that is intercepting the request and not dropping it in a way that lets the client know.
5. There's some SSL protocol problems that aren't being immediately rejected.

I'd put my money on #1.

Now, WP uses cURL by default for HTTPS requests, so let's simplify this and just create a simple script that tries to replicate what you have on the command line. Can you provide the EXACT cURL command line you're using and redact nothing EXCEPT for the inner parts of the domain name and any password/key values? For example, if your request is:

curl https://www.innovationtechnologysomething.com/api/rest.php?key=2M4nyS3cret5&request=fetchpage

Then provide us with:

curl https://www.i[REDACTED]g.com/api/rest.php?key=2[REDACTED]5&request=fetchpage

Notice the first and last character of the redacted values are still there, but the request isn't exposed in any significant way.

Just to confirm, the WP instance is running from the exact same server where you're doing the command line cURL, correct? And you're not changing the URL for the command line (e.g. a test remote server vs. production remote server) ?

I'd also confirm versions. On the command line, do:
curl -V

Then create a separate PHP script with just these 2 lines:
<?php
print_r(curl_version());

Run both and provide the output here.
0
 
Tim BunchAuthor Commented:
This all occurs on a single EC2 instance. The CLI command uses the same PHP function to initiate the cURL operation. Nothing is different.

When I remove the authorization header, I get a response back saying it's missing. So I know my request is making it out. When I add the Authorization back in, I get a plain and simple timeout (unless the function is ran from the CLI):

{"errors":{"http_request_failed":["cURL error 28: Operation timed out after 15001 milliseconds with 313 out of 15051 bytes received"]},"error_data":[]}

Open in new window


My cURL version:

curl 7.51.0 (x86_64-redhat-linux-gnu) libcurl/7.51.0 NSS/3.28.4 zlib/1.2.8 libidn2/0.16 libpsl/0.6.2 (+libicu/50.1.2) libssh2/1.4.2
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtsp scp sftp smb smbs smtp smtps telnet tftp 
Features: AsynchDNS IDN IPv6 Largefile GSS-API Kerberos SPNEGO NTLM NTLM_WB SSL libz UnixSockets PSL 

Open in new window


"version_number":471808,"age":3,"features":2000797,"ssl_version_number":0,"version":"7.51.0","host":"x86_64-redhat-linux-gnu","ssl_version":"NSS\/3.28.4","libz_version":"1.2.8","protocols":["dict","file","ftp","ftps","gopher","http","https","imap","imaps","ldap","ldaps","pop3","pop3s","rtsp","scp","sftp","smb","smbs","smtp","smtps","telnet","tftp"]

Open in new window


It's an unusual puzzle, this one. I normally crack these things fairly quickly.
0
 
gr8gonzoConsultantCommented:
The CLI command uses the same PHP function to initiate the cURL operation.
So when you're talking about CLI, you're not running the cURL binary directly? You're running a PHP script that calls the cURL functions? If not, let's take all the overhead of WP's libraries out of the picture. Can you create a basic PHP script that has nothing but the bare minimum cURL call along?

<?php
$ch = curl_init('https://...your URL...');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_USERPWD, 'USERNAME_HERE:PASSWORD_HERE'); // Populates the Authorization HTTP header
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);

// Just for debugging purposes
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false); 
curl_setopt($ch, CURLOPT_VERBOSE, true);

$result = curl_exec($ch);
var_dump($result);
var_dump(curl_error($result));
curl_close($ch);

Open in new window


If that still gives you the timeout, then can you elaborate on the service you're calling? For example, is it some service that has to do things like database lookups and return data and if so, are you giving it a lot of parameters?
0
 
Tim BunchAuthor Commented:
The CORS configuration on the remote server was configured in such a way that caused the Origin to not be recognized by the PHP script unless ran from the CLI. The solution here is not on the requesting server's side.
0
 
Tim BunchAuthor Commented:
Resolution discovered through further testing of my own.
0
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

Join & Write a Comment

Featured Post

Cloud Class® Course: Microsoft Office 2010

This course will introduce you to the interfaces and features of Microsoft Office 2010 Word, Excel, PowerPoint, Outlook, and Access. You will learn about the features that are shared between all products in the Office suite, as well as the new features that are product specific.

  • 5
  • 4
Tackle projects and never again get stuck behind a technical roadblock.
Join Now