Solved

How do you combine cURL and Regex in PHP

Posted on 2011-02-24
7
589 Views
Last Modified: 2012-05-11
Experts,

How do I corectly structure this cURL and Regex to pull some data from a website then insert it in my database.

Thanks for your help!
<?php


mysql_connect("Host","UN","PW") or die("Unable to connect to SQL server");
mysql_select_db('DB') or die("Unable to SELECT DB");

echo "Connected to DB";
echo "<BR>";

$query = "SELECT * FROM temptable";
$result = mysql_query($query) or die('Error: '.mysql_error());
while($row = mysql_fetch_array($result))
  {
$sku=mysql_real_escape_string($row['sku']);
echo $sku."<br />";


  
    
	$pages = array('home' => 
'http://www.mysite.com/', 
               'login' => 
'https://www.mysite.com/signin.asp', 
               'schedule' => 
'http://www.mysite.com/product.asp?&sku=' . $sku . ''); 
    $ch = curl_init(); 
    //Set options for curl session 
    $options = array(CURLOPT_USERAGENT => 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.1) Gecko/20061204 Firefox/2.0.0.1)', 
             CURLOPT_SSL_VERIFYPEER => FALSE, 
             CURLOPT_SSL_VERIFYHOST => 2, 
             CURLOPT_HEADER => TRUE, 
             //CURLOPT_RETURNTRANSFER => TRUE, 
             CURLOPT_COOKIEFILE => '/cookie.txt', 
             CURLOPT_COOKIEJAR => '/cookies.txt'); 
    
    //Hit home page for session cookie 
    $options[CURLOPT_URL] = $pages['home']; 
    curl_setopt_array($ch, $options); 
    curl_exec($ch); 
    
    //Login 
    $options[CURLOPT_URL] = $pages['login']; 
    $options[CURLOPT_POST] = TRUE; 
    $options[CURLOPT_POSTFIELDS] = 'email=MyEmail&password=PW&submitBtn'; 
    $options[CURLOPT_FOLLOWLOCATION] = FALSE; 
    curl_setopt_array($ch, $options); 
    curl_exec($ch); 
    
    //Hit schedule page 
    $options[CURLOPT_URL] = $pages['schedule']; 
    curl_setopt_array($ch, $options); 
    $schedule = curl_exec($ch); 
    
    //Output schedule 
    //echo $schedule;
	
	preg_match_all('%<title>(.*?)</title>%s',$schedule,$matches0);

$title=$matches0[1];
echo $title."<br />";


mysql_query("INSERT INTO temptable2 (title) VALUES('".mysql_real_escape_string($title)."') ");



	}	 
    
    //Close curl session 
    curl_close($ch);
	 
?>

Open in new window

0
Comment
Question by:rlb1
  • 4
  • 3
7 Comments
 
LVL 34

Expert Comment

by:Beverley Portlock
ID: 34977718
It looks OK on inspection. What is wrong with it?
0
 

Author Comment

by:rlb1
ID: 34980628
The cURL functions are working properly and I am getting data returned from all $pages (home, login, schedule).   The REGEX is not picking up the 'Title' which is the same code that is working perfectly on another script.  The script is not inserting the 'Title' to the database.  

I am not sure what is wrong.  I have spent several hours working on it...  



0
 
LVL 34

Expert Comment

by:Beverley Portlock
ID: 34981084
Change

$title=$matches0[1];

to

$title = $matches0[1][0];
0
Efficient way to get backups off site to Azure

This user guide provides instructions on how to deploy and configure both a StoneFly Scale Out NAS Enterprise Cloud Drive virtual machine and Veeam Cloud Connect in the Microsoft Azure Cloud.

 

Author Comment

by:rlb1
ID: 34983657
bportlock: Thanks.  It seems that the REGEX statement may not be recognizing   $schedule (which is the 3rd page in the array).  The database is receiving some blank inserts with no data.    I have adjusted several things in the script and cannot come up with a solution.   The login is working and the pages are appearing correctly.  Also, echo $title."<br />"; is not returning any data either.

preg_match('%<title>(.*?)</title>%',$schedule,$matches0);

$title=$matches0[1];
echo $title."<br />";

Thanks for your help!!
0
 

Author Comment

by:rlb1
ID: 34984633
8 (continuous working on it) Hours Later...  I started completely over, rewrote the script and got it to work!!  


<?php

$ch = curl_init();
curl_setopt($ch, CURLOPT_COOKIEJAR, "..cookie.txt");
curl_setopt($ch, CURLOPT_URL,"https://www.mywebsite.com/signin.asp?");
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, "email=myemail&password=mypassword&submitBtn");

ob_start();      // prevent any output
curl_exec ($ch); // execute the curl command
ob_end_clean();  // stop preventing output

curl_close ($ch);
unset($ch);

$ch = curl_init();
curl_setopt($ch, CURLOPT_RETURNTRANSFER,1);
curl_setopt($ch, CURLOPT_COOKIEFILE, "../cookie.txt");
curl_setopt($ch, CURLOPT_URL,"http://www.mywebsite.com/product.asp?sku=03170");

$buf2 = curl_exec ($ch);

curl_close ($ch);

//echo "<PRE>".htmlentities($buf2);

//echo $buf2;


preg_match('%<title>(.*?)<\/title>%',$buf2,$matches0);

$title=$matches0[1];
echo $title."<br />";







?>

Open in new window

0
 
LVL 34

Accepted Solution

by:
Beverley Portlock earned 500 total points
ID: 34986961
:-)

0
 

Author Closing Comment

by:rlb1
ID: 35021720
Thanks!
0

Featured Post

Migrating Your Company's PCs

To keep pace with competitors, businesses must keep employees productive, and that means providing them with the latest technology. This document provides the tips and tricks you need to help you migrate an outdated PC fleet to new desktops, laptops, and tablets.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Nothing in an HTTP request can be trusted, including HTTP headers and form data.  A form token is a tool that can be used to guard against request forgeries (CSRF).  This article shows an improved approach to form tokens, making it more difficult to…
Introduction This article is intended for those who are new to PHP error handling (https://www.experts-exchange.com/articles/11769/And-by-the-way-I-am-New-to-PHP.html).  It addresses one of the most common problems that plague beginning PHP develop…
The viewer will learn how to create and use a small PHP class to apply a watermark to an image. This video shows the viewer the setup for the PHP watermark as well as important coding language. Continue to Part 2 to learn the core code used in creat…
This tutorial will teach you the core code needed to finalize the addition of a watermark to your image. The viewer will use a small PHP class to learn and create a watermark.

831 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question