Solved

How can I get dynamic content from another page to be displayed on my frontpage?

Posted on 2009-07-08
19
314 Views
Last Modified: 2013-11-18
Our company has weekly Intranews which is published on our external site and so currently someone has to manually copy and paste these news articles into a template and publish them to our Intranet.  I have to figure there would be a way to use a SSI script to pull the relevant data into the current template automatically every time a new article is published.  There is the main site which contains the last 5 published articles, and then a link which goes to the complete articles.  If there are any suggestions or ideas to follow I would greatly appreciate it!  Any code you may need just post and I will put it up for analyzing as needed.  Thanks.
0
Comment
Question by:PhillipsPlastics
  • 10
  • 9
19 Comments
 
LVL 4

Expert Comment

by:eejones
ID: 24804931
Can you use an iframe, and the source of the iframe would be the page that contains the 5 articles?
0
 

Author Comment

by:PhillipsPlastics
ID: 24804972
Currently the site is setup in a 3 column design and on the frontpage the articles rotate through every 15 min using a little php code.  I would like to continue to use this method since it allows for the article to be displayed on the whole middle column and makes the site look more dynamic per the CIO's request.
0
 
LVL 4

Expert Comment

by:eejones
ID: 24805165
Please forgive me if I have misunderstood the task. It would help to see the external website (can you post the url?)

Could the middle column on the intranet site contain an iframe? Are you trying to capture the whole frontpage in this intranet page middle column?
0
 

Author Comment

by:PhillipsPlastics
ID: 24805277
Unfortunately the external site is limited to company employees as it is password protected since it deals with sales etc...  I can post the relevant code after I clean it up a bit however if you would like me to.

In regards to the second question I would just want to glean 1 story at a time without the rest of the page being displayed.  Whether that means saving the content in another html document (as is done now) or merely rotating the call to get the current page doesn't matter so long as it works consistently.  

Every story ends with a "<!-- post footer links -->" before closing 2 div classes and beginning a new div class and adding another article on the external site.  I was thinking that a call to grab all content within that parameter and putting it into a different html page might be possible but due to my own inexperience I wasn't sure how to do so.
0
 
LVL 4

Expert Comment

by:eejones
ID: 24805387
I wonder if this will help - instructions on how to scrape from another website using PHP.

http://www.oooff.com/php-scripts/basic-php-scrape-tutorial/basic-php-scraping.php

Once you get the whole page content into a variable you can extract what you need and display it where you want.
0
 

Author Comment

by:PhillipsPlastics
ID: 24805482
This may be able to work... the only problem I run into is how to add authentication to the scrape request?
0
 
LVL 4

Expert Comment

by:eejones
ID: 24805638
Ack, that might be a problem. If you have to login to the external site to see the articles, I do not see how you can do this unless the login script uses method="get" in the <form> tag,  in which case you can pass the username and password in the url. The webmaster of the external site could easily tweak their code to accept the username:password either as "post" or as "get." Then you could use this for the url:

http://externalwebsite.com?username=chocolate&password=vanilla

assuming they are using the variable names 'username' and 'password'

0
 

Author Comment

by:PhillipsPlastics
ID: 24814410
I had to go ahead and use the curl extension using php...  however now that I have imported the data properly I need to pull each individual article into a separate htm file.  Any suggestions?
<?php

$userAgent = 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.8.1.6) Gecko/20070725 Firefox/3.5';

$target_url = "http://xxxxxxx.com";

$ch = curl_init();

curl_setopt($ch, CURLOPT_USERPWD, 'login:password');

curl_setopt($ch, CURLOPT_USERAGENT, $userAgent);

curl_setopt($ch, CURLOPT_URL,$target_url);

curl_setopt($ch, CURLOPT_FAILONERROR, true);

curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);

curl_setopt($ch, CURLOPT_AUTOREFERER, true);

curl_setopt($ch, CURLOPT_RETURNTRANSFER,true);

curl_setopt($ch, CURLOPT_TIMEOUT, 10);

$html = curl_exec($ch);

echo $html;

if (!$html) {

	echo "<br />cURL error number:" .curl_errno($ch);

	echo "<br />cURL error:" . curl_error($ch);

	exit;

}

?>

Open in new window

0
 
LVL 4

Expert Comment

by:eejones
ID: 24815556
Are all the articles concatenated and stored in the $html variable?
0
How to improve team productivity

Quip adds documents, spreadsheets, and tasklists to your Slack experience
- Elevate ideas to Quip docs
- Share Quip docs in Slack
- Get notified of changes to your docs
- Available on iOS/Android/Desktop/Web
- Online/Offline

 

Author Comment

by:PhillipsPlastics
ID: 24815620
Yes and I did successfully add php code to output everything to a file which may or may not be needed to search and grab the content I need.
0
 
LVL 4

Expert Comment

by:eejones
ID: 24815707
I think you could use the php file manipulation features to parse $html and write several html files, and the include links to them in your main page. Or maybe you could put all the articles into one html file and separate the articles with <a href name="article1"> tags and link to those anchors?
0
 

Author Comment

by:PhillipsPlastics
ID: 24823340
What kind of statements would I need to parse the $html and write the serveral html files?  This seems to be my best option for how the site is currently setup.
0
 
LVL 4

Expert Comment

by:eejones
ID: 24823960
To open and read the source file I think you need the PHP functions

fopen()
fgets()
fwrite()
fclose()

This might be useful as a reference:
http://us.php.net/manual/en/ref.filesystem.php
http://www.w3schools.com/PHP/php_file.asp

You might need some php string but probably not, since you are just reading/writing lines "as is" from the source file.

Some rough code is below. There may be some errors in it but I cannot test it right now. HTH.
Something like this to read the source file and break it up into chunks of code for the new pages, stored in an array $mypages: 
 

<?php

$file = fopen("sourcefile.html", "r") or exit("cannot open file");

//Initialize a variable to hold the contents for a new html file

$contents = "";

$startnewpage=0;

//Read lines until end of file

$mypages = array();

while(!feof($file)) {

$thisline = fgets($file);

// Test if this line of code marks the start or end of a page. 

// You may have to use substr_count() for the comparison 

// in case $thisline includes carriage return, trailing spaces etc.

if ($thisline  == 'text that marks the start of a new page') {

$startnewpage=1;

$contents = "";

}

if ($startnewpage) {

$contents = $contents . $thisline;

} 

if ($thisline == 'text that marks the end of a page') {

$startnewpage=0;

array_push($mypages,$content);

$content="";

}

}

fclose($file);

?>
 

Then you would loop through the array $mypages, and for each element in the array, open a new file, write into it the HTML header and opening body tags, write into it the content stored in $mypages[$n], write the HTML closing tags, and close the file.(You may have to have extracted some header information from the source file, such as a link to a stylesheet.) 
 

Here is, in a nutshell, how to open a file for writing, write to it and close it:
 

$myFile = "newfile.html";

$fh = fopen($myFile, 'w') or die("cannot open file");

$content = "Hello Kitty\n";

fwrite($fh, $content);

$content = "Hello Duckies\n";

fwrite($fh, $content);

fclose($fh);

Open in new window

0
 
LVL 4

Expert Comment

by:eejones
ID: 24823979
I already see an error in that snippet: $contents and $content should be the same variable name.
0
 

Author Comment

by:PhillipsPlastics
ID: 24824554
I can't seem to get any output echoing the variables to debug and make sure I can get the first part of the code working...
The first unique entry for each would be <h3 class="entry-header">
The ending unique would be <div class="entry" id="
OR the new page could start again just at the <h3 class="entry-header">
Currently the only thing that IS working is echoing out $thisline at line 10 shows the entire webpage....
<?php

$file = fopen("./output.txt", "r") or exit("cannot open file");

//Initialize a variable to hold the contents for a new html file

$content = "";

$startnewpage=0;

//Read lines until end of file

$mypages = array();

while(!feof($file)) {

$thisline = fgets($file);

// Test if this line of code marks the start or end of a page. 

// You may have to use substr_count() for the comparison 

// in case $thisline includes carriage return, trailing spaces etc.

if ($thisline  == 'entry-header') {

$startnewpage=1;

$content = "";

}

if ($startnewpage) {

$content = $content . $thisline;

} 

if ($thisline == 'entry') {

$startnewpage=0;

array_push($mypages,$content);

$content="";

}

}

fclose($file);

?>

Open in new window

0
 

Author Comment

by:PhillipsPlastics
ID: 24827311
ok so I got it working so it echoes out the correct things and debugged it thus far however I can't seem to figure out how to get it to properly output to a file within my loop....  suggestions?
<?php

$file = fopen("./output.txt", "r") or exit("cannot open file"); //Initialize a variable to hold the contents for a new html file
 

$content = "";

$startnewpage = 0;

$data = "";

c = 1;
 

$mypages = array();		//Read lines until end of file

while(!feof($file)) {

$data = fgets($file);

						// Test if this line of code marks the start or end of a page. 

						// You may have to use substr_count() for the comparison 

						// in case $data includes carriage return, trailing spaces etc.

		if ((substr_count($data,"h3"))  > 0) {

			echo $data;

			while(((substr_count($data,"technorati") == 0))) {

			$data = fgets($file);

			echo $data;

			}

			echo "<br><br><br><br><br><br><br><br>";

	}

	if ($data != "newline1") {

		/* echo "fail <br>"; */

		$data = "";

	}
 

}

fclose($file);

?>

Open in new window

0
 

Author Comment

by:PhillipsPlastics
ID: 24827336
ugh forgot the "$" before the counter var... but still working on output...
0
 

Author Comment

by:PhillipsPlastics
ID: 24827366
yay! got it working final code posted below...
<?php

$file = fopen("./output.txt", "r") or exit("cannot open file"); //Initialize a variable to hold the contents for a new html file
 

$content = "";

$startnewpage = 0;

$data = "";

$c = 0;
 

$mypages = array();		//Read lines until end of file

while(!feof($file)) {

$data = fgets($file);

						// Test if this line of code marks the start or end of a page. 

						// You may have to use substr_count() for the comparison 

						// in case $data includes carriage return, trailing spaces etc.

		if ((substr_count($data,"h3"))  > 0) {

			echo $data;

			while(((substr_count($data,"technorati") == 0))) {

			$data = fgets($file);

			echo $data;

				$myFile = "newfile$c.html";

				$fh = fopen($myFile, 'a') or die("cannot open file");

				$content = $data;

				fwrite($fh, $content);

				fclose($fh);

			}

						$c++;

	}

	if ($data != "newline1") {

		/* echo "fail <br>"; */

		$data = "";

	}
 

}

fclose($file);

?>

Open in new window

0
 
LVL 4

Accepted Solution

by:
eejones earned 500 total points
ID: 24827374
To open a new file:

$myFile = "newfile.html";
$fh = fopen($myFile, 'w') or die("cannot open file");

You are getting a line of data into $data in your loop so to get $data into your new file:

fwrite($fh, $data);

To close the file:

fclose($fh);
0

Featured Post

Find Ransomware Secrets With All-Source Analysis

Ransomware has become a major concern for organizations; its prevalence has grown due to past successes achieved by threat actors. While each ransomware variant is different, we’ve seen some common tactics and trends used among the authors of the malware.

Join & Write a Comment

Browsers only know CSS so your awesome SASS code needs to be translated into normal CSS. Here I'll try to explain what you should aim for in order to take full advantage of SASS.
Have you tried to learn about Unicode, UTF-8, and multibyte text encoding and all the articles are just too "academic" or too technical? This article aims to make the whole topic easy for just about anyone to understand.
In this tutorial viewers will learn how to style a corner ribbon overlay for an image using CSS Create a new class by typing ".Ribbon":  Define the class' "display:" as "inline-block": Define its "position:" as "relative": Define its "overflow:" as …
HTML5 has deprecated a few of the older ways of showing media as well as offering up a new way to create games and animations. Audio, video, and canvas are just a few of the adjustments made between XHTML and HTML5. As we learned in our last micr…

707 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

14 Experts available now in Live!

Get 1:1 Help Now