Link to home
Start Free TrialLog in
Avatar of Jon Imms
Jon ImmsFlag for United States of America

asked on

Wordpress Cron - Send XML Data to external server FTP

Hello,

We currently have a Wordpress XML file which has upwards of 200,000 entries in it,  which is too big to render on our server.

This is a custom feed, which an external job site will pull, and display the listings on their website.

As the XML Data is so huge,  Is it possible to create a Cron job, which every night will send the data to an external server via FTP, and then the job site can pull from the FTP location.

Here is the current XML Template we are using.  (I have $numposts as 5 at the moment, just so i know the feed works, and memory is the issue).
https://cretecarrierjobs.com/cretecarrier-xml-feed/
<?php
/**
 * Template Name: Custom RSS Template - cretefeed - RR-JI
 */
$numposts = 500000; // number of posts in feed
$posts = query_posts('showposts='.$numposts.'');
$more = 1;
header('Content-Type: '.feed_content_type('rss-http').'; charset='.get_option('blog_charset'), true);
echo '<?xml version="1.0" encoding="'.get_option('blog_charset').'"?'.'>';
?>
<rss version="2.0"
     xmlns:content="http://purl.org/rss/1.0/modules/content/"
     xmlns:wfw="http://wellformedweb.org/CommentAPI/"
     xmlns:dc="http://purl.org/dc/elements/1.1/"
     xmlns:atom="http://www.w3.org/2005/Atom"
     xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
     xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
    <?php do_action('rss2_ns'); ?>>
    <channel>
        <title><?php bloginfo_rss('name'); ?> - Feed</title>
        <atom:link href="<?php self_link(); ?>" rel="self" type="application/rss+xml" />
        <link><?php bloginfo_rss('url') ?></link>
        <description><?php bloginfo_rss('description') ?></description>
        <lastBuildDate><?php echo mysql2date('D, d M Y H:i:s +0000', get_lastpostmodified('GMT'), false); ?></lastBuildDate>
        <?php the_generator( 'rss2' ); ?>
        <language><?php echo get_option('rss_language'); ?></language>
        <sy:updatePeriod><?php echo apply_filters( 'rss_update_period', 'hourly' ); ?></sy:updatePeriod>
        <sy:updateFrequency><?php echo apply_filters( 'rss_update_frequency', '1' ); ?></sy:updateFrequency>
        <?php do_action('rss2_head'); ?>
        <?php while(have_posts()) : the_post(); ?>
            <?php $zips = get_field('zipcode');
            $zips_array = explode(' ',$zips);
            foreach($zips_array as $zip){ ?>
                <job>
                    <title><?php the_title_rss(); ?></title>
                    <date><?php echo mysql2date('D, d M Y H:i:s +0000', get_post_time('Y-m-d H:i:s', true), false); ?></date>
                    <referencenumber><?php the_id(); ?><?php echo $zip; ?></referencenumber>
                    <url><?php the_permalink_rss(); ?></url>
                    <company><![CDATA[Crete Carrier]]></company>
                    <city><![CDATA[<?php the_field('city'); ?>]]></city>
                    <state><![CDATA[<?php the_field('state'); ?>]]></state>
                    <country><![CDATA[US]]></country>
                    <postalcode><![CDATA[<?php echo $zip; ?>]]></postalcode>

                    <?php if (get_option('rss_use_excerpt')) : ?>
                        <description><![CDATA[<?php the_excerpt() ?>]]></description>
                        <applylink><?php echo ('https://intelliapp2.driverapponline.com/c/cretecarrier?r=RandallReilly-CreteXMLFeed'); ?></applylink>
                    <?php else : ?>
                        <description><![CDATA[<?php the_excerpt() ?>]]></description>
                        <applylink><?php echo ('https://intelliapp2.driverapponline.com/c/cretecarrier?r=RandallReilly-CreteXMLFeed'); ?></applylink>
                        <?php if ( strlen( $post->post_content ) > 0 ) : ?>
                            <content:encoded><![CDATA[<?php the_content() ?>]]></content:encoded>
                        <?php else : ?>
                            <content:encoded><![CDATA[<?php the_excerpt_rss() ?>]]></content:encoded>
                        <?php endif; ?>
                    <?php endif; ?>

                    <salary><?php echo do_shortcode( get_post_meta( get_the_id(), 'salary', true ) ); ?></salary>
                    <experience><?php echo do_shortcode( '[Qualifications]' ); ?></experience>
                    <jobtype><![CDATA[<?php the_field('jobtype'); ?>]]></jobtype>
                    <category><?php the_category( ' ' ); ?></category>
                    <tags><?php the_tags( ' ' ); ?></tags>
                    <?php rss_enclosure(); ?>
                    <?php do_action('rss2_item'); ?>
                </job>
            <?php } ?>
        <?php endwhile; ?>
    </channel>
</rss>

Open in new window


So basically we will be writing this file(s) to FTP nightly to the external server, and then the other party can pick it up from their ftp server.
ASKER CERTIFIED SOLUTION
Avatar of Ray Paseur
Ray Paseur
Flag of United States of America image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Avatar of Jon Imms

ASKER

Thanks Ray.

The problem i am having, is that the XML feed has got upwards of 200,000 jobs in the xml. https://cretecarrierjobs.com/cretecarrier-xml-feed/  The feed does not display because of the size of it (error : PHP Fatal error:  Allowed memory size of 536870912 bytes exhausted (tried to allocate 375128064 bytes) )

Because of this, the job board people want me to send the output via FTP to their server, daily, and then they can pull from that. But i'm not sure how to do that in wordpress.
Or, to make things easier,  write to an XML file on my server.  then i can just give them permission to FTP that one file from my server.