asked on

Why multiple Twitter RSS feeds cannot be fetched sometimes from one webpage?

Hello,

I have two questions. I made two projects, and they do not work sometimes. All project uses multiple twitter rss services in one web page.

1) project # 1 uses jquery ui tab (accordion), where they are 7 parts. One part is showing Anderson Cooper's Twitts, One part is showing CNN's Twitts, One part is showing Microsoft's Twitts, and One part is showing Yahoo's Twitts, etc ( Image attached).
[language is asp .net C#]
Everything works fine, but if I refresh the page multiple times( say I click refresh 10 times in 10 seconds), the rss feeds don't comes and a error 400 happens. The whole code is at below, and after one hour, everything works again.

Do you know why is this happening?

...............................................................................................................................................

2) project # 2 uses uses jquery ui tab (accordion), where they are 7 parts. One part is showing Anderson Cooper's Twitts, one part is showing CNN's Twitts, One part is showing Microsoft's Twitts, One part is showing Yahoo's Twitts, etc (same as # 1 case).
[Here the language is javascript and php]
In this program, I first try to fetch rss data by using yahoo pipe by using javascript; sometimes the yahoo pipes fetch data and sometimes, the pipes don't. When a pipe cannot fetch data, I call a php file to fetch those rss data, and I have observed, sometimes those rss feeds get fetched and sometimes

they don't get fetched, or sometimes the rss feeds get fetched but get displayed in wrong order. For example, Microsoft's Rss feed is at CNN's place.

Do you know what is the reason. The whole working code is attached. To make this error work, refresh the page couple times, and you will see
the error.

Do you know what is the reason?

Note: if you test my code at your computer, and when you will see the error scenarios that I am describing ( case # 1), please try to go to Anderson Cooper's or CNN's or Microsoft's twitter rss feed address, and you will not able to go there for one hour as well
(http://twitter.com/statuses/user_timeline/15224867.rss<-----Rss feed will be unavailable to you for one hour.)

Please give me your best guess. I also have guesses but I am not sure whether my guesses are correct.

*** The codes are attached for both case

# 1 Case: ASP .NET C# and jquery ui tab
-------------------------------------------------------
ASPX File
........................
<%@ Page Language="C#" AutoEventWireup="true" CodeFile="ASP_Jquery.aspx.cs" Inherits="ASP_Jquery" %>

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">

<html xmlns="http://www.w3.org/1999/xhtml">
<head runat="server">
    <title></title>
     <link rel="stylesheet" type="text/css" href="AJAXPHP/style.css" />


      <script type="text/javascript">
          $(document).ready(function () {
             //alert("TEST2");
          });
      </script>
      	<link type="text/css" href="css/dot-luv/jquery-ui-1.8.4.custom.css" rel="stylesheet" />	
		<script type="text/javascript" src="js/jquery-1.4.2.min.js"></script>
		<script type="text/javascript" src="js/jquery-ui-1.8.4.custom.min.js"></script>
        <script type="text/javascript" src="http://swip.codylindley.com/jquery.DOMWindow.js"></script>


		<script type="text/javascript">
		    $(function () {
		        // alert("TEST");
		        // Accordion
		        $("#accordion").accordion({ header: "h3" });
		        $("#me").css("display", "block"); $("#cnn").css("display", "block"); $("#yahoo").css("display", "block");
		        $("#nelp").css("display", "block"); $("#obama").css("display", "block"); $("#ms").css("display", "block");
		        $("#anderson").hide().fadeIn(2000); $("#andersonImage").hide().fadeIn(2000);


		    });
		</script>
    <style type="text/css">
        .style1
        {
            width: 400px;
        }
    </style>
</head>
<body>
<form id="form1" runat="server">
<div id="page-wrap">
		<div id="inside">
			<div id="header">
        Daily Twitter

            </div>
            <div id="subHeader"><a href="http://twitter.com/andersoncooper">AC</a>&nbsp; &nbsp;&nbsp; 
                                    <a href="http://twitter.com/cnn">cnn</a> &nbsp &nbsp; &nbsp; 
                                    <a href="http://twitter.com/cnn">CNN</a>&nbsp &nbsp; &nbsp; 
                                    <a href="http://twitter.com/yahoo">Yahoo!</a> &nbsp &nbsp; &nbsp; 
                                      <a href="http://twitter.com/cnn">Barack Obama</a>&nbsp &nbsp; &nbsp; 
                                    <a href="http://twitter.com/nelp">Nelp</a> &nbsp &nbsp; &nbsp; 
                                    <a href="http://twitter.com/microsoft">Microsoft</a></div>	
			
			<!-- Tabs -->
       
		<!-- Accordion -->
	
<div id="accordion">
			<div>
				<h3><a href="#">Reporter Anderson Cooper's Twitts</a></h3>
				<div id ="div1">
                    <table width="98%"><tr>
                    <td valign="top"> <a href="http://ac360.blogs.cnn.com/" class="absoluteIframeDOMWindow"><img src="andersoncooper.jpg" alt="ac" height="400" width="300" id = "andersonImage" style="display:none;" /></a>
                    
                    <script type="text/javascript">
                        $('.absoluteIframeDOMWindow').openDOMWindow({
                            height: 400,
                            width: 700,
                            positionType: 'absolute',
                            positionTop: 150,
                            eventType: 'click',
                            positionLeft: 300,
                            windowSource: 'iframe',
                            windowPadding: 0,
                            loader: 1,
                            loaderImagePath: 'animationProcessing.gif',
                            loaderHeight: 16,
                            loaderWidth: 17
                        });

</script>
                    
                    
                    
                    </td>
                    <td valign="top" style="padding: 5px 10px 20px 30px;">
                    <span id = "anderson" style="display:none;">
                        <asp:Label ID="Label1" Visible="false" runat="server"/>
                        <asp:Image ID="Image1" Visible="false" runat="server" ImageUrl="http://blogs.ft.com/techblog/files/2009/08/twitter_fail_whale.jpg" AlternateText="Twitter Service is broken" />
                        <asp:repeater id="Repeater1" runat="server">
                                <HeaderTemplate></HeaderTemplate>
                                <ItemTemplate>
                                <a href='<%#((System.Xml.XmlNode)Container.DataItem).SelectSingleNode("link").InnerText%>'>
                                <%#((System.Xml.XmlNode)Container.DataItem).SelectSingleNode("pubDate").InnerText%></a><br />
                                <%#((System.Xml.XmlNode)Container.DataItem).SelectSingleNode("description").InnerText%><br /><br />
                
                                </ItemTemplate>
                                <FooterTemplate></FooterTemplate>
                         </asp:repeater>
                    </span>
                    </td>
                    </tr></table>
                </div>
			</div>
			<div>
				<h3><a href="#" id="me" style="display:none;">Parrottweet</a></h3>
				<div id ="div2" style="display:none;">
                    <table width="98%"><tr>
                    <td valign="top"><img src="me.jpg" alt="ac" height="400" width="300" /></td>
                    <td valign="top" style="padding: 5px 10px 20px 30px;">
                     <asp:Label ID="Label2" Visible="false" runat="server"/>
                    <asp:Image ID="Image2" Visible="false" runat="server" ImageUrl="http://blogs.ft.com/techblog/files/2009/08/twitter_fail_whale.jpg" AlternateText="Twitter Service is broken" />
                         
                            <asp:repeater id="Repeater2" runat="server">
                            <HeaderTemplate></HeaderTemplate>
                            <ItemTemplate>
                            <a href='<%#((System.Xml.XmlNode)Container.DataItem).SelectSingleNode("link").InnerText%>'>
                            <%#((System.Xml.XmlNode)Container.DataItem).SelectSingleNode("pubDate").InnerText%></a><br />
                            <%#((System.Xml.XmlNode)Container.DataItem).SelectSingleNode("description").InnerText%><br /><br />
                
                            </ItemTemplate>
                            <FooterTemplate></FooterTemplate>
                            </asp:repeater>
                    </td>
                    </tr></table>
                </div>
           </div>
		   <div>
				<h3><a href="#" id="cnn" style="display:none;">CNN's Twitts</a></h3>
				<div id ="div3" style="display:none;">
                        <table width="98%"><tr>
                        <td valign="top"><img src="cnn1.jpg" alt="ac" height="400" width="300" /></td>
                        <td valign="top" style="padding: 5px 10px 20px 30px;">
                        <asp:Label ID="Label3" Visible="false" runat="server"/>
                        <asp:Image ID="Image3" Visible="false" runat="server" ImageUrl="http://blogs.ft.com/techblog/files/2009/08/twitter_fail_whale.jpg" AlternateText="Twitter Service is broken" />
                          
                                <asp:repeater id="Repeater3" runat="server">
                                <HeaderTemplate></HeaderTemplate>
                                <ItemTemplate>
                                <a href='<%#((System.Xml.XmlNode)Container.DataItem).SelectSingleNode("link").InnerText%>'>
                                <%#((System.Xml.XmlNode)Container.DataItem).SelectSingleNode("pubDate").InnerText%></a><br />
                                <%#((System.Xml.XmlNode)Container.DataItem).SelectSingleNode("description").InnerText%><br /><br />
                
                                </ItemTemplate>
                                <FooterTemplate></FooterTemplate>
                                </asp:repeater>
                        </td>
                        </tr></table>
                </div>
           </div>
               <div>
				<h3><a href="#" id="yahoo" style="display:none;">Yahoo!'s Twitts</a></h3>
				<div id ="div4" style="display:none;">
                    <table width="98%"><tr>
                    <td valign="top"><img src="yahoo.png" alt="ac" height="400" width="300" /></td>
                    <td valign="top" style="padding: 5px 10px 20px 30px;">
                    <asp:Label ID="Label4" Visible="false" runat="server" />
                    <asp:Image ID="Image4" Visible="false" runat="server" ImageUrl="http://blogs.ft.com/techblog/files/2009/08/twitter_fail_whale.jpg" AlternateText="Twitter Service is broken" />
                          
                            <asp:repeater id="Repeater4" runat="server">
                            <HeaderTemplate></HeaderTemplate>
                            <ItemTemplate>
                            <a href='<%#((System.Xml.XmlNode)Container.DataItem).SelectSingleNode("link").InnerText%>'>
                            <%#((System.Xml.XmlNode)Container.DataItem).SelectSingleNode("pubDate").InnerText%></a><br />
                            <%#((System.Xml.XmlNode)Container.DataItem).SelectSingleNode("description").InnerText%><br /><br />
                
                            </ItemTemplate>
                            <FooterTemplate></FooterTemplate>
                            </asp:repeater>
                    </td>
                    </tr></table>
                </div>
           </div>
              <div>
				<h3><a href="#" id="nelp" style="display:none;">Nelp's Twitts</a></h3>
				<div id ="div5" style="display:none;">
                    <table width="98%"><tr>
                    <td valign="top"><img src="nelp-mini.png" alt="ac" height="400" width="300" /></td>
                    <td valign="top" style="padding: 5px 10px 20px 30px;">
                    <asp:Label ID="Label5" Visible="false" runat="server"/>
                    <asp:Image ID="Image5" Visible="false" runat="server" ImageUrl="http://blogs.ft.com/techblog/files/2009/08/twitter_fail_whale.jpg" AlternateText="Twitter Service is broken" />
                          
                            <asp:repeater id="Repeater5" runat="server">
                            <HeaderTemplate></HeaderTemplate>
                            <ItemTemplate>
                            <a href='<%#((System.Xml.XmlNode)Container.DataItem).SelectSingleNode("link").InnerText%>'>
                            <%#((System.Xml.XmlNode)Container.DataItem).SelectSingleNode("pubDate").InnerText%></a><br />
                            <%#((System.Xml.XmlNode)Container.DataItem).SelectSingleNode("description").InnerText%><br /><br />
                
                            </ItemTemplate>
                            <FooterTemplate></FooterTemplate>
                            </asp:repeater>
                    </td>
                    </tr></table>
                </div>
           </div>
              <div>
				<h3><a href="#" id="obama" style="display:none;">Barack Obama's Twitts</a></h3>
				<div id ="div6" style="display:none;">
                    <table width="98%"><tr>
                    <td valign="top"><img src="barackobama.jpg" alt="ac" height="400" width="300" /></td>
                    <td valign="top" style="padding: 5px 10px 20px 30px;">
                       <asp:Label ID="Label6" Visible="false" runat="server"/>
                    <asp:Image ID="Image6" Visible="false" runat="server" ImageUrl="http://blogs.ft.com/techblog/files/2009/08/twitter_fail_whale.jpg" AlternateText="Twitter Service is broken" />
                          
                       
                            <asp:repeater id="Repeater6" runat="server">
                            <HeaderTemplate></HeaderTemplate>
                            <ItemTemplate>
                            <a href='<%#((System.Xml.XmlNode)Container.DataItem).SelectSingleNode("link").InnerText%>'>
                            <%#((System.Xml.XmlNode)Container.DataItem).SelectSingleNode("pubDate").InnerText%></a><br />
                            <%#((System.Xml.XmlNode)Container.DataItem).SelectSingleNode("description").InnerText%><br /><br />
                
                            </ItemTemplate>
                            <FooterTemplate></FooterTemplate>
                            </asp:repeater>
                    </td>
                    </tr></table>
                </div>
           </div>
              <div>
				<h3><a href="#" id="ms" style="display:none;">Microsoft's Twitts</a></h3>
				<div id ="div7" style="display:none;">
                        <table width="98%"><tr>
                        <td valign="top"><img src="microsoft.png" alt="ac" height="400" width="300" /></td>
                        <td valign="top" style="padding: 5px 10px 20px 30px;">
                          <asp:Label ID="Label7" Visible="false" runat="server"/>
                        <asp:Image ID="Image7" Visible="false" runat="server" ImageUrl="http://blogs.ft.com/techblog/files/2009/08/twitter_fail_whale.jpg" AlternateText="Twitter Service is broken" />
                        
                                <asp:repeater id="Repeater7" runat="server">
                                <HeaderTemplate></HeaderTemplate>
                                <ItemTemplate>
                                <a href='<%#((System.Xml.XmlNode)Container.DataItem).SelectSingleNode("link").InnerText%>'>
                                <%#((System.Xml.XmlNode)Container.DataItem).SelectSingleNode("pubDate").InnerText%></a><br />
                                <%#((System.Xml.XmlNode)Container.DataItem).SelectSingleNode("description").InnerText%><br /><br />
                
                                </ItemTemplate>
                                <FooterTemplate></FooterTemplate>
                                </asp:repeater>
                        </td>
                        </tr></table>
                </div>
           </div>
 </div>
	
	
                <div style="clear: both;"></div>
			    <div id="footer"><h3>Information at Footer</h3></div>            
        </div>
</div>
</form>
</body>
</html>


..................................................................
Code Behind---C#
......................
using System;using System.Collections.Generic;using System.Linq;using System.Web;using System.Web.UI;using System.Web.UI.WebControls;using System.Xml;

public partial class ASP_Jquery : System.Web.UI.Page{
    protected void Page_Load(object sender, EventArgs e){
        string[] urls = new string[7] { "http://twitter.com/statuses/user_timeline/15224867.rss", "http://twitter.com/statuses/user_timeline/15224867.rss", 
        "http://twitter.com/statuses/user_timeline/759251.rss", "http://twitter.com/statuses/user_timeline/19380829.rss", "http://twitter.com/statuses/user_timeline/78461452.rss",
        "http://twitter.com/statuses/user_timeline/813286.rss","http://twitter.com/statuses/user_timeline/24741685.rss" };
        HttpCookie MyCookie = new HttpCookie("LastVisit");
        DateTime now = DateTime.Now;

        MyCookie.Value = now.ToString();
        MyCookie.Expires = now.AddHours(1);

        Response.Cookies.Add(MyCookie);
   


        if (!IsPostBack)
        {
            for (int i = 0; i < 7; i++)
            {
                XmlDocument xml = new XmlDocument();
                XmlNodeList nodes = xml.SelectNodes("/rss/channel/item");
                
                try
                {
                    xml.Load(urls[i]);
                    if (i == 0){
                        xml.Load(urls[i]);
                        Repeater1.DataSource = nodes;
                        Repeater1.DataBind();
                    }
                    else if (i == 1){
                        xml.Load(urls[i]);
                        Repeater2.DataSource = nodes;
                        Repeater2.DataBind();
                    }
                    else if (i == 2){
                        xml.Load(urls[i]);
                        Repeater3.DataSource = nodes;
                        Repeater3.DataBind();
                    }
                    else if (i == 3){
                        xml.Load(urls[i]);
                        Repeater4.DataSource = nodes;
                        Repeater4.DataBind();
                    }
                    else if (i == 4){
                        xml.Load(urls[i]);
                        Repeater5.DataSource = nodes;
                        Repeater5.DataBind();
                    }
                    else if (i == 5){
                        xml.Load(urls[i]);
                        Repeater6.DataSource = nodes;
                        Repeater6.DataBind();
                    }
                    else if (i == 6){
                        xml.Load(urls[i]);
                        Repeater7.DataSource = nodes;
                        Repeater7.DataBind();
                    }
                }
                catch (Exception error){
                    string errorMessage = "Either Twitter rss service or main web service is down<br/>Error Message: ";
                    Image1.Visible = true; Image2.Visible = true; Image3.Visible = true; Image4.Visible = true; Image5.Visible = true; Image6.Visible = true;Image7.Visible = true;
                    Label1.Visible = true; Label2.Visible = true; Label3.Visible = true; Label4.Visible = true; Label5.Visible = true; Label6.Visible = true;Label7.Visible = true;
                    Label1.Text = errorMessage + error.Message; Label2.Text = errorMessage + error.Message; Label3.Text = errorMessage + error.Message;
                    Label4.Text = errorMessage + error.Message; Label5.Text = errorMessage + error.Message; Label6.Text = errorMessage + error.Message; Label7.Text = errorMessage + error.Message;
                }

            }
        }
    }
}
-------------------------------------------------------------
#2 Case ( Yahoo Pipe, JavaScript, and PHP)
.................

JS file

flag = 0; var i = 0; var myArray = new Array(7); 
var names = new Array("ac", "me", "cnn", "yahoo", "nelp", "obama", "ms");
function getFeed() {
    var newScript = document.createElement('script');
    newScript.type = 'text/javascript';
    myArray[0] = "http://pipes.yahoo.com/pipes/pipe.run?_id=0fe4e5149d6346e748e4b0e5a424197e&_render=json&_callback=myfunction";
    myArray[1] = "http://pipes.yahoo.com/pipes/pipe.run?_id=0fe4e5149d6346e748e4b0e5a424197e&_render=json&_callback=myfunction";
    myArray[2] = "http://pipes.yahoo.com/pipes/pipe.run?_id=74b96e2551b4c024a1300803e09d0401&_render=json&_callback=myfunction";
    myArray[3] = "http://pipes.yahoo.com/pipes/pipe.run?_id=fffd88d2544ad48b0352f22414a30afb&_render=json&_callback=myfunction";
    myArray[4] = "http://pipes.yahoo.com/pipes/pipe.run?_id=16b91210f014087d5e0ccaab0aec0c7e&_render=json&_callback=myfunction";
    myArray[5] = "http://pipes.yahoo.com/pipes/pipe.run?_id=895e6164c61b2ee9f7fd73b8e16a71b1&_render=json&_callback=myfunction";
    myArray[6] = "http://pipes.yahoo.com/pipes/pipe.run?_id=490fc765697c7bac6412ba62a6c71473&_render=json&_callback=myfunction"; 

    if (i < 7) {
        newScript.src = myArray[i];
    }
    else if (i == 7) {
        return;
    }
    document.getElementsByTagName("head")[0].appendChild(newScript);
}
function fetchDataByPHP(i) {
    myStr = "";
    if (i == 0) {
        myStr = names[i];
    } else if (i == 1) {
        myStr = names[i];
    }
    else if (i == 2) {
        myStr = names[i];
    }
    else if (i == 3) {
        myStr = names[i];
    }
    else if (i == 4) {
        myStr = names[i];
    }
    else if (i == 5) {
        myStr = names[i];
    }
    else if (i == 6) {
        myStr = names[i];
    }

    if (window.XMLHttpRequest) {// code for IE7+, Firefox, Chrome, Opera, Safari
        xmlhttp = new XMLHttpRequest();
    }
    else {// code for IE6, IE5
        xmlhttp = new ActiveXObject("Microsoft.XMLHTTP");
    } 
    var j = i;
    j = j + 1;
    xmlhttp.onreadystatechange = function () {
        if (xmlhttp.readyState == 4 && xmlhttp.status == 200) {
           // alert(xmlhttp.responseText);
            myStr = xmlhttp.responseText;
            tempDiv = "div" + j;
            document.getElementById(tempDiv).innerHTML = myStr;
        }
    }
    xmlhttp.open("GET", "fetchDataByCurl.php?q=" + myStr, true);
    xmlhttp.send();
}
function myfunction(feed) {
    str = "";
    if (feed.value.items.length == 0) {
        fetchDataByPHP(i);
        i++;
        getFeed();
    }
    else {
       // alert(feed.value.items[0].description);
        for (j = 0; j <= 8; j++) {
            var tmpDate = feed.value.items[j].pubDate;
            tmpDate = formatDate(tmpDate);
            str += "At <a href='" + feed.value.items[j].link + "'>" + tmpDate + "</a><br/>";
            str += feed.value.items[j].description + "<br/><br/>";
        }
        displayData(str);
        i++; 
        getFeed();
    }
}
function displayData(str) {
    var ac = "<table border='2' width='100%'>";
    if (i == 0) {
        ac += "<tr><td valign='top' class='style2'><img src='andersoncooper.jpg'/></td><td>";
        document.getElementById('div1').innerHTML = ac + str + "</td></tr></table>";
    }
    if (i == 1) {
        ac += "<tr><td valign='top' class='style2'><img src='andersoncooper.jpg' height='100%'/></td><td>";
        document.getElementById('div2').innerHTML = ac + str + "</td></tr></table>";
    }
    if (i == 2) {
        ac += "<tr><td valign='top' class='style2'><img src='cnn1.jpg'/></td><td>";
        document.getElementById('div3').innerHTML = ac + str + "</td></tr></table>";
    }
    if (i == 3) {
        ac += "<tr><td valign='top' class='style2'><img src='yahoo.png' height='400px' width='304px'/></td><td>";
        document.getElementById('div4').innerHTML = ac + str + "</td></tr></table>";
    }
    if (i == 4) {
        ac += "<tr><td valign='top' class='style2'><img src='nelp-mini.png' height='100%'/></td><td>";
        document.getElementById('div5').innerHTML = ac + str + "</td></tr></table>";
    }
    if (i == 5) {
        ac += "<tr><td valign='top' class='style2'><img src='barackobama.jpg'/></td><td>";
        document.getElementById('div6').innerHTML = ac + str + "</td></tr></table>";
    }
    if (i == 6) {
        ac += "<tr><td valign='top' class='style2'><img src='Microsoft.png' height='400px' width='304px'/></td><td>";
        document.getElementById('div7').innerHTML = ac + str + "</td></tr></table>";
    }
}
function formatDate(tmpDate) {
    var t = Date.parse(tmpDate);
   // var t = Date.parse("2010-08-13T00:20:45+00:00");   
    var d = new Date(t); 
    var weekday = new Array("Sunday", "Monday", "Tuesday", "Wednesday", "Thursday", "Friday", "Saturday");
    var month = new Array("January", "February", "March", "April", "May", "June", "July", "August", "September", "October", "November", "December");
    var h = d2(d.getHours()); var m = d2(d.getMinutes());
    var ampm = "AM";
    if (h > 12) {
        h = d2(h - 12);
        ampm = "PM";
    }
    var mfDate = weekday[d.getDay()] + ", " + month[d.getMonth()] + " " + tmpDate.substr(5, 2) + ", " + d.getFullYear() + " " + h + ":" + m + " " + ampm;
   // alert(mfDate);
    return mfDate;
}

function d2(n) {
    if (n < 10) return "0" + n;
    return n;
}


..............................................................................................
///PHP file

<?php
$q=$_GET["q"];
$j = 0;


$content= "";
$content .= '<table border="2" width="100%"><tr>';
$content .= '<td valign="top" class="style2">';

if($q == "ac"){
  $xml=("http://twitter.com/statuses/user_timeline/15224867.rss");
  $content.= '<img src="andersoncooper.jpg"/></td><td>';
}
elseif($q == "me"){
    $xml=("http://twitter.com/statuses/user_timeline/15224867.rss");
    $content.= '<img src="me.png"/></td><td>';
}
elseif($q == "cnn"){
    $xml=("http://twitter.com/statuses/user_timeline/759251.rss"); 
    $content.= '<img src="cnn1.jpg"/></td><td>';
}  
elseif($q == "yahoo"){
    $xml=("http://twitter.com/statuses/user_timeline/19380829.rss"); 
    $content.= '<img src="yahoo.png"/></td><td>';
} 
elseif($q == "nelp"){
    $xml=("http://twitter.com/statuses/user_timeline/78461452.rss");  
    $content.= '<img src="nelp-mini.png"/></td><td>';
} 
elseif($q == "obama"){
    $xml=("http://twitter.com/statuses/user_timeline/813286.rss"); 
    $content.= '<img src="barackobama.jpg"/></td><td>';
} 
elseif($q == "ms"){
    $xml=("http://twitter.com/statuses/user_timeline/24741685.rss"); 
    $content.= '<img src="Microsoft.png"/></td><td>';
} 


  $curl_handle=curl_init();
  curl_setopt($curl_handle,CURLOPT_URL,$xml);
  curl_setopt($curl_handle,CURLOPT_CONNECTTIMEOUT,2);
  curl_setopt($curl_handle,CURLOPT_RETURNTRANSFER,1);
  $buffer = curl_exec($curl_handle);
  $xmlText = new SimpleXmlElement($buffer, LIBXML_NOCDATA);
  curl_close($curl_handle);  
  
  $cnt = count($xmlText->channel->item);
  
 
     for($i=0; $i<$cnt; $i++){
        $url 	= $xmlText->channel->item[$i]->link;
        $title 	= $xmlText->channel->item[$i]->title;
        $date = $xmlText->channel->item[$i]->pubDate;
        $this_date = $date;
        $this_ts = strtotime($this_date);
        $this_date = strftime("%A, %B %d, %Y %I:%M %p",$this_ts);
        $date = ucfirst($this_date);
        $content .= 'Atttt <a href="'.$url.'">'.$date.'</a>'.'<br/>'.$title.'<br/><br/>';
     } 
   
   echo $content."</td></tr></table>";   
?>


..........................
HTML file
..............
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
    <title>AJAX NON Page Load Demo</title>
    <link rel="stylesheet" type="text/css" href="style.css" />
    <script src="jquery-1.3.1.min.js" type="text/javascript"></script>
    <script src="myJs.js" type="text/javascript"></script>
      <script type="text/javascript">
          $(document).ready(function () {
             // alert("Calling Feed");
              getFeed();
          });
      </script>

      	<link type="text/css" href="css/dot-luv/jquery-ui-1.8.4.custom.css" rel="stylesheet" />	
		<script type="text/javascript" src="js/jquery-1.4.2.min.js"></script>
		<script type="text/javascript" src="js/jquery-ui-1.8.4.custom.min.js"></script>

		<script type="text/javascript">
		    $(function () {

		        // Accordion
		        $("#accordion").accordion({ header: "h3" });

		    });
		</script>
    <style type="text/css">
        .style1
        {
            width: 400px;
        }
    </style>
</head>
<body>
<div id="page-wrap">

		<div id="inside">
			<div id="header"><table width="100%">
            <tr><td align="left" class="style1">
            <img src="http://sbctoday.com/wp-content/uploads/2009/05/twitter-logo.png" alt="img" />
            </td><td align="right"><h1>Your Daily Twitter Source is  ***HERE***</h1>
            </td></tr></table>
            </div>
            <div id="subHeader"><h2><a href="http://twitter.com/andersoncooper">AC</a>&nbsp; &nbsp;&nbsp; 
                                    <a href="http://twitter.com/cnn">CNN</a> &nbsp; &nbsp; &nbsp; 
                                    <a href="http://twitter.com/cnn">CNN</a>&nbsp; &nbsp; &nbsp; 
                                    <a href="http://twitter.com/yahoo">Yahoo!</a> &nbsp; &nbsp; &nbsp; 
                                      <a href="http://twitter.com/cnn">Barack Obama</a>&nbsp; &nbsp; &nbsp; 
                                    <a href="http://twitter.com/yahoo">Obama</a> &nbsp; &nbsp; &nbsp; 
                                    <a href="http://twitter.com/microsoft">NELP</a></h2></div>	
			
			<!-- Tabs -->

       
		<!-- Accordion -->
	
		<div id="accordion">
			<div>
				<h3><a href="#">Reporter Anderson Cooper's Twitts</a></h3>

				<div id ="div1"></div>
			</div>
			<div>
				<h3><a href="#"> *** Twitts</a></h3>

				<div id ="div2"></div>
			</div>
			<div>
				<h3><a href="#">CNN's Twitts</a></h3>
				<div id ="div3"></div>
			</div>
            <div>
				<h3><a href="#">Yahoo!'s Twitts</a></h3>

				<div id ="div4"></div>
			</div>
            <div>
                <h3><a href="#">Nelp's Twitts</a></h3>
				<div id ="div5"></div>
				
			</div>
            <div>
				<h3><a href="#">Barack Obama's Twitts</a></h3>

				<div id ="div6"></div>
			</div>
            <div>
				<h3><a href="#">Microsoft's Twitts</a></h3>
				<div id ="div7"></div>
			</div>
		</div>
	
	
                <div style="clear: both;"></div>

			    <div id="footer"><h3>Information at Footer</h3></div>            
        </div>
       
</div>
</body>
</html>

Open in new window

ashley2009

ASKER

Please respond; I prefer multiple people will respond and will give their best guess about the problems that are going on in these two project. My guesses are at below:

Case #1 ( AJAX and PHP implementation):

> As AJAX is sending 7 requests from the javascript to the php file one after another, some requests are getting lost and/or getting mixed with incoming responses that are coming back from the php file(server). Therefore, 60% of the time, there are errors like empty div cell or wrong information in div cell (i.e. cnn's information is at microsoft's information)

Case# 2 ( C# ASP .NET)

> What is the error?

If someone refreshes the page multiple time( i.e. 10 times in 10 seconds), an Error 400 happens, and as a result all the divs show Error code 400 and also from that computer or IP address, any rss feed of twitter becomes inaccessible.

I believe that this is happening as, I think, Twitter company, blocks that IP address for an hour that sends too much request ( maybe related to denial of service attack ). Maybe Twitter company thinks that I am tring to slow their server by sending too much requests by clicking refresh 10 times in 10 seconds, which is ( 10*7= 70 requests in 10 seconds).

However, I am not sure about my guesses as I never read anything about these. I just created these two projects, and I am facing this.

Please answer your best guesses and also please tell is there any way to do this type of project where there won't be errors like this situation.

I will hope to hear your guesses, your advise, and your comments. Please comment.

kawzaki

I have tried the second approach, it worked with me even with 10 times fast F5 (refresh)

but anyhow, i guess the problem is in the HTTP protocol , and thats why you got 400.
more details about HTTP protocol below.

also, the server might consider you an attacker trying DoS (denial of service) attack.
and the firewall bans requests from your IP for an HOUR.

details about http protocol:

* Obtain an IP address from the IP name of the site (the site URL without the leading 'http://'). This lookup (conversion of IP name to IP address) is provided by domain name servers (DNSs).
* Open an IP socket connection to that IP address.
* Write an HTTP data stream through that socket.
* Receive an HTTP data stream back from the Web server in response. This data stream contains status codes whose values are determined by the HTTP protocol. Parse this data stream for status codes and other useful information.

This error occurs in the final step above when the client receives an HTTP status code it recognises as '400'.

Hope this adds to your debugging efforts.

ashley2009

ASKER

Thank you for your response, but if I have one twitter feed instead 7, it works no matter how many time I click refresh.

I can understand somewhat the http protocol that you have talked about, but I do not understand when you said
"i guess the problem is in the HTTP protocol"

What is the problem?

http://twitter.com/statuses/user_timeline/15224867.rss <---- is a valid url

You said---
"Obtain an IP address from the IP name of the site (the site URL without the leading 'http://'). This lookup (conversion of IP name to IP address) is provided by domain name servers (DNSs)."

"This error occurs in the final step above when the client receives an HTTP status code it recognises as '400'."

How error can happen where
http://twitter.com/statuses/user_timeline/15224867.rss <-----

is a valid url.

I think it might be denial of service problem and twitter is perceiving me as a DOS attacker.

Am I right? Am i 100% right when I say that twitter is thinking maybe I may be a DOS attacker?

kawzaki

you are welcome.

yes, my first guess actually was DoS prevention by the server's firewall,
and the clue was getting banned for one hour.

Error 400 will be happening as explained.
This process of converting the Domain Name to an IP, takes fractions of seconds.
however, if the server thinks you're trying to attack, and process less number of requests (to the same domain, twitter in your case), HTTP protocol gets broken.

Basically, DoS leads to getting 400 in your browser.

ashley2009

ASKER

Hello kawzaki,

thank you; therefore, you and me, we are both sure that case# 2 is related to DOS issue, and, as a result, later HTTP protocol issue/problem appears.

Will you please see or look in my case # 1 and try to confirm your guess on my case # 1. I have also posted my guess on case # 1.

What do you think about my guess for case # 1?

Other Experts, please provide your guesses on case # 1 or if you want to add more on case # 2.

kawzaki

It's the same, changing the back-end programming language wont change the fact about overwhelming the server with several requests in specified interval.

:)

ashley2009

ASKER

What do you mean when you say:

"It's the same, changing the back-end programming language wont change the fact about overwhelming the server with several requests in specified interval."

Are you telling that as I am sending too much requests in case # 1, therefore, problem is happening, and case# 1's problem is related to case# 2's problem ?

I thought that at case # 1 the problem is happening as javascript is sending too much requests asynchronously ( one after another) while the responses are coming back from sever.

Responses are coming back too ate or too fast as ajax is sending more requests, and there is only one var object to get the responseText

myStr = xmlhttp.responseText;

For all these reason, problem is happening. However, these are just my guess. I think that I need 7 var object to hold incoming xmlhttp.responseText, or maybe I should look into and should try to use jquery to do this type of project, but that is a different subject.

Now I am just trying to be confirmed whether my guesses are correct.

ashley2009

ASKER

ASKER CERTIFIED SOLUTION

kawzaki

membership

This solution is only available to members.

To access this solution, you must be a member of Experts Exchange.

Start Free Trial

ashley2009

ASKER

Thank you for your response. How to make cached version of the responses? By java script or by php?

I think that you are trying to tell the following:

1) There will be 7 vars, in the javascript, for 7 xmlhttp.responseText, that will come from php
var1 = xmlhttp.responseText
var2 = xmlhttp.responseText
var3= xmlhttp.responseText
var4 = xmlhttp.responseText, etc

And when you say "adding cached version of the twitter", I think that you mean either
> to save those vars in cookie or
> in hidden variables/ cache?( and the variables/cache will be valid for 1 minute or for 1 hour)

Am I right? What do you mean by cache? I mean I know cache ( cached webpage, which browsers do), but I never created cache before though I created cookies. Please tell me more and/or provide information java script cache or php cache to save information for use....

I believe when you said cache, you said that I need to create cache in javascript or in php? I am not that clear about cache. Please comment.

kawzaki

you are welcome.

it is getting more technical now.
1- for caching,
you can cache the data in a cookie (small data) or in a file on the server (size is not limited).
my preference is of course on the server as a file.
it will serve all visitors, not only one visitor.

adding this cache means adding a layer of abstraction
changing the logic from fetching directly from source sites
to displaying already fetched data.

using php,
simply fetch the rss result, and save it as a file
yahoo.html
cnn.html
etc..

displaying the rss will be calling fetchDataByPhp.php
which must keep track of time and when to fetch data from the source to update the cache.

if no need to update, just get_file_contents() of the requested feed and return to be displayed.
otherwise, fetch the data, update the cache, and return result.

2- for using variable
it will be similar to that but not exactly as you wrote it.

DRY ( dont repeat yourself )
so think of reusing your code

function getPageByXMLHTTP( url )
{
// the code to send xmlhttp request and return result
}

mystr1 = getPageByXMLHTTP( cnnURL )
mystr2 = getPageByXMLHTTP( yahooURL )

and so on..

ashley2009

ASKER

I believe that I understand what you are saying. Now not only I have confirmed my guesses with your knowledge and more input, I have alternative ways for approaching this type of scenarios.

I am going to close this question in 2-4 hours in case you or other experts want to say more.