Need SID Killer code or workaround

Posted on 2005-04-28
Last Modified: 2010-04-09
Does anyone know of any code that can be used to get the search engines to ignore the SID that is created at the start of browsing.  The site is writing in HTML with a couple "shop" files written in Perl.  The shopping cart we are using is the Hassan cart, but we have certainly modified it.  

We have a site that has lots of content in html, but once google gets in there, a SID is attached to everything and it sure isn't helping our rankings.  We are trying to create a work around using a SID killer code, or if that doesn't work, routing the spider through a static sitemap link where the spider can browse the website (and we'll tell users not to, because it will dump their SID and their cart).  A

Anyone with any code or workaround suggestions.  If you need to see some code, let me know.

Question by:battlepigeons
    LVL 49

    Accepted Solution

    Personally I think that a webcrawler should not be able to spider your web basket, and should be limited to informational pages, and products pages only. Possibly you could the robots.txt to limit the search reach of the search engine spiders.

    LVL 1

    Author Comment

    Problem is, all product pages must be accessed through the Cgi-bin for example.  When you enter the website, the first button you click has to go through the cgi-bin to create a session.  That way, you can put one thing in the cart and then browse around to anyother page on the website (other products, about us, home) and the session isn't dropped.

    So you start off at:

    then you push any button on the website and it has the token that is through a perl file in the cgi-bin:


    Out comes:

    I've robots.txt'd it so that it can't enter the cgi-bin, but that still won't get my content spidered.  It will just stop the cacheing the SIDs.  So unless I want to create a sitemap for ONLY the spider, that will link to pages and content that will be pure html.....I need to find a way to deal with the spider/SID.
    LVL 4

    Assisted Solution

    Would it be possible to tell your CGI script that is initializing the sessions to not do so if the browser agent is googlebot etc?  I'm not a cgi/perl person at all, but this is how a similar thing can be done using PHP, which is what I use. Just a thought.


    LVL 1

    Author Comment

    Good idea...anyone know how to do that in perl?
    LVL 36

    Assisted Solution

    I'm no expert in Perl, but after searching around a bit, (as is often the case with Perl), someone has done the work already :)

    You might think about using HTTP::BrowserDetect

    Write Comment

    Please enter a first name

    Please enter a last name

    We will never share this with anyone.

    Featured Post

    Course: HTML5 Mobile App Development with PhoneGap

    PhoneGap can help you leverage your already existing HTML5, JavaScript, and CSS skills in order to create and deploy cross-platform mobile apps.This program comprehensively covers HTML5 mobile app development from top to bottom.

    Suggested Solutions

    "I want to put my photos online, but I don't want them stolen.  What settings should I use?" When You Put Photos Online First and foremost, any digital file published on the WWW can be copied, stored, modified, retransmitted, etc.  Remember Naps…
    Deprecated and Headed for the Dustbin By now, you have probably heard that some PHP features, while convenient, can also cause PHP security problems.  This article discusses one of those, called register_globals.  It is a thing you do not want.  …
    In this tutorial viewers will learn how to position overlapping items using z-index in CSS. They will also learn the restrictions on the z-index property.  Create a new HTML document with an internal stylesheet.: Create a div in CSS and name it Red.…
    In this tutorial viewers will learn how to style a corner ribbon overlay for an image using CSS Create a new class by typing ".Ribbon":  Define the class' "display:" as "inline-block": Define its "position:" as "relative": Define its "overflow:" as …

    759 members asked questions and received personalized solutions in the past 7 days.

    Join the community of 500,000 technology professionals and ask your questions.

    Join & Ask a Question

    Need Help in Real-Time?

    Connect with top rated Experts

    10 Experts available now in Live!

    Get 1:1 Help Now