?
Solved

How do you detect in Visual Basic when new questions are posted on the Experts Exchange website, like EE quickpost does ??

Posted on 2005-03-24
22
Medium Priority
?
374 Views
Last Modified: 2013-12-04
Can anyone point me in the right direction with regards on how to do this ? Preferably I want visual basic 6.0 tutorials or help in that area to do this :) However any help is appreicated !
0
Comment
Question by:gecko_au2003
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 9
  • 6
  • 3
  • +3
22 Comments
 
LVL 3

Expert Comment

by:r_a_j_e_s_h
ID: 13628042
hi
    u can get the source(HTML) of the webpage from VB.. u have to compare that with the previous version for the new questions table part... that will do...

regards
rajesh

0
 
LVL 23

Author Comment

by:gecko_au2003
ID: 13628216
Isnt there an easier way or another way of doing it ?
0
 
LVL 86

Assisted Solution

by:Mike Tomlinson
Mike Tomlinson earned 800 total points
ID: 13628218
As far as I know...

The way QuickPost works is it tracks the last known question number and then it increments that number and checks the response to see if a page is generated or not.  For example, this question is #21364302.  If you use this address:

    http://www.experts-exchange.com/Q_21364302.htm

You will be redirected to this address instead:

    http://www.experts-exchange.com/Programming/Q_21364302.html

So the next question after this one would be #21364303:

    http://www.experts-exchange.com/Q_21364303.htm

which resolves to:

    http://www.experts-exchange.com/Web/WebDevSoftware/ColdFusion/Q_21364303.html

So basically he uses a timer and whenever it fires he increments the question number and then parses part of the response string to determine whether that next number is open, closed, deleted or does not exist yet.

~IM

~IM
0
Learn how to optimize MySQL for your business need

With the increasing importance of apps & networks in both business & personal interconnections, perfor. has become one of the key metrics of successful communication. This ebook is a hands-on business-case-driven guide to understanding MySQL query parameter tuning & database perf

 
LVL 23

Author Comment

by:gecko_au2003
ID: 13628230
I increased points to 500, as I would like an example of how to do this in visual basic 6, in terms of checking for the question's status, ie open, closed, deleted or does not exist yet as well as doing the other stuff you explained IM, if that is ok:)
0
 
LVL 86

Expert Comment

by:Mike Tomlinson
ID: 13631120
I'm on vacation so I can't give you any specific code until late next week or the week after that.  I'm guessing you can simply use a WebBrowser control and look for keywords in the response strings.

Stay tuned...
0
 
LVL 23

Author Comment

by:gecko_au2003
ID: 13631166
ok, no problem IM, Have fun on your vacation :)
0
 
LVL 25

Expert Comment

by:InteractiveMind
ID: 13640838
I understand that you're after a VB solution -- but if you're not *too* fussed about the language, then I've just spent the past couple of hours creating this for you:



:======8<--------------[  EEPostScanner.java ]----------------
   
    /**
     *
     *  EE Spider.
     *  By R. Darkins -- InteractiveMind
     *  
     *  # Allows you to scan for the latest questions on the site.
     *  
     *  Time-taken to create: a few hours - lol.
     *  
     */
   
    import java.io.*;
    import java.net.*;
    import java.util.*;
   
    import java.awt.*;
    import java.awt.event.*;
    import javax.swing.*;
    import javax.swing.text.html.*;
    import javax.swing.text.*;
   
   
    public class EEPostScanner extends JFrame implements ActionListener
    {
       
        private int     iScrX           = (int ) Toolkit.getDefaultToolkit().getScreenSize().getWidth();
        private int     iScrY           = (int ) Toolkit.getDefaultToolkit().getScreenSize().getHeight();
       
        private int     iAppX           = 600;
        private int     iAppY           = 400;
       
        private JButton bGo             = new JButton("Go");
        private JButton bStop           = new JButton("Stop");
        private JButton bPause          = new JButton("Pause");
        private JButton bClear          = new JButton("Clear");
       
        private DefaultListModel model  = new DefaultListModel();
        private JList       list        = new JList(model);
        private JTextField  tThread     = new JTextField();
       
        private Thread      t;
        private Vector      v           = new Vector(10, 5);
       
        private boolean     paused      = false;
       
       
        public EEPostScanner ()
        {
            super( "EE QuickPost" );
            setBounds( (iScrX-iAppX)/2, (iScrY-iAppY)/2, iAppX, iAppY );
            setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
           
            JPanel pane = new JPanel();
            setContentPane(pane);
           
            GridBagLayout gridbag = new GridBagLayout();
            GridBagConstraints constraints = new GridBagConstraints();
           
            pane.setLayout(gridbag);
           
            JLabel label_start = new JLabel( "Starting Q: #" );
            buildConstraints( constraints, 0, 0, 1, 1, 40, 15 );
            constraints.fill = GridBagConstraints.NONE;
            constraints.anchor = GridBagConstraints.EAST;
            gridbag.setConstraints( label_start, constraints );
            pane.add( label_start );
           
            tThread.setForeground(new Color(60, 112, 182));         // RGB
            buildConstraints( constraints, 1, 0, 2, 1, 40, 15 );
            constraints.fill = GridBagConstraints.HORIZONTAL;
            gridbag.setConstraints( tThread, constraints );
            pane.add( tThread );
           
            buildConstraints( constraints, 3, 0, 1, 1, 20, 15 );
            constraints.fill = GridBagConstraints.NONE;
            constraints.anchor = GridBagConstraints.CENTER;
            gridbag.setConstraints( bGo, constraints );
            pane.add( bGo );
           
            JScrollPane scroller = new JScrollPane(list);
            buildConstraints( constraints, 0, 1, 4, 1, 100, 70 );
            constraints.fill = GridBagConstraints.BOTH;
            gridbag.setConstraints( scroller, constraints );
            pane.add( scroller );
           
            buildConstraints( constraints, 0, 2, 1, 1, 40, 15 );
            constraints.fill = GridBagConstraints.NONE;
            constraints.anchor = GridBagConstraints.CENTER;
            gridbag.setConstraints( bStop, constraints );
            pane.add( bStop );
           
            buildConstraints( constraints, 1, 2, 1, 1, 20, 15 );
            gridbag.setConstraints( bPause, constraints );
            pane.add( bPause );
           
            buildConstraints( constraints, 2, 2, 1, 1, 20, 15 );
            gridbag.setConstraints( bClear, constraints );
            pane.add( bClear );
           
            bGo.addActionListener(this);
            bStop.addActionListener(this);
            bPause.addActionListener(this);
            bClear.addActionListener(this);
           
            list.addMouseListener(new MouseAdapter() {
                public void mouseClicked(MouseEvent evt) {
                    JList list = (JList)evt.getSource();
                    if (evt.getClickCount() == 2) {          // Double-click
                        // Get item index
                        int index = list.locationToIndex(evt.getPoint());
                       
                        try
                        {
                            java.lang.Process proc = Runtime.getRuntime().exec( "rundll32 url.dll,FileProtocolHandler \"http://www.experts-exchange.com/Q_" + v.get(index) + ".html\"" );
                            proc.waitFor();
                        } catch (Exception e){}
                    }
                }
            });
           
           
            setVisible(true);
        }
       
        public void actionPerformed( ActionEvent evt )
        {
            Object source = evt.getSource();
           
            if ( source == bGo )
            {
                String sQ = tThread.getText();
                int iQ;
                try{
                    iQ = Integer.parseInt(sQ);
                    t = new Process( model, iQ, v );
                    t.start();
                }catch (Exception e) { e.printStackTrace(); }
            } else if ( source == bStop )
            {
                if ( t.isAlive() )
                {
                    t.stop();
                }
            } else if ( source == bPause )
            {
                if ( t.isAlive() )
                {
                    if ( paused )
                    {
                        t.resume();
                        paused =! paused;
                        bPause.setText("Pause");
                    }
                    else
                    {
                        t.suspend();
                        paused =! paused;
                        bPause.setText("Resume");
                    }
                }
            } else if ( source == bClear )
            {
                if ( v.size() > 0 )
                {
                    v.clear();
                    model.clear();
                }
            }
        }
       
        public static void main ( String [] args )
        {
            new EEPostScanner();
        }
       
       
        private void buildConstraints( GridBagConstraints gbc, int gx, int gy, int gw, int gh, int wx, int wy )
        {
            gbc.gridx = gx;
            gbc.gridy = gy;
            gbc.gridwidth = gw;
            gbc.gridheight = gh;
            gbc.weightx = wx;
            gbc.weighty = wy;
        }
       
       
    class Process extends Thread implements ActionListener
    {
       
        private DefaultListModel model;
        private int              startingThread;
        private Vector           v;
        private JButton          cont;
        private boolean          add = false;
       
        public Process( DefaultListModel model, int startingThread, Vector v )
        {
            this.model = model;
            this.startingThread = startingThread;
            this.v = v;
        }
       
        public void run ()
        {
            for ( int i = startingThread; ; i++ )
            {
                String url      = "http://www.experts-exchange.com/Q_" + i + ".html";
                String someHtml = getText( url );
               
                if ( someHtml != "deleted" )
                {
                    if ( someHtml != null )
                    {
                        String titleStartTag = "<title>";
                        String titleEndTag = "</title>";
                       
                        int start = someHtml.indexOf(titleStartTag);
                        int end = someHtml.indexOf(titleEndTag);
                       
                        if (start != -1 && end !=-1)
                        {
                            String titleText = someHtml.substring(start + titleStartTag.length(), end);
                           
                            model.add( model.getSize(), new String(titleText) );
                            v.add(v.size(), i);
                        }
                    } else
                    {
                        JWindow win = new JWindow();
                        win.setBounds( ((int ) Toolkit.getDefaultToolkit().getScreenSize().getWidth()-300)/2,
                                       ((int ) Toolkit.getDefaultToolkit().getScreenSize().getHeight()-200)/2,
                                       300,
                                       200 );
                        JPanel pane = new JPanel();
                        win.setContentPane(pane);
                        JLabel mess = new JLabel("Question: #" + i + " cannot be found.");
                        mess.setForeground(Color.WHITE);
                        pane.add(mess);
                        cont = new JButton("Continue");
                        pane.add(cont);
                        cont.addActionListener(this);
                        pane.setBackground(Color.RED);
                        win.pack();
                        win.setVisible(true);
                        try{Thread.sleep(5000);} catch (Exception e){}
                       
                        if ( add )
                        {
                            i++;
                            add = false;
                        }
                       
                        win.setVisible(false);
                       
                        i--;
                    }
                }
               
                try{ Thread.sleep(300); } catch (Exception e){}
               
            }
           
        }
       
        public void actionPerformed( ActionEvent evt )
        {
            if ( evt.getSource() == cont )
                add = true;
        }
       
        public String getText(String uriStr) {
           
            URLConnection conn = null;
            InputStreamReader in;
            BufferedReader data;
            String line;
            try
            {
                URL page = new URL(uriStr);
               
                conn = page.openConnection();
                conn.connect();
               
                in = new InputStreamReader(conn.getInputStream());
                data = new BufferedReader(in);
               
                while ( (line = data.readLine()) != null )
                {
                    if ( line.indexOf("<title>") > 0 )
                        return line;
                }
            } catch (Exception e)
            {
                String sRet = e.getMessage();
                if ( sRet.indexOf("code: 403") > 0 )
                    return "deleted";
            }
            return null;
        }
       
    }
   
    }
   
:======8<--------------[  EEPostScanner.java ]----------------


It's just over 300 lines.

But despite "doing it for you", I'll probably use it also... So, only accept my answer, if you really think that it's good, and it helps some what..

Best of luck :)
>> IM
0
 
LVL 25

Expert Comment

by:InteractiveMind
ID: 13640839
*.. As you can see, it's in Java ..*
0
 
LVL 86

Expert Comment

by:Mike Tomlinson
ID: 13640970
LOL...you have an Idle and an Interactive mind helping you out...
0
 
LVL 23

Author Comment

by:gecko_au2003
ID: 13642138
I would prefer a VB solution so that I can add other features to it, however what compiler can you reccomend that I use to get that code in java to work ?

Thanks for the help !

Just to let everyone know I am travelling back to england in about 10 hours time so I wont be able to reply for a few days ! I hope this isnt a problem !
0
 
LVL 25

Expert Comment

by:InteractiveMind
ID: 13642400
lol -- I'll sign myself as "Inter" in this thread, from now on... because your "~IM" and my ">> IM" are very, very similar  lol.

That's no problem, Shane. Have a good flight (although, by the time that I've submitted this message, you're probably already here lol).

To compile that, and to run it (and also to develop your own Java programs one day), all you need is the JDK (Java Development Kit). Just go to this link, and you can download it: http://192.18.97.239/ECom/EComTicketServlet/BEGIN69AA94FED7F9E76CA98AEA990ADD4CFC/-2147483648/790021419/1/573074/ +
572810/790021419/2ts+/westCoastFSEND/j2sdk-1.4.2_07-oth-JPR/j2sdk-1.4.2_07-oth-JPR:14/j2sdk-1_4_2_07-windows-i586-p-iftw.exe
(It would seem obvious to me that you're on Windows.. but for anyone who want the JDK, that are *not* on Windows, then go here: https://jsecom16k.sun.com/ECom/EComActionServlet/LegalPage:~:com.sun.sunit.sdlc.content.LegalWebPageInfo; + jsessionid=69AA94FED7F9E76CA98AEA990ADD4CFC;jsessionid=69AA94FED7F9E76CA98AEA990ADD4CFC ).

If I remember correctly, on dial-up, it will take about an hour (not so long ago, I had dial-up.. it ran at 46kb/s  LOL -- but finally, an ISP started to offer 1mp/s in my area...... I doubt that many people appreciate BB as much as me - lol - after a few years on 46k!).

I'll just quickly re-cap what my Java program does/can do:

It has a textfield, that you type in a number (the Thread ID) to start at. For example: 21364302, then you hit the "Go" button - and it will list the Title and Topic Area, of every thread that carries on from that number, 1 by 1.. It skips all "Deleted" threads, by detecting a Server Error code of 403 (which indicates a deleted thread).. It will also detect once a number/thread ID has been reached, which does not yet exist... In this case, it does a few things:
   It displays a small window on the screen, which says "Question #(the Thread ID) cannot be found." It then has a button called "Continue", if you click that, it will check the next Thread ID....
   If you don't click the "Continue" button, it will keep looping, checking to see if this thread has been created yet.. once it has, it will again, be added to the List, in the application.
With this List, you can double click any of the Qs (in the List), it will then open that Q in a browser..

There's also a "Stop" button, so that you can stop it from scanning from Qs, and change the number to start scanning from... btw: You can get it to scan, several ranges at once (it's multi-threaded), so this speeds up the scanning process - however, in this case, the "Stop" button becomes useless... because my intentions were only for you to use 1 additional thread to scan, at a time.. but if you use several, the "Stop" button will only stop one of the threads, then be rendered useless.
There's also a "Pause" button, which as you can probably guess, pauses the Scanning thread, incase you need to evaluate each thread as it's been detected, but need to quickly leave the room - you can pause it.. the pause button will then become a "Resume" button - which I think is obvious what it does. :)
And finally, you have a "Clear" button - which clears all of the detected threads so far, from the List, and from memory.. so if there's *loads* appearing, and they're annoying you, just click on the "Clear" button, and what has appeared so far, will dissapear :)  (I recommend that you do this at least once an hour... because everytime a thread is found, it adds a number to memory, to be recalled later.. So, the "Clear" button also flushes the memory being used..

That's about it..

Oh, before compiling it, change this line:

    private Vector      v           = new Vector(10, 5);

(nearish the top of the code -- it's a class instance).. Change it to:

    private Vector      v           = new Vector(30, 15);

That should make it a little more efficient, and suitable for this type of application.


Good luck! :)
>> IM
0
 
LVL 25

Expert Comment

by:InteractiveMind
ID: 13642403
Oops... I mean:

Good luck! :)
_Inter

hehe ^_^
0
 
LVL 15

Expert Comment

by:Ralf Klatt
ID: 13645237
Hi Gecko,

... hve a good flight!


Best regards,
Raisor
0
 
LVL 11

Expert Comment

by:bingie
ID: 13646325
Gecko, why reinvent the wheel? Quickpost already does this...



0
 
LVL 23

Author Comment

by:gecko_au2003
ID: 13655040
Thanks Raisor, I want to re make it because I am always having problems with EE Quickpost saying I am not logged in which is un true because I have logged in to the website in an internet explorer window behind the application.

It says about clearing temp files, making sure that your cookies are enabled and I went through all of that to no avail. So I wanted to make it with a web control instead of what they did so that I could attempt to do it so that I would not have any errors with regards to logging in, I also wanted to make it so that it would give the user the option of either having the URL they can click on to reply to it or a submit box like they have provided which posts straight to the questoin which you have selected from the list of questions it found that were new( Had just been posted ).

BTW - I am back now and I will await any further answers / suggestions , if any and will go from there :) Thanks for everyones help btw :)
0
 
LVL 23

Author Comment

by:gecko_au2003
ID: 13655211
If I do eventually make it I will post it in the support Area and see what the admins think to it before I release it as I do not want to go against any EE rules or anything like that ! I will take a look at that URL after I post this :)
0
 
LVL 25

Expert Comment

by:InteractiveMind
ID: 13662324
> I've edited InteractiveMind's post above to get rid of the horizonal scrolling
Cheers Netminder :)

If the use of this application lags the servers at all, then if anyone insists on using it, I recommend that you change this line:

     try{ Thread.sleep(300); } catch (Exception e){}

to this:

     try{ Thread.sleep(1000); } catch (Exception e){}

Or, if you're feeling really kind, then increase the "1000" to something even higher :)   (That is the delay in Milliseconds, between each connection attempt).

>> IM
0
 
LVL 23

Author Comment

by:gecko_au2003
ID: 13662387
I am going to wait until I get a visual basic reply to see if I can do it in visual basic because I have no clue how I will link 2 programs together, ie the one in java to detect the new questions and then the browser in visual basic 6.0.
0
 
LVL 23

Author Comment

by:gecko_au2003
ID: 13744625
Are there any suggestions at all with regards to this in a visual basic 6.0 sense ie a visual basic 6.0 reply with pointers to examples, source code or at least pointers in the correct direction with regards to what API's or even what controls I would need to use or anything that would be of use to me making this ?
0
 
LVL 25

Accepted Solution

by:
InteractiveMind earned 1200 total points
ID: 13746512
I suppose that you just need to search EE's VB Knowledge Base for the following:

  Get webpage code
  Parse text
  Open browser
  GUI list

that sort of thing.. then just put it all together....? ;)
0
 
LVL 23

Author Comment

by:gecko_au2003
ID: 13824645
I can't figure it out exactly, but when I get time to sit down and actually do some hands on planning , coding , etc. I will post a Q related to this :) thanks for the help everyone !
0

Featured Post

Optimize your web performance

What's in the eBook?
- Full list of reasons for poor performance
- Ultimate measures to speed things up
- Primary web monitoring types
- KPIs you should be monitoring in order to increase your ROI

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Entering a date in Microsoft Access can be tricky. A typo can cause month and day to be shuffled, entering the day only causes an error, as does entering, say, day 31 in June. This article shows how an inputmask supported by code can help the user a…
Since upgrading to Office 2013 or higher installing the Smart Indenter addin will fail. This article will explain how to install it so it will work regardless of the Office version installed.
In this fourth video of the Xpdf series, we discuss and demonstrate the PDFinfo utility, which retrieves the contents of a PDF's Info Dictionary, as well as some other information, including the page count. We show how to isolate the page count in a…
Introduction to Processes
Suggested Courses

752 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question