Solved

http web request

Posted on 2011-03-25
17
456 Views
Last Modified: 2013-11-27
guys i want a list of url to loop by http web request and get the source code in text box one by one


please check the code cause if i click on each url in list box one by one i do get source code in text box but in loop i does not gives so how can i get that fixed and how can i add a delay between to requests?
For a = 0 To ListBox1.Items.Count - 1

            Dim i As String

            i = ListBox1.Text

            Dim request As System.Net.HttpWebRequest = System.Net.HttpWebRequest.Create("http://" & ListBox1.Text)

            Dim response As System.Net.HttpWebResponse = request.GetResponse
            request.Timeout = 8


            Dim sr As System.IO.StreamReader = New System.IO.StreamReader(response.GetResponseStream())





            Dim source As String = sr.ReadToEnd

            TextBox1.Text = source
        
            ListBox1.SelectedIndex = ListBox1.SelectedIndex + 1



        Next

Open in new window

0
Comment
Question by:WILLS88
  • 9
  • 8
17 Comments
 
LVL 23

Expert Comment

by:Jens Fiederer
ID: 35218971
You could use a Sleep method (see the Thread functions) to hesitate in between, or you could concatenate the different source codes in the textbox instead of wiping the previous one out each time (TextBox1.Text = TextBox1.Text + source) or you could use a whole bunch of text boxes, putting one source in each....
0
 

Author Comment

by:WILLS88
ID: 35220946
nop i have to wipe out the old source each time in where i should add sleep funtion i know that
but even when i add the sleep function it does not shows source code in box.
0
 

Author Comment

by:WILLS88
ID: 35220950
see this is the code and GUI i want it like

1.make request for a web in listbox
2.gets is source code in text box
3.after getting source code wait for  few seconds
4.then make request for next web in listbox and loop the whole process for all webs in listbox
loop.jpg
0
How our DevOps Teams Maximize Uptime

Our Dev teams are like yours. They’re continually cranking out code for new features/bugs fixes, testing, deploying, responding to production monitoring events and more. It’s complex. So, we thought you’d like to see what’s working for us. Read the use case whitepaper.

 
LVL 23

Expert Comment

by:Jens Fiederer
ID: 35222621
This here works for me.

Remember to actually iterate through the items of the list box, and to refresh the textbox so the user can see the contents.
Private Sub Form1_Load(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles MyBase.Load
        ListBox1.Items.Add("www.google.com")
        ListBox1.Items.Add("www.yahoo.com")
        ListBox1.Items.Add("www.facebook.com")
        ListBox1.Items.Add("www.bing.com")
        ListBox1.Items.Add("www.myspace.com")
    End Sub

    Private Sub Button1_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Button1.Click
        For a = 0 To ListBox1.Items.Count - 1

            Dim request As System.Net.HttpWebRequest = System.Net.HttpWebRequest.Create("http://" & ListBox1.Items(a))
            Dim response As System.Net.HttpWebResponse = request.GetResponse

            Dim sr As System.IO.StreamReader = New System.IO.StreamReader(response.GetResponseStream())


            Dim source As String = sr.ReadToEnd

            TextBox1.Text = source
            TextBox1.Refresh()

            ListBox1.SelectedIndex = ListBox1.SelectedIndex + 1
            Threading.Thread.Sleep(5000)



        Next
    End Sub

Open in new window

0
 

Author Comment

by:WILLS88
ID: 35223051
ok this worked fine bro the next issue is exceptions sometimes i get 403 or 500 http error and project does not get the source code of page i want it like even if response is 403 or 500 project keeps on going and the source of that page on which 403 or 500 showed comes in text box
0
 
LVL 23

Expert Comment

by:Jens Fiederer
ID: 35223075
You can use a try - catch block to keep going when an error occurs - but that is really a new question.
0
 

Author Comment

by:WILLS88
ID: 35223230
o ok i`ll make aother thread for this the loop is working fine but the condition is not working i added a condition
how to make that work?
Private Sub Button1_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Button1.Click
        For a = 0 To ListBox1.Items.Count - 1

            Dim request As System.Net.HttpWebRequest = System.Net.HttpWebRequest.Create("http://" & ListBox1.Items(a))
            Dim response As System.Net.HttpWebResponse = request.GetResponse

            Dim sr As System.IO.StreamReader = New System.IO.StreamReader(response.GetResponseStream())


            Dim source As String = sr.ReadToEnd

            TextBox1.Text = source
            TextBox1.Refresh()

if texbox1.text.contains("keyword") then
listbox2.items.add(listbox1.selecteditem)
endif

            ListBox1.SelectedIndex = ListBox1.SelectedIndex + 1
            Threading.Thread.Sleep(5000)



        Next
    End Sub

Open in new window

0
 

Author Comment

by:WILLS88
ID: 35223286
well even though the text box contains the keyword it jumps to next web and gets its source i want to add that web which have that keyword in sourcecode and then move to next site get its code and check for keyword and loop on all sites
0
 
LVL 23

Expert Comment

by:Jens Fiederer
ID: 35223313
Not sure what you are trying to do here, but all your condition does is to add an item (the selected item, not necessarily the item you were on) to listbox2, not affecting the flow of control any other way.
0
 

Author Comment

by:WILLS88
ID: 35223399
well its not adding the selected item


and i want to do in whole project is

1.go to website in listbox
2.get source code in text box
3.check the condition for text in textbox
4.if condition is true then add that website in listbox2
5.wait for  like 5 sec
6.jump to next website and loop the process

does that makes sense?
0
 
LVL 23

Expert Comment

by:Jens Fiederer
ID: 35223416
Maybe there isn't any selected item when you try to add?

You really should do listbox2.items.add(ListBox1.Items(a))
0
 

Author Comment

by:WILLS88
ID: 35223540
nop not working that way even
0
 
LVL 23

Expert Comment

by:Jens Fiederer
ID: 35223563
are you refreshing the listbox2 in between loop iterations?
0
 

Author Comment

by:WILLS88
ID: 35223820
no listbox2 is not to be refreshed i should work like each new website with the keyword comes under the first one like

www.site1.com
www.site2.com

and so on
0
 
LVL 23

Expert Comment

by:Jens Fiederer
ID: 35223825
I mean to make sure it is displayed, just like you needed to refresh the textbox.
0
 

Author Comment

by:WILLS88
ID: 35225680
i did not get your point :(
0
 
LVL 23

Accepted Solution

by:
Jens Fiederer earned 50 total points
ID: 35226330
Like attached.  Of course, you might simply not be finding "keyword"
If source.Contains("keyword") Then
                ListBox2.Items.Add(ListBox1.Items(a))
                ListBox2.Refresh()
            End If

Open in new window

0

Featured Post

Master Your Team's Linux and Cloud Stack!

The average business loses $13.5M per year to ineffective training (per 1,000 employees). Keep ahead of the competition and combine in-person quality with online cost and flexibility by training with Linux Academy.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

This article describes some techniques which will make your VBA or Visual Basic Classic code easier to understand and maintain, whether by you, your replacement, or another Experts-Exchange expert.
If you need to start windows update installation remotely or as a scheduled task you will find this very helpful.
Get people started with the process of using Access VBA to control Outlook using automation, Microsoft Access can control other applications. An example is the ability to programmatically talk to Microsoft Outlook. Using automation, an Access applic…
Get people started with the utilization of class modules. Class modules can be a powerful tool in Microsoft Access. They allow you to create self-contained objects that encapsulate functionality. They can easily hide the complexity of a process from…

839 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question