?
Solved

http web request

Posted on 2011-03-25
17
Medium Priority
?
461 Views
Last Modified: 2013-11-27
guys i want a list of url to loop by http web request and get the source code in text box one by one


please check the code cause if i click on each url in list box one by one i do get source code in text box but in loop i does not gives so how can i get that fixed and how can i add a delay between to requests?
For a = 0 To ListBox1.Items.Count - 1

            Dim i As String

            i = ListBox1.Text

            Dim request As System.Net.HttpWebRequest = System.Net.HttpWebRequest.Create("http://" & ListBox1.Text)

            Dim response As System.Net.HttpWebResponse = request.GetResponse
            request.Timeout = 8


            Dim sr As System.IO.StreamReader = New System.IO.StreamReader(response.GetResponseStream())





            Dim source As String = sr.ReadToEnd

            TextBox1.Text = source
        
            ListBox1.SelectedIndex = ListBox1.SelectedIndex + 1



        Next

Open in new window

0
Comment
Question by:WILLS88
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 9
  • 8
17 Comments
 
LVL 23

Expert Comment

by:Jens Fiederer
ID: 35218971
You could use a Sleep method (see the Thread functions) to hesitate in between, or you could concatenate the different source codes in the textbox instead of wiping the previous one out each time (TextBox1.Text = TextBox1.Text + source) or you could use a whole bunch of text boxes, putting one source in each....
0
 

Author Comment

by:WILLS88
ID: 35220946
nop i have to wipe out the old source each time in where i should add sleep funtion i know that
but even when i add the sleep function it does not shows source code in box.
0
 

Author Comment

by:WILLS88
ID: 35220950
see this is the code and GUI i want it like

1.make request for a web in listbox
2.gets is source code in text box
3.after getting source code wait for  few seconds
4.then make request for next web in listbox and loop the whole process for all webs in listbox
loop.jpg
0
Cloud Training Guides

FREE GUIDES: In-depth and hand-crafted Linux, AWS, OpenStack, DevOps, Azure, and Cloud training guides created by Linux Academy instructors and the community.

 
LVL 23

Expert Comment

by:Jens Fiederer
ID: 35222621
This here works for me.

Remember to actually iterate through the items of the list box, and to refresh the textbox so the user can see the contents.
Private Sub Form1_Load(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles MyBase.Load
        ListBox1.Items.Add("www.google.com")
        ListBox1.Items.Add("www.yahoo.com")
        ListBox1.Items.Add("www.facebook.com")
        ListBox1.Items.Add("www.bing.com")
        ListBox1.Items.Add("www.myspace.com")
    End Sub

    Private Sub Button1_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Button1.Click
        For a = 0 To ListBox1.Items.Count - 1

            Dim request As System.Net.HttpWebRequest = System.Net.HttpWebRequest.Create("http://" & ListBox1.Items(a))
            Dim response As System.Net.HttpWebResponse = request.GetResponse

            Dim sr As System.IO.StreamReader = New System.IO.StreamReader(response.GetResponseStream())


            Dim source As String = sr.ReadToEnd

            TextBox1.Text = source
            TextBox1.Refresh()

            ListBox1.SelectedIndex = ListBox1.SelectedIndex + 1
            Threading.Thread.Sleep(5000)



        Next
    End Sub

Open in new window

0
 

Author Comment

by:WILLS88
ID: 35223051
ok this worked fine bro the next issue is exceptions sometimes i get 403 or 500 http error and project does not get the source code of page i want it like even if response is 403 or 500 project keeps on going and the source of that page on which 403 or 500 showed comes in text box
0
 
LVL 23

Expert Comment

by:Jens Fiederer
ID: 35223075
You can use a try - catch block to keep going when an error occurs - but that is really a new question.
0
 

Author Comment

by:WILLS88
ID: 35223230
o ok i`ll make aother thread for this the loop is working fine but the condition is not working i added a condition
how to make that work?
Private Sub Button1_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Button1.Click
        For a = 0 To ListBox1.Items.Count - 1

            Dim request As System.Net.HttpWebRequest = System.Net.HttpWebRequest.Create("http://" & ListBox1.Items(a))
            Dim response As System.Net.HttpWebResponse = request.GetResponse

            Dim sr As System.IO.StreamReader = New System.IO.StreamReader(response.GetResponseStream())


            Dim source As String = sr.ReadToEnd

            TextBox1.Text = source
            TextBox1.Refresh()

if texbox1.text.contains("keyword") then
listbox2.items.add(listbox1.selecteditem)
endif

            ListBox1.SelectedIndex = ListBox1.SelectedIndex + 1
            Threading.Thread.Sleep(5000)



        Next
    End Sub

Open in new window

0
 

Author Comment

by:WILLS88
ID: 35223286
well even though the text box contains the keyword it jumps to next web and gets its source i want to add that web which have that keyword in sourcecode and then move to next site get its code and check for keyword and loop on all sites
0
 
LVL 23

Expert Comment

by:Jens Fiederer
ID: 35223313
Not sure what you are trying to do here, but all your condition does is to add an item (the selected item, not necessarily the item you were on) to listbox2, not affecting the flow of control any other way.
0
 

Author Comment

by:WILLS88
ID: 35223399
well its not adding the selected item


and i want to do in whole project is

1.go to website in listbox
2.get source code in text box
3.check the condition for text in textbox
4.if condition is true then add that website in listbox2
5.wait for  like 5 sec
6.jump to next website and loop the process

does that makes sense?
0
 
LVL 23

Expert Comment

by:Jens Fiederer
ID: 35223416
Maybe there isn't any selected item when you try to add?

You really should do listbox2.items.add(ListBox1.Items(a))
0
 

Author Comment

by:WILLS88
ID: 35223540
nop not working that way even
0
 
LVL 23

Expert Comment

by:Jens Fiederer
ID: 35223563
are you refreshing the listbox2 in between loop iterations?
0
 

Author Comment

by:WILLS88
ID: 35223820
no listbox2 is not to be refreshed i should work like each new website with the keyword comes under the first one like

www.site1.com
www.site2.com

and so on
0
 
LVL 23

Expert Comment

by:Jens Fiederer
ID: 35223825
I mean to make sure it is displayed, just like you needed to refresh the textbox.
0
 

Author Comment

by:WILLS88
ID: 35225680
i did not get your point :(
0
 
LVL 23

Accepted Solution

by:
Jens Fiederer earned 200 total points
ID: 35226330
Like attached.  Of course, you might simply not be finding "keyword"
If source.Contains("keyword") Then
                ListBox2.Items.Add(ListBox1.Items(a))
                ListBox2.Refresh()
            End If

Open in new window

0

Featured Post

CHALLENGE LAB: Troubleshooting Connectivity Issues

Goal: Fix the connectivity issue in the lab's AWS environment so that you can SSH into the provided EC2 instance.  

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Today I had a very interesting conundrum that had to get solved quickly. Needless to say, it wasn't resolved quickly because when we needed it we were very rushed, but as soon as the conference call was over and I took a step back I saw the correct …
This article describes some techniques which will make your VBA or Visual Basic Classic code easier to understand and maintain, whether by you, your replacement, or another Experts-Exchange expert.
As developers, we are not limited to the functions provided by the VBA language. In addition, we can call the functions that are part of the Windows operating system. These functions are part of the Windows API (Application Programming Interface). U…
Get people started with the process of using Access VBA to control Excel using automation, Microsoft Access can control other applications. An example is the ability to programmatically talk to Excel. Using automation, an Access application can laun…
Suggested Courses
Course of the Month12 days, 3 hours left to enroll

752 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question