Solved

http web request

Posted on 2011-03-25
17
454 Views
Last Modified: 2013-11-27
guys i want a list of url to loop by http web request and get the source code in text box one by one


please check the code cause if i click on each url in list box one by one i do get source code in text box but in loop i does not gives so how can i get that fixed and how can i add a delay between to requests?
For a = 0 To ListBox1.Items.Count - 1

            Dim i As String

            i = ListBox1.Text

            Dim request As System.Net.HttpWebRequest = System.Net.HttpWebRequest.Create("http://" & ListBox1.Text)

            Dim response As System.Net.HttpWebResponse = request.GetResponse
            request.Timeout = 8


            Dim sr As System.IO.StreamReader = New System.IO.StreamReader(response.GetResponseStream())





            Dim source As String = sr.ReadToEnd

            TextBox1.Text = source
        
            ListBox1.SelectedIndex = ListBox1.SelectedIndex + 1



        Next

Open in new window

0
Comment
Question by:WILLS88
  • 9
  • 8
17 Comments
 
LVL 23

Expert Comment

by:Jens Fiederer
Comment Utility
You could use a Sleep method (see the Thread functions) to hesitate in between, or you could concatenate the different source codes in the textbox instead of wiping the previous one out each time (TextBox1.Text = TextBox1.Text + source) or you could use a whole bunch of text boxes, putting one source in each....
0
 

Author Comment

by:WILLS88
Comment Utility
nop i have to wipe out the old source each time in where i should add sleep funtion i know that
but even when i add the sleep function it does not shows source code in box.
0
 

Author Comment

by:WILLS88
Comment Utility
see this is the code and GUI i want it like

1.make request for a web in listbox
2.gets is source code in text box
3.after getting source code wait for  few seconds
4.then make request for next web in listbox and loop the whole process for all webs in listbox
loop.jpg
0
 
LVL 23

Expert Comment

by:Jens Fiederer
Comment Utility
This here works for me.

Remember to actually iterate through the items of the list box, and to refresh the textbox so the user can see the contents.
Private Sub Form1_Load(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles MyBase.Load
        ListBox1.Items.Add("www.google.com")
        ListBox1.Items.Add("www.yahoo.com")
        ListBox1.Items.Add("www.facebook.com")
        ListBox1.Items.Add("www.bing.com")
        ListBox1.Items.Add("www.myspace.com")
    End Sub

    Private Sub Button1_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Button1.Click
        For a = 0 To ListBox1.Items.Count - 1

            Dim request As System.Net.HttpWebRequest = System.Net.HttpWebRequest.Create("http://" & ListBox1.Items(a))
            Dim response As System.Net.HttpWebResponse = request.GetResponse

            Dim sr As System.IO.StreamReader = New System.IO.StreamReader(response.GetResponseStream())


            Dim source As String = sr.ReadToEnd

            TextBox1.Text = source
            TextBox1.Refresh()

            ListBox1.SelectedIndex = ListBox1.SelectedIndex + 1
            Threading.Thread.Sleep(5000)



        Next
    End Sub

Open in new window

0
 

Author Comment

by:WILLS88
Comment Utility
ok this worked fine bro the next issue is exceptions sometimes i get 403 or 500 http error and project does not get the source code of page i want it like even if response is 403 or 500 project keeps on going and the source of that page on which 403 or 500 showed comes in text box
0
 
LVL 23

Expert Comment

by:Jens Fiederer
Comment Utility
You can use a try - catch block to keep going when an error occurs - but that is really a new question.
0
 

Author Comment

by:WILLS88
Comment Utility
o ok i`ll make aother thread for this the loop is working fine but the condition is not working i added a condition
how to make that work?
Private Sub Button1_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Button1.Click
        For a = 0 To ListBox1.Items.Count - 1

            Dim request As System.Net.HttpWebRequest = System.Net.HttpWebRequest.Create("http://" & ListBox1.Items(a))
            Dim response As System.Net.HttpWebResponse = request.GetResponse

            Dim sr As System.IO.StreamReader = New System.IO.StreamReader(response.GetResponseStream())


            Dim source As String = sr.ReadToEnd

            TextBox1.Text = source
            TextBox1.Refresh()

if texbox1.text.contains("keyword") then
listbox2.items.add(listbox1.selecteditem)
endif

            ListBox1.SelectedIndex = ListBox1.SelectedIndex + 1
            Threading.Thread.Sleep(5000)



        Next
    End Sub

Open in new window

0
 

Author Comment

by:WILLS88
Comment Utility
well even though the text box contains the keyword it jumps to next web and gets its source i want to add that web which have that keyword in sourcecode and then move to next site get its code and check for keyword and loop on all sites
0
Maximize Your Threat Intelligence Reporting

Reporting is one of the most important and least talked about aspects of a world-class threat intelligence program. Here’s how to do it right.

 
LVL 23

Expert Comment

by:Jens Fiederer
Comment Utility
Not sure what you are trying to do here, but all your condition does is to add an item (the selected item, not necessarily the item you were on) to listbox2, not affecting the flow of control any other way.
0
 

Author Comment

by:WILLS88
Comment Utility
well its not adding the selected item


and i want to do in whole project is

1.go to website in listbox
2.get source code in text box
3.check the condition for text in textbox
4.if condition is true then add that website in listbox2
5.wait for  like 5 sec
6.jump to next website and loop the process

does that makes sense?
0
 
LVL 23

Expert Comment

by:Jens Fiederer
Comment Utility
Maybe there isn't any selected item when you try to add?

You really should do listbox2.items.add(ListBox1.Items(a))
0
 

Author Comment

by:WILLS88
Comment Utility
nop not working that way even
0
 
LVL 23

Expert Comment

by:Jens Fiederer
Comment Utility
are you refreshing the listbox2 in between loop iterations?
0
 

Author Comment

by:WILLS88
Comment Utility
no listbox2 is not to be refreshed i should work like each new website with the keyword comes under the first one like

www.site1.com
www.site2.com

and so on
0
 
LVL 23

Expert Comment

by:Jens Fiederer
Comment Utility
I mean to make sure it is displayed, just like you needed to refresh the textbox.
0
 

Author Comment

by:WILLS88
Comment Utility
i did not get your point :(
0
 
LVL 23

Accepted Solution

by:
Jens Fiederer earned 50 total points
Comment Utility
Like attached.  Of course, you might simply not be finding "keyword"
If source.Contains("keyword") Then
                ListBox2.Items.Add(ListBox1.Items(a))
                ListBox2.Refresh()
            End If

Open in new window

0

Featured Post

Enabling OSINT in Activity Based Intelligence

Activity based intelligence (ABI) requires access to all available sources of data. Recorded Future allows analysts to observe structured data on the open, deep, and dark web.

Join & Write a Comment

Suggested Solutions

Since upgrading to Office 2013 or higher installing the Smart Indenter addin will fail. This article will explain how to install it so it will work regardless of the Office version installed.
Real-time is more about the business, not the technology. In day-to-day life, to make real-time decisions like buying or investing, business needs the latest information(e.g. Gold Rate/Stock Rate). Unlike traditional days, you need not wait for a fe…
As developers, we are not limited to the functions provided by the VBA language. In addition, we can call the functions that are part of the Windows operating system. These functions are part of the Windows API (Application Programming Interface). U…
Show developers how to use a criteria form to limit the data that appears on an Access report. It is a common requirement that users can specify the criteria for a report at runtime. The easiest way to accomplish this is using a criteria form that a…

772 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

10 Experts available now in Live!

Get 1:1 Help Now