asked on

Clicking on a link using VBScript

I try to automate some handling on a webpage. I have read all the articles I could find, but it just does not fit.

This is partial the webpage:

This is the HTML beneath:

<div class="rgWrap rgArrPart2">
<a title="Go to Next Page" href="javascript:__doPostBack('ctl00j','')">Volgende</a>
<input type="submit" name="ctl003" value=" " title="Next Page" class="rgPageNext" /> 
<a title="Go to Last Page" href="javascript:__doPostBack('ctl004','')">Laatste</a>
<input type="submit" name="ctl005" value=" " title="Last Page" class="rgPageLast" />

Open in new window

"Volgende" en "Laatste" are Dutch for "Next" and "Last".

The code which I try to build needs to click on the "Volgende" so I get the next page with records.

I have tried the following code:

GetElementById():

   IE.Document.GetElementById("ctl00j").Click()

    GetElementsByTagName() and filter by value:

    Set oInputs = IE.Document.GetElementsByTagName("input")
    For Each elm In oInputs
        If elm.Value = "Go to Next Page" Then
            elm.Click
            Exit For
        End If
    Next

Open in new window

GetElementsByClassName():

    For Each elem In IE.Document.GetElementsByClassName("rgPageNext")
        If elem.innerText = "Go to Next Page" Then
            elem.Focus
            elem.Click
        End If
    Next

Open in new window

GetElementsByTagName() and filter by class attribute:

    For Each a In IE.document.getElementsByTagName("a")
        If a.GetAttribute("class") = "rgPageNext" Then
            a.Click
            Exit For
        End If
    Next

Open in new window

GetElementsByTagName() and filter by title attribute:

    For Each a In IE.document.getElementsByTagName("a")
      If a.getAttribute("title") = "Go to Next Page" Then
        a.Click
        Exit For
      End If
    Next

Open in new window

The first approach gives this error message:

    Script: K:\temp\TT\mftt.vbs
    Line: 37
    Char: 2
    Error: Object required: 'IE.Document.getElementByID(...)'
    Code: 800A01A8
    Source: Microsoft VBScript runtime error

Open in new window

The other attempts are not giving error messages but also no reaction on the link to click. I'm missing here the OnClick event. With other links in this project the OnClick event was present. I can't change the HTML, so I'm stuck.

UPDATE1: I have added the script file so far. I have not attempt to build the logic. First aim is to read all the fields and click the right buttons. If that is successful I will build the iteration process,
UPDATE2: due to privacy reasons I can't give you access to the site. I have added the DOM from the page and removed any names.
mftt2.txt
DOM.txt
Transactiehistorie.txt

Chinmay Patel

Hi John,

I am curious to know why you are trying to do this with Script? Also instead of IE.Document... just try Document. And if possible, post the entire script with the tags so I can check if there is anything else that might be the cause of the error.

Also I see that your script is independent, how does it load the page? Is this a part of some page? or some testing or automation app?

Also do you have control over the actual ASPX page? I was thinking this control might be a part of a container hence you should try using
ClientIDMode=Static to generate a static id for example "ButtonNext" and access it via your script.
https://docs.microsoft.com/en-us/dotnet/api/system.web.ui.control.clientidmode?view=netframework-4.7.2

Regards,
Chinmay.

Kimputer

Personally, I don't like to use IE automation that much. I use it for emergency reasons, and it's very crash prone (stray IE threads staying in memory). Usually I like to really sit for it and start a VB project. Since you already know vbscript, it shouldn't be that hard. Also Visual Studio Community is free, and the IDE just simple kicks ass.

To know the logic of the site, I use Fiddler4 and really analyze all the clicks I do, and find out how the cookies works, and all the posted/returned data. Those posts and returns I will mimic in VB with Webclient or httprequests. Your final product will be very very stable (of course, depends on your code)

CroetOne

ASKER

Thanks for your swift reaction.

I know it is not the best way of doing this, but it is for one time use only. I need the data of 8000 customers from this site which is ours, but not de code of the site. Both the clients and the data are from us, the website code and the export function of the database are having limited access and it will cost us a fortune the extract the data otherwise.
Because I did this before with other sites, I thought this was doable, but it seems more complex.

@Chinmay: no, I do not have control over the actual ASPX page. I could post a copy of the DOM where I only leave the personal data out.

Chinmay Patel

:D This is a grey area but may I suggest using something totally different? Why don't you create a VB.Net app and then use WebBrowser control (https://docs.microsoft.com/en-us/dotnet/api/system.windows.forms.webbrowser?view=netframework-4.7.2)or even better use HTML Agility pack https://html-agility-pack.net/to do it without writing a ton of custom code?

CroetOne

ASKER

@Chibway: I'm always eager to learn new tools, but for now I need a quick and dirty solution. Have you any idee why the webpages does not react on the attempts to click the link?

Chinmay Patel

Fair enough. But I am not sure how exactly you are loading this page? If you can explain then I will able to guide you further. And btw the one I am mentioning is really really quick and dirty solution with the guarantee that it will work. If you can share a sample URL(sans your sensitive data) I can write a quick code that will do the scrapping for you.

CroetOne

ASKER

@Chinmay: I have provided the DOM and the script file so far (uploaded this morning). As you can see in the script file you need to login, which is no problem at all.
But I can't give any URL, you would need to login, and then you have access to all the data. Sorry....

CroetOne

ASKER

@Chinway:...but you making the scraper would be very helpful!

Chinmay Patel

Hi John,

No need to be sorry. I appreciate you posting even this much of data. And to tell you the truth, I am happy that you care for the privacy. And what you have posted is more than enough for me to get started. I hope this is not time sensitive, I am neck deep in couple of other questions plus my day to day tasks. :(

Regards,
Chinmay.

Chinmay Patel

If I give you C# code for the scrapper, will you be able to build and run it?

CroetOne

ASKER

I wouldn't. I have limited experience with c#. And...this is not the only part. What you see now are records for one account of one customer. The process logic is not in place. I have no idea if I'm able to integrate your scraper with the process iteration of tapping 'next' until ready and moving to the next account by going to the previous screen and select then the account and move to the transaction report screen to scrape all the screens within this transaction info of this account. Then going back and if no account anymore go an other step back and select the next costumer, until all customers has been handled. I would able to write this in VBScript, but handling a C# scraper within that code...
I could on the other hand provide you with the DOM's of the previous screens, but then you would write the program and I sit back. That's not what I had in mind when I asked the question. So let me know how we can work together in solving this problem.

Kind regards,
-John

CroetOne

ASKER

Hi, I saw your first post later and no, it is not very time sensitive but a bit. I should need a working solution somewhere next week...
Is that possible for you?

Chinmay Patel

:D VBScript it is. I will try to close other open items and get to you as soon as I can. And EE is HUGE, I am sure if I am slacking someone else will pick up the pace.

Chinmay Patel

The accessing DOM Part I can promise, beyond that unless and until I see the screens and understand the co-relation between elements, it will be hard form e to come up with the entire thing. I will make sure that I clear the issues on accessing DOM and clicking on elements today/tomorrow (IST)

CroetOne

ASKER

Thanks! You're very kind.

Chinmay Patel

Naah... don't thank me :D.. I am very selfish. This is how I keep learning new stuff - I find reading - sometimes - very borrring... :D

So I ran your script on some other page. I checked your page it's DOM is simple, nothing special. Your code should have worked. I targeted Bing.com and it worked - up to an extent.

' // Login procedure for ... //
Dim WshShell
Dim IE
Dim strLoginName
Dim strPassword

Set WshShell = WScript.CreateObject("WScript.Shell")
SET IE = Nothing

Call LogIn

Function LogIn
    Set IE = WScript.CreateObject("InternetExplorer.Application", "IE_")
    IE.Visible = True
    IE.Navigate "https://www.bing.com/"
	Wait IE,500
    
	'now we have the page where we can select the clients
	Wait IE, 500
    With IE.Document
		debugger
        'by clicking the 'search' button, we create a client list 
		.getElementByID("sb_form_q").value = "TEST"
        .getElementByID("sb_form_go").click()
        
	End With
    'We go to the transction overview page
    End Function

Sub Wait(IE, SleepInterval)
    Do
        WScript.Sleep SleepInterval
    Loop While IE.ReadyState < 4 Or IE.Busy
End Sub

SET IE = Nothing

Open in new window

Do you have access to Visual Studio? or VS Code? I think there might be something else at work here. I was thinking we can debug the script at your end by putting MsgBox at right places. If that does not work then we can use //X switch from command line to select a debugger like VS.

Thoughts?

CroetOne

ASKER

Thanks! I will look in to it when I'm back at the office!
Regards,
-John

CroetOne

ASKER

Thoughts....yes...some where in the question I had noticed that I had little experience with HTML and therefor with scraping web pages.
When I wrote the code for the fourth approach, I wrote `tittle` as the label name. Not aware that in English there is also a `title`, which in this case should be the choice.
Now I have got that of my back (the fourth approach works), I stumbled in to the next problem. Trying to iterate through the detail records, I need to click on the `>` sign left of the records in the page table. I again have tried several approaches. The last one I have tried gives me most info.

Dim Counter :  Counter = 0
Set SetTo1 = IE.Document.getElementByID("ctl00_mainContentPlaceHolder_usercontrols_piramide_mutatielist_ascx1_RadGrid1_ctl00")
Set SetTo2 = SetTo1.getElementsByClassName("rgExpand") 
Msgbox SetTo2.length 'will print how many items are present in the collection
While Counter < SetTo2.length
	SetTo2.Item(Counter).Click
	Wait IE, 1500
        msgBox "One open"
	Counter = Counter + 1   ' Increment Counter
Wend

Open in new window

I have also tried to use `For Each` but both ways I get a permission denied.

---------------------------
Windows Script Host
---------------------------
Script:      K:\temp\TT\mftt2.vbs
Line:      44
Char:      2
Error:      Permission denied: 'SetTo2.length'
Code:      800A0046
Source: Microsoft VBScript runtime error

I can't understand why I get a Permission denied the second time the script wants to click on the link. The first iteration works. The message `one open` displays and the details are shown. Closing them after opening does not solves this. That is what I have tried also.

CroetOne

ASKER

I have added in the question the DOM after the first successful click on the &gt sign....

Chinmay Patel

Do you get a value of SetTo2 in MsgBox?

CroetOne

ASKER

Yes, I get this value indeed. I noticed that when I close and reopen the object again in the loop, the permission denied is gone...
So I can iterate the transaction page. Chatting with you about this project is helpful, but I'm unsure whether or not this is will lead to a definable question with a adequate answer...:-)

CroetOne

ASKER

Please close this question

This question needs an answer!

Become an EE member today

7 DAY FREE TRIAL

Members can start a 7-Day Free trial then enjoy unlimited access to the platform.

View membership options

Learn why we charge membership fees

We get it - no one likes a content blocker. Take one extra minute and find out why we block content.