Find the redirected website name

I really don't understand the question.

old: http://www.airdefense.net/products/index.shtm
new: http://www.airdefense.net/products/servicesplatform/index.php
old: http://www.agileit.com/blog/lists/posts/post.aspx?id=265
new:http://www.agileit.com/news/bypass-uac-and-elevate-permissions-of-scripts-runas-administrator-at-the-command-line/

Select all Open in new window

Q-27980557.txt

How about this, using input/output.txt like in one of your earlier questions:

Const strInputFile = "input.txt"
Const strOutputFile = "output.txt"

Set objFSO = CreateObject("Scripting.FileSystemObject")

Set objInput = objFSO.OpenTextFile(strInputFile, 1, False)
Set objOutput = objFSO.CreateTextFile(strOutputFile, True)

Set objIE = CreateObject("InternetExplorer.Application")
'objIE.Visible = True
While Not objInput.AtEndOfStream
	strUrl = objInput.ReadLine
    objIE.Navigate strUrl
    While objIE.busy = True Or objIE.ReadyState <> 4
        WScript.Sleep 100
    Wend
    objOutput.WriteLine strUrl & vbTab & objIE.LocationURL
Wend
objIE.Quit
MsgBox "done"

Select all Open in new window

To see what's going on uncomment line 10. The 2 url's you posted were done in a few seconds. Maybe it's hung up on something? You could try adding

objIE.Silent = True

Select all Open in new window

There may be another way but I would have to investigate. Using IE window may not be necessary. Using XMLHTTP object for example may just connect to the url and receive the 'redirect' response that tells the browser where to go. That should be much faster than actually going there (and load the whole page).

I tried with XMLHTTP but the first example you gave uses javascript to redirect so that's going to be a lot harder to detect. Actually there's a number of ways a page can redirect (meta tags, server side code) so we better go back to basics. I don't understand what could be making your browser so slow, have you checked what happens if you just go to one of these redirecting sites manually? Of course the other option is (like you said in your other question) to investigate other browsers ability for COM control, haven't found that yet though...

That seems to be a rather generic error indicating something is not available. Impossible for me to tell what's missing I'm afraid. Since you say it's a fresh system my first thought would be to check permissions, on drive/folder/file level as well as for IE. Also it depends on whether you added the line I suggested (or other things) what's on line 14... Finally (maybe not for this problem but in general), it would help to know if you are testing with the same websites.

I've made a version of the script that does some more checks and logging. Now on the fresh machine it may not make a difference because something else is going on but on your original machine this may skip problematic sites (after trying a number of times) and show where the problem is.

Option Explicit

Const strInputFile = "input.txt"
Const strOutputFile = "output.txt"
Const strLogFile = "log.txt"

Dim objFSO, objInput, objOutput, objLog, objIE, strUrl, intCount, blnBusy, intReadyState

Set objFSO = CreateObject("Scripting.FileSystemObject")

Set objInput = objFSO.OpenTextFile(strInputFile, 1, False)
Set objOutput = objFSO.CreateTextFile(strOutputFile, True)
Set objLog = objFSO.CreateTextFile(strLogFile, True)

Set objIE = CreateObject("InternetExplorer.Application")
objIE.Visible = True
objIE.Silent = True
While Not objInput.AtEndOfStream
    strUrl = objInput.ReadLine
    objLog.WriteLine Now() & " url = " & strUrl
    objIE.Navigate strUrl
    intCount = 0
    blnBusy = objIE.busy
    intReadyState = objIE.ReadyState
    While (blnBusy = True Or intReadyState <> 4) And intCount < 20
        WScript.Sleep 1000 ' increased to 1 second
        intCount = intCount + 1
        objLog.WriteLine Now() & " count = " & intCount
        If blnBusy <> objIE.busy Then
            objLog.WriteLine Now() & " busy changed from " & blnBusy & " to " & objIE.busy
            blnBusy = objIE.busy
            intCount = 0
        End If
        If intReadyState <> objIE.ReadyState Then
            objLog.WriteLine Now() & " readystate changed from " & intReadyState & " to " & objIE.ReadyState
            intReadyState = objIE.ReadyState
            intCount = 0
        End If
    Wend
    If strUrl = objIE.LocationURL Then
        objLog.WriteLine Now() & " no redirect detected"
    Else
        objLog.WriteLine Now() & " success, writing output and continue to next line"
        objOutput.WriteLine strUrl & vbTab & objIE.LocationURL
    End If
Wend
objLog.WriteLine Now() & " done"

objInput.Close
Set objInput = Nothing

objOutput.Close
Set objOutput = Nothing

objLog.Close
Set objLog = Nothing

Set objFSO = Nothing

objIE.Quit
Set objIE = Nothing

MsgBox "done"

Select all Open in new window

Were there valid redirects in output.txt?

Error on line 28 seems very strange, that's a line I added to write logging and now you get that same error as before on that new line...? It does seem to correspond with the output as that is cut off during a loop (count = 16). Very strange. Maybe some error catching is in order.

Can you also try it on your other system please?

Great! sure, here is the output. I think there's a problem to do with showing the previous url when something goes wrong but since that doesn't happen on my system I haven't dived into the exact how and what yet...
log.txt
Redirection-output.txt

Yes, looks like it, that could be caught by saving a copy of the previous value in another variable but I think more important to understand what's going on. Can you see (now that the IE window is visible again) what's happening, any errors or hanging?

Weird. I have run it a number of times and no problems like that. Would only make sense if the user running the script has permission to create the file but not overwrite it somehow. Can't even think of a way to reproduce it...

I haven't been able to find anything on automating use of other browsers, other than for testing changes to the sources, which is not what you would need. Maybe that could be a subject for a new question?

I really don't see what's messing this up for you. The script is easy enough, and works for me without any hitch on 2 different computers. I can only think security barrier, firewall? Not something I can check for you though, you will have to try and find that part out yourself I'm afraid, maybe using Fiddler?

Let me add that check I was talking about to make sure it doesn't report a failure as success. Back in a jiffy.

Can you try this?

Option Explicit

Const strInputFile = "Redirection-input.txt"
Const strOutputFile = "Redirection-output.txt"
Const strLogFile = "log.txt"

Dim objFSO, objInput, objOutput, objLog, objIE, strUrl, strPrevUrl, intCount, blnBusy, intReadyState

Set objFSO = CreateObject("Scripting.FileSystemObject")

Set objInput = objFSO.OpenTextFile(strInputFile, 1, False)
Set objOutput = objFSO.CreateTextFile(strOutputFile, True)
Set objLog = objFSO.CreateTextFile(strLogFile, True)

Set objIE = CreateObject("InternetExplorer.Application")
objIE.Visible = True
objIE.Silent = True
While Not objInput.AtEndOfStream
    strUrl = objInput.ReadLine
    objLog.WriteLine Now() & " url = " & strUrl
    objIE.Navigate strUrl
    intCount = 0
    blnBusy = objIE.busy
    intReadyState = objIE.ReadyState
    While (blnBusy = True Or intReadyState <> 4) And intCount < 20
        WScript.Sleep 1000 ' increased to 1 second
        intCount = intCount + 1
        objLog.WriteLine Now() & " count = " & intCount
        If blnBusy <> objIE.busy Then
            objLog.WriteLine Now() & " busy changed from " & blnBusy & " to " & objIE.busy
            blnBusy = objIE.busy
            intCount = 0
        End If
        If intReadyState <> objIE.ReadyState Then
            objLog.WriteLine Now() & " readystate changed from " & intReadyState & " to " & objIE.ReadyState
            intReadyState = objIE.ReadyState
            intCount = 0
        End If
    Wend
    If strPrevUrl = objIE.LocationURL Then ' navigation failed!
        objLog.WriteLine Now() & " website not available"
    Else
        If strUrl = objIE.LocationURL Then
            objLog.WriteLine Now() & " no redirect detected"
        Else
            objLog.WriteLine Now() & " success, writing output and continue to next line"
            objOutput.WriteLine strUrl & vbTab & objIE.LocationURL
        End If
        strPrevUrl = objIE.LocationURL
    End If
Wend
objLog.WriteLine Now() & " done"

objInput.Close
Set objInput = Nothing

objOutput.Close
Set objOutput = Nothing

objLog.Close
Set objLog = Nothing

Set objFSO = Nothing

objIE.Quit
Set objIE = Nothing

MsgBox "done"

Select all Open in new window

Yes, that would mean the new code works ;-)

For some reason when the objIE is told to navigate to some url's on your machine it doesn't go anywhere. That's why I added the code to check for the _previously_ successfully retrieved URL. When it's still the same, apparently nothing happened and instead of logging the incorrect url's they are skipped.

I'll repeat my earlier question: what happens if you go to one of those skipped sites (like the 2nd or 3rd from your last post) in a regular IE window? Do you see any errors or does it seem to hang?

Or, if you're now ready to check on another machine...?

Well I can only repeat what I said before, you get this same error on yet another line so in my mind it has got to be something external, maybe your script is being intercepted by a virus scanner, I don't know but as far as I can see it has nothing to do with the actual line being reported. Obviously I would almost say as it is a different line each time... Time for bed here so hope you can look at extinguishing external factors.

Sure, still no problem found.

EDIT: forgot the logging:

13/01/2013 10:28:37 url = http://answers.microsoft.com/en-us/windows/forum/windowsvista-system/blue-screen-error-memory-management-stop/f9aff112-da38-46e7-9697-a206a182758f?msgId=7940c4d1-afac-4787-abb5-b2d8165a2c39
13/01/2013 10:28:38 count = 1
13/01/2013 10:28:39 busy changed from True to False
13/01/2013 10:28:39 readystate changed from 1 to 4
13/01/2013 10:28:39 success, writing output and continue to next line
13/01/2013 10:28:39 done

Select all Open in new window

Redirection-output.txt

now I get the same error but it's different than before: line 21 is the navigate. That is failing because the next input line (the last one you posted above) is not a valid website address (shouldn't contain the @).

Yep, was already working on that ;-)

Option Explicit

Const strInputFile = "Redirection-input.txt"
Const strOutputFile = "Redirection-output.txt"
Const strLogFile = "log.txt"
Const strErrFile = "errors.txt"

Const C_MAX_LOOP = 20
Const C_WAIT_MSEC = 1000 ' 1000 = 1 sec

Dim objFSO, objInput, objOutput, objLog, objErr, objIE, strUrl, strPrevUrl, intCount, blnBusy, intReadyState

Set objFSO = CreateObject("Scripting.FileSystemObject")

Set objInput = objFSO.OpenTextFile(strInputFile, 1, False)
Set objOutput = objFSO.CreateTextFile(strOutputFile, True)
Set objLog = objFSO.CreateTextFile(strLogFile, True)
Set objErr = objFSO.CreateTextFile(strErrFile, True)

Set objIE = CreateObject("InternetExplorer.Application")
objIE.Visible = True
objIE.Silent = True

On Error Resume Next

While Not objInput.AtEndOfStream
    strUrl = objInput.ReadLine
    objLog.WriteLine Now() & " url = " & strUrl
    intCount = 0

    objIE.Navigate strUrl
    If Err.Number <> 0 Then
        objErr.WriteLine strUrl
        objLog.WriteLine Now() & " ERROR" ' "ERROR #" & Err.Number & " (0x" & Hex(Err.Number) & "): " & Err.Description
        intCount = C_MAX_LOOP ' skip loop
        Err.Clear
    End If

    blnBusy = objIE.busy
    intReadyState = objIE.ReadyState
    While (blnBusy = True Or intReadyState <> 4) And intCount < C_MAX_LOOP
        WScript.Sleep C_WAIT_MSEC
        intCount = intCount + 1
        objLog.WriteLine Now() & " count = " & intCount
        If blnBusy <> objIE.busy Then
            objLog.WriteLine Now() & " busy changed from " & blnBusy & " to " & objIE.busy
            blnBusy = objIE.busy
            intCount = 0
        End If
        If intReadyState <> objIE.ReadyState Then
            objLog.WriteLine Now() & " readystate changed from " & intReadyState & " to " & objIE.ReadyState
            intReadyState = objIE.ReadyState
            intCount = 0
        End If
    Wend
    If strPrevUrl = objIE.LocationURL Then ' navigation failed!
        objLog.WriteLine Now() & " website not available"
    Else
        If strUrl = objIE.LocationURL Then
            objLog.WriteLine Now() & " no redirect detected"
        Else
            objLog.WriteLine Now() & " success, writing output and continue to next line"
            objOutput.WriteLine strUrl & vbTab & objIE.LocationURL
        End If
        strPrevUrl = objIE.LocationURL
    End If
Wend
objLog.WriteLine Now() & " done"

objInput.Close
Set objInput = Nothing

objOutput.Close
Set objOutput = Nothing

objLog.Close
Set objLog = Nothing

objErr.Close
Set objErr = Nothing

Set objFSO = Nothing

objIE.Quit
Set objIE = Nothing

MsgBox "done"

Select all Open in new window

Note that I usually don't like using "On Error" unless in a confined subroutine, because it's possible that you get stuck in an endless loop. In this case it seems safe.

Hmm, you mean you get a "Save as" dialog? Not sure if that's possible to detect. What happens after 20 seconds by the way? I would try myself but I suspect it depends on your environment. Obviously it would be possible to skip links ending in .pdf or .zip for example but that may not be every case.

the script ends? oh that's unexpected. so do you have an example that's not an obvious download? (those obvious ones you could just exclude from your selection I hope)

Ah I see what happens: the IE window closes but the script runs in the background, but can't do anything useful after that.

EDIT: took out optimistic remark...

I think I've seen something to disable file downloads in an IE window, have to check.

Nope, no go there.

I have seen some other options (in Excel VBA, using 'Dim with events' for example) but am afraid these will have the same problems specifically with erroneous/non-existing sites and downloads.

I want to run a completely different solution by you: would you be ok if it is done in a C# webforms application? I'm not sure it will work, but I'm willing to try.

Ok, my own first impression is good, I'm curious what you will think. Main code is below and also the full project, hope you can open it (I use Visual C# 2010 Express).

        private void DoProcess() {
            foreach (string u in textBox1.Lines) {
                string sts = "?";
                string u2 = string.Empty;
                try {
                    toolStripStatusLabel1.Text = string.Format("Processing url: {0}", u);
                    Application.DoEvents();
                    HttpWebRequest hwr = WebRequest.Create(new Uri(u)) as HttpWebRequest;
                    hwr.AllowAutoRedirect = true;
                    WebResponse wr = hwr.GetResponse();
                    u2 = wr.ResponseUri.AbsoluteUri;
                    wr.Close();
                    if (u2.Equals(u)) {
                        sts = "no redirect detected";
                    } else if (true) {
                        sts = "redirect ok";
                    }
                } catch (Exception ex) {
                    u2 = string.Empty;
                    sts = "error: " + ex.Message;
                }
                dataGridView1.Rows.Add(u, u2, sts);
            }
            toolStripStatusLabel1.Text = "Processing complete.";
        }

Select all Open in new window

EE-Q-27980557.ZIP

Great! Now to convert all those other scripts and excel sheets...? ;-)

That's a lot of history on those questions. I have difficulty ascertaining the current status and actually more important it's not up to me to decide whether they have been answered or not. But especially if you are asking for a speedup by making a change in programming language it would be more logical to me that you close those questions as they seem to have been answered but just rather slow. But that's probably not those experts' fault entirely since you have been asking specifically for VBA/vbscript solutions. In the above solution I only came up with changing the environment because there seemed to have been a number of problems on your PC/network with exactly those environments and I couldn't figure out why, so that could easily be part of the reason it's not working very quickly as well.

I would suggest to re-ask those questions, but only after you download Visual Studio C# 2010 (or newer) because it's going to be rough if everybody has to start posting links to exe's that may some day stop working (but in the above case the source code and entire project is available so shouldn't be a problem). So it would be nice if you have a basis to work off of, feel free to use the project I posted to refer to or post in new questions, but please do so in the C# topic area giving everybody a chance to have a go at this but also I urge you to only do so after you have played around with Visual Studio a bit so you can easily test out code that experts post. Ultimately I would expect that in a small project like this you can easily make changes yourself (using the WebResponse) to download files, check for strings in url's etc etc. It can seem like a daunting step from scripting to C# but I feel the rewards for you will be worthwile.