Check website for specific string after login using Powershell

After I have used a script to log into a website I need to check the site for a specific string.

Any suggestion on how to do this?
LVL 1
jimmylew52Asked:
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

x
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Jason RybergTechnical Consultant IVCommented:
Where is the string located?  Within a file, or dynamically created upon login on the screen?

You might be able to use this:

$client = new-object System.Net.WebClient
$client.DownloadFile("http://www.contoso.com/file.html","C:\tmp\file.html")
$file = Get-Content C:\tmp\file.html
$file.Contains("string text here")

Open in new window

0
jimmylew52Author Commented:
Created on the screen
0
jimmylew52Author Commented:
I need to check the page I have logged into
0
Challenges in Government Cyber Security

Has cyber security been a challenge in your government organization? Are you looking to improve your government's network security? Learn more about how to improve your government organization's security by viewing our on-demand webinar!

jimmylew52Author Commented:
gives me an error saying downloadfile has 2 arguments
0
Qlemo"Batchelor", Developer and EE Topic AdvisorCommented:
It wouldn't work anyway. The script just downloads a (static) web page.
$ie = New-Object -com InternetExplorer.Application
$ie.Visible = $true
$ie.Navigate2('http://something')

while ($ie.ReadyState -ne 4) { sleep -m 10 }
while ($ie.Document.body.innerText -notlike '* Logged in*') { sleep 5 } 
if ($ie.Document.body.innerText -like '* WhatYouSearchFor *')
{
  # action to take
}

Open in new window

The first while waits for the page to be stable ("ready").
The second while waits for an indication you have been logged in successfully.
And then the check for your search string is made.
0
Jason RybergTechnical Consultant IVCommented:
I asked the author if he was requesting static or dynamic content.  My earlier script would work on a static page.  

You need to create a session using PowerShell in order to retrieve the string after the login process; this can be done using the Invoke-WebRequest cmdlet.  This cmdlet will allow you to pass login information to your website.  Without the details of how you process logins on your site, it won't be possible to get the appropriate string after the login process.  The following website provides more information (and an explaination of a login process):

http://technet.microsoft.com/en-us/library/hh849901.aspx

If you provide more details on your login method (what variables are passed for instance), I should be able to help out more.
0
jimmylew52Author Commented:
This is the login script. It works fine. From here I want to check to see if the login succeeded or failed. So far  the only way I know of to tell is to look for text on the page after login.

Any suggestions would be appreciated.

# Find the username field and set the value to that of our variable
    $usernameField = $doc.getElementById('userName')
      #write-host $usernameField
    $usernameField.value = $username
      write-host $username
    # Find the password field and set the value to that of the result
    # of a call to the get-password function with the paramter defined at top
    $passwordField = $doc.getElementById('password')
      $passwordField.value = $pass
      write-host $pass
    # Find and click the submit button
    $submitButton = $doc.getElementById('login')
      write-host $submitButton
    $submitButton.click()
    # Wait until login is complete
    do {sleep 1} until (-not ($ie.Busy))
0
jimmylew52Author Commented:
This is the problem with your script for me.

$ie.Navigate2('http://something')

I am already logged into the site. I need to search the site  after login.
0
Qlemo"Batchelor", Developer and EE Topic AdvisorCommented:
You already have the Documents object in $doc, so you can just apply .body.innerHtml to that and compare as string, as s posted by me earlier.
0
Jason RybergTechnical Consultant IVCommented:
Agree with Qlemo with one exception.  Wrap the string in a div or some other tag that allows an ID attribute and then get it and its innerHTML property:


$doc.Document.getElementById("LoginNotice").innerHTML

Open in new window

0
jimmylew52Author Commented:
$doc.Document.getElementById("LoginNotice").innerHTML


The eliment ID I cannot find. As near as I can tell there is not an eliment ID like on the login page.
0
Jason RybergTechnical Consultant IVCommented:
You will need to wrap the post-login string in a div with an id attribute, (e.g. <div id="loginstring">login successful</div>) and then get the div by ID using the Powershell method above.
0
jimmylew52Author Commented:
You will need to wrap the post-login string in a div with an id attribute, (e.g. <div id="loginstring">login successful</div>) and then get the div by ID using the Powershell method above.

Sorry, went about a mile over my head. I have no idea what it is you are trying to tell me to do.
0
Jason RybergTechnical Consultant IVCommented:
I have to assume that you're not developing the website at this point.  

Most modern web browsers have an "inspect element" feature that will allow you to find the element's ID.  I would suggest logging in manually and determining the string you want to check  using the inspect element method.
Right click - Inspect ElementThis is what the tool looks like in Chrome:
Chrome Inspection toolYou'll use the ID attribute of the element to find the the string.
0
jimmylew52Author Commented:
ID= "tbam"

string I want to find = ActivityMonitor

Would this be correct?

$doc.Document.getElementById("tbam").innerHTML
if ($ie.Document.body.innerText -like 'ActivityMonitor')
0
Qlemo"Batchelor", Developer and EE Topic AdvisorCommented:
if ($doc.Document.getElementById("tbam").innerText -eq 'ActivityMonitor')

Open in new window

If you use like, you'll need to have wildcards (*) in the search pattern.
0
Jason RybergTechnical Consultant IVCommented:
I would change the second line to

if ($doc.Document.getElementById("tbam").innerHTML.Contains('ActivityMonitor')) {

  #do stuff

}

Open in new window

0
jimmylew52Author Commented:
Keeps erroring because of a null value, should this work?

If ($doc.Document.getElementById("tbam").innerText -ne "null")
0
jimmylew52Author Commented:
This does not error but it does not write to the test file either.

If ($doc.Document.getElementById("tbam").innerText -ne "null")

            {$loginSucceeded, $WebSite -join " " | out-file -filepath c:\MonitorFolder\ServersUp.txt -append}
0
Qlemo"Batchelor", Developer and EE Topic AdvisorCommented:
If the result is null, the ID you provided does not exist, and nothing will be written to the file.
0
jimmylew52Author Commented:
Here is the line I got the ID from:

<a id="tbam" href="/t2/tbam/activityMonitor.htl?method=showActivityMonitor&screenId=tbam">ActivityMonitor  </a>

Is there maybe a way to download the current page to a file and search the file for test?
0
Jason RybergTechnical Consultant IVCommented:
Since you're looking at an A element, the code would be:

if ($doc.Document.getElementById("tbam").textContent.contains("ActivityMonitor")) {

  #do stuff

}

Open in new window

0
jimmylew52Author Commented:
error:

You cannot call a method on a null-valued expression.
At C:\MonitorFolder\test6.ps1:56 char:34
+     If ($doc.Document.getElementById <<<< ('tbam').innerHTML.Contains('Activi
tyMonitor'))
    + CategoryInfo          : InvalidOperation: (getElementById:String) [], Ru
   ntimeException
    + FullyQualifiedErrorId : InvokeMethodOnNull
0
Jason RybergTechnical Consultant IVCommented:
Please re-read my newest post; you're no longer calling the innerHTML property, you need to call the textContent property.
0
jimmylew52Author Commented:
You cannot call a method on a null-valued expression.
At C:\MonitorFolder\test6.ps1:56 char:34
+     If ($doc.Document.getElementById <<<< ("tbam").textContent.contains("Acti
vityMonitor"))
    + CategoryInfo          : InvalidOperation: (getElementById:String) [], Ru
   ntimeException
    + FullyQualifiedErrorId : InvokeMethodOnNull
0
Jason RybergTechnical Consultant IVCommented:
Can you post your full script?
0
jimmylew52Author Commented:
ForEach($WebSite in Get-Content C:\MonitorFolder\websites.txt){
$WebSite | % {
 try {$web = New-Object System.Net.WebClient
    $out = $web.DownloadString("$WebSite")}
      catch {$WebSite, $Down -join " " | out-file -filepath c:\MonitorFolder\ServersDown.txt -append
                  break}
   
    write-host $website

# Create the IE com object
$ie = new-object -com InternetExplorer.Application
# Navigate to the login page
$ie.navigate($WebSite)
# Wait for the page to finish loading
do {sleep 1} until (-not ($ie.Busy))
$ie.visible = $true #Uncomment this for debugging
 
# Assign the DOM to the $doc variable
$doc = $ie.document

# Login to the Web Page.
Try {
    # Find the username field and set the value to that of our variable
    $usernameField = $doc.getElementById('userName')
      #write-host $usernameField
    $usernameField.value = $username
      write-host $username
    # Find the password field and set the value to that of the result
    # of a call to the get-password function with the paramter defined at top
    $passwordField = $doc.getElementById('password')
      $passwordField.value = $pass
      write-host $pass
    # Find and click the submit button
    $submitButton = $doc.getElementById('login')
      #write-host $submitButton
    $submitButton.click()
    # Wait until login is complete
    do {sleep 1} until (-not ($ie.Busy))
      } catch {$null}
            
      If ($doc.Document.getElementById("tbam").textContent.contains("ActivityMonitor"))      
      

            {$loginSucceeded, $WebSite -join " " | out-file -filepath c:\MonitorFolder\ServersUp.txt -append}
            else
          {$LoginFailed, $WebSite, $Check -join " " | out-file -filepath c:\MonitorFolder\ServersDown.txt -append}
 } #catch {$LoginFailed, $WebSite, $Check -join " " | out-file -filepath c:\MonitorFolder\ServersDown.txt -append}

}
0
Jason RybergTechnical Consultant IVCommented:
Change:

If ($doc.Document.getElementById("tbam").textContent.contains("ActivityMonitor"))

Open in new window


To:

If ($doc.getElementById("tbam").textContent.contains("ActivityMonitor"))

Open in new window

0
jimmylew52Author Commented:
You cannot call a method on a null-valued expression.
At C:\MonitorFolder\test6.ps1:56 char:54
+     If ($doc.getElementById("tbam").textContent.contains <<<< ("ActivityMonit
or"))
    + CategoryInfo          : InvalidOperation: (contains:String) [], RuntimeE
   xception
    + FullyQualifiedErrorId : InvokeMethodOnNull
0
Jason RybergTechnical Consultant IVCommented:
Can you post the return from running:

$doc.getElementById("tbam") | gm

Open in new window

0
jimmylew52Author Commented:
Get-Member : No object has been specified to the get-member cmdlet.
At C:\MonitorFolder\test6.ps1:57 char:34
+     $doc.getElementById("tbam") | gm <<<<
    + CategoryInfo          : CloseError: (:) [Get-Member], InvalidOperationEx
   ception
    + FullyQualifiedErrorId : NoObjectInGetMember,Microsoft.PowerShell.Command
   s.GetMemberCommand
0
Jason RybergTechnical Consultant IVCommented:
Ok, I see what's going on now.

You're storing the IE object in the $doc variable, so when you go to reference the tbam element later, it doesn't exist, because in the $doc variable, you haven't logged in yet!  So, you need to remove the line where you store the IE object (DOM, as you refer to it), and keep referencing the $ie.Document variable instead:


Remove these lines:
# Assign the DOM to the $doc variable 
$doc = $ie.document 

Open in new window


And just refer to the IE object directly:

# Login to the Web Page. 
Try {
    # Find the username field and set the value to that of our variable 
    $usernameField = $ie.Document.getElementById('userName') 
      #write-host $usernameField
    $usernameField.value = $username 
      write-host $username
    # Find the password field and set the value to that of the result 
    # of a call to the get-password function with the paramter defined at top 
    $passwordField = $ie.Document.getElementById('password')
      $passwordField.value = $pass 
      write-host $pass
    # Find and click the submit button 
    $submitButton = $ie.Document.getElementById('login') 
      #write-host $submitButton
    $submitButton.click() 
    # Wait until login is complete 
    do {sleep 1} until (-not ($ie.Busy))
      } catch {$null}
            
      If ($ie.Document.getElementById("tbam").textContent.contains("ActivityMonitor"))      
      

            {$loginSucceeded, $WebSite -join " " | out-file -filepath c:\MonitorFolder\ServersUp.txt -append}
            else
          {$LoginFailed, $WebSite, $Check -join " " | out-file -filepath c:\MonitorFolder\ServersDown.txt -append}
 } #catch {$LoginFailed, $WebSite, $Check -join " " | out-file -filepath c:\MonitorFolder\ServersDown.txt -append}

}

Open in new window

0
jimmylew52Author Commented:
The term 'ie.Document.getElementById' is not recognized as the name of a cmdlet
, function, script file, or operable program. Check the spelling of the name, o
r if a path was included, verify that the path is correct and try again.
At C:\MonitorFolder\test6.ps1:59 char:32
+     If (ie.Document.getElementById <<<< ("tbam").textContent.contains("Activi
ty"))
    + CategoryInfo          : ObjectNotFound: (ie.Document.getElementById:Stri
   ng) [], CommandNotFoundException
    + FullyQualifiedErrorId : CommandNotFoundException
0
Jason RybergTechnical Consultant IVCommented:
Looks like you missed the "$" before the $ie variable:

If (<$>ie.Document.getElementById <<<< ("tbam").textContent.contains("Activi
ty"))
0
jimmylew52Author Commented:
You cannot call a method on a null-valued expression.
At C:\MonitorFolder\test6.ps1:59 char:62
+     If ($ie.Document.getElementById("tbam").textContent.contains <<<< ("Activ
ityMonitor"))
    + CategoryInfo          : InvalidOperation: (contains:String) [], RuntimeE
   xception
    + FullyQualifiedErrorId : InvokeMethodOnNull
0
Jason RybergTechnical Consultant IVCommented:
Ok, the IE page should be visible on screen because you set the $ie.visible property to true.

Can you see the string you're wanting to check on the screen?
0
jimmylew52Author Commented:
Yes it is there
0
Jason RybergTechnical Consultant IVCommented:
Can you post the source of the page after you login, with the string visible?
0
jimmylew52Author Commented:
<html>
<head>
<body marginwidth="2" marginheight="2" leftmargin="2" topmargin="2">
<form id="menuBarForm" method="POST" action="/t2/login/chooser.jsp" name="menuBarForm">
<header id="mast">
<nav id="navbar">
<table>
<tbody>
<tr>
<td colspan="1">
T
<sup>2</sup>
Com
</td>
</tr>
<tr>
<script>
<td width="20px" align="left">
<td>
<a id="tbam" href="/t2/tbam/activityMonitor.htl?method=showActivityMonitor&screenId=tbam">ActivityMonitor  </a>
</td>
<td>
</tr>
</tbody>
</table>
</nav>
</form>
<div class="container">
<script type="text/javascript">
<meta content="text/html; charset=UTF-8" http-equiv="Content-Type">
<p> </p>
<blockquote>Choose an area from the menu bar.</blockquote>
<p> </p>
</div>
<div style="clear:both;"></div>
<div id="footerId">
<script language="JavaScript">
</body>
</html>
0
Jason RybergTechnical Consultant IVCommented:
Are you developing this webapp?  If you are, what are those <script> tags for?  Are you using ASP.net or JavaScript?  Your <script> tags are never closed.

When I remove the <script> tag prior to the A tag, I can view and retrieve it and its properties.
0
jimmylew52Author Commented:
I am not the developer. I can talk to them and see if that can be done.

It is in JAVA.
0
jimmylew52Author Commented:
Expanded the code a little more. It looks like the <script[/> tag gets closed. I'm not a programmer so I an not that sure of what I see.

<html>
<head>
<body marginwidth="2" marginheight="2" leftmargin="2" topmargin="2">
<form id="menuBarForm" method="POST" action="/t2/login/chooser.jsp" name="menuBarForm">
<header id="mast">
<nav id="navbar">
<table>
<tbody>
<tr>
<tr>
<script>
saveUserActivityLog(logincheck,'LogIn','LogedIn','UserLogin Time');
</script>
<td width="20px" align="left">
<div id="left_menu_header">
<img id="btn" class="hideButton" src="/t2/images/nav/2.jpg" value="<" name="btn" title="Hide">
</div>
</td>
<td>
<a id="tbam" href="/t2/tbam/activityMonitor.htl?method=showActivityMonitor&screenId=tbam">ActivityMonitor  </a>
</td>
<td>
<a id="customer" href="/t2/reports/redirect-chooser.jsp?screenId=customer">Reports</a>
</td>
</tr>
</tbody>
</table>
</nav>
</form>
<div class="container">
<div style="clear:both;"></div>
<div id="footerId">
<script language="JavaScript">
</body>
</html>
0
jimmylew52Author Commented:
Here is what I have found out:

$doc = $ie.document has to be in the script or I cannot login
$ie.document is the original page before login and that is why I get null for tbam.

Looks like I need to be able to access the page after login to be able to find tbam and get the script to work.

I tool a try with this but the page downloaded is still the page before login.

$WebSite2 = ($WebSite + "/login.htl?method=showLogin")
      write-host $WebSite2
$ie = new-object -com InternetExplorer.Application
# Navigate to the login page
$ie.navigate($WebSite2)
# Wait for the page to finish loading
do {sleep 1} until (-not ($ie.Busy))
$doc2 = $ie.document
$doc2 | out-file "C:\monitorFolder\test62.txt"
0
Jason RybergTechnical Consultant IVCommented:
Are you able to post the website, or is it internal only?  I have tested inputting values into textbox fields and button click functions on several other websites, and it seems to work on nearly every other page.  There might be some backend JavaScript that is killing the PowerShell retrieval of objects though, and at this point, it seems like the easiest thing to do is for me to try to work it out on the actual site.
0
jimmylew52Author Commented:
Website I can post

https://t2libd1i174.systrends.com/t2/login/chooser.jsp

Not sure how much good it will do without login credentials. I cannot post those, might give access to confidential information.
0
Jason RybergTechnical Consultant IVCommented:
$ie.document is the original page before login and that is why I get null for tbam.

This isn't true.  $ie.document represents the active IE window.  I have been able to login to my gmail account using PowerShell and the $ie.Document object. (Source)

When I went to your site linked above, I didn't see any login textboxes, so I can't verify using your page.  It is almost impossible at this point for me to try to pinpoint the actual issue, since I can't replicate the actual process your developers are using.  As I've said, I've logged into gmail and performed searches on Google and Bing using the method I've described above.  It seems likely that there might be some issues on the JavaScript side that could be preventing PowerShell from reporting the objects correctly.

Another method you could try is to search for all A tags and report back the one that contains the search string:


$ie.Document.body.getElementsByTagName("a") | ? textcontent -eq "ActivityMonitor"

Open in new window


OR

$ie.Document.links | ? textcontent -eq "ActivityMonitor"

Open in new window

0
jimmylew52Author Commented:
Making some progress.

$source = $ie.Document.documentElement.outerHTML | out-file "C:\monitorFolder\test6.html"

Downloads most of the page.

Trying to get the page searched for text now.
0
jimmylew52Author Commented:
This worked:

$WebSiteCon = "ActivityMonitor"

$Source = $ie.Document.documentElement.outerHTML | out-file "C:\monitorFolder\test6.html"

      $Search = "C:\monitorFolder\test6.html"
      
      $result = Get-Content $Search | Select-String $WebSiteCon -quiet
       write-host $result
      If ($result -eq $True)
        
            {$loginSucceeded, $WebSite -join " " | out-file -filepath c:\MonitorFolder\ServersUp.txt -append}
            else
          {$LoginFailed, $WebSite, $Check -join " " | out-file -filepath c:\MonitorFolder\ServersDown.txt -append}
 }
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
Qlemo"Batchelor", Developer and EE Topic AdvisorCommented:
      If ($result -eq $True) 

Open in new window

is nonsense, because the result of the IF expression again is $true or $false ;-). The correct usage is a simple
      If ($result) 

Open in new window

0
jimmylew52Author Commented:
I don't understand. I posted the solution and tried to award the points to the experts. I am happy with the help they provided. Did I click a wrong button?
0
jimmylew52Author Commented:
Experts may not have provided the final answer but they did keep me encouraged to keep trying.

Thank you so much for all the effort.
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Powershell

From novice to tech pro — start learning today.