[Okta Webinar] Learn how to a build a cloud-first strategyRegister Now

x
?
Solved

Powershell need to find this value

Posted on 2014-08-09
27
Medium Priority
?
2,829 Views
Last Modified: 2014-08-15
OK, I need to find "Run by:" on web page so I need to if it exists return   "Run by:" or True.
$RunByTest = $ie.Document.getElementsByTagName("span") | ? { $_.InnerText -like 'Run by:*' }  |  select  -First 1 -expandProperty classname

Open in new window


right now it returns blank or false because it cant find value on page and value does exists on page.

sample
or here is some sample code
<table style="border-collapse:collapse;width:100%;" class="table" cellpadding="0" cellspacing="0"><tr class="tableRow"><td class="reportTitle"><span class="textItem">Daily Employee Overtime Report</span></td></tr></table></div><table style="border-collapse:collapse;width:100%;" class="table" cellpadding="0" cellspacing="0"><tr class="tableRow"><td class="tableCell"><table style="border-collapse:collapse;width:100%;" class="reportSubtitle" cellpadding="0" cellspacing="0"><tr class="tableRow"><td class="tableCell"><span class="textItem">Run by:</span></td><td class="tableCell"><span class="textItem">WORKBRAIN</span></td></tr></table></td><td class="tableCell"><table style="border-collapse:collapse;width:100%;" class="reportSubtitle" cellpadding="0" cellspacing="0"><tr class="tableRow">

Open in new window

0
Comment
Question by:Leo Torres
  • 11
  • 11
  • 2
24 Comments
 
LVL 16

Expert Comment

by:Dale Harris
ID: 40251298
Have you tried this:
stackoverflow.com/questions/20334928/issue-parsing-html-using-powershell-and-xpath
0
 
LVL 8

Author Comment

by:Leo Torres
ID: 40251313
I have never used item object. Not sure how that link helps sorry I am still very green on powershell

How do i Even get my result in the item object?
0
 
LVL 71

Expert Comment

by:Qlemo
ID: 40251651
Well, XPath is just another access method for web content, and Invoke-WebRequest just simplifies the access to a page in IE. Both doesn't change the need of knowledge for the internals of the web page to parse, though the path to get there is different.

Up to now (viewing the questions posted prior) using $ie wasn't hitting a road block, and so I would stay with it. Invoke-WebRequest however is worth it to use - if you are on PS 3 or later, as it waits for the web page to be built completely before returning, sparing you some code lines for waiting.
0
 The Evil-ution of Network Security Threats

What are the hacks that forever changed the security industry? To answer that question, we created an exciting new eBook that takes you on a trip through hacking history. It explores the top hacks from the 80s to 2010s, why they mattered, and how the security industry responded.

 
LVL 71

Expert Comment

by:Qlemo
ID: 40251658
Your code line should give you the string "textItem", as that is the class name of the table cell containing "Run by:". Doesn't make much sense that way, you only want to check existence, so this would be better:
@($ie.Document.getElementsByTagName("span") | ? { $_.InnerText -like 'Run by:*' }).Count -gt 0

Open in new window

But, using "span" as filter tag is clumsy, and should only be used if the page does not contain many "span" entries. On the other hand the HTML snippet does not provide unique tags to search for, and that means you might have no better means.
0
 
LVL 8

Author Comment

by:Leo Torres
ID: 40252055
Your Code suggestion above is not returning a value. See line 57 below.
Add-Type -Path "C:\Windows\System32\WindowsPowerShell\v1.0\Modules\HtmlAgilityPack.1.4.6\Net45\HtmlAgilityPack.dll"

CLS

$url = "https://yazaki.company.com"
#$url = "https://psecu.company.com"
#$url = "https://quickenloans.company.com"

$ie = New-Object -comobject InternetExplorer.Application
$ie.visible = $true
$ie.silent = $true
$ie.Navigate( $url )
while($ie.busy){Start-Sleep 1}
 
Start-Sleep -s 5
 
#$ie.Document.getElementsByTagName("span") | ft -a classname, tagname, innerText
 
$ie.Document.getElementsByTagName("input") | ? { $_.Id -eq 'loginField' } | % { $_.value = "User" }
$ie.Document.getElementsByTagName("input") | ? { $_.Id -eq 'passwordField' } | % { $_.value = "pass" }
$ie.Document.getElementsByTagName("button") | ? { $_.Type -eq 'button' } | % { $_.Click() }
 
 
$ReportPart = "/interface/folderTree.jsp?rootId=216&expandLevel=1&clearUIPath=true&uiPathLabel=Reports"
$ReportLink = $url + $ReportPart
 
$ie.navigate($ReportLink)
 
#$ie.Document.getElementsByTagName("a") | ft -a classname, tagname, innerText
 
Start-Sleep -s 5

$link = $ie.Document.getElementsByTagName("a") | ? { $_.InnerText -like '*Overtime Report' }  |  select  -First 1 -expandProperty href
 
        
if($link) {          
   $link = $link          
}
else {          
	$link = $ie.Document.getElementsByTagName("a") | ? { $_.InnerText -like '*Count' }  |  select  -First 1 -expandProperty href           
}

$ie.navigate($link)

Start-Sleep -s 5
 
$ie.Document.getElementsByTagName("a") | ft -a classname, tagname, innerText
$ie.Document.getElementsByTagName("button") | ? { $_.innerText -eq 'Go' } | % { $_.Click() }

Start-Sleep -s 25
 
$ie.Document.getElementsByTagName("span") | ft -a classname, tagname, innerText #Gets you error
#$ie.Document.getElementsByTagName("span") | select className
#$RunByTest = $ie.Document.getElementsByTagName("span") | ? { $_.InnerText -like 'Run by:*' }  |  select  -First 1 -expandProperty innerText
 
#$RunByTest = $ie.Document.getElementsByTagName("span") | ?  { $_.InnerText -like 'Run*' }  | %  select  -First 1 -expandProperty classname
$RunByTest = @($ie.Document.getElementsByTagName("span") | ? { $_.InnerText -like 'Run by:*' }).Count -gt 0

if($RunByTest) {
    $TestValue = 200          
}
else {          
   $TestValue = -200
}
Write-Host $TestValue 

Open in new window



Just so you understand the Run By: string tells me that the page loaded correctly by Automated user and that the page works.

Else I set integer set to -200 and collect the error string from page and I will store values into array and display array as report so see what Links are down. Just so you have an over view.
0
 
LVL 16

Expert Comment

by:Dale Harris
ID: 40252101
Here's a quick breakdown of what I was referring to in: stackoverflow.com/questions/20334928/issue-parsing-html-using-powershell-and-xpath

$url = 'http://www.example.com/path/to/some.html'
$html = (Invoke-Webrequest $url).ParsedHTML
$html.getElementsByTagName('p') | ? { $_.className -eq 'row' } | % {
  $ID   = $_.getAttributeNode('data-pid').value
  $Date = $_.getElementsByTagName('span') | ? { $_.className -eq 'date' } |
          % { $_.innerText }

  # do stuff with $ID and $Date
  "{0}: {1}" -f $ID, $Date
}

Open in new window


It looks very similar to what you're trying to accomplish.  This is the same way I parsed HTML for a Dell Service Tag look-up for Warranty information and thought it would lead you down the path to success.
0
 
LVL 8

Author Comment

by:Leo Torres
ID: 40252522
OK, looks promising but as the code above shows when I am looking for Run by: string I dont know the site url is. I land on this page after a series of button clicks. Unless I can grab it from $ie object I don't know what it is.
0
 
LVL 71

Expert Comment

by:Qlemo
ID: 40252962
As said, you can interchange (Invoke-WebRequest $url).ParseHTML and $ie.Navigate($url); <# wait until ready #> $ie.Document arbitrarily. It doesn't change anything.

If line 52 throws out something useful, line 57 should be valid. Only reasons I can see for not working is
a) the page is still not ready to full extend
b) the search string is incorrect.
The comment in line 52 is "Gets you error" - does that mean it displays the error, if the page is not working (at all) and hence the condition to check for, or is there an error when trying to execute that line?
0
 
LVL 8

Author Comment

by:Leo Torres
ID: 40254089
You would think that your last statement would be true but its not. If page errors out I got the Error text on that span I noticed this by mistake when page errored I got page error in that Item. How ever when page was successful that line returns NULL. As far as time goes if you notice I am waiting 25 seconds to make sure report is ready for scan.

I have attached a copy of html Page that is displayed when successful   and a page with a failure so you can see the differences.
Html-Test.txt
Html-Test-error.txt
0
 
LVL 71

Expert Comment

by:Qlemo
ID: 40254485
Sorry having to say that, but Html-Test.txt results in $true with
@($ie.Document.getElementsByTagName("span") | ? { $_.InnerText -like 'Run by:*' }).Count -gt 0

Open in new window

The error page does not.
0
 
LVL 8

Author Comment

by:Leo Torres
ID: 40255289
Here is my result I get false.
ULogin7.JPG
I cant have this password on the open internet. If you email me at «removed» I will send you user and password. so you can run the code in its entirety.
0
 
LVL 71

Expert Comment

by:Qlemo
ID: 40257140
(received real-life URL and credentials)

The issue is that we have a frameset consisting of two frames, of which the second contains the string we search for. I can get to the frame, but not further, with
$ie.Document.body.childnodes.item(1)

Open in new window

and have to give up here.
But you should be able to post that HTML code of the report here, maybe with some obfuscation in regard of names and URLs.
0
 
LVL 8

Author Comment

by:Leo Torres
ID: 40257161
Qlemo,  where did you place the code line you specified above.

Just curious in terms of array wouldn't 1 be in the second array. If array is ie.. 0,1,2,3...

Aren't you in the right Frame with item(1)?
0
 
LVL 71

Expert Comment

by:Qlemo
ID: 40257185
That line is executed at about line 52 of http:#a40252055 (and instead of all following lines).

And yes, we are in the second array element (second frame). The first contains some "header" stuff, the second the report data.
0
 
LVL 8

Author Comment

by:Leo Torres
ID: 40257340
Oh FYI

The HTML of the report is already here its post ID 40254089
http://www.experts-exchange.com/Programming/Languages/Scripting/Powershell/Q_28494276.html#a40254089

You said you got success? I dont understand when you did real test thru web and could not produce same result. On the file you said you got true.
0
 
LVL 71

Expert Comment

by:Qlemo
ID: 40257771
No, it is not. Search in the HTML you posted for "frameset" and you won't find anything. The live code of the report result is completely different, at least for me. I got the report, and it immediately starts with a frameset, while the code you posted starts with some scripting code.
0
 
LVL 8

Author Comment

by:Leo Torres
ID: 40258796
You know I am looking for the Run By: string for confirmation that report has ran. May be we could look for something else that would help me make that same determination. For example 2 Framesets I didnt see that in error HTML. Even something Unique in the header that would give the indication that report has ran successfully.

Your thoughts for alternatives are welcome.
0
 
LVL 71

Expert Comment

by:Qlemo
ID: 40258958
That should be feasible and easier to achieve. You can check for the classname (or is it type?) of $ie.Document.body, which needs to be "frameset" for success.
But what to do after that? Calling the report without storing the results does not make sense to me.
0
 
LVL 8

Author Comment

by:Leo Torres
ID: 40259341
I dont what to see report results. That's not my objective.

We have 272 of these pages and every Saturday night Sunday Morning these servers have updates and reboots done. We need to test sites manually at this point. This process you see the code doing is being done Site by Site by a human 272 sites all the way down to the report. All the tester does is verify that they can run the report. If the report runs then the site is up and functional and they move on to next site to test. We need to know site is down before the client finds out and going thru 272 sites really delays finds bad sites.

That True/false number will be passed thru more code to display which site is up and what sites are down.

This is the objective of this code. Report result are irrelevant and done me anything to me.

Hope this help you understand the code and what I need to accomplish better.
0
 
LVL 71

Expert Comment

by:Qlemo
ID: 40259500
Got it. For making the tests robust, it would be good to know if you only need to change the URL for each site, or there is more. It would be tedious to implement 272 ways to test each site individually ;-).
But since you provided three URLs, I will check myself with those.

BTW, in http:#a40252055 you import the HTML Agility Pack, but we are not using any feature of it. I will take the liberty and simplify the code, removing all the test stuff, and making it suitable for testing that lot of sites.
0
 
LVL 71

Accepted Solution

by:
Qlemo earned 2000 total points
ID: 40259565
How about this? It might need some refinement in regards of reintroducing sleeps after clicking buttons. Of course I was not able to test for errors.
Calling a function is much more convinient, as you can see. You could even use a foreach going thru a string array with URLs (hardcoded or read from a text file or ...).
CLS

if (!(Get-Variable ie -Scope global -ea SilentlyContinue)) { $global:ie = New-Object -comobject InternetExplorer.Application }
$ie.visible = $true
$ie.silent  = $true

$global:ReportPart = "/interface/folderTree.jsp?rootId=216&expandLevel=1&clearUIPath=true&uiPathLabel=Reports"


function Test-ReportSite ([String] $url, [String] $login, [String] $password)
{
  $ie.Navigate2($url)
  while ($ie.busy)                               { Start-Sleep -m 100 }
  while ($ie.Document.readyState -ne 'Complete') { Start-Sleep -m 100 }
   
  $ie.Document.getElementsByTagName("input")  | ? { $_.Id   -eq 'loginField'    } | % { $_.value = $login    }
  $ie.Document.getElementsByTagName("input")  | ? { $_.Id   -eq 'passwordField' } | % { $_.value = $password }
  $ie.Document.getElementsByTagName("button") | ? { $_.Type -eq 'button'        } | % { $_.Click() }
  while ($ie.Document.readyState -ne 'Complete') { Start-Sleep -m 100 }
  
  $ie.Navigate2($url + $ReportPart)
  while ($ie.Document.readyState -ne 'Complete') { Start-Sleep -m 100 }

  $link = $ie.Document.getElementsByTagName("a") | ? { $_.InnerText -like '*Overtime Report' }  |  select  -First 1 -expandProperty href
  if(!$link) {          
  	$link = $ie.Document.getElementsByTagName("a") | ? { $_.InnerText -like '*Count' }  |  select  -First 1 -expandProperty href           
  }
  $ie.Navigate2($link)
  while ($ie.Document.readyState -ne 'Complete') { Start-Sleep -m 100 }
   
  $ie.Document.getElementsByTagName("button") | ? { $_.innerText -eq 'Go' } | % { $_.Click() }
  Start-Sleep 5

<#  Error message evaluation ignored ATM   
  $ie.Document.getElementsByTagName("span") | ft -a classname, tagname, innerText #Gets you error
#>

  $RunByTest = $ie.Document.body.tagname -eq 'FRAMESET' -and $ie.Document.body.id -eq 'reportViewerFrame'

  if($RunByTest) {
      $TestValue = 200          
  }
  else {          
     $TestValue = -200
  }
  Write-Host "$URL: $TestValue" 
}

Test-ReportSite "https://yazaki.company.com"        "user" "pass"
Test-ReportSite "https://psecu.company.com"         "user" "pass"
Test-ReportSite "https://quickenloans.company.com"  "user" "pass"

Open in new window

0
 
LVL 8

Author Comment

by:Leo Torres
ID: 40259700
Changed:
Write-Host "$URL: $TestValue"
to
Write-Host "URL: $TestValue"

it was causing an error.

OK where do I save this function so i can call it from any ISE window or any command line at anytime.

This looks promising.
0
 
LVL 71

Expert Comment

by:Qlemo
ID: 40261980
Sorry for that. The colon has a special meaning ("drive" resp. "provider"). My intention was to use Write_host "$URL`: $TestValue", so you can see each URL with the result (instead of only results).
You can put that directly into ISE, storing it to your favourite location. If you do as I did, function and calling code are in the same script, and you do not need to consider anything.

However, if you want to have the function in an own script, and then be able to call it from comand line, you can store it into (again) an arbitrary location (script name does not matter either), together with the preceeding code. Then import the script via dot operator, e.g.
  . C:\Scripts\FunctionForTesting.ps1
Doing that once will keep anything defined available in the current shell; would you just call the script without the dot operator, no vars and other definitions should be available outside of the script.
0
 
LVL 8

Author Closing Comment

by:Leo Torres
ID: 40264754
This looks like it works but I cant see the process in IE. I need to see it to validate.
I opened new question here.
http://www.experts-exchange.com/Programming/Languages/Scripting/Powershell/Q_28499027.html
0

Featured Post

Technology Partners: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

My attempt to use PowerShell and other great resources found online to simplify the deployment of Office 365 ProPlus client components to any workstation that needs it, regardless of existing Office components that may be needing attention.
High user turnover can cause old/redundant user data to consume valuable space. UserResourceCleanup was developed to address this by automatically deleting user folders when the user account is deleted.
The viewer will learn how to count occurrences of each item in an array.
The viewer will receive an overview of the basics of CSS showing inline styles. In the head tags set up your style tags: (CODE) Reference the nav tag and set your properties.: (CODE) Set the reference for the UL element and styles for it to ensu…
Suggested Courses

834 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question