Link to home
Start Free TrialLog in
Avatar of SQL Guru
SQL Guru

asked on

how to get refreshed page_source from selenium -Pyhton platform

1. I load the page
2.input some data
3.click on submit button
4.It does the calculation , and result are shown below in the same page.

In-spite of simplicity wait, page_source doesn't have newly loaded content.

My test code:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.ui import Select
import time 
from selenium.webdriver.common.by import By
from selenium.webdriver.support.wait import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from bs4 import BeautifulSoup

driver = webdriver.Chrome('C:\\Users\\Vivekanandhan\\Downloads\\Compressed\\chromedriver_win32\\chromedriver.exe')


driver.get('http://srirangaminfo.com/porutham.php')
#Input Value to the text box
inputElement = driver.find_element_by_id("name2")
inputElement.send_keys('1')

#Select value from the list box 
select = Select(driver.find_element_by_id('fdd2'))
select.select_by_value('4')

select = Select(driver.find_element_by_id('fmm2'))
select.select_by_value('6')

select = Select(driver.find_element_by_id('fyy2'))
select.select_by_value('1986')

select = Select(driver.find_element_by_id('fhr2'))
select.select_by_value('1')

select = Select(driver.find_element_by_id('fmin2'))
select.select_by_value('49')

select = Select(driver.find_element_by_id('fsec2'))
select.select_by_value('0')

select = Select(driver.find_element_by_id('fam2'))
select.select_by_value('1')
######################################################
inputElement = driver.find_element_by_id("name")
inputElement.send_keys('2')

select = Select(driver.find_element_by_id('fdd'))
select.select_by_value('4')

select = Select(driver.find_element_by_id('fmm'))
select.select_by_value('6')

select = Select(driver.find_element_by_id('fyy'))
select.select_by_value('1986')

select = Select(driver.find_element_by_id('fhr'))
select.select_by_value('1')

select = Select(driver.find_element_by_id('fmin'))
select.select_by_value('49')

select = Select(driver.find_element_by_id('fsec'))
select.select_by_value('0')

select = Select(driver.find_element_by_id('fam'))
select.select_by_value('1')

driver.find_element_by_name('cal1').click()
########################################################################

driver.implicitly_wait(10)


html = driver.page_source
soup = BeautifulSoup(html, 'lxml')

print (soup)

Open in new window

ASKER CERTIFIED SOLUTION
Avatar of Flabio Gates
Flabio Gates

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Avatar of SQL Guru
SQL Guru

ASKER

I tried to get the text from class 'tot blft' but it's giving me an error.

tot = div.find_element_by_class_name('tot blft')
print (tot.text)

InvalidSelectorException: Message: invalid selector: Compound class names not permitted
  (Session info: chrome=60.0.3112.113)
  (Driver info: chromedriver=2.31.488763 (092de99f48a300323ecf8c2a4e2e7cab51de5ba8),platform=Windows NT 10.0.15063 x86_64)

Open in new window

That's a different issue.
Am away from my laptop until tomorrow; left it at the office but try:

tot = div.find_element_by_xpath(".//*[@class='tot blft']")

Open in new window