asked on

how i can take the information from the website and convert it to json format

hello, expert.
there is one home listings site.
I would like to receive some information within this site.
this site's url is https: = //www.hurriyetemlak.com/satilik
I marked the information I want with a red square.
I want to get the data I marked with beautifulsoup.
and I want to get this data in json format.
I'm new to python.

import bs4
import requests

r=requests.get("https://www.hurriyetemlak.com/satilik")
print(r.status_code)
soup=bs4.BeautifulSoup(r.content,"html.parser")
#print(soup.find_all("div",attrs={"class":"col-lg-12 mt-1 box list-container"}))
adresler=(soup.find_all("div",attrs={"class":"col-lg-12 mt-1 box list-container"}))
for x in adresler:
    print(x)

Open in new window

ı did something in its own right.
but I didn't make much progress
I don't know how.
I need your help.
thanks
Ads-z.png

David Favor

Beautiful Soup is likely the wrong tool, as this site uses Javascript for rendering.

Best start with either PhantomJS or Selenium or some code which executes Javascript + CSS, else your code may work initially, then fail down the road.

ali şahin

ASKER

ok thanks.
I installed selenium on my computer.
I tried to write the codes.
but I was able to go this far.

iimport time
from selenium import webdriver
driver = webdriver.Chrome('C:\chromedriver')  #
driver.get('https://www.hurriyetemlak.com/satilik')
time.sleep(5) 

#time.sleep(5) 
evler=driver.find_elements_by_xpath('//span[@class="title"]')
konum=driver.find_elements_by_xpath('//li[@class="location"]')
oda=driver.find_elements_by_xpath('//li[@class="room"]')
sayi=len(evler)

dizi=[]
#print(buyers.text)
for  i in  range(sayi):
    sozluk = {}
    sozluk['adres']=evler[i].text
    sozluk['lokasyon']=konum[i].text
    sozluk['oda']=oda[i].text
    dizi.append(sozluk)

print(dizi)
driver.quit()

Open in new window

python gives me this eror.
[sozluk['lokasyon']=konum[i].text
IndexError: list index out of range
do I convert data to json format?

ASKER CERTIFIED SOLUTION

David Favor

membership

This solution is only available to members.

To access this solution, you must be a member of Experts Exchange.

Start Free Trial