Want to protect your cyber security and still get fast solutions? Ask a secure question today.Go Premium

x
  • Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 286
  • Last Modified:

how to download raw post data json python

hello
i would like download playlist json files how to downloads this files?

1 load url www.somsite.com/playliste/1
2 download json raw http://somesite/index.php?option=com_play&view=playlist&format=raw



ty
0
Gaaara
Asked:
Gaaara
  • 14
  • 11
2 Solutions
 
aikimarkCommented:
Have you used urllib, mechanize, or request?
0
 
GaaaraAuthor Commented:
I want to know how to get files I am one thus begin wanted to know what to use and also to have some documentation with visible demo
0
 
aikimarkCommented:
is there a specific web site you are opening?
0
Technology Partners: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 
GaaaraAuthor Commented:
no just get the playlis with a cron job for get a émail In every new links for 2 -3 web site To miss nothing
0
 
aikimarkCommented:
please post an actual working URL that returns a JSON file.
0
 
aikimarkCommented:
I'm getting a 404 on the link that is supposed to return JSON
0
 
GaaaraAuthor Commented:
the json link Is activated in the load of the page with a free account or premium account with cookie

exemple with curl copied with  firebug
curl 'http://animedigitalnetwork.fr/index.php?option=com_vodvideo&view=playlist&format=raw' -H 'Host: animedigitalnetwork.fr' -H 'User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:34.0) Gecko/20100101 Firefox/34.0' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8' -H 'Accept-Language: fr,fr-fr;q=0.8,en-us;q=0.5,en;q=0.3' -H 'Accept-Encoding: gzip, deflate' -H 'X-Requested-With: XMLHttpRequest' -H 'Content-Type: application/x-www-form-urlencoded; charset=UTF-8' -H 'Referer: http://animedigitalnetwork.fr/video/naruto-shippuden' -H 'Cookie: _ga=GA1.2.2143731636.1420256453; 18acd9b63ecbf50de0b8c010c2b7289f=m9sbtbulg7mciu7b2f1gpt3337; _gat=1' --data 'playlist=265&season=&order=DESC'

Open in new window

the script need to collect all the cookie and get the file

i dont no have a 404 with any link ?
0
 
aikimarkCommented:
you have signed up for an account with that site.  That is why your URLs get results.  Log out (sign off) and retry your URL to see what I'm seeing.
0
 
GaaaraAuthor Commented:
is ok for me :) i dont no have a 404 error What do you want to know about the site
0
 
aikimarkCommented:
I like to test code prior to posting it.
0
 
GaaaraAuthor Commented:
If you have some difficulty post the code i test it :)
0
 
aikimarkCommented:
Let me rephrase my prior comment.  I don't feel comfortable posting untested code.  Maybe one of the other Python experts will feel better about your testing proposal.
0
 
GaaaraAuthor Commented:
mm test with my test compte of this site

user    testpseudo
pass   KytmEgsdKQa9

you have a vpn ?
0
 
aikimarkCommented:
I was able to sign in, but your second link doesn't return anything.  I added &playlist=265 and didn't see any JSON.
0
 
GaaaraAuthor Commented:
0
 
GaaaraAuthor Commented:
I've requested that this question be deleted for the following reason:

...
0
 
aikimarkCommented:
Looks like (rendered) HTML to me
0
 
GaaaraAuthor Commented:
with BeautifulSoup is possible to passing this part ?

	

    import urllib2
    from BeautifulSoup import BeautifulSoup
     
    page = urllib2.urlopen('http://animedigitalnetwork.fr/video/naruto-shippuden')
    soup = BeautifulSoup(page)
    first_div = soup.find('div',{"class" : "adn-video"})
    print  first_div 

Open in new window


i got this résult

<div class="adn-video"> <div class="adn-video_screenshot">
<img src="http://image.animedigitalnetwork.fr/license/claymore/tv/web/eps1_328x184.jpg" alt="Claymore 1" /><span class="adn_video_play-button"></span> </div><div class="adn-video_text"><div class="adn-video_title">
<h4>Claymore</h4><span>Épisode 1</span><div class="adn-rating mobile-hide" itemprop="aggregateRating" itemscope="itemscope" itemtype="http://schema.org/AggregateRating"><meta itemprop="ratingValue" content="4.6667" /><meta itemprop="ratingCount" content="10" /><div id="adn-rating"><ul class="adn-rating_empty"><li>&#xe002;</li><li>&#xe002;</li><li>&#xe002;</li><li>&#xe002;</li><li>&#xe002;</li></ul><ul class="adn-rating_rating"><li>&#xe002;</li><li>&#xe002;</li><li>&#xe002;</li><li>&#xe002;</li><li>&#xe002;</li></ul></div><p class="adn-rating-message"></p></div></div><div class="adn-video_link">
<a title="Claymore 1" href="/video/claymore/1849-episode-1-la-claymore">Voir la vidéo</a>
</div></div></div>

Open in new window


is not naruto !?
0
 
aikimarkCommented:
There are several ways to parse the data out of HTML and beautifulsoup is certainly one of those ways.  What you have posted in your latest comment is HTML and not JSON.


Try this URL:
http://animedigitalnetwork.fr/video/naruto-shippuden

Then do a view source or inspect element to see the HTML.
0
 
GaaaraAuthor Commented:
is resolved

from bs4 import BeautifulSoup
import requests

headers = {'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.95 Safari/537.36'}

# initialize session
session = requests.Session()

# getting playlist
response = session.get('http://animedigitalnetwork.fr/video/naruto-shippuden', headers=headers)
soup = BeautifulSoup(response.content)
playlist = soup.find('a', {'data-playlist': True})['data-playlist']

# getting list of videos
url = 'http://animedigitalnetwork.fr/index.php?option=com_vodvideo&view=playlist&format=raw'
response = session.post(url, data={
    'playlist': playlist,
    'season': '',
    'order': 'DESC'
}, headers=headers)

soup = BeautifulSoup(response.content)
for video in soup.select('div.adn-video'):
    print video.a.get('href')

Open in new window

0
 
GaaaraAuthor Commented:
is ok
0
 
GaaaraAuthor Commented:
I've requested that this question be closed as follows:

Accepted answer: 0 points for Gaaara's comment #a40537186
Assisted answer: 500 points for aikimark's comment #a40537116

for the following reason:

i give you the 500 points for the efforts
0
 
aikimarkCommented:
There is no need to give me 'effort' points.  You can accept your comment as the solution.
0
 
GaaaraAuthor Commented:
i give you the 500 points for the efforts
0

Featured Post

Concerto Cloud for Software Providers & ISVs

Can Concerto Cloud Services help you focus on evolving your application offerings, while delivering the best cloud experience to your customers? From DevOps to revenue models and customer support, the answer is yes!

Learn how Concerto can help you.

  • 14
  • 11
Tackle projects and never again get stuck behind a technical roadblock.
Join Now