Solved

How to: Write to a Text File Python

Posted on 2015-01-07
23
137 Views
Last Modified: 2015-01-19
i have a script created recently and i would like copy information on a text files and add a information

Code

from bs4 import BeautifulSoup
import requests

headers = {'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.95 Safari/537.36'}

# initialize session
session = requests.Session()

# getting playlist
response = session.get('http://animedigitalnetwork.fr/video/naruto-shippuden', headers=headers)
soup = BeautifulSoup(response.content)
playlist = soup.find('a', {'data-playlist': True})['data-playlist']

# getting list of videos
url = 'http://animedigitalnetwork.fr/index.php?option=com_vodvideo&view=playlist&format=raw'
response = session.post(url, data={
    'playlist': playlist,
    'season': '',
    'order': 'DESC'
}, headers=headers)

soup = BeautifulSoup(response.content)
for video in soup.select('div.adn-video'):
    print video.a.get('href')

Open in new window


Demo
the code generate a list of link

/video/naruto-shippuden/3809-episode-7-cours-kankuro

Open in new window


and i would like add this information

http://www.animedigitalnetwork.fr

Open in new window


to get this on a text files

http://www.animedigitalnetwork.fr/video/naruto-shippuden/3809-episode-7-cours-kankuro

Open in new window


ty :)
0
Comment
Question by:Gaaara
  • 13
  • 10
23 Comments
 
LVL 28

Accepted Solution

by:
pepr earned 500 total points
Comment Utility
Just replace the last three lines by
soup = BeautifulSoup(response.content)
with open('links.txt', 'w') as f:
    for video in soup.select('div.adn-video'):
        s = 'http://www.animedigitalnetwork.fr' + video.a.get('href')
        f.write(s + '\n')

Open in new window

The s = 'http://www.animedigitalnetwork.fr' + video.a.get('href') prepends the wanted prefix. Now the link is in the s variable. The f.write(s + '\n') writes the line to the open text file (plus newline to separate the links to the lines). The open('links.txt', 'w') opens the links.txt file for writing in the text mode. The with open('links.txt', 'w') as f: wraps it so that the open file object is named f, and ensures that the file is closed automatically after the construct.
0
 

Author Comment

by:Gaaara
Comment Utility
it work and i forget a question

I would want to know how to add a link in a line of command(order) example python test.py link

i add this on the script

urladn =sys.argv

Open in new window

and how to a a variable on this line
response = session.get('variable', headers=headers)
0
 
LVL 28

Expert Comment

by:pepr
Comment Utility
I am not sure if I understand you well. Do you mean something like this?
#!python2

from bs4 import BeautifulSoup
import os
import requests

headers = {'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.95 Safari/537.36'}

# initialize session
session = requests.Session()

# getting playlist
response = session.get('http://animedigitalnetwork.fr/video/naruto-shippuden', headers=headers)
soup = BeautifulSoup(response.content)
playlist = soup.find('a', {'data-playlist': True})['data-playlist']

# getting list of videos
url = 'http://animedigitalnetwork.fr/index.php?option=com_vodvideo&view=playlist&format=raw'
response = session.post(url, data={
    'playlist': playlist,
    'season': '',
    'order': 'DESC'
}, headers=headers)

soup = BeautifulSoup(response.content)
for video in soup.select('div.adn-video'):
    remote_path = video.a.get('href')
    url = 'http://www.animedigitalnetwork.fr' + remote_path

    # Get the content of the url.
    response = session.get(url,
        data={
            'playlist': playlist,
            'season': '',
            'order': 'DESC'},
        headers=headers)

    # Transform the remote name to the local name.
    local_path, basename = os.path.split(remote_path)
    local_path = 'responses' + local_path
    fname = os.path.join(local_path, basename)
    print basename

    # Make the local directory if it does not exist.
    if not os.path.isdir(local_path):
        os.makedirs(local_path)

    # Write the response content to the local file.
    with open(fname, 'w') as f:
        f.write(response.content)

Open in new window

It creates the responses subdirectory with the response contents stored in the separate files.
0
 

Author Comment

by:Gaaara
Comment Utility
no :)

ce que je veut faire c'est créer un input directement dans  le terminal
 I want to make a input directly in the terminal

exemple
python test.py -l http://animedigitalnetwork.fr/video/naruto-shippuden -n naruto

Open in new window


is a good idea for the name automatically the text files :)
0
 
LVL 28

Expert Comment

by:pepr
Comment Utility
OK. I do not speak French ;)

I assume that you want to add the -l option to pass the URL from command line instead of having it explicitly written in the script, right?

What the -n option means? And what should be generated by the script?
0
 

Author Comment

by:Gaaara
Comment Utility
option n is to personalise a name for text files
0
 

Author Comment

by:Gaaara
Comment Utility
i dont no have a news ?
0
 
LVL 28

Expert Comment

by:pepr
Comment Utility
You can import the sys module and use sys.argv to get the command line arguments. It is easier to use fixed positions of the arguments (without options -l and -n). See https://docs.python.org/2.7/library/sys.html#sys.argv
0
 

Author Closing Comment

by:Gaaara
Comment Utility
ty
0
 
LVL 28

Expert Comment

by:pepr
Comment Utility
Do you understand what to do with the sys.argv? If not, continue to ask here.

Basically, you should first test if there is enough arguments passed to the script, and then you extract the wanted values into variables that will be later used instead of hardcoded strings:
if len(sys.argv) < 3:
    print 'Usage:', sys.argv[0], '<URL> <directory>'

givenURL = sys.argv[1]
outputdir = sys.argv[2]

Open in new window

0
 

Author Comment

by:Gaaara
Comment Utility
I accepted the answer because you'd give me documentation and so I would analyze it to continue :)


thank you for the explanation ^^
0
Do You Know the 4 Main Threat Actor Types?

Do you know the main threat actor types? Most attackers fall into one of four categories, each with their own favored tactics, techniques, and procedures.

 
LVL 28

Expert Comment

by:pepr
Comment Utility
Feel free to continue here when you have the question related to the code.
0
 

Author Comment

by:Gaaara
Comment Utility
no problem :) thank you
0
 

Author Comment

by:Gaaara
Comment Utility
I have successfully completed the script :)
0
 

Author Comment

by:Gaaara
Comment Utility
hello

how to print the First line  on a text files with python  ?
0
 
LVL 28

Expert Comment

by:pepr
Comment Utility
The easy way is to read the file in the loop and use the break command when you want to stop:
#!python2

# Generate the demo file.
fname = 'testdata.txt'
with open(fname, 'w') as f:
    for n in xrange(1, 10):
        f.write('line {}\n'.format(n))


# Read the file, and break after printing the first line.        
with open(fname) as f:
    for line in f:
        print line.rstrip()
        break

Open in new window

0
 

Author Comment

by:Gaaara
Comment Utility
cool ty :)
0
 
LVL 28

Expert Comment

by:pepr
Comment Utility
It can be easily modified to get the first n lines:
# Read the file, and break after printing the first n lines.        
with open(fname) as f:
    for n, line in enumerate(f, 1):
        print line.rstrip()
        if n == 5:  # break after the fifth line
            break

Open in new window


The built-in enumerate() function adds counting of anything that can be iterated. Here the optional second argument was used to count lines from 1 instead from the default zero.
0
 

Author Comment

by:Gaaara
Comment Utility
hello you have a complete website to learn python ?
0
 
LVL 28

Expert Comment

by:pepr
Comment Utility
No. I do recommend the official web pages to find some. The official tutorial is good for beginners.
0
 

Author Comment

by:Gaaara
Comment Utility
hello i have 2 question for you is the and of my script ! :)


my exemple is a bash :)  i would like get the same ting with python

[ "$(echo $REPLY | tr [:upper:] [:lower:])" == "y" ] || exit
           links=$(awk -F\" '/meta/ {if($4=="") $4="dummy"; print $2, $4}' video.smil)
           link1=${links% *}
           link2=${links#* }
           [[ $link2 = "dummy" ]] && unset link2

Open in new window


rez=$(awk -F= '$1 ~ /^RES$/{print $2}' res.cfg)
    case $rez in
        480* )  python $files/SD.py;;
        720* )  python $files/HD.py;;
        1080* ) python $files/FHD.py;;
        * ) echo "Bad response $rez"; exit 1;;
    esac

Open in new window


ty ^^
0
 
LVL 28

Expert Comment

by:pepr
Comment Utility
This is unrelated to the original question.
0
 

Author Comment

by:Gaaara
Comment Utility
no just  i need help :)  http://www.experts-exchange.com/Programming/Languages/Scripting/Python/Q_28599864.html

i create  a new question ^^ a try  To make him
0

Featured Post

Why You Should Analyze Threat Actor TTPs

After years of analyzing threat actor behavior, it’s become clear that at any given time there are specific tactics, techniques, and procedures (TTPs) that are particularly prevalent. By analyzing and understanding these TTPs, you can dramatically enhance your security program.

Join & Write a Comment

Plenty of writing has gone on the web trying to compare Python with other competitive programming languages and vice versa. However, not much has been put into a wholistic perspective. This article should help you decide whether to adopt Python as a…
Variable is a place holder or reserved memory locations to store any value. Which means whenever we create a variable, indirectly we are reserving some space in the memory. The interpreter assigns or allocates some space in the memory based on the d…
Learn the basics of lists in Python. Lists, as their name suggests, are a means for ordering and storing values. : Lists are declared using brackets; for example: t = [1, 2, 3]: Lists may contain a mix of data types; for example: t = ['string', 1, T…
Learn the basics of while and for loops in Python.  while loops are used for testing while, or until, a condition is met: The structure of a while loop is as follows:     while <condition>:         do something         repeate: The break statement m…

772 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

9 Experts available now in Live!

Get 1:1 Help Now