Python Issues

michael ogunbiyi
michael ogunbiyi used Ask the Experts™
on
i appear to be having issues using anaconda.
i have copied the code into Jupyter note as follows:
In [1]:
def CryptoDataCSV(symbol, frequency):

#Params: String symbol, int frequency = 300,900,1800,7200,14400,86400

#Returns: df from first available date

url ='https://poloniex.com/public?command=returnChartData¤cyPair='+symbol+'&end=9999999999&period='+str(frequency)+'&start=0'

df = pd.read_json(url)

df.set_index('date',inplace=True)

df.to_csv(symbol + '.csv')

print('Processed: ' + symbol)
In [2]:
import pandas as pd
import matplotlib.pyplot as plt
%matplotlib inline
In [3]:
tickers = ['USDT_BTC','USDT_ETC','USDT_XMR','USDT_ETH','USDT_DASH',
'USDT_XRP','USDT_LTC','USDT_NXT','USDT_STR','USDT_REP','USDT_ZEC']
In [4]:
crypto_df = pd.DataFrame()
for ticker in tickers:
crypto_df[ticker] = pd.read_csv(ticker+'.csv', index_col = 'date')['close']
crypto_df.dropna(inplace=True)
---------------------------------------------------------------------------
FileNotFoundError                         Traceback (most recent call last)
<ipython-input-4-b59f4116fa1a> in <module>()
      1 crypto_df = pd.DataFrame()
      2 for ticker in tickers:
----> 3     crypto_df[ticker] = pd.read_csv(ticker+'.csv', index_col = 'date')['close']
      4 crypto_df.dropna(inplace=True)
~\Anaconda3\lib\site-packages\pandas\io\parsers.py in parser_f(filepath_or_buffer, sep, delimiter, header, names, index_col, usecols, squeeze, prefix, mangle_dupe_cols, dtype, engine, converters, true_values, false_values, skipinitialspace, skiprows, nrows, na_values, keep_default_na, na_filter, verbose, skip_blank_lines, parse_dates, infer_datetime_format, keep_date_col, date_parser, dayfirst, iterator, chunksize, compression, thousands, decimal, lineterminator, quotechar, quoting, escapechar, comment, encoding, dialect, tupleize_cols, error_bad_lines, warn_bad_lines, skipfooter, skip_footer, doublequote, delim_whitespace, as_recarray, compact_ints, use_unsigned, low_memory, buffer_lines, memory_map, float_precision)
    653                     skip_blank_lines=skip_blank_lines)
    654
--> 655         return _read(filepath_or_buffer, kwds)
    656
    657     parser_f.__name__ = name
~\Anaconda3\lib\site-packages\pandas\io\parsers.py in _read(filepath_or_buffer, kwds)
    403
    404     # Create the parser.
--> 405     parser = TextFileReader(filepath_or_buffer, **kwds)
    406
    407     if chunksize or iterator:
~\Anaconda3\lib\site-packages\pandas\io\parsers.py in __init__(self, f, engine, **kwds)
    762             self.options['has_index_names'] = kwds['has_index_names']
    763
--> 764         self._make_engine(self.engine)
    765
    766     def close(self):
~\Anaconda3\lib\site-packages\pandas\io\parsers.py in _make_engine(self, engine)
    983     def _make_engine(self, engine='c'):
    984         if engine == 'c':
--> 985             self._engine = CParserWrapper(self.f, **self.options)
    986         else:
    987             if engine == 'python':
~\Anaconda3\lib\site-packages\pandas\io\parsers.py in __init__(self, src, **kwds)
   1603         kwds['allow_leading_cols'] = self.index_col is not False
   1604
-> 1605         self._reader = parsers.TextReader(src, **kwds)
   1606
   1607         # XXX
pandas/_libs/parsers.pyx in pandas._libs.parsers.TextReader.__cinit__ (pandas\_libs\parsers.c:4209)()
pandas/_libs/parsers.pyx in pandas._libs.parsers.TextReader._setup_parser_source (pandas\_libs\parsers.c:8873)()
FileNotFoundError: File b'USDT_BTC.csv' does not exist

I GET THE ABOVE ERROR. BASICALLY NOTHING HAPPENS!
i would appreciate help. My aim is to use this to download data for top 50 currencies to run in a forecasting and risk analysis model.
Comment
Watch Question

Do more with

Expert Office
EXPERT OFFICE® is a registered trademark of EXPERTS EXCHANGE®
Can you re-post with both the source code and the traceback wrapped in CODE tags? It is difficult to read as it is now.

Author

Commented:
import matplotlib.pyplot as plt
In [1]:
 
 
%matplotlib inline
In [2]:
 
 
def CryptoDataCSV(symbol, frequency):

    #Params: String symbol, int frequency = 300,900,1800,7200,14400,86400

    #Returns: df from first available date

    url ='https://poloniex.com/public?command=returnChartData¤cyPair='+symbol+'&end=9999999999&period='+str(frequency)+'&start=0'

    df = pd.read_json(url)

    df.set_index('date',inplace=True)

    df.to_csv(symbol + '.csv')

    print('Processed: ' + symbol)

In [3]:
 
 
tickers = ['USDT_BTC','USDT_ETC','USDT_XMR','USDT_ETH','USDT_DASH',
 'USDT_XRP','USDT_LTC','USDT_NXT','USDT_STR','USDT_REP','USDT_ZEC']

In [4]:
 
x
for ticker in tickers:
    CryptoDataCSV(ticker, 86400)['close']
---------------------------------------------------------------------------
HTTPError                                 Traceback (most recent call last)
<ipython-input-6-46cec03c9a07> in <module>()
      1 for ticker in tickers:
----> 2     CryptoDataCSV(ticker, 86400)['close']

<ipython-input-4-1ad6182137ea> in CryptoDataCSV(symbol, frequency)
      7     url ='https://poloniex.com/public?command=returnChartData¤cyPair='+symbol+'&end=9999999999&period='+str(frequency)+'&start=0'
      8
----> 9     df = pd.read_json(url)
     10
     11     df.set_index('date',inplace=True)

~\Anaconda3\lib\site-packages\pandas\io\json\json.py in read_json(path_or_buf, orient, typ, dtype, convert_axes, convert_dates, keep_default_dates, numpy, precise_float, date_unit, encoding, lines)
    320
    321     filepath_or_buffer, _, _ = get_filepath_or_buffer(path_or_buf,
--> 322                                                       encoding=encoding)
    323     if isinstance(filepath_or_buffer, compat.string_types):
    324         try:

~\Anaconda3\lib\site-packages\pandas\io\common.py in get_filepath_or_buffer(filepath_or_buffer, encoding, compression)
    184     if _is_url(filepath_or_buffer):
    185         url = str(filepath_or_buffer)
--> 186         req = _urlopen(url)
    187         content_encoding = req.headers.get('Content-Encoding', None)
    188         if content_encoding == 'gzip':

~\Anaconda3\lib\urllib\request.py in urlopen(url, data, timeout, cafile, capath, cadefault, context)
    221     else:
    222         opener = _opener
--> 223     return opener.open(url, data, timeout)
    224
    225 def install_opener(opener):

~\Anaconda3\lib\urllib\request.py in open(self, fullurl, data, timeout)
    530         for processor in self.process_response.get(protocol, []):
    531             meth = getattr(processor, meth_name)
--> 532             response = meth(req, response)
    533
    534         return response

~\Anaconda3\lib\urllib\request.py in http_response(self, request, response)
    640         if not (200 <= code < 300):
    641             response = self.parent.error(
--> 642                 'http', request, response, code, msg, hdrs)
    643
    644         return response

~\Anaconda3\lib\urllib\request.py in error(self, proto, *args)
    568         if http_err:
    569             args = (dict, 'default', 'http_error_default') + orig_args
--> 570             return self._call_chain(*args)
    571
    572 # XXX probably also want an abstract factory that knows when it makes

~\Anaconda3\lib\urllib\request.py in _call_chain(self, chain, kind, meth_name, *args)
    502         for handler in handlers:
    503             func = getattr(handler, meth_name)
--> 504             result = func(*args)
    505             if result is not None:
    506                 return result

~\Anaconda3\lib\urllib\request.py in http_error_default(self, req, fp, code, msg, hdrs)
    648 class HTTPDefaultErrorHandler(BaseHandler):
    649     def http_error_default(self, req, fp, code, msg, hdrs):
--> 650         raise HTTPError(req.full_url, code, msg, hdrs, fp)
    651
    652 class HTTPRedirectHandler(BaseHandler):

HTTPError: HTTP Error 403: Forbidden
Can you give me a sample of a valid value for "symbol?"
Oh and when I said CODE tags, I meant use one of the buttons above the area where you type your comments.
(on my phone, these buttons are to the left of the text area)
Introduction to R

R is considered the predominant language for data scientist and statisticians. Learn how to use R for your own data science projects.

Found a sample URL for the returnChartData call:
https://poloniex.com/public?command=returnChartData&currencyPair=BTC_XMR&start=1405699200&end=9999999999&period=14400

Open in new window

Can you modify your code accordingly? I.e., change
url ='https://poloniex.com/public?command=returnChartData¤cyPair='+symbol+'&end=9999999999&period='+str(frequency)+'&start=0'

Open in new window

to
url ='https://poloniex.com/public?command=returnChartData&currencyPair='+symbol+'&end=9999999999&period='+str(frequency)+'&start=0'

Open in new window

Note the ampersand after returnChartData and currencyPair instead of cyPair.

Author

Commented:
I PASTED:

url ='https://poloniex.com/public?command=returnChartData¤cyPair='+symbol+'&end=9999999999&period='+str(frequency)+'&start=0'

IT NOW BRINGS THE FOLLOWING ERROR.


  File "<ipython-input-1-c7ee74ec9df5>", line 9
    df = pd.read_json(url)
    ^
IndentationError: unexpected indent
you did not change the part of the URL I told you to change
command=returnChartData¤cyPair=

Open in new window

should be
command=returnChartData&currencyPair=

Open in new window

you should retain what indentation you already had.
you should really use CODE tags when posting code.
alternatively, you can Attach your source code so we can see your properly indented code.
(see the "Attach File" link below the comment text area)
the code in the link you shared shows the correct URL parameters
def CryptoDataCSV(symbol, frequency):
    #Params: String symbol, int frequency = 300,900,1800,7200,14400,86400
    #Returns: df from first available date
    url ='https://poloniex.com/public?command=returnChartData&currencyPair='+symbol+'&end=9999999999&period='+str(frequency)+'&start=0'
    df = pd.read_json(url)
    ...

Open in new window

I.e., ...command=returnChartData&currencyPair=...
Your version is not the same.

Author

Commented:
i really don't knows what i am doing wrong here.

see attached.
Untitled--2-.html
Ok, let's try isolating the problem. Can you try running the following code:
import urllib
import json
from pprint import pprint

frequency = 14400

tickers = ['USDT_BTC','USDT_BCH','USDT_ETC','USDT_XMR','USDT_ETH','USDT_DASH', 'USDT_XRP','USDT_LTC','USDT_NXT','USDT_STR','USDT_REP','USDT_ZEC']

for symbol in tickers:
    url ='https://poloniex.com/public?command=returnChartData&currencyPair='+symbol+'&end=9999999999&period='+str(frequency)+'&start=0'

    f = urllib.urlopen(url)
    d = json.load(f)
    f.close()
    len(d)
    print symbol
    pprint(d[:2])

Open in new window


Are you, by the way, using a proxy server to connect to the internet?

Author

Commented:
now gives this error:



  File "<ipython-input-49-98f46ec6a299>", line 16
    print symbol
               ^
SyntaxError: Missing parentheses in call to 'print'. Did you mean print(int symbol)?
so you're using version 3. use
print(symbol)

Open in new window

no int
wait... will update code for version 3 of python
updated for python 3:
import urllib.request
import json
from pprint import pprint

frequency = 14400

tickers = ['USDT_BTC','USDT_BCH','USDT_ETC','USDT_XMR','USDT_ETH','USDT_DASH', 'USDT_XRP','USDT_LTC','USDT_NXT','USDT_STR','USDT_REP','USDT_ZEC']

for symbol in tickers:
    url ='https://poloniex.com/public?command=returnChartData&currencyPair='+symbol+'&end=9999999999&period='+str(frequency)+'&start=0'

    f = urllib.request.urlopen(url)
    d = json.load(f)
    f.close()
    len(d)
    print(symbol)
    pprint(d[:2])

Open in new window

Author

Commented:
thanks.

much appreciated

Author

Commented:
now back to http error
Untitled--3-.html
Can you try running the script from the command prompt and not from Anaconda?
from urllib.request import Request, urlopen
import json
from pprint import pprint

user_agent = 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.7) Gecko/2009021910 Firefox/3.0.7'
headers = {'User-Agent':user_agent}

frequency = 14400

tickers = ['USDT_BTC','USDT_BCH','USDT_ETC','USDT_XMR','USDT_ETH','USDT_DASH', 'USDT_XRP','USDT_LTC','USDT_NXT','USDT_STR','USDT_REP','USDT_ZEC']

for symbol in tickers:
    url ='https://poloniex.com/public?command=returnChartData&currencyPair='+symbol+'&end=9999999999&period='+str(frequency)+'&start=0'
    req = Request(url, headers=headers)
    try:
        f = urlopen(req)
        d = json.load(f)
        f.close()
    except urllib.request.HTTPError as e:
        print(e.read())
        raise
    else:
        len(d)
        print(symbol)
        pprint(d[:2])

Open in new window


Again, are you behind a proxy server?

Author

Commented:
command prompt on windows?

cmd?

Author

Commented:
the code cleaned up?
Untitled.htm
"cmd?"
yes
the code cleaned up?
What do you mean by this?

Author

Commented:
i had removed the duplicated code note. and run in sequence.
Untitled.htm
i had removed the duplicated code note. and run in sequence.
You lost me. I don't get what you are saying.
have you tried from CMD?
from urllib.request import Request, urlopen, HTTPError
import json
from pprint import pprint

user_agent = 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.7) Gecko/2009021910 Firefox/3.0.7'
headers = {'User-Agent':user_agent}

frequency = 14400

tickers = ['USDT_BTC','USDT_BCH','USDT_ETC','USDT_XMR','USDT_ETH','USDT_DASH', 'USDT_XRP','USDT_LTC','USDT_NXT','USDT_STR','USDT_REP','USDT_ZEC']

for symbol in tickers:
    url ='https://poloniex.com/public?command=returnChartData&currencyPair='+symbol+'&end=9999999999&period='+str(frequency)+'&start=0'
    print(url)
    req = Request(url, headers=headers)
    try:
        f = urlopen(req)
        d = json.load(f)
        f.close()
    except HTTPError as e:
        print(e.read())
        raise
    else:
        print(symbol)
        pprint(d[:2])

Open in new window

You have not answered my question regarding your possible use of a proxy server to connect to the internet.
Are you or are you not behind a proxy server?
I have some errands to run but will try to get back to this afterward.

Author

Commented:
not behind any proxy as far as i am aware?

Author

Commented:
command prompt doesn't understand the commands.

i have copied and pasted each line separately and then together.

Author

Commented:
the last file is the latest code and error.
Syntax for command prompt is e.g.,
python script1.py
Where script1.py contains your python code
Please learn to use CODE tags.
Attachments are not viewable from my phone.

Author

Commented:
import urllib.request
import json
from pprint import pprint

frequency = 14400

tickers = ['USDT_BTC','USDT_ETC','USDT_XMR','USDT_ETH','USDT_DASH', 'USDT_XRP','USDT_LTC','USDT_NXT','USDT_STR','USDT_REP','USDT_ZEC']

for symbol in tickers:
    url ='https://poloniex.com/public?command=returnChartData&currencyPair='+symbol+'&end=9999999999&period='+str(frequency)+'&start=0'
    f = urllib.request.urlopen(url)
    d = json.load(f)
    f.close()
    len(d)
    print(symbol)
    pprint(d[:2])

Open in new window

Try from the commnaD prompt and try to find out definitively about a possible proxy server.

Author

Commented:
wins proxy enabled ....:No

Author

Commented:
i don't think i know what i am doing in command prompt.

Author

Commented:
cmd prompt gives the following response:

C:\WINDOWS\system32>C:\Users\Michael\Anaconda3\pkgs\python-3.6.3-h9e2ca53_1\python.exe
Python 3.6.3 |Anaconda, Inc.| (default, Oct 15 2017, 03:27:45) [MSC v.1900 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import urllib.request
>>> import json
>>> from pprint import pprint
>>>
>>> frequency = 14400
>>>
>>> tickers = ['USDT_BTC','USDT_ETC','USDT_XMR','USDT_ETH','USDT_DASH', 'USDT_XRP','USDT_LTC','USDT_NXT','USDT_STR','USDT_REP','USDT_ZEC']
>>>
>>> for symbol in tickers:
...     url ='https://poloniex.com/public?command=returnChartData¤cyPair='+symbol+'&end=9999999999&period='+str(frequency)+'&start=0'
...     f = urllib.request.urlopen(url)
...     d = json.load(f)
...     f.close()
...     len(d)
...     print(symbol)
...     pprint(d[:2])
...     pprint(d[:2])
...
Traceback (most recent call last):
  File "<stdin>", line 3, in <module>
  File "C:\Users\Michael\Anaconda3\pkgs\python-3.6.3-h9e2ca53_1\lib\urllib\request.py", line 223, in urlopen
    return opener.open(url, data, timeout)
  File "C:\Users\Michael\Anaconda3\pkgs\python-3.6.3-h9e2ca53_1\lib\urllib\request.py", line 532, in open
    response = meth(req, response)
  File "C:\Users\Michael\Anaconda3\pkgs\python-3.6.3-h9e2ca53_1\lib\urllib\request.py", line 642, in http_response
    'http', request, response, code, msg, hdrs)
  File "C:\Users\Michael\Anaconda3\pkgs\python-3.6.3-h9e2ca53_1\lib\urllib\request.py", line 570, in error
    return self._call_chain(*args)
  File "C:\Users\Michael\Anaconda3\pkgs\python-3.6.3-h9e2ca53_1\lib\urllib\request.py", line 504, in _call_chain
    result = func(*args)
  File "C:\Users\Michael\Anaconda3\pkgs\python-3.6.3-h9e2ca53_1\lib\urllib\request.py", line 650, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 403: Forbidden
Well then ask your network admins about your internet connection as the exact same code works from my laptop.
I noticed you did not try my latest code.
Copy the code and save to your disk as flabio.py
Then from the command prompt, type
python flabio.py
save the attached to your disk then from the command prompt, run:
python flabio.py 1> output.txt 2>&1

Open in new window


this will create a file named output.txt
please post the contents of output.txt here
flabio.py
For reference, here's my output.txt showing the code works just fine from here.
output.txt

Author

Commented:
is the enclosed not the latest code?

i have run this using python flabio.py and it gives the following error:

(C:\Users\Michael\Anaconda3) C:\Users\Michael>python.exe
Python 3.6.3 |Anaconda, Inc.| (default, Oct 15 2017, 03:27:45) [MSC v.1900 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> python flabio.py
  File "<stdin>", line 1
    python flabio.py
                ^
SyntaxError: invalid syntax
flabio.py.txt
Command line invocation of python accepts parameters.
So you type:
python.exe flabio.py
Then hit the enter key
The .exe is optional
Oh and that the url works from a browser does not rule out a proxy server.
Get help from someone on your network team to determine your internet connection settings.
I was not asking if you had a proxy server running on your machine.

Author

Commented:
still getting an error using command prompt.

C:\WINDOWS\system32>C:\Users\Michael\Anaconda3\PYTHON.EXE
Python 3.6.3 |Anaconda, Inc.| (default, Oct 15 2017, 03:27:45) [MSC v.1900 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> python.exe flabio.py
  File "<stdin>", line 1
    python.exe flabio.py
                    ^
SyntaxError: invalid syntax

Author

Commented:
with regards to using a proxy server. i am doing this on a private subscription through an ISP in Nigeria. can I get the proxy settings logging into my modem?
You type that at the command prompt.
You do not launch python, then type python again.
Ex.,
C:\users\flabio>python.exe flabio.py
Read up on command prompt invocation of python. You need to run the script flabio.py then send the output. I've sent how to capture this earlier.
Read up on command prompts.
Are you running some sort of anonymizer?
Can you try from a different machine/location?
Try asking your ISP for help.

Author

Commented:
does python.exe and flabio.py have to be in the same folder? if so does it matter where?
Why do you not follow instructions?
I said launch python like:
C:\users\flabio>python.exe flabio.py

You however are doing:
C:\users\flabio>python.exe
Then typing:
python.exe flabio.py

There's a huge difference!
If you cannot follow instructions then I cannot help you anymore. Are you alone? Try getting somebody to help you.

Author

Commented:
apologies if i am not getting it.

when i do this command (assuming i have it right now) i get a blue pop up saying "this app cant run on your PC", to find a version for your PC, check  with your PC publisher.

then in CMD it says:

C:\Users\Michael>python.exe flabio.py
Access is denied.
Question inactive

Do more with

Expert Office
Submit tech questions to Ask the Experts™ at any time to receive solutions, advice, and new ideas from leading industry professionals.

Start 7-Day Free Trial