• Status: Solved
  • Priority: Low
  • Security: Public
  • Views: 294
  • Last Modified:

Python Issues

i appear to be having issues using anaconda.
i have copied the code into Jupyter note as follows:
In [1]:
def CryptoDataCSV(symbol, frequency):

#Params: String symbol, int frequency = 300,900,1800,7200,14400,86400

#Returns: df from first available date

url ='https://poloniex.com/public?command=returnChartData¤cyPair='+symbol+'&end=9999999999&period='+str(frequency)+'&start=0'

df = pd.read_json(url)

df.set_index('date',inplace=True)

df.to_csv(symbol + '.csv')

print('Processed: ' + symbol)
In [2]:
import pandas as pd
import matplotlib.pyplot as plt
%matplotlib inline
In [3]:
tickers = ['USDT_BTC','USDT_ETC','USDT_XMR','USDT_ETH','USDT_DASH',
'USDT_XRP','USDT_LTC','USDT_NXT','USDT_STR','USDT_REP','USDT_ZEC']
In [4]:
crypto_df = pd.DataFrame()
for ticker in tickers:
crypto_df[ticker] = pd.read_csv(ticker+'.csv', index_col = 'date')['close']
crypto_df.dropna(inplace=True)
---------------------------------------------------------------------------
FileNotFoundError                         Traceback (most recent call last)
<ipython-input-4-b59f4116fa1a> in <module>()
      1 crypto_df = pd.DataFrame()
      2 for ticker in tickers:
----> 3     crypto_df[ticker] = pd.read_csv(ticker+'.csv', index_col = 'date')['close']
      4 crypto_df.dropna(inplace=True)
~\Anaconda3\lib\site-packages\pandas\io\parsers.py in parser_f(filepath_or_buffer, sep, delimiter, header, names, index_col, usecols, squeeze, prefix, mangle_dupe_cols, dtype, engine, converters, true_values, false_values, skipinitialspace, skiprows, nrows, na_values, keep_default_na, na_filter, verbose, skip_blank_lines, parse_dates, infer_datetime_format, keep_date_col, date_parser, dayfirst, iterator, chunksize, compression, thousands, decimal, lineterminator, quotechar, quoting, escapechar, comment, encoding, dialect, tupleize_cols, error_bad_lines, warn_bad_lines, skipfooter, skip_footer, doublequote, delim_whitespace, as_recarray, compact_ints, use_unsigned, low_memory, buffer_lines, memory_map, float_precision)
    653                     skip_blank_lines=skip_blank_lines)
    654
--> 655         return _read(filepath_or_buffer, kwds)
    656
    657     parser_f.__name__ = name
~\Anaconda3\lib\site-packages\pandas\io\parsers.py in _read(filepath_or_buffer, kwds)
    403
    404     # Create the parser.
--> 405     parser = TextFileReader(filepath_or_buffer, **kwds)
    406
    407     if chunksize or iterator:
~\Anaconda3\lib\site-packages\pandas\io\parsers.py in __init__(self, f, engine, **kwds)
    762             self.options['has_index_names'] = kwds['has_index_names']
    763
--> 764         self._make_engine(self.engine)
    765
    766     def close(self):
~\Anaconda3\lib\site-packages\pandas\io\parsers.py in _make_engine(self, engine)
    983     def _make_engine(self, engine='c'):
    984         if engine == 'c':
--> 985             self._engine = CParserWrapper(self.f, **self.options)
    986         else:
    987             if engine == 'python':
~\Anaconda3\lib\site-packages\pandas\io\parsers.py in __init__(self, src, **kwds)
   1603         kwds['allow_leading_cols'] = self.index_col is not False
   1604
-> 1605         self._reader = parsers.TextReader(src, **kwds)
   1606
   1607         # XXX
pandas/_libs/parsers.pyx in pandas._libs.parsers.TextReader.__cinit__ (pandas\_libs\parsers.c:4209)()
pandas/_libs/parsers.pyx in pandas._libs.parsers.TextReader._setup_parser_source (pandas\_libs\parsers.c:8873)()
FileNotFoundError: File b'USDT_BTC.csv' does not exist

I GET THE ABOVE ERROR. BASICALLY NOTHING HAPPENS!
i would appreciate help. My aim is to use this to download data for top 50 currencies to run in a forecasting and risk analysis model.
0
michael ogunbiyi
Asked:
michael ogunbiyi
  • 32
  • 24
1 Solution
 
Flabio GatesCommented:
Can you re-post with both the source code and the traceback wrapped in CODE tags? It is difficult to read as it is now.
0
 
michael ogunbiyiAuthor Commented:
import matplotlib.pyplot as plt
In [1]:
 
 
%matplotlib inline
In [2]:
 
 
def CryptoDataCSV(symbol, frequency):

    #Params: String symbol, int frequency = 300,900,1800,7200,14400,86400

    #Returns: df from first available date

    url ='https://poloniex.com/public?command=returnChartData¤cyPair='+symbol+'&end=9999999999&period='+str(frequency)+'&start=0'

    df = pd.read_json(url)

    df.set_index('date',inplace=True)

    df.to_csv(symbol + '.csv')

    print('Processed: ' + symbol)

In [3]:
 
 
tickers = ['USDT_BTC','USDT_ETC','USDT_XMR','USDT_ETH','USDT_DASH',
 'USDT_XRP','USDT_LTC','USDT_NXT','USDT_STR','USDT_REP','USDT_ZEC']

In [4]:
 
x
for ticker in tickers:
    CryptoDataCSV(ticker, 86400)['close']
---------------------------------------------------------------------------
HTTPError                                 Traceback (most recent call last)
<ipython-input-6-46cec03c9a07> in <module>()
      1 for ticker in tickers:
----> 2     CryptoDataCSV(ticker, 86400)['close']

<ipython-input-4-1ad6182137ea> in CryptoDataCSV(symbol, frequency)
      7     url ='https://poloniex.com/public?command=returnChartData¤cyPair='+symbol+'&end=9999999999&period='+str(frequency)+'&start=0'
      8
----> 9     df = pd.read_json(url)
     10
     11     df.set_index('date',inplace=True)

~\Anaconda3\lib\site-packages\pandas\io\json\json.py in read_json(path_or_buf, orient, typ, dtype, convert_axes, convert_dates, keep_default_dates, numpy, precise_float, date_unit, encoding, lines)
    320
    321     filepath_or_buffer, _, _ = get_filepath_or_buffer(path_or_buf,
--> 322                                                       encoding=encoding)
    323     if isinstance(filepath_or_buffer, compat.string_types):
    324         try:

~\Anaconda3\lib\site-packages\pandas\io\common.py in get_filepath_or_buffer(filepath_or_buffer, encoding, compression)
    184     if _is_url(filepath_or_buffer):
    185         url = str(filepath_or_buffer)
--> 186         req = _urlopen(url)
    187         content_encoding = req.headers.get('Content-Encoding', None)
    188         if content_encoding == 'gzip':

~\Anaconda3\lib\urllib\request.py in urlopen(url, data, timeout, cafile, capath, cadefault, context)
    221     else:
    222         opener = _opener
--> 223     return opener.open(url, data, timeout)
    224
    225 def install_opener(opener):

~\Anaconda3\lib\urllib\request.py in open(self, fullurl, data, timeout)
    530         for processor in self.process_response.get(protocol, []):
    531             meth = getattr(processor, meth_name)
--> 532             response = meth(req, response)
    533
    534         return response

~\Anaconda3\lib\urllib\request.py in http_response(self, request, response)
    640         if not (200 <= code < 300):
    641             response = self.parent.error(
--> 642                 'http', request, response, code, msg, hdrs)
    643
    644         return response

~\Anaconda3\lib\urllib\request.py in error(self, proto, *args)
    568         if http_err:
    569             args = (dict, 'default', 'http_error_default') + orig_args
--> 570             return self._call_chain(*args)
    571
    572 # XXX probably also want an abstract factory that knows when it makes

~\Anaconda3\lib\urllib\request.py in _call_chain(self, chain, kind, meth_name, *args)
    502         for handler in handlers:
    503             func = getattr(handler, meth_name)
--> 504             result = func(*args)
    505             if result is not None:
    506                 return result

~\Anaconda3\lib\urllib\request.py in http_error_default(self, req, fp, code, msg, hdrs)
    648 class HTTPDefaultErrorHandler(BaseHandler):
    649     def http_error_default(self, req, fp, code, msg, hdrs):
--> 650         raise HTTPError(req.full_url, code, msg, hdrs, fp)
    651
    652 class HTTPRedirectHandler(BaseHandler):

HTTPError: HTTP Error 403: Forbidden
0
 
Flabio GatesCommented:
Can you give me a sample of a valid value for "symbol?"
Oh and when I said CODE tags, I meant use one of the buttons above the area where you type your comments.
(on my phone, these buttons are to the left of the text area)
0
Cloud Class® Course: C++ 11 Fundamentals

This course will introduce you to C++ 11 and teach you about syntax fundamentals.

 
Flabio GatesCommented:
Found a sample URL for the returnChartData call:
https://poloniex.com/public?command=returnChartData&currencyPair=BTC_XMR&start=1405699200&end=9999999999&period=14400

Open in new window

Can you modify your code accordingly? I.e., change
url ='https://poloniex.com/public?command=returnChartData¤cyPair='+symbol+'&end=9999999999&period='+str(frequency)+'&start=0'

Open in new window

to
url ='https://poloniex.com/public?command=returnChartData&currencyPair='+symbol+'&end=9999999999&period='+str(frequency)+'&start=0'

Open in new window

Note the ampersand after returnChartData and currencyPair instead of cyPair.
0
 
michael ogunbiyiAuthor Commented:
I PASTED:

url ='https://poloniex.com/public?command=returnChartData¤cyPair='+symbol+'&end=9999999999&period='+str(frequency)+'&start=0'

IT NOW BRINGS THE FOLLOWING ERROR.


  File "<ipython-input-1-c7ee74ec9df5>", line 9
    df = pd.read_json(url)
    ^
IndentationError: unexpected indent
0
 
Flabio GatesCommented:
you did not change the part of the URL I told you to change
command=returnChartData¤cyPair=

Open in new window

should be
command=returnChartData&currencyPair=

Open in new window

you should retain what indentation you already had.
you should really use CODE tags when posting code.
0
 
Flabio GatesCommented:
alternatively, you can Attach your source code so we can see your properly indented code.
(see the "Attach File" link below the comment text area)
0
 
Flabio GatesCommented:
the code in the link you shared shows the correct URL parameters
def CryptoDataCSV(symbol, frequency):
    #Params: String symbol, int frequency = 300,900,1800,7200,14400,86400
    #Returns: df from first available date
    url ='https://poloniex.com/public?command=returnChartData&currencyPair='+symbol+'&end=9999999999&period='+str(frequency)+'&start=0'
    df = pd.read_json(url)
    ...

Open in new window

I.e., ...command=returnChartData&currencyPair=...
Your version is not the same.
0
 
michael ogunbiyiAuthor Commented:
i really don't knows what i am doing wrong here.

see attached.
Untitled--2-.html
0
 
Flabio GatesCommented:
Ok, let's try isolating the problem. Can you try running the following code:
import urllib
import json
from pprint import pprint

frequency = 14400

tickers = ['USDT_BTC','USDT_BCH','USDT_ETC','USDT_XMR','USDT_ETH','USDT_DASH', 'USDT_XRP','USDT_LTC','USDT_NXT','USDT_STR','USDT_REP','USDT_ZEC']

for symbol in tickers:
    url ='https://poloniex.com/public?command=returnChartData&currencyPair='+symbol+'&end=9999999999&period='+str(frequency)+'&start=0'

    f = urllib.urlopen(url)
    d = json.load(f)
    f.close()
    len(d)
    print symbol
    pprint(d[:2])

Open in new window


Are you, by the way, using a proxy server to connect to the internet?
0
 
michael ogunbiyiAuthor Commented:
now gives this error:



  File "<ipython-input-49-98f46ec6a299>", line 16
    print symbol
               ^
SyntaxError: Missing parentheses in call to 'print'. Did you mean print(int symbol)?
0
 
Flabio GatesCommented:
so you're using version 3. use
print(symbol)

Open in new window

no int
0
 
Flabio GatesCommented:
wait... will update code for version 3 of python
0
 
Flabio GatesCommented:
updated for python 3:
import urllib.request
import json
from pprint import pprint

frequency = 14400

tickers = ['USDT_BTC','USDT_BCH','USDT_ETC','USDT_XMR','USDT_ETH','USDT_DASH', 'USDT_XRP','USDT_LTC','USDT_NXT','USDT_STR','USDT_REP','USDT_ZEC']

for symbol in tickers:
    url ='https://poloniex.com/public?command=returnChartData&currencyPair='+symbol+'&end=9999999999&period='+str(frequency)+'&start=0'

    f = urllib.request.urlopen(url)
    d = json.load(f)
    f.close()
    len(d)
    print(symbol)
    pprint(d[:2])

Open in new window

0
 
michael ogunbiyiAuthor Commented:
thanks.

much appreciated
0
 
michael ogunbiyiAuthor Commented:
now back to http error
Untitled--3-.html
0
 
Flabio GatesCommented:
Can you try running the script from the command prompt and not from Anaconda?
from urllib.request import Request, urlopen
import json
from pprint import pprint

user_agent = 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.7) Gecko/2009021910 Firefox/3.0.7'
headers = {'User-Agent':user_agent}

frequency = 14400

tickers = ['USDT_BTC','USDT_BCH','USDT_ETC','USDT_XMR','USDT_ETH','USDT_DASH', 'USDT_XRP','USDT_LTC','USDT_NXT','USDT_STR','USDT_REP','USDT_ZEC']

for symbol in tickers:
    url ='https://poloniex.com/public?command=returnChartData&currencyPair='+symbol+'&end=9999999999&period='+str(frequency)+'&start=0'
    req = Request(url, headers=headers)
    try:
        f = urlopen(req)
        d = json.load(f)
        f.close()
    except urllib.request.HTTPError as e:
        print(e.read())
        raise
    else:
        len(d)
        print(symbol)
        pprint(d[:2])

Open in new window


Again, are you behind a proxy server?
0
 
michael ogunbiyiAuthor Commented:
command prompt on windows?

cmd?
0
 
michael ogunbiyiAuthor Commented:
the code cleaned up?
Untitled.htm
0
 
Flabio GatesCommented:
"cmd?"
yes
0
 
Flabio GatesCommented:
the code cleaned up?
What do you mean by this?
0
 
michael ogunbiyiAuthor Commented:
i had removed the duplicated code note. and run in sequence.
Untitled.htm
0
 
Flabio GatesCommented:
i had removed the duplicated code note. and run in sequence.
You lost me. I don't get what you are saying.
0
 
Flabio GatesCommented:
have you tried from CMD?
from urllib.request import Request, urlopen, HTTPError
import json
from pprint import pprint

user_agent = 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.7) Gecko/2009021910 Firefox/3.0.7'
headers = {'User-Agent':user_agent}

frequency = 14400

tickers = ['USDT_BTC','USDT_BCH','USDT_ETC','USDT_XMR','USDT_ETH','USDT_DASH', 'USDT_XRP','USDT_LTC','USDT_NXT','USDT_STR','USDT_REP','USDT_ZEC']

for symbol in tickers:
    url ='https://poloniex.com/public?command=returnChartData&currencyPair='+symbol+'&end=9999999999&period='+str(frequency)+'&start=0'
    print(url)
    req = Request(url, headers=headers)
    try:
        f = urlopen(req)
        d = json.load(f)
        f.close()
    except HTTPError as e:
        print(e.read())
        raise
    else:
        print(symbol)
        pprint(d[:2])

Open in new window

0
 
Flabio GatesCommented:
You have not answered my question regarding your possible use of a proxy server to connect to the internet.
Are you or are you not behind a proxy server?
0
 
Flabio GatesCommented:
I have some errands to run but will try to get back to this afterward.
0
 
michael ogunbiyiAuthor Commented:
not behind any proxy as far as i am aware?
0
 
michael ogunbiyiAuthor Commented:
command prompt doesn't understand the commands.

i have copied and pasted each line separately and then together.
0
 
michael ogunbiyiAuthor Commented:
0
 
michael ogunbiyiAuthor Commented:
the last file is the latest code and error.
0
 
Flabio GatesCommented:
Syntax for command prompt is e.g.,
python script1.py
Where script1.py contains your python code
0
 
Flabio GatesCommented:
Please learn to use CODE tags.
Attachments are not viewable from my phone.
0
 
michael ogunbiyiAuthor Commented:
import urllib.request
import json
from pprint import pprint

frequency = 14400

tickers = ['USDT_BTC','USDT_ETC','USDT_XMR','USDT_ETH','USDT_DASH', 'USDT_XRP','USDT_LTC','USDT_NXT','USDT_STR','USDT_REP','USDT_ZEC']

for symbol in tickers:
    url ='https://poloniex.com/public?command=returnChartData&currencyPair='+symbol+'&end=9999999999&period='+str(frequency)+'&start=0'
    f = urllib.request.urlopen(url)
    d = json.load(f)
    f.close()
    len(d)
    print(symbol)
    pprint(d[:2])

Open in new window

0
 
Flabio GatesCommented:
Try from the commnaD prompt and try to find out definitively about a possible proxy server.
0
 
michael ogunbiyiAuthor Commented:
wins proxy enabled ....:No
0
 
michael ogunbiyiAuthor Commented:
i don't think i know what i am doing in command prompt.
0
 
michael ogunbiyiAuthor Commented:
cmd prompt gives the following response:

C:\WINDOWS\system32>C:\Users\Michael\Anaconda3\pkgs\python-3.6.3-h9e2ca53_1\python.exe
Python 3.6.3 |Anaconda, Inc.| (default, Oct 15 2017, 03:27:45) [MSC v.1900 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import urllib.request
>>> import json
>>> from pprint import pprint
>>>
>>> frequency = 14400
>>>
>>> tickers = ['USDT_BTC','USDT_ETC','USDT_XMR','USDT_ETH','USDT_DASH', 'USDT_XRP','USDT_LTC','USDT_NXT','USDT_STR','USDT_REP','USDT_ZEC']
>>>
>>> for symbol in tickers:
...     url ='https://poloniex.com/public?command=returnChartData¤cyPair='+symbol+'&end=9999999999&period='+str(frequency)+'&start=0'
...     f = urllib.request.urlopen(url)
...     d = json.load(f)
...     f.close()
...     len(d)
...     print(symbol)
...     pprint(d[:2])
...     pprint(d[:2])
...
Traceback (most recent call last):
  File "<stdin>", line 3, in <module>
  File "C:\Users\Michael\Anaconda3\pkgs\python-3.6.3-h9e2ca53_1\lib\urllib\request.py", line 223, in urlopen
    return opener.open(url, data, timeout)
  File "C:\Users\Michael\Anaconda3\pkgs\python-3.6.3-h9e2ca53_1\lib\urllib\request.py", line 532, in open
    response = meth(req, response)
  File "C:\Users\Michael\Anaconda3\pkgs\python-3.6.3-h9e2ca53_1\lib\urllib\request.py", line 642, in http_response
    'http', request, response, code, msg, hdrs)
  File "C:\Users\Michael\Anaconda3\pkgs\python-3.6.3-h9e2ca53_1\lib\urllib\request.py", line 570, in error
    return self._call_chain(*args)
  File "C:\Users\Michael\Anaconda3\pkgs\python-3.6.3-h9e2ca53_1\lib\urllib\request.py", line 504, in _call_chain
    result = func(*args)
  File "C:\Users\Michael\Anaconda3\pkgs\python-3.6.3-h9e2ca53_1\lib\urllib\request.py", line 650, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 403: Forbidden
0
 
Flabio GatesCommented:
Well then ask your network admins about your internet connection as the exact same code works from my laptop.
0
 
Flabio GatesCommented:
I noticed you did not try my latest code.
Copy the code and save to your disk as flabio.py
Then from the command prompt, type
python flabio.py
0
 
Flabio GatesCommented:
save the attached to your disk then from the command prompt, run:
python flabio.py 1> output.txt 2>&1

Open in new window


this will create a file named output.txt
please post the contents of output.txt here
flabio.py
0
 
Flabio GatesCommented:
For reference, here's my output.txt showing the code works just fine from here.
output.txt
0
 
michael ogunbiyiAuthor Commented:
is the enclosed not the latest code?

i have run this using python flabio.py and it gives the following error:

(C:\Users\Michael\Anaconda3) C:\Users\Michael>python.exe
Python 3.6.3 |Anaconda, Inc.| (default, Oct 15 2017, 03:27:45) [MSC v.1900 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> python flabio.py
  File "<stdin>", line 1
    python flabio.py
                ^
SyntaxError: invalid syntax
flabio.py.txt
0
 
michael ogunbiyiAuthor Commented:
0
 
Flabio GatesCommented:
Command line invocation of python accepts parameters.
So you type:
python.exe flabio.py
Then hit the enter key
The .exe is optional
0
 
Flabio GatesCommented:
Oh and that the url works from a browser does not rule out a proxy server.
Get help from someone on your network team to determine your internet connection settings.
I was not asking if you had a proxy server running on your machine.
0
 
michael ogunbiyiAuthor Commented:
still getting an error using command prompt.

C:\WINDOWS\system32>C:\Users\Michael\Anaconda3\PYTHON.EXE
Python 3.6.3 |Anaconda, Inc.| (default, Oct 15 2017, 03:27:45) [MSC v.1900 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> python.exe flabio.py
  File "<stdin>", line 1
    python.exe flabio.py
                    ^
SyntaxError: invalid syntax
0
 
michael ogunbiyiAuthor Commented:
with regards to using a proxy server. i am doing this on a private subscription through an ISP in Nigeria. can I get the proxy settings logging into my modem?
0
 
Flabio GatesCommented:
You type that at the command prompt.
You do not launch python, then type python again.
Ex.,
C:\users\flabio>python.exe flabio.py
0
 
Flabio GatesCommented:
Read up on command prompt invocation of python. You need to run the script flabio.py then send the output. I've sent how to capture this earlier.
Read up on command prompts.
Are you running some sort of anonymizer?
Can you try from a different machine/location?
Try asking your ISP for help.
0
 
michael ogunbiyiAuthor Commented:
does python.exe and flabio.py have to be in the same folder? if so does it matter where?
0
 
Flabio GatesCommented:
Why do you not follow instructions?
I said launch python like:
C:\users\flabio>python.exe flabio.py

You however are doing:
C:\users\flabio>python.exe
Then typing:
python.exe flabio.py

There's a huge difference!
0
 
Flabio GatesCommented:
If you cannot follow instructions then I cannot help you anymore. Are you alone? Try getting somebody to help you.
0
 
michael ogunbiyiAuthor Commented:
apologies if i am not getting it.

when i do this command (assuming i have it right now) i get a blue pop up saying "this app cant run on your PC", to find a version for your PC, check  with your PC publisher.

then in CMD it says:

C:\Users\Michael>python.exe flabio.py
Access is denied.
0
 
Flabio GatesCommented:
Question inactive
0
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

Join & Write a Comment

Featured Post

Free Tool: Site Down Detector

Helpful to verify reports of your own downtime, or to double check a downed website you are trying to access.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

  • 32
  • 24
Tackle projects and never again get stuck behind a technical roadblock.
Join Now