Link to home
Start Free TrialLog in
Avatar of coder
coderFlag for Australia

asked on

connecting to postgres, elastic search on ubuntu with dev machine on windows

Hi Experts,

How to connect to postgres, elastic search server in linux, ubuntu on AWS to  with Dango-cms or Wagtail cms from dev machine on windows

Please tell me how elastic search details are stored in database and how to connect to do a elastic search which is in postgres database.

All indexes of elastic search is stored in postgres sql.  How will i connect to postgres from windows dev machine.

And How can I access the postgres database in ubuntu aws to development machine have django-cms or wagtail cms on windows.

 ES_CONNECTION=172.17.0.1:9200,

database connection string is created with dj_database_url  the connection string is DATABASE=postgres://user:password@server/database.

Please help me,

I get the following error when I build the wagatail cms - based on python
python manage.py makemigrations

throws the below error.

django.db.utils.OperationalError: could not connect to server: Connection timed out (0x0000274C/10060)
        Is the server running on host "172.17.0.1" and accepting
        TCP/IP connections on port 5432?

I connect string sample postgres sample is postgres://test1:test2@172.17.0.1/database

When I go inside the aws machine through putty (ssh) 54.*.*.*:22

then

telnet 172.17.0.1 5432

It is connection is success full.

but not able to connect from outside aws machine from a windows pc (development machine).

With Many Thanks,
Bharath AK
ASKER CERTIFIED SOLUTION
Avatar of Marcin Karkocha
Marcin Karkocha
Flag of Poland image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
SOLUTION
Avatar of earth man2
earth man2
Flag of United Kingdom of Great Britain and Northern Ireland image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Avatar of coder

ASKER

Hi there,

           In elasticsearch.yml [on ubuntu machine - aws ] is network.host=172.17.0.1 Do I have to provide external ip address for this network.host? or this 172.17.0.1 is okay?   For postgressql  I am connecting like DATABASE=postgres://user:password@172.17.0.1/database,  by using dj-database-url.py

I had tried by providing the external ip address to network.host in elastic search.yml and postgres database connection DATABASE=postgres://user:password@externalip/database  i am getting the same error.

Still getting the same error, with ipaddress changed?  Please correct where I am wrong

With Many Thanks
Bharath AK
Avatar of coder

ASKER

Hi Experts,

        I had added  the local pc's external address in pg_hba.conf

host all all x.x.x.x/24 trust               ->(windows pc external ip)
listen_addresses = '*' in postgresql.conf

es_connection to external-ip address linux pc

database connection in local pc(windows) to external ipaddres of linux (server's external address)

Please correct where i am wrong.

With Many thanks,
Bharath AK
Avatar of coder

ASKER

Hi There,

     I had changed the port number to 5432 ES_CONNECTION TO 172.17.0.1:5432  still not working.  Please help in resolving this.

With Many thanks,
Bharath AK
Avatar of coder

ASKER

Hi Experts,

        I need clarity on how to store the elastic search index on postgres and what is network host settings in elastic search configuration file. (yml file).  Please throw some light on it.

With Many Thanks,
Bharath AK
SOLUTION
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Avatar of coder

ASKER

Hi Experts,

       The endpoint ports are already open in aws.  

With Many Thanks,
Bharath AK
Avatar of coder

ASKER

Hi Experts,

     Now that error got fixed I had commented tcpip_socket = true in postgresql.conf.

Now that error has vanished but not able see the page on the browser.

    I had tried python3 manage.py runserver 0.0.0.0:8000

I had tried https://54.*.*.*:8000 

I get the following error.

code 400, message Bad request syntax ('\x16\x03\x01\x00®\x01\x00\x00ª\x03\x03\x80éî·SÙq~|\\\xa0\x1b\x82 O)\xa0Ç\x8e²\x17æ0k©\x12\x17IÍ\x06j \x00\x00\x1c\x8a\x8aÀ+À/À,À0̨̩À\x13À\x14\x00\x9c\x00\x9d\x00/\x005\x00')
You're accessing the development server over HTTPS, but it only supports HTTP.

code 400, message Bad HTTP/0.9 request type ('\x16\x03\x01\x00®\x01\x00\x00ª\x03\x03RÅ*\x0e\x8c\x7fó<p9ÁI=\x92\x8a\x8fÞj\x988-Ó=Y\x96ð\x9b]Ãf\x14¶\x00\x00')
code 400, message Bad request version ('JJÀ+À/À,À0̨̩À\x13À\x14\x00\x9c\x00\x9d\x00/\x005\x00')
You're accessing the development server over HTTPS, but it only supports HTTP.

You're accessing the development server over HTTPS, but it only supports HTTP.

You're accessing the development server over HTTPS, but it only supports HTTP.

When I try http://*.*.*.*/  I don't errors on console.  But I get site cannot be reached and refused to connect on the browser.

I don't know what setting I miss.

now when I tried once again

I got this following error.

Environment:


Request Method: GET
Request URL: http://54.*.*.*:8000/

Django Version: 1.11.1
Python Version: 3.6.4
Installed Applications:
['django.contrib.admin.apps.SimpleAdminConfig',
 'django.contrib.auth',
 'django.contrib.contenttypes',
 'django.contrib.sessions',
 'django.contrib.messages',
 'django.contrib.staticfiles',
 'django.contrib.sitemaps',
 'cms',
 'search',
 'events',
 'wagtail.wagtailforms',
 'wagtail.wagtailredirects',
 'wagtail.wagtailembeds',
 'wagtail.wagtailsites',
 'wagtail.wagtailusers',
 'wagtail.wagtailsnippets',
 'wagtail.wagtaildocs',
 'wagtail.wagtailimages',
 'wagtail.wagtailsearch',
 'wagtail.wagtailadmin',
 'wagtail.wagtailcore',
 'modelcluster',
 'taggit',
 'adminplus',
 'pipeline',
 'webpack_loader',
 'rest_framework',
 'django_ses',
 'wagtailfontawesome',
 'storages',
 'debug_toolbar']
Installed Middleware:
['django.middleware.gzip.GZipMiddleware',
 'debug_toolbar.middleware.DebugToolbarMiddleware',
 'ntdl.middleware.ExceptionLoggingMiddleware',
 'django.middleware.security.SecurityMiddleware',
 'django.contrib.sessions.middleware.SessionMiddleware',
 'django.middleware.common.CommonMiddleware',
 'django.middleware.csrf.CsrfViewMiddleware',
 'django.contrib.auth.middleware.AuthenticationMiddleware',
 'django.contrib.messages.middleware.MessageMiddleware',
 'django.middleware.clickjacking.XFrameOptionsMiddleware',
 'wagtail.wagtailcore.middleware.SiteMiddleware',
 'wagtail.wagtailredirects.middleware.RedirectMiddleware']



Traceback:

File "/usr/local/lib/python3.6/dist-packages/django/core/handlers/exception.py" in inner
  41.             response = get_response(request)

File "/usr/local/lib/python3.6/dist-packages/django/utils/deprecation.py" in __call__
  138.             response = self.process_request(request)

File "/usr/local/lib/python3.6/dist-packages/django/middleware/security.py" in process_request
  25.             host = self.redirect_host or request.get_host()

File "/usr/local/lib/python3.6/dist-packages/django/http/request.py" in get_host
  105.         if domain and validate_host(domain, allowed_hosts):

File "/usr/local/lib/python3.6/dist-packages/django/http/request.py" in validate_host
  579.         if pattern == '*' or is_same_domain(host, pattern):

File "/usr/local/lib/python3.6/dist-packages/django/utils/http.py" in is_same_domain
  291.     pattern = pattern.lower()

Exception Type: AttributeError at /
Exception Value: 'ellipsis' object has no attribute 'lower'


If I try once again http://54.*.*.*:800

I get error connection refused. on browser. without the above logs

Please tell me where I missing.

With Many Thanks,

Bharath AK
Avatar of coder

ASKER

Hi There,

     complete error list on the top of the broweser

Request Method:      GET
Request URL:      http://54.*.*.*:8000/
Django Version:      1.11.1
Exception Type:      AttributeError
Exception Value:      
'ellipsis' object has no attribute 'lower'
Exception Location:      /usr/local/lib/python3.6/dist-packages/django/utils/http.py in is_same_domain, line 291
Python Executable:      /usr/bin/python3
Python Version:      3.6.4
Python Path:      
['/home/ubuntu/workarea/sourcecode/ntdl',
 '/usr/lib/python36.zip',
 '/usr/lib/python3.6',
 '/usr/lib/python3.6/lib-dynload',
 '/usr/local/lib/python3.6/dist-packages',
 '/usr/lib/python3/dist-packages']

With Many Thanks,
Bharath AK
Avatar of coder

ASKER

Hi Experts,

        I am able to resolve this issue.  I went inside the http.py and print the value of  the variable which was throwing error.  It was printing test_site, test_site2, then '...'.  Then I had checked the test_site in my project, it was in settings.py file for the variable ALLOWED_HOSTS.
once I change the value of ALLOWED_HOSTS ellipses error vanished.  Then I ran the url in postman.  I got few errors for not initializing  the variables on environment variables like Facebook id, and search path, Once I Initialized all the variables errors vanished I am able to see the application on the browser.

With Many Thanks,
Bharath AK
Avatar of coder

ASKER

Hi Experts,

Thank you all.  I am able to resolve this issue.

Kind Regards,
Bharath AK