Want to protect your cyber security and still get fast solutions? Ask a secure question today.Go Premium

x
?
Solved

Installing Sphinx on a master-slave setup

Posted on 2011-10-02
11
Medium Priority
?
1,583 Views
Last Modified: 2012-05-12
Hi all,

On my quest to find the best search solution I have come across Sphinx which comes highly recommended.

My question is about the setup structure on the following server setup:

2 x webservers load balanced
1 x master (2gb mem, 2 cores)
1 x slave (3gb mem, 4 cores)

All ubuntu.

I have mysql replication across master and slave and now I want to introduce Sphinx.

Ideally I'd want to seed the index on the master and have the sphinx-queries sent to the slave via the php api.

Has anyone got any experience doing this? Is the index r-synced across to the slave to keep it in sync? Do I just install sphinx on the master? What about the web-servers and the php-api, do they both need sphinx installed too?

Any help getting me started is much appreciated!
0
Comment
Question by:dolythgoe
  • 6
  • 4
11 Comments
 
LVL 111

Expert Comment

by:Ray Paseur
ID: 36900986
In my experience Sphinx only comes highly recommended by people who have not really used Sphinx very much.  It may not be the answer you want to hear, but it is a plain fact that Google knows more about search than you and I do.  I recommend that you go with the mainstream algorithms from Google.
0
 

Author Comment

by:dolythgoe
ID: 36901012
This is true but I'm not trying to build Google or use it's index - in fact, I think Google lacks a lot in some areas - e.g filtering and search of technical articles tends to bring back stuff dated in 2001 which isn't that relevant for mysql searches if you want more recent stuff. Goolge does not have social tagging and only recently put +1 in their index. For most things, it's the best but some things it isn't.

Sphinx is there to act as a 'piece' of the overall puzzle because it does a better job than mysql for a specific query type. Of course, if I wanted Google power then it would be the first place to go but in this case, it's not required.

0
 
LVL 24

Accepted Solution

by:
johanntagle earned 2000 total points
ID: 36901032
On the contrary I've used Sphinx heavily on some projects and I'm happy with it.  I agree that Google is a much more powerful search (that may be the understatement of the decade), but custom search engines have their place in the world.  A lot of times it is easier to implement in a custom search engine to search only specific database fields than to find out how to do it in google, if it's even possible.  Yes, you'll end up having to handle additional load on your servers, but a lot of times its worth it.

The sphinx engine does not have to be on any of the database servers or web servers actually.  It can be anywhere - on big sites it has its own machine dedicated to it.  It just needs to access the database to read what needs to be indexed, then it stores the index data wherever it is installed - the exact path depends on your configuration.  Your PHP application will then need to know the address of the server where sphinx resides and the port it is running on (default is 9312).

Given your setup I would recommend that you set up sphinx on the slave and have it read data from the slave.  This is because I believe the slave would be the least loaded machine right now and also so that in case the master fails, you still have both database and sphinx ready.
0
Technology Partners: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 

Author Comment

by:dolythgoe
ID: 36901404
Cheers.

So I'm going to take your advice and go with Sphinx on slave - for indexing and readonly queries to sphinx/mysql and keep master for inserts and other tasks.

I'm setting it up on ubuntu but I think I might need to open some ports between the machines. Is it a case of setting sphinx up to listen to the webservers? Not quite sure how to do that - I'm guessing the sphinx.conf but no idea about the ports!

The setup is not live and a testing rig without load so breaking it doesn't matter too much ;)

Cheers
David
0
 
LVL 24

Expert Comment

by:johanntagle
ID: 36901413
9312 is the default port sphinx uses.  Just make sure that the server-based firewall (if any) allows it.
0
 

Author Comment

by:dolythgoe
ID: 36901568
Thanks for your help.

So I got this all setup but went to run searchd and got:

listening on 10.2.7.10:9312
bind() failed on 10.2.7.10, retrying...
bind() failed on 10.2.7.10, retrying...
bind() failed on 10.2.7.10, retrying...
bind() failed on 10.2.7.10, retrying...
bind() failed on 10.2.7.10, retrying...
bind() failed on 10.2.7.10, retrying...
bind() failed on 10.2.7.10, retrying...
bind() failed on 10.2.7.10, retrying...
bind() failed on 10.2.7.10, retrying...
bind() failed on 10.2.7.10, retrying...
bind() failed on 10.2.7.10, retrying...
bind() failed on 10.2.7.10, retrying...
FATAL: bind() failed on 10.2.7.10: Cannot assign requested address

10.2.7.10 being private IP for web server 1 (10.2.7.11 is webserver 2 but it never got that far)

I put this in the sphinx.conf but I guess it's wrong!

        #Listen on the 2 webservers
      listen                  = 10.2.7.10:9312
      listen                  = 10.2.7.11:9312
      # listen                  = 9312
      # listen                  = /var/run/searchd.sock
      listen                  = 9312
      listen                  = 9306:mysql41

Hopefully this is obvious to you!
0
 
LVL 24

Expert Comment

by:johanntagle
ID: 36901581
No it should be the IP of the slave in your sphinx.conf, not the IP's of the webservers.
0
 

Author Comment

by:dolythgoe
ID: 36901616
Thanks for that - searchd is now running - is there anything needed to do to the webservers apart from specify host (which I assume is the slave private IP) and port 9312?

Cheers
0
 
LVL 24

Expert Comment

by:johanntagle
ID: 36901646
I'm not too familiar with PHP (I use mainly Ruby on Rails) but check this out
http://www.sanisoft.com/blog/2010/01/11/sphinx-search-engine-and-php-installation-indexing/

And of course, there are the docs:
http://sphinxsearch.com/docs/ 
http://sphinxsearch.com/wiki/doku.php?id=php_api_docs
0
 

Author Comment

by:dolythgoe
ID: 36901755
Thanks for those - I've had the php scrip setup for a little while but just getting nothing back from my tests.

Really confusing, no errors shown, nothing in the logs just blank. It's a difficult doc to traverse and google is giving me rubbish again!

I've set the server IP and port but something must be missing. I think I might have to hunt down to see if a firewall is causing the issue.

Will keep you posted!
0
 

Author Closing Comment

by:dolythgoe
ID: 36901924
All done and working :) thanks for your help!
0

Featured Post

Free Tool: Port Scanner

Check which ports are open to the outside world. Helps make sure that your firewall rules are working as intended.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Containers like Docker and Rocket are getting more popular every day. In my conversations with customers, they consistently ask what containers are and how they can use them in their environment. If you’re as curious as most people, read on. . .
Backups and Disaster RecoveryIn this post, we’ll look at strategies for backups and disaster recovery.
Explain concepts important to validation of email addresses with regular expressions. Applies to most languages/tools that uses regular expressions. Consider email address RFCs: Look at HTML5 form input element (with type=email) regex pattern: T…
In this video, Percona Solution Engineer Rick Golba discuss how (and why) you implement high availability in a database environment. To discuss how Percona Consulting can help with your design and architecture needs for your database and infrastr…
Suggested Courses

581 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question