Homo-Adminus Blog - Latest Comments in Using Sphinx for Non-Fulltext Queries

Re: Using Sphinx for Non-Fulltext Queries

bulid8701056 — Fri, 30 Jan 2009 18:34:12 -0000

Thanks for the info. May God have mercy on us all.

Re: Using Sphinx for Non-Fulltext Queries

Barry Hunter — Mon, 25 Aug 2008 13:45:52 -0000

Yes I agree sphinx makes for a great general purpose index. We put a huge database in sphinx for full text, but realised it could be used an index on many of the other fields too!

@Pavel

Sphinx can search multiple indexes in one query, so the basic idea is to create a main index, and the a delta index, you just more regually reindex the delta index. You can even have multiple delta indexes - basically constantly reindexing the changes. You can also use line attribute updates to 'delete' items from the main index without reindexing.

Re: Using Sphinx for Non-Fulltext Queries

Brian Moon — Thu, 24 Jul 2008 21:55:19 -0000

Andrew has said that online add and remove is coming in 0.9.9. When will it be ready? Don't know. But that will be a big, big jump in usefulness.

Re: Using Sphinx for Non-Fulltext Queries

Pavel — Thu, 24 Jul 2008 15:13:22 -0000

Hello Alexei,
I have one comment regarding this:
From what I've read so far Sphynx is great only for non-constantly-changing-data... Am I correct.
If you have thousands of writes per second and you want the info to be available within 1-3 second span how would you do that with sphynx??? if reindexing takes 20-40 seconds of 3Mil rows as you said...

Re: Using Sphinx for Non-Fulltext Queries

lix — Sun, 20 Jul 2008 12:42:15 -0000

Hi!

First of all, you blog is very interesting and I like it, salute from Ireland to you! ;)

The problem is with MySQL(like with every DBMS written by folks wo/ corporate experience) still the relative small amount of features to handle something bigger than the average.

Let me show you what i am thinking about:

http://www.ibm.com/developerworks/db2/library/t...

http://www.oracle.com/technology/products/oracl...

http://msdn.microsoft.com/en-us/library/ms34514...

And just for the record, innodb is an Oracle product.

Maybe you can try this:
http://dev.mysql.com/doc/refman/5.1/en/partitio...

Regards,
lix

Re: Using Sphinx for Non-Fulltext Queries

Scoundrel — Tue, 15 Jul 2008 02:05:16 -0000

2W. Andrew Loe III: No, for browse index we use documents' attributes only.

Re: Using Sphinx for Non-Fulltext Queries

W. Andrew Loe III — Thu, 10 Jul 2008 12:46:37 -0000

Do you guys use Sphinx to index the contents of your documents, or only metadata about them that is stored in the database?

Re: Using Sphinx for Non-Fulltext Queries

Карл — Wed, 04 Jun 2008 22:27:22 -0000

Good блог at you, cognitive:)

Re: Using Sphinx for Non-Fulltext Queries

Peter Zaitsev — Thu, 22 May 2008 21:02:38 -0000

Brian,

There are two different techniques - one is adding pseudo keywords and other is using filters and "Full Table Scan" query.

Pseudo keywords works if there are few of them in the query and they are well selective.

If you will have 10 keywords each with 90% selectivity this would not work well.

Filters w full table scan however can be slower with good selectivity but you can have a lot of non selective filters with very low penalty.

Also note Sphinx "full table scan" has optimizations so it often can skip a lot of "blocks" based on filters which makes it kind of similar to indexed lookup

Re: Using Sphinx for Non-Fulltext Queries

Brian Moon — Mon, 19 May 2008 12:52:27 -0000

We use this technique for filtering searches on categories, price, etc. We just stick cat_39 in a deal and add that to the sphinx query. Works like a charm.