We use this technique for filtering searches on categories, price, etc. We just stick cat_39 in a deal and add that to the sphinx query. Works like a charm.
Peter Zaitsev
· 1 year ago
Brian,
There are two different techniques - one is adding pseudo keywords and other is using filters and "Full Table Scan" query.
Pseudo keywords works if there are few of them in the query and they are well selective.
If you will have 10 keywords each with 90% selectivity this would not work well.
Filters w full table scan however can be slower with good selectivity but you can have a lot of non selective filters with very low penalty.
Also note Sphinx "full table scan" has optimizations so it often can skip a lot of "blocks" based on filters which makes it kind of similar to indexed lookup
Карл
· 1 year ago
Good блог at you, cognitive:)
W. Andrew Loe III
· 1 year ago
Do you guys use Sphinx to index the contents of your documents, or only metadata about them that is stored in the database?
Scoundrel
· 1 year ago
2W. Andrew Loe III: No, for browse index we use documents' attributes only.
lix
· 1 year ago
Hi!
First of all, you blog is very interesting and I like it, salute from Ireland to you! ;)
The problem is with MySQL(like with every DBMS written by folks wo/ corporate experience) still the relative small amount of features to handle something bigger than the average.
Hello Alexei, I have one comment regarding this: From what I've read so far Sphynx is great only for non-constantly-changing-data... Am I correct. If you have thousands of writes per second and you want the info to be available within 1-3 second span how would you do that with sphynx??? if reindexing takes 20-40 seconds of 3Mil rows as you said...
Brian Moon
· 1 year ago
Andrew has said that online add and remove is coming in 0.9.9. When will it be ready? Don't know. But that will be a big, big jump in usefulness.
Barry Hunter
· 1 year ago
Yes I agree sphinx makes for a great general purpose index. We put a huge database in sphinx for full text, but realised it could be used an index on many of the other fields too!
@Pavel
Sphinx can search multiple indexes in one query, so the basic idea is to create a main index, and the a delta index, you just more regually reindex the delta index. You can even have multiple delta indexes - basically constantly reindexing the changes. You can also use line attribute updates to 'delete' items from the main index without reindexing.
Bulimia
· 9 months ago
Thanks for the info. May God have mercy on us all.
All Comments
There are two different techniques - one is adding pseudo keywords and other is using filters and "Full Table Scan" query.
Pseudo keywords works if there are few of them in the query and they are well selective.
If you will have 10 keywords each with 90% selectivity this would not work well.
Filters w full table scan however can be slower with good selectivity but you can have a lot of non selective filters with very low penalty.
Also note Sphinx "full table scan" has optimizations so it often can skip a lot of "blocks" based on filters which makes it kind of similar to indexed lookup
First of all, you blog is very interesting and I like it, salute from Ireland to you! ;)
The problem is with MySQL(like with every DBMS written by folks wo/ corporate experience) still the relative small amount of features to handle something bigger than the average.
Let me show you what i am thinking about:
http://www.ibm.com/developerworks/db2/library/t...
http://www.oracle.com/technology/products/oracl...
http://msdn.microsoft.com/en-us/library/ms34514...
And just for the record, innodb is an Oracle product.
Maybe you can try this:
http://dev.mysql.com/doc/refman/5.1/en/partitio...
Regards,
lix
I have one comment regarding this:
From what I've read so far Sphynx is great only for non-constantly-changing-data... Am I correct.
If you have thousands of writes per second and you want the info to be available within 1-3 second span how would you do that with sphynx??? if reindexing takes 20-40 seconds of 3Mil rows as you said...
@Pavel
Sphinx can search multiple indexes in one query, so the basic idea is to create a main index, and the a delta index, you just more regually reindex the delta index. You can even have multiple delta indexes - basically constantly reindexing the changes. You can also use line attribute updates to 'delete' items from the main index without reindexing.