How does Search work in Get Satisfaction?

  • Question
  • Updated 9 months ago
  • Answered
Photo of Amy Muller

Amy Muller, Co-founder

  • 56,076 Points 50k badge 2x thumb

Posted 4 years ago

  • 23
Photo of Amy Muller

Amy Muller, Co-founder

  • 56,076 Points 50k badge 2x thumb
I’m going to do my best to break down how our public search within a given community currently works.

We index the Topic Title, Topic Description, tags, products and replies.

We use “Stemming”. Stemming is the process by which a popular keyword can be associated with a prefix, suffix or pluralization, so that more matches can be found. Without stemming, a search for "work" would not return results that contained the word "works" or "working", for example.

We have about 650 “Stop” words listed. Stop words are popular words that add no value to search queries and thus should be ignored. For instance, imagine there are two topics in a community: "How do I make coffee?" and "How do I eat lasagna?". Searching for "How do I make coffee?" would bring up the topic about lasagna because the words "how do I" are found in it, even though they add no value to the search. When indexing occurs, we manipulate the content to remove stop words so that only relevant content is available for searching.

There are two different ways public search happens. One is the basic Topic Search function which utilizes the features listed above. The other is the “Topic intercept” that our system performs when someone submits something through the “Start a Topic” box at the top of the Community Home Page and within the Feedback Widget. In addition to the main Topic Search functionality mentioned above, the “topic intercept” results have some ranking algorithms applied to them. The weight of “answered” Questions, “solved” Problems, and Company Updates is ranked higher than than the weight of other topics.

We do not currently support exact matches or logical terms (allows matches to obey AND, OR and NOT commands -- also known as “Boolean”), however those updates are on our future improvements list, as is Analytics around search data. Unfortunately, there is no current date set for these items yet, but we do desperately want them.

If you’re listed as an Employee of a community in Get Satisfaction, you’ll have access to the Management View which has a topic search that functions the same way it does in the public search but includes many additional filtering and sorting options (such as specific product, specific tag, specific statuses, date ranges, among other things) as well as people search to enhance the searching capabilities.
Photo of rootsmusicR

rootsmusic

  • 1,826 Points 1k badge 2x thumb
Amy, can I disable stemming of a keyword? Can I force searching of a "stop word"? Thanks.
Photo of Amy Muller

Amy Muller, Co-founder

  • 56,076 Points 50k badge 2x thumb
Hi rootsmusic. Sorry for the delay in answering your question!
Unfortunately, no, there's no way to customize this. It's a predetermined list used by our search algorithm.
Photo of Wayne (wsmwk)W(

Wayne (wsmwk)

  • 5,772 Points 5k badge 2x thumb
In ranking "users'" getsatisfaction search results, does search results algorithm favor found in topic and/or tags and/or company responses and articles?

In other words, do results found for in topic bodies posted by users get ranked lower?

Likewise, for the search results presented in the management interface.

Also, I've looked at
* http://getsatisfaction.com/getsatisfa... and
* http://getsatisfaction.com/getsatisfa...

and it's not clear that new functionality was rolled out. Is there a rough time frame?

Thanks

additional reference: http://getsatisfaction.com/getsatisfa...

This reply was created from a merged topic originally titled
Does search results algorithm favor text in tags and topics?.
Photo of Morgan Sherwood

Morgan Sherwood

  • 34,940 Points 20k badge 2x thumb
Thanks for clearing things up Amy. I know we're all looking forward to Boolean logic to be added!
Photo of Amy Muller

Amy Muller, Co-founder

  • 56,076 Points 50k badge 2x thumb
You're welcome, Morgan! :)
Photo of Angry Black Man

Angry Black Man

  • 170 Points 100 badge 2x thumb
Why are exact matches not supported????? Why would you force people to get useless, unrelated, unhelpful results when all they're trying to do is NOT have to create new topics by searching!!! It DOESN'T MAKE ANY SENSE!! thats wonderful that you dont support boolean.. fine... But to search for "desktop alerts" (for example) and for the first result to have nothing to do with query is stupid!!!

How is this optimal?? What purpose could this stupidity POSSIBLY SERVE?!!!??
Photo of Geysa

Geysa

  • 15,452 Points 10k badge 2x thumb
Hi there, sorry you are frustrated with our search. I'd like to offer an interim solution that works great, while we're building out our search. If you go to Google and search for "desktop alerts" site:getsatisfaction.com you'll see the results you want.
Photo of Angry Black Man

Angry Black Man

  • 170 Points 100 badge 2x thumb
This only brings the problem full circle. The reason i'm searching for "desktop alerts" is because this is a feature of Mozilla Thunderbid. Their site, "mozilla messaging" uses this ..[expletive should be here].. software. I tried to use search operators and realized you guys PURPOSEFULLY designed the software to be broken and to intentionally spew out bloated, useless, intentionally misleading and incorrect results.

Therefore, the next workaround that i tried was your belated suggestion. To illustrate its failings, here is a random thread from 2010:

http://getsatisfaction.com/mozilla_me...

here's the syntax:

site:getsatisfaction.com/mozilla_messaging "cancel alerts"

after you put those two together, you'll see why your belated workaround is UNACCEPTABLE. Having already tried what you belatedly suggest, this is why i started posting here in the first place!!! I wanted to vilify mozilla, but their only problem was choosing this software that you guys wrote!

WHY IS THIS SOFTWARE INTENTIONALLY HANDICAPPED!? ITS 2011. THE ONLY EXPLANATION FOR THIS TO OCCUR IS THAT IT WAS INTENTIONAL. I JUST WANT TO KNOW WHY!!!!
Photo of Angry Black Man

Angry Black Man

  • 170 Points 100 badge 2x thumb
as an aside, you guys should do what facebook shows is a good idea: if someone marks a comment as a "good answer" (like), it should show a list of all those who marked it that way.
Photo of Amy Muller

Amy Muller, Co-founder

  • 56,076 Points 50k badge 2x thumb
Hi. Please know that we are not intentionally handicapping our search. We did a big upgrade to our search earlier this year and switched search servers. The new server we're using is more robust but came with it's own complexities. There are some specific technical hairballs around integrating exact matches and Boolean search which we just haven't had a chance to tackle yet. Our search upgrade is a big project and is being done in a phased approach. The next phase will include addressing this, but due to other priorities I'm not sure where on the current roadmap it is.

Again, I apologize for your frustrating experience here. We get frustrated by our search functionality as well and really hope that the additional improvements can get on the schedule soon.
Photo of Angry Black Man

Angry Black Man

  • 170 Points 100 badge 2x thumb
"really hope that the additional improvements can get on the schedule soon"

wow, that sentence is a perfect example of people TRYING to schedule "meetings about meetings".

nice dodge. good one.
Photo of Chad Woolley

Chad Woolley

  • 252 Points 250 badge 2x thumb
Thank you very much for the update and transparency. Looking forward to boolean search!
Photo of Amy Muller

Amy Muller, Co-founder

  • 56,076 Points 50k badge 2x thumb
You're welcome. Thanks for your patience and understanding! :)
Photo of Chad Woolley

Chad Woolley

  • 252 Points 250 badge 2x thumb
I'd also love to see tags as a first-class search field in the admin search (ideally with regex / wildcard support). Is that on the roadmap?
Photo of Amy Muller

Amy Muller, Co-founder

  • 56,076 Points 50k badge 2x thumb
Hi Chad. So now that we know you're talking about tags :), you mean that you'd like a search field in the Management View that is for searching tags only? Our topic search does search tags as they *are* indexed, but you're right that there's no specific tag search. You can also click on any given tag to see all the topics that have that tag and get a direct URL to that tag. So that means that you could just put a URL in the address field of your browser that looks like this: http://getsatisfaction.com/[COMPANY]/.... For instance, I can do that for the tag "search" in our community: http://getsatisfaction.com/getsatisfa...

I know that's not exactly what you're asking for, but maybe it helps a bit. Otherwise, your request for a tag search field in the Mgmt View is noted. :)
Photo of Chad Woolley

Chad Woolley

  • 252 Points 250 badge 2x thumb
Thanks for the response. Right, you can currently do a search on a tag, but that includes everything, for only one tag, with no other filtering options.

I'm saying that since tags are a non-ambiguous field, I'd like to be able to better leverage them in admin power searches.

For example, see all 'search' tags, but further filter by other search fields such as keywords or topic status.

Or support boolean search for tags - search AND api, or search AND !api

Of course, these could also be solved gmail- or tracker-style, with prefixes in the standard free-form search box - e.g. tag:search tag:api

Thanks!
Photo of Amy Muller

Amy Muller, Co-founder

  • 56,076 Points 50k badge 2x thumb
Ok, thanks for the additional feedback, Chad!
Photo of Roland Tanglao

Roland Tanglao, Champion

  • 4,684 Points 4k badge 2x thumb
Hi Chad:
What do you mean by "labels"? Do you mean tags or user defined codes or something else ? My guess :-) is tags!
Photo of Amy Muller

Amy Muller, Co-founder

  • 56,076 Points 50k badge 2x thumb
I had the same question! :)
Photo of Chad Woolley

Chad Woolley

  • 252 Points 250 badge 2x thumb
yeah, I meant tags :)
Photo of Roland Tanglao

Roland Tanglao, Champion

  • 4,684 Points 4k badge 2x thumb
Until Get Satisfaction rolls out their boolean search, here's a workaround.

If you are technical or know somebody technical who can hack my scripts (I know that rules out 99% of you reading this :-( but for the 1% :-) who know Ruby and MongoDB it's not that hard!) all you need to do is:

1. slurp your Get Satisfaction topics into a MongoDB database (change "mozilla_messaging" and "mozilla_thunderbird" to your company and product names):
https://github.com/rtanglao/momogs/bl...

2. run my crude brute force search script which supports search of both the text and the tags using regex and Boolean i.e. AND, OR and NOT
https://github.com/rtanglao/momogs/bl...
Photo of Morgan Sherwood

Morgan Sherwood

  • 34,950 Points 20k badge 2x thumb
Roland, you are a god amongst community managers.
Photo of Amy Muller

Amy Muller, Co-founder

  • 56,076 Points 50k badge 2x thumb
So true.
Photo of Phil Scanlan

Phil Scanlan

  • 1,356 Points 1k badge 2x thumb
We do not currently support exact matches or logical terms (allows matches to obey AND, OR and NOT commands -- also known as “Boolean”), however those updates are on our future improvements list, as is Analytics around search data. Unfortunately, there is no current date set for these items yet, but we do desperately want them.


We desperately want them to
Photo of rsx11mR

rsx11m

  • 3,296 Points 3k badge 2x thumb
I had a wish list in http://gsfn.us/t/15fml which for some reason got merged into this topic here. The most attractive workaround for a user without having access to the management view (which is a bit better than the options for a regular user but still much less than possible with other search engines) is to simply go to Google and add site:getsatisfaction.com/your_community and use their search and sorting tools.
Photo of Amy Muller

Amy Muller, Co-founder

  • 56,076 Points 50k badge 2x thumb
I have to agree with you on that one. I sometimes use Google to search our community. Some day, this will not be necessary! :)
Photo of Amy Muller

Amy Muller, Co-founder

  • 56,076 Points 50k badge 2x thumb
Oh, and that other topic of yours wasn't merged into this one. I just added a "redirect" at the top which is basically just adding a link saying "here's some useful info". I did that with all our search related topics since I was able to give a comprehensive explanation in this topic about how our search currently functions. Don't worry, though, we are still tracking your Idea topic and all your suggestions.
Photo of avhfjA

avhfj

  • 60 Points
Any news? The lack of search capability is so frustrating! How about just using a third party search, eg Google.
Photo of Caty

Caty, Employee

  • 43,222 Points 20k badge 2x thumb
Hello!

Thanks so much for taking the time to send in this feedback and sorry for all the frustrations it has caused. I'm meeting with our Product Manager about Search later this week and will update you on any progress that I am able to share. I'll definitely be sure to ask about Google.
Photo of Peter KlugherzPK

Peter Klugherz

  • 60 Points
Amy,
If the capability exists for employees to filter on specific products why can't that be made available to the rest of us? Of course, the ability to use Boolean operators would be preferred.
Photo of Caty

Caty, Employee

  • 43,222 Points 20k badge 2x thumb
Hi Peter,

I just wanted to let you know that I'm meeting with our Product Manager later this week, and we're going to be talking about search. I'll ask about the filter capabilities, and will pass along any progress that I'm able to share publicly. Thanks in advance for your patience!
Photo of Philip Hotchkiss

Philip Hotchkiss

  • 552 Points 500 badge 2x thumb
The feedback on our search functionality is much appreciated. There's not doubt, we have much to improve. Elevating the relevance of results and adding Boolean logic functionality is on our product roadmap for the second half of this year.

Our engineering roadmap calls for search scalability work to support the growth of our business and the volume of search queries. These growing pains are frustrating, but necessary for us to move through so that we build our next generation search algorithm on a solid foundation.

In the meantime, as our product managers & engineers finalize the requirements for the next gen version of Get Sat search - we are on the lookout for incremental improvements that can be made along the way. As we spot these opportunities, we will do our best to bring them into the product in an agile fashion.

We apologize for the frustration many of you, our valued customers and community members have expressed and we'll keep you updated in the second half of the year as we move closer to a release of the new search functionality. In the meantime, please keep the ideas and feature requests flowing. We capture these requests and review them carefully as a product, support and engineering team.

Thanks,

Phil
Photo of Chad Woolley

Chad Woolley

  • 252 Points 250 badge 2x thumb
Thanks for the transparency! Glad you were able to find this topic again! ;)
Photo of rootsmusicR

rootsmusic

  • 1,826 Points 1k badge 2x thumb
Please enable boolean syntax in your search engine. Thanks.

This reply was created from a merged topic originally titled
advanced search.
Photo of rootsmusicR

rootsmusic

  • 1,826 Points 1k badge 2x thumb
How can I search for a specific URL posted in the body of a topic? How can I include a "stop word" in my search keywords? Thanks.
Photo of Adam

Adam, Champion

  • 18,422 Points 10k badge 2x thumb
Hey rootsmusic.

Thanks for your question.

Currently there isn't a way to include "stop words" in your search keywords, and our search algorithm strips ".com" from search keywords, which can interfere when searching for URLs. As our Chief Product Officer noted above in his official response, we're sensitive to the fact that there's room for improvement in our search functionality, and improving search result relevancy and adding Boolean syntax is on our product roadmap for the second half of this year.

Thanks!

Adam
Photo of Caty

Caty, Employee

  • 43,222 Points 20k badge 2x thumb
Official Response
Last week we released a few significant improvements to our Search functionality, specifically:

- Support for Boolean Operators: AND, OR
Note: AND and OR are the only Boolean operators currently supported, and must be capitalized when searching.

- Support for search within quotations, aka exact match

- Algorithmic improvements

Coupled with this new release, our search still follows some the existing logic that Amy outlined out in an earlier post:

We index the Topic Title, Topic Description, tags, products and replies.

We use “Stemming”. Stemming is the process by which a popular keyword can be associated with a prefix, suffix or pluralization, so that more matches can be found. Without stemming, a search for "work" would not return results that contained the word "works" or "working", for example.

We have roughly 650 “Stop” words. Stop words are popular words that add no value to search queries and thus should be ignored. For instance, imagine there are two topics in a community: "How do I make coffee?" and "How do I eat lasagna?". Searching for "How do I make coffee?" would bring up the topic about lasagna because the words "how do I" are found in it, even though they add no value to the search. When indexing occurs, we manipulate the content to remove stop words so that only relevant content is available for searching.

There are two different ways public search happens. One is the basic Topic Search function which utilizes the features listed above. The other is the “Topic intercept” that our system performs when someone submits something through the “Start a Topic” box at the top of the Community Home Page and within the Feedback Widget. In addition to the main Topic Search functionality mentioned above, the “topic intercept” results have some ranking algorithms applied to them. The weight of “answered” Questions, “solved” Problems, and Company Updates is ranked higher than than the weight of other topics.

If you’re listed as an Employee of a community in Get Satisfaction, you’ll have access to the Management View which has a topic search that functions the same way it does in the public search but includes many additional filtering and sorting options (such as specific product, specific tag, specific statuses, date ranges, among other things) as well as people search to enhance the searching capabilities.


Enjoy!
Photo of Wayne (wsmwk)W(

Wayne (wsmwk)

  • 5,772 Points 5k badge 2x thumb
seems to work great. tags too! THANKS, THANKS, THANKS

QUESTION: does your mentioning of OR imply that search terms can be grouped with () ? Or that AND and OR can be ordered to achieve results similar to using () ?

Note - I find that "and" works even in lower case, despite "Note: AND and OR must be capitalized when searching. "
Photo of Ramya

Ramya, Employee

  • 4,906 Points 4k badge 2x thumb
Official Response
AND and OR can be ordered to achieve results similar to using ()
For example if
Search = removing topics OR deleting topics
Results = topics that either contain both terms removing topics or contain both terms deleting topics.
If your search is
Search = change AND password
Results = Will contain topics that contain both terms change and password.

By default our searches are implicit AND which means that when you do not put any Boolean operator (AND/OR) in between terms it will look for topics that contain all the terms in the query. eg. change password will return all topics that contain both the terms change and password.
In the case of lower case "and" what you are noticing is this behavior. Lowercase "and" and "or" are stopwords hence eliminated. The terms in the query are then joined by implicit AND and the response is returned.
Photo of Nathan Roth

Nathan Roth

  • 472 Points 250 badge 2x thumb
Better search would have the largest ROI for my company as well given that 99%+ of our visitors do not post, but just browse.

Please consider live search results like Lithium does. Here's an example from a huge brand with a ton of content: http://forums.att.com/

At the end of the day, customers just want to get an accurate answer fast. What better way of doing that than perfecting search!
Photo of Jose Corona

Jose Corona

  • 528 Points 500 badge 2x thumb
Hi,
Yo say "We index the Topic Title, Topic Description, tags, products and replies"
Question: Does the search also finds word matches at the body of the post or only on those fiedls being indexed?
The reason is that we have some jpg files and I was thinking that we shouldl change it to html so search could also find the text. But I wuold liketo be sure.
Thanks
Photo of Camilla

Camilla

  • 13,684 Points 10k badge 2x thumb
Hey JR

I am not sure what the answer is to your questions. I am going to speak to a few people on my team and get back to you soon! Thanks for your patience.
Photo of Jose Corona

Jose Corona

  • 528 Points 500 badge 2x thumb
Perfect...thanks
Photo of Caty

Caty, Employee

  • 43,222 Points 20k badge 2x thumb
Topic Description is the "body" of the post, so yes, all the text within the body of a topic is being indexed. I believe this includes HTML tags for images.
Photo of Jose Corona

Jose Corona

  • 528 Points 500 badge 2x thumb
Great... Thanks a lot!
Photo of Emily Phinney

Emily Phinney

  • 110 Points 100 badge 2x thumb
Hi,

Is there any way to change or update stop words? For example, I'm finding if I search "What is banana pudding?"my results come up with every conversation that asks a question starting with "What is"

Thank you
Photo of Stephen Lee

Stephen Lee, QA Manager

  • 11,230 Points 10k badge 2x thumb
Hi Emily - we're actually working on that very problem as we speak :) Look for an update in the coming weeks... 

Thanks for the feedback,
Steve
Photo of Emily Phinney

Emily Phinney

  • 110 Points 100 badge 2x thumb
Steve- 

May I please have a list of the stop words that are currently being implemented?

Thanks,
Emily
Photo of Stephen Lee

Stephen Lee, QA Manager

  • 11,230 Points 10k badge 2x thumb
Sure, here's the list we're currently using. We're working on potentially expanding this list and are open to suggestions:

a, an, and, are, as, at, be, but, by, for, if, in, into, is, it, no, not, of, on, or, such, that, the, their, then, there, these, they, this, to, was, will, with

Steve