Login to access your dashboard here.
Blog : Previous : Information Technology (IT) : Birth of Searchy 2.0: How to Rank Results?

Birth of Searchy 2.0: How to Rank Results?

Birth of Searchy 2.0: How to Rank Results? KnowledgeToWork.com
Wednesday, August 3, 2016
Posted in category Information Technology (IT) by

To meet our March 2016 site launch date, the original KnowledgeToWork.com search results were provided from a series of SQL select calls though PHP. This meant passing back and forth large amounts of data between the Apache web and MySQL servers. Although it sounds clunky, in hindsight it was a much needed development piece for learning the steps needed to provide the user with an efficacious response to their queries. As we moved from just over 1,000 learning resources in March to around 5,000 by the end of April, something needed to be done to keep the search working quickly. It was obvious that by combining steps into one stored procedure that much of the processing could be pushed to the MySQL server, resulting in more balance of resources, less data movement waste, and a better user experience.

The change process also provided an opportunity to rethink how search results were ranked or ordered. The original process was simplistic and employed a flat full text index strategy across all the fields our digital librarian had identified as important for search indexing. This meant that words matched in various categories like keyword, subject, title, and description all contributed equally toward the rank score. During the requirements gathering process it became apparent that there existed multiple use case scenarios needing solutions. From this, I developed a system to weight clusters of search metadata, to then be the aggregate of full text ranking scores based on the weighting adjustment. After testing, we concluded that some types of searches were expecting very specific results while others were more explorative searches. This lead to further development of search modes to provide a way to switch between an exact word match mode and matches based on related words. A consequence of this is that it then required two sets of score weights to account for differences in mode methods ranking scores.

Pathways were also needed to provide the ability to generate clickable metadata hyperlinks, to create search results sets based solely around that metadata value (example: all resources by an author or that have a specific keyword). Due to the way we had arranged the data in the architecture, we could pass to the same search procedure the ID numbers and through some parameters bypass the need for any full text indexes, and use the numerical indexes for super-fast results.

It was decided along the way that the search procedure needed a name and Searchy stuck. After the revisions were complete, Searchy 2.0 was born. As this project continues to grow we may need to consider a solution like LuceneSphinxElasticSearch, or Solr, but for now we have a solid solution that is in alignment with our goal to provide software that relies on open source tools, in this case a clean PHP/MySQL process. There are already plans to include a Boolean mode for use cases where users need to use search methods that must include words or exclude results with specific words. Searchy 3.x is due to be released sometime in Q4 of 2016.

-J DBA knowledgetowork.com & highered.org

Follow me on LinkedIn

Grapple with development by embracing change, uncover use cases, gather requirements, and provide efficacious search results using open source tools.


Click tag to search related posts

searchy
MySQL
metadata
search
SQL
stored procedure
open source
change

Search other posts in category "Information Technology (IT)"

Comments
Login
or register to comment on this blog post.

 Be the first to comment on this blog post.
Permalink:
Sign up free, learn at your own pace, track your progress.