Characterizing Usability Through Query Log Analysis
Search query log analysisAdam Fourney, Richard Mann and Michael Terry
People naturally turn to search engines whenever they encounter a problem using software or any interactive system. In this project, we are exploring how search engine query logs can be mined to understand the primary needs and issues users face when using interactive systems.
Example: Getting Firefox's Menu Bar BackApproximately once every 2.1 hours, a variation of the query "firefox get menu bar back" is issued to Google.com (United States locale). Similarly, once every 1.7 hours, a variation of the query, "restore menu bar firefox", is issued . In fact, this pattern of searches is so popular that it appears in Google Suggest's top ten list for completing the phrase ``firefox how to''.
An inspection of the Firefox user interface (version 3.6 on Windows), reveals that the top-level menu bar is easily hidden by deactivating the ``Menu bar'' item in Firefox's ``View --> Toolbars'' sub-menu. However, once this action is taken, it is not easily reversed: the top-level menuing system is now hidden, removing the very means the user would employ to attempt to re-instate the menu bar.
What is noteworthy about this example is that we quickly moved from data derived from query logs to a testable hypothesis regarding the usability of the software.
Exploring Query LogsBelow, we provide links to a pair of interactive visualizations that demonstrate the overall utility of this approach for two open source projects: Ubuntu Linux, and Firefox. The Ubuntu data set contains over 120,000 unique queries, while the Firefox data set contains about 75,000 distinct queries. Click on the images below to start exploring these data sets.
|Ubuntu [en-US Locale] [Empty Locale]||Firefox [en-US Locale] [Empty Locale]|
|GIMP [en-US Locale] [Empty Locale]||Kindle [en-US Locale] [Empty Locale]|