Mining Web Activity Logs

Web mining has been a major research topic during the last decade. Traditional sources of information on the Web include the text of web pages, the structure of the pages as organized by the web page creators, as well as the link structure of the Web. Another emerging source of information is the tags for web pages as collected and organized by collaborative tagging systems (e.g. del.icio.us, StumbleUpon). Search engines are interested in organizing all the information provided by these sources so that they are able to respond accurately to different search tasks. However, the ample use of web search engines by billions of people worldwide has created another interesting source of information. The search and browsing activity of these users implicitly captures human knowledge and reveals the most interesting places on the web. By looking at the search queries people issue, we can extract a succinct, user-generated summary for pages appearing in the search results. For example, such a summary for Google’s main page could be: google, search engine, larry page, pagerank. These summaries tend to be accurate given the fact that web users have been trained all these years to formulate queries with an emphasis on selecting the least ambiguous and most meaningful words. In a way, in addition to requesting new information, search users are also revealing new knowledge (that perhaps does not even exist yet on the web) through the keywords they select for their queries and the navigational paths they follow in response. The paths users follow are also implicit votes on the interesting parts on the Web. Search engines are the first to record the users’ search history in order to be able to analyze it and improve their services in the future. These query logs contain the terms users issue in a search engine along with the search results people click on in response. In addition, through browser toolbars, search engines can record web activity logs that contain the history of pages a user visits. Although web activity and query logs contain invaluable information for the search engines, it is also the web users who can explicitly benefit from such data. However, we have to identify meaningful ways to mine the web activity logs and make them available back to the web users. The primary goal of my research has been to investigate novel uses of web activity logs in formulating and solving novel problems related to the Web. The research on query and web activity logs that we have performed, with collaborators from Yahoo! and Microsoft Research, apply to problems related to sponsored search, recommendation systems, web information extraction systems, and web navigation systems. Section 1 outlines the research I have performed to date, while Section 2 describes my current research and plans for the future.