Tag Weighting and Its Applications

Information explosion on the internet popularly known as the web has led to a phenomenon of the user unable to find the information relevant to him or her. There is plethora of information available and thousands of pages are being added everyday. Hence more and more information is available on each and every subject. Hence, it has become increasing difficult for users to remain up to date on one’s own areas of interest. This has increased the need for indexing of documents drastically. Indexing can take place at the user end or at the search engine end. However traditionally, indexing used to take place at the middle level by authorities or librarians. With the vast amount of content available on the internet, it is practically impossible for any authority to index the data. This led to the evolution of social bookmarking systems where users are able to maintain individual collections of resources and index them using tags for retrieval purposes. This model has evolved into a collaborative model where users share their resource collection and tags with others leading to the emergence of collaborative tagging systems. These systems are also referred to as “Folksonomies”. Due to the flexibility and ease of use, the usage of these systems has increased very rapidly. Many tagging systems like Delicious, Flickr, Bibsonomy, Citeseer etc. have evolved and became very popular. This has created a new form of rich user generated content which can be exploited to good use in a wide range of applications. The data in these systems typically consists of three dimensions resources, users and tags. In this thesis, we have focused on some of the basic metrics for extracting information from folksonomies Tag weight and Tag similarity and the usage of them in applications like Tag Search and Web document summarization. First, we propose a model of tag weighting extending the traditional Vector Space Model for term weighting. We show how tag weighting can be used in tag similarity and also show how we compute tag similarity. We evaluate our proposed tag similarity measure by grounding them against semantic lexicon WordNet using correlation measures Kendall’s tau and Spearman’s rho correlation coefficients. We further demonstrate the application of our tag weighting in summarization of web documents. We have proposed a summarization feature based on tag weights which can be used with some of the existing features in generating summaries. We have evaluated the generated summaries by comparing them with manually generated summaries to find how close they are to the manual summaries and hence more readable. Further, we also demonstrated the application of our tag similarity in tag search. Our experimental results proved that our proposed approaches are able to achieve good improvement over our baselines.