Publications

2017

• Fernando, Z.T., Marenzi, I., Nejdl, W.: ArchiveWeb: collaboratively extending and exploring web archive collections---How would you like to work with your collections? International Journal on Digital Libraries. 1--17 (2017).

  Abstract BibTeX EndNote Download URL

  Curated web archive collections contain focused digital content which is collected by archiving organizations, groups, and individuals to provide a representative sample covering specific topics and events to preserve them for future exploration and analysis. In this paper, we discuss how to best support collaborative construction and exploration of these collections through the ArchiveWeb system. ArchiveWeb has been developed using an iterative evaluation-driven design-based research approach, with considerable user feedback at all stages. The first part of this paper describes the important insights we gained from our initial requirements engineering phase during the first year of the project and the main functionalities of the current ArchiveWeb system for searching, constructing, exploring, and discussing web archive collections. The second part summarizes the feedback we received on this version from archiving organizations and libraries, as well as our corresponding plans for improving and extending the system for the next release.

  

  
• Singh, J., Anand, A.: Designing Search Tasks for Archive Search. Proceedings of the 2017 Conference on Conference Human Information Interaction and Retrieval. p. 361--364. ACM, Oslo, Norway (2017).

  Abstract BibTeX EndNote URL

  Longitudinal corpora like legal, corporate and newspaper archives are of immense value to a variety of users, and time as an important factor strongly influences their search behavior in these archives. While many systems have been developed to support users' temporal information needs, questions remain over how users utilize these advances to satisfy their needs. Analyzing their search behavior will provide us with novel insights into search strategy, guide better interface and system design and highlight new problems for further research. In this paper we propose a set of search tasks, with varying complexity, that IIR researchers can utilize to study user search behavior in archives. We discuss how we created and refined these tasks as the result of a pilot study using a temporal search engine. We not only propose task descriptions but also pre and post-task evaluation mechanisms that can be employed for a large-scale study (crowdsourcing). Our initial findings show the viability of such tasks for investigating search behavior in archives.

  

  
• Setty, V., Anand, A., Mishra, A., Anand, A.: Modeling Event Importance for Ranking Daily News Events. Proceedings of the Tenth ACM International Conference on Web Search and Data Mining. p. 231--240. ACM, Cambridge, United Kingdom (2017).

  Abstract BibTeX EndNote URL

  We deal with the problem of ranking news events on a daily basis for large news corpora, an essential building block for news aggregation. News ranking has been addressed in the literature before but with individual news articles as the unit of ranking. However, estimating event importance accurately requires models to quantify current day event importance as well as its significance in the historical context. Consequently, in this paper we show that a cluster of news articles representing an event is a better unit of ranking as it provides an improved estimation of popularity, source diversity and authority cues. In addition, events facilitate quantifying their historical significance by linking them with long-running topics and recent chain of events. Our main contribution in this paper is to provide effective models for improved news event ranking. To this end, we propose novel event mining and feature generation approaches for improving estimates of event importance. Finally, we conduct extensive evaluation of our approaches on two large real-world news corpora each of which span for more than a year with a large volume of up to tens of thousands of daily news articles. Our evaluations are large-scale and based on a clean human curated ground-truth from Wikipedia Current Events Portal. Experimental comparison with a state-of-the-art news ranking technique based on language models demonstrates the effectiveness of our approach.

  

  
• Gottschalk, S., Demidova, E.: MultiWiki: Interlingual Text Passage Alignment in Wikipedia. ACM Trans. Web. 11, 6:1--6:30 (2017).

  Abstract BibTeX EndNote URL

  In this article, we address the problem of text passage alignment across interlingual article pairs in Wikipedia. We develop methods that enable the identification and interlinking of text passages written in different languages and containing overlapping information. Interlingual text passage alignment can enable Wikipedia editors and readers to better understand language-specific context of entities, provide valuable insights in cultural differences, and build a basis for qualitative analysis of the articles. An important challenge in this context is the tradeoff between the granularity of the extracted text passages and the precision of the alignment. Whereas short text passages can result in more precise alignment, longer text passages can facilitate a better overview of the differences in an article pair. To better understand these aspects from the user perspective, we conduct a user study at the example of the German, Russian, and English Wikipedia and collect a user-annotated benchmark. Then we propose MultiWiki, a method that adopts an integrated approach to the text passage alignment using semantic similarity measures and greedy algorithms and achieves precise results with respect to the user-defined alignment. The MultiWiki demonstration is publicly available and currently supports four language pairs.

  

  
• Lee, R.K.-W., Hoang, T.-A., Lim, E.-P.: On Analyzing User Topic-Specific Platform Preferences Across Multiple Social Media Sites. Proceedings of the 26th International Conference on World Wide Web. p. 1351--1359. International World Wide Web Conferences Steering Committee, Perth, Australia (2017).

  Abstract BibTeX EndNote URL

  Topic modeling has traditionally been studied for single text collections and applied to social media data represented in the form of text documents. With the emergence of many social media platforms, users find themselves using different social media for posting content and for social interaction. While many topics may be shared across social media platforms, users typically show preferences of certain social media platform(s) over others for certain topics. Such platform preferences may even be found at the individual level. To model social media topics as well as platform preferences of users, we propose a new topic model known as MultiPlatform-LDA (MultiLDA). Instead of just merging all posts from different social media platforms into a single text collection, MultiLDA keeps one text collection for each social media platform but allowing these platforms to share a common set of topics. MultiLDA further learns the user-specific platform preferences for each topic. We evaluate MultiLDA against TwitterLDA, the state-of-the-art method for social media content modeling, on two aspects: (i) the effectiveness in modeling topics across social media platforms, and (ii) the ability to predict platform choices for each post. We conduct experiments on three real-world datasets from Twitter, Instagram and Tumblr sharing a set of common users. Our experiments results show that the MultiLDA outperforms in both topic modeling and platform choice prediction tasks. We also show empirically that among the three social media platforms, "Daily matters" and "Relationship matters" are dominant topics in Twitter, "Social gathering", Öuting" and "Fashion" are dominant topics in Instagram, and "Music", "Entertainment" and "Fashion" are dominant topics in Tumblr.

  

  

2016

• Gossen, G., Demidova, E., Risse, T.: Analyzing Web Archives Through Topic and Event Focused Sub-collections. Proceedings of the 8th ACM Conference on Web Science. p. 291--295. ACM, Hannover, Germany (2016).

  Abstract BibTeX EndNote Download URL

  Web archives capture the history of the Web and are therefore an important source to study how societal developments have been reflected on the Web. However, the large size of Web archives and their temporal nature pose many challenges to researchers interested in working with these collections. In this work, we describe the challenges of working with Web archives and propose the research methodology of extracting and studying sub-collections of the archive focused on specific topics and events. We discuss the opportunities and challenges of this approach and suggest a framework for creating sub-collections.

  

  
• Kanhabua, N., Kemkes, P., Nejdl, W., Nguyen, T.N., Reis, F., Tran, N.K.: How to Search the Internet Archive Without Indexing It. In: Fuhr, N., Kovács, L., Risse, T., and Nejdl, W. (eds.) Research and Advanced Technology for Digital Libraries - 20th International Conference on Theory and Practice of Digital Libraries, TPDL 2016, Hannover, Germany, September 5-9, 2016, Proceedings. p. 147--160. Springer (2016).

  BibTeX EndNote Download URL

  

  
• Rokicki, M., Zerr, S., Siersdorfer, S.: Just in Time: Controlling Temporal Performance in Crowdsourcing Competitions. Proceedings of the 25th International Conference on World Wide Web. p. 817--827. International World Wide Web Conferences Steering Committee, Montr&\#233;al, Qu&\#233;bec, Canada (2016).

  BibTeX EndNote Download URL

  

  
• Gottschalk, S., Demidova, E.: Analysing Temporal Evolution of Interlingual Wikipedia Article Pairs. Proceedings of the 39th International ACM SIGIR Conference on Research and Development in Information Retrieval. p. 1089--1092. ACM, Pisa, Italy (2016).

  Abstract BibTeX EndNote Download URL

  Wikipedia articles representing an entity or a topic in different language editions evolve independently within the scope of the language-specific user communities. This can lead to different points of views reflected in the articles, as well as complementary and inconsistent information. An analysis of how the information is propagated across the Wikipedia language editions can provide important insights in the article evolution along the temporal and cultural dimensions and support quality control. To facilitate such analysis, we present MultiWiki -- a novel web-based user interface that provides an overview of the similarities and differences across the article pairs originating from different language editions on a timeline. MultiWiki enables users to observe the changes in the interlingual article similarity over time and to perform a detailed visual comparison of the article snapshots at a particular time point.

  

  
• Fulantelli, G., Marenzi, I., Ahmad, Q.A.I., Taibi, D.: SaR-Web - A Tool to Support Search as Learning Processes. In: Gwizdka, J., Hansen, P., Hauff, C., He, J., and Kando, N. (eds.) Proceedings of the Second International Workshop on Search as Learning, SAL 2016, co-located with the 39th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2016, Pisa, Italy, July 21st, 2016. CEUR-WS.org (2016).

  BibTeX EndNote Download URL

  

  
• Holzmann, H., Runnwerth, M., Sperber, W.: Linking Mathematical Software in Web Archives. In: Greuel, G.-M., Koch, T., Paule, P., and Sommese, A. (eds.) Mathematical Software -- ICMS 2016: 5th International Conference, Berlin, Germany, July 11-14, 2016, Proceedings. p. 419--422. Springer International Publishing, Cham (2016).

  BibTeX EndNote Download URL

  

  
• Singh, J., Hoffart, J., Anand, A.: Discovering Entities with Just a Little Help from You. Proceedings of the 25th ACM International on Conference on Information and Knowledge Management. p. 1331--1340. ACM, Indianapolis, Indiana, USA (2016).

  Abstract BibTeX EndNote URL

  Linking entities like people, organizations, books, music groups and their songs in text to knowledge bases (KBs) is a fundamental task for many downstream search and mining applications. Achieving high disambiguation accuracy crucially depends on a rich and holistic representation of the entities in the KB. For popular entities, such a representation can be easily mined from Wikipedia, and many current entity disambiguation and linking methods make use of this fact. However, Wikipedia does not contain long-tail entities that only few people are interested in, and also at times lags behind until newly emerging entities are added. For such entities, mining a suitable representation in a fully automated fashion is very difficult, resulting in poor linking accuracy. What can automatically be mined, though, is a high-quality representation given the context of a new entity occurring in any text. Due to the lack of knowledge about the entity, no method can retrieve these occurrences automatically with high precision, resulting in a chicken-egg problem. To address this, our approach automatically generates candidate occurrences of entities, prompting the user for feedback to decide if the occurrence refers to the actual entity in question. This feedback gradually improves the knowledge and allows our methods to provide better candidate suggestions to keep the user engaged. We propose novel human-in-the-loop retrieval methods for generating candidates based on gradient interleaving of diversification and textual relevance approaches. We conducted extensive experiments on the FACC dataset, showing that our approaches convincingly outperform carefully selected baselines in both intrinsic and extrinsic measures while keeping users engaged.

  

  
• Holzmann, H., Anand, A.: Tempas: Temporal Archive Search Based on Tags. Proceedings of the 25th International Conference Companion on World Wide Web. p. 207--210. International World Wide Web Conferences Steering Committee, Montr&\#233;al, Qu&\#233;bec, Canada (2016).

  BibTeX EndNote Download URL

  

  
• Fernando, Z.T., Marenzi, I., Nejdl, W., Kalyani, R.: ArchiveWeb: Collaboratively Extending and Exploring Web Archive Collections. In: Fuhr, N., Kovács, L., Risse, T., and Nejdl, W. (eds.) Research and Advanced Technology for Digital Libraries - 20th International Conference on Theory and Practice of Digital Libraries, TPDL 2016, Hannover, Germany, September 5-9, 2016, Proceedings. p. 107--118. Springer (2016).

  BibTeX EndNote Download URL

  

  
• Holzmann, H., Nejdl, W., Anand, A.: On the Applicability of Delicious for Temporal Search on Web Archives. Proceedings of the 39th International ACM SIGIR Conference on Research and Development in Information Retrieval. p. 929--932. ACM, Pisa, Italy (2016).

  Abstract BibTeX EndNote Download URL

  Web archives are large longitudinal collections that store webpages from the past, which might be missing on the current live Web. Consequently, temporal search over such collections is essential for finding prominent missing webpages and tasks like historical analysis. However, this has been challenging due to the lack of popularity information and proper ground truth to evaluate temporal retrieval models. In this paper we investigate the applicability of external longitudinal resources to identify important and popular websites in the past and analyze the social bookmarking service Delicious for this purpose. The timestamped bookmarks on Delicious provide explicit cues about popular time periods in the past along with relevant descriptors. These are valuable to identify important documents in the past for a given temporal query. Focusing purely on recall, we analyzed more than 12,000 queries and find that using Delicious yields average recall values from 46% up to 100%, when limiting ourselves to the best represented queries in the considered dataset. This constitutes an attractive and low-overhead approach for quick access into Web archives by not dealing with the actual contents.

  

  
• Zhou, Y., Demidova, E., Cristea, A.I.: Who Likes Me More?: Analysing Entity-centric Language-specific Bias in Multilingual Wikipedia. Proceedings of the 31st Annual ACM Symposium on Applied Computing. p. 750--757. ACM, Pisa, Italy (2016).

  Abstract BibTeX EndNote Download URL

  In this paper we take an important step towards better understanding the existence and extent of entity-centric language-specific bias in multilingual Wikipedia, and any deviation from its targeted neutral point of view. We propose a methodology using sentiment analysis techniques to systematically extract the variations in sentiments associated with real-world entities in different language editions of Wikipedia, illustrated with a case study of five Wikipedia language editions and a set of target entities from four categories.

  

  
• Holzmann, H., Goel, V., Anand, A.: ArchiveSpark: Efficient Web Archive Access, Extraction and Derivation. Proceedings of the 16th ACM/IEEE-CS on Joint Conference on Digital Libraries. p. 83--92. ACM, Newark, New Jersey, USA (2016).

  Abstract BibTeX EndNote Download URL

  Web archives are a valuable resource for researchers of various disciplines. However, to use them as a scholarly source, researchers require a tool that provides efficient access to Web archive data for extraction and derivation of smaller datasets. Besides efficient access we identify five other objectives based on practical researcher needs such as ease of use, extensibility and reusability. Towards these objectives we propose ArchiveSpark, a framework for efficient, distributed Web archive processing that builds a research corpus by working on existing and standardized data formats commonly held by Web archiving institutions. Performance optimizations in ArchiveSpark, facilitated by the use of a widely available metadata index, result in significant speed-ups of data processing. Our benchmarks show that ArchiveSpark is faster than alternative approaches without depending on any additional data stores while improving usability by seamlessly integrating queries and derivations with external tools.

  

  
• Holzmann, H., Sperber, W., Runnwerth, M.: Archiving Software Surrogates on the Web for Future Reference. In: Fuhr, N., Kovács, L., Risse, T., and Nejdl, W. (eds.) Research and Advanced Technology for Digital Libraries: 20th International Conference on Theory and Practice of Digital Libraries, TPDL 2016, Hannover, Germany, September 5--9, 2016, Proceedings. p. 215--226. Springer International Publishing, Cham (2016).

  BibTeX EndNote URL

  

  
• Springstein, M., Ewerth, R.: On the Effects of Spam Filtering and Incremental Learning for Web-Supervised Visual Concept Classification. Proceedings of the 2016 ACM on International Conference on Multimedia Retrieval. p. 377--380. ACM, New York, New York, USA (2016).

  Abstract BibTeX EndNote Download URL

  Deep neural networks have been successfully applied to the task of visual concept classification. However, they require a large number of training examples for learning. Although pre-trained deep neural networks are available for some domains, they usually have to be fine-tuned for an envisaged target domain. Recently, some approaches have been suggested that are aimed at incrementally (or even endlessly) learning visual concepts based on Web data. Since tags of Web images are often noisy, normally some filtering mechanisms are employed in order to remove ``spam'' images that are not appropriate for training. In this paper, we investigate several aspects of a web-supervised system that has to be adapted to another target domain: 1.) the effect of incremental learning, 2.) the effect of spam filtering, and 3.) the behavior of particular concept classes with respect to 1.) and 2.). The experimental results provide some insights under which conditions incremental learning and spam filtering are useful.

  

  
• Müller, E., Otto, C., Ewerth, R.: Semi-supervised Identification of Rarely Appearing Persons in Video by Correcting Weak Labels. Proceedings of the 2016 ACM on International Conference on Multimedia Retrieval. p. 381--384. ACM, New York, New York, USA (2016).

  Abstract BibTeX EndNote Download URL

  Some recent approaches for character identification in movies and TV broadcasts are realized in a semi-supervised manner by assigning transcripts and/or subtitles to the speakers. However, the labels obtained in this way achieve only an accuracy of $80\% - 90\%$ and the number of training examples for the different actors is unevenly distributed. In this paper, we propose a novel approach for person identification in video by correcting and extending the training data with reliable predictions to reduce the number of annotation errors. Furthermore, the intra-class diversity of rarely speaking characters is enhanced. To address the imbalance of training data per person, we suggest two complementary prediction scores. These scores are also used to recognize whether or not a face track belongs to a (supporting) character whose identity does not appear in the transcript etc. Experimental results demonstrate the feasibility of the proposed approach, outperforming the current state of the art.

  

  
• Fetahu, B., Markert, K., Nejdl, W., Anand, A.: Finding News Citations for Wikipedia. Proceedings of the 25th ACM International on Conference on Information and Knowledge Management. p. 337--346. ACM, Indianapolis, Indiana, USA (2016).

  Abstract BibTeX EndNote Download URL

  An important editing policy in Wikipedia is to provide citations for added statements in Wikipedia pages, where statements can be arbitrary pieces of text, ranging from a sentence to a paragraph. In many cases citations are either outdated or missing altogether. In this work we address the problem of finding and updating news citations for statements in entity pages. We propose a two-stage supervised approach for this problem. In the first step, we construct a classifier to find out whether statements need a news citation or other kinds of citations (web, book, journal, etc.). In the second step, we develop a news citation algorithm for Wikipedia statements, which recommends appropriate citations from a given news collection. Apart from IR techniques that use the statement to query the news collection, we also formalize three properties of an appropriate citation, namely: (i) the citation should entail the Wikipedia statement, (ii) the statement should be central to the citation, and (iii) the citation should be from an authoritative source. We perform an extensive evaluation of both steps, using 20 million articles from a real-world news collection. Our results are quite promising, and show that we can perform this task with high precision and at scale.

  

  
• Shaltev, M., Zab, J.-H., Kemkes, P., Siersdorfer, S., Zerr, S.: Cobwebs from the Past and Present: Extracting Large Social Networks Using Internet Archive Data. Proceedings of the 39th International ACM SIGIR Conference on Research and Development in Information Retrieval. p. 1093--1096. ACM, Pisa, Italy (2016).

  Abstract BibTeX EndNote Download URL

  Social graph construction from various sources has been of interest to researchers due to its application potential and the broad range of technical challenges involved. The World Wide Web provides a huge amount of continuously updated data and information on a wide range of topics created by a variety of content providers, and makes the study of extracted people networks and their temporal evolution valuable for social as well as computer scientists. In this paper we present SocGraph - an extraction and exploration system for social relations from the content of around 2 billion web pages collected by the Internet Archive over the 17 years time period between 1996 and 2013. We describe methods for constructing large social graphs from extracted relations and introduce an interface to study their temporal evolution.

  

  
• Singh, J., Nejdl, W., Anand, A.: Expedition: A Time-Aware Exploratory Search System Designed for Scholars. Proceedings of the 39th International ACM SIGIR Conference on Research and Development in Information Retrieval. p. 1105--1108. ACM, Pisa, Italy (2016).

  Abstract BibTeX EndNote Download URL

  Archives are an important source of study for various scholars. Digitization and the web have made archives more accessible and led to the development of several time-aware exploratory search systems. However these systems have been designed for more general users rather than scholars. Scholars have more complex information needs in comparison to general users. They also require support for corpus creation during their exploration process. In this paper we present Expedition - a time-aware exploratory search system that addresses the requirements and information needs of scholars. Expedition possesses a suite of ad-hoc and diversity based retrieval models to address complex information needs; a newspaper-style user interface to allow for larger textual previews and comparisons; entity filters to more naturally refine a result list and an interactive annotated timeline which can be used to better identify periods of importance.

  

  
• Holzmann, H., Nejdl, W., Anand, A.: The Dawn of Today's Popular Domains: A Study of the Archived German Web over 18 Years. Proceedings of the 16th ACM/IEEE-CS on Joint Conference on Digital Libraries. p. 73--82. ACM, Newark, New Jersey, USA (2016).

  Abstract BibTeX EndNote Download URL

  The Web has been around and maturing for 25 years. The popular websites of today have undergone vast changes during this period, with a few being there almost since the beginning and many new ones becoming popular over the years. This makes it worthwhile to take a look at how these sites have evolved and what they might tell us about the future of the Web. We therefore embarked on a longitudinal study spanning almost the whole period of the Web, based on data collected by the Internet Archive starting in 1996, to retrospectively analyze how the popular Web as of now has evolved over the past 18 years. For our study we focused on the German Web, specifically on the top 100 most popular websites in 17 categories. This paper presents a selection of the most interesting findings in terms of volume, size as well as age of the Web. While related work in the field of Web Dynamics has mainly focused on change rates and analyzed datasets spanning less than a year, we looked at the evolution of websites over 18 years. We found that around 70% of the pages we investigated are younger than a year, with an observed exponential growth in age as well as in size up to now. If this growth rate continues, the number of pages from the popular domains will almost double in the next two years. In addition, we give insights into our data set, provided by the Internet Archive, which hosts the largest and most complete Web archive as of today.

  

  
• Singh, J., Nejdl, W., Anand, A.: History by Diversity: Helping Historians Search News Archives. Proceedings of the 2016 ACM on Conference on Human Information Interaction and Retrieval. p. 183--192. ACM, Carrboro, North Carolina, USA (2016).

  Abstract BibTeX EndNote Download URL

  Longitudinal corpora like newspaper archives are of immense value to historical research, and time as an important factor for historians strongly influences their search behaviour in these archives. While searching for articles published over time, a key preference is to retrieve documents which cover the important aspects from important points in time which is different from standard search behavior. To support this search strategy, we introduce the notion of a Historical Query Intent to explicitly model a historian's search task and define an aspect-time diversification problem over news archives. We present a novel algorithm, HistDiv, that explicitly models the aspects and important time windows based on a historian's information seeking behavior. By incorporating temporal priors based on publication times and temporal expressions, we diversify both on the aspect and temporal dimensions. We test our methods by constructing a test collection based on The New York Times Collection with a workload of 30 queries of historical intent assessed manually. We find that HistDiv outperforms all competitors in subtopic recall with a slight loss in precision. We also present results of a qualitative user study to determine wether this drop in precision is detrimental to user experience. Our results show that users still preferred HistDiv's ranking.

  

  
• Vo, K.D., Tran, T., Nguyen, T.N., Zhu, X., Nejdl, W.: Can We Find Documents in Web Archives Without Knowing Their Contents? Proceedings of the 8th ACM Conference on Web Science. p. 173--182. ACM, Hannover, Germany (2016).

  Abstract BibTeX EndNote Download URL

  Recent advances of preservation technologies have led to an increasing number of Web archive systems and collections. These collections are valuable to explore the past of the Web, but their value can only be uncovered with effective access and exploration mechanisms. Ideal search and ranking methods must be robust to the high redundancy and the temporal noise of contents, as well as scalable to the huge amount of data archived. Despite several attempts in Web archive search, facilitating access to Web archive still remains a challenging problem. In this work, we conduct a first analysis on different ranking strategies that exploit evidences from metadata instead of the full content of documents. We perform a first study to compare the usefulness of non-content evidences to Web archive search, where the evidences are mined from the metadata of file headers, links and URL strings only. Based on these findings, we propose a simple yet surprisingly effective learning model that combines multiple evidences to distinguish "good" from "bad" search results. We conduct empirical experiments quantitatively as well as qualitatively to confirm the validity of our proposed method, as a first step towards better ranking in Web archives taking metadata into account.

  

  
• Taibi, D., Rogers, R., Marenzi, I., Nejdl, W., Ahmad, Q.A.I., Fulantelli, G.: Search As Research Practices on the Web: The SaR-Web Platform for Cross-language Engine Results Analysis. Proceedings of the 8th ACM Conference on Web Science. p. 367--369. ACM, Hannover, Germany (2016).

  Abstract BibTeX EndNote Download URL

  Search engines are the most utilized tools to access information on the Web. The success of large companies such as Google owes to their capacity to conduct users through the vast troves of knowledge and information online. Recently, the concept of search as research has been used to shift the research focus from workings of information-seeking tools towards methods for the social study of Web and particularly the social meanings of engine results. In this paper, we present SaR-Web, a web search tool that provides an automatic means to carry out search as research on the Web. It compares the results of same (translated) queries across search engine language domains, thereby enabling cross-linguistic and cross-cultural comparisons of results. SaR-Web outputs enable the comparative study of cultural mores as well as societal associations and concerns, interpreted through search engine results.

  

  

2015

• Gossen, G., Demidova, E., Risse, T.: iCrawl: Improving the Freshness of Web Collections by Integrating Social Web and Focused Web Crawling. Proceedings of the 15th ACM/IEEE-CS Joint Conference on Digital Libraries. p. 75--84. ACM, Knoxville, Tennessee, USA (2015).

  Abstract BibTeX EndNote Download URL

  Researchers in the Digital Humanities and journalists need to monitor, collect and analyze fresh online content regarding current events such as the Ebola outbreak or the Ukraine crisis on demand. However, existing focused crawling approaches only consider topical aspects while ignoring temporal aspects and therefore cannot achieve thematically coherent and fresh Web collections. Especially Social Media provide a rich source of fresh content, which is not used by state-of-the-art focused crawlers. In this paper we address the issues of enabling the collection of fresh and relevant Web and Social Web content for a topic of interest through seamless integration of Web and Social Media in a novel integrated focused crawler. The crawler collects Web and Social Media content in a single system and exploits the stream of fresh Social Media content for guiding the crawler.

  

  
• Gossen, G., Demidova, E., Risse, T.: The iCrawl Wizard – Supporting Interactive Focused Crawl Specification. Proceedings of the European Conference on Information Retrieval (ECIR) 2015 (2015).

  BibTeX EndNote Download URL

  

  
• Tran, A.T., Tran, N.K., Hadgu, A.T., Jäschke, R.: Semantic Annotation for Microblog Topics Using Wikipedia Temporal Information. In: Màrquez, L., Callison-Burch, C., Su, J., Pighin, D., and Marton, Y. (eds.) Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, EMNLP 2015, Lisbon, Portugal, September 17-21, 2015. p. 97--106. The Association for Computational Linguistics (2015).

  BibTeX EndNote Download URL

  

  
• Holzmann, H., Tahmasebi, N., Risse, T.: Named Entity Evolution Recognition on the Blogosphere. International Journal on Digital Libraries. 15, 209-235 (2015).

  BibTeX EndNote Download URL

  

  
• Singh, J., Anand, A., Setty, V., Anand, A.: Exploring Long Running News Stories Using Wikipedia. Proceedings of the ACM Web Science Conference. p. 57:1--57:2. ACM, Oxford, United Kingdom (2015).

  Abstract BibTeX EndNote Download URL

  A significant portion of today's news articles are part of long running stories. To better understand the context of these stories journalists, social scientists and other scholars use news collections to find temporal and topical insights. However these insights are devoid of user impressions, derived from click-through data and query logs, and are only reliable if the collection is complete and consistent. In this work we introduce the notion of combining user impressions from Wikipedia with news collection based insights for long running news story exploration and outline promising new research directions. We also demonstrate our initial attempts with a prototype system called NewsEX.

  

  
• Tahmasebi, N., Borin, L., Capannini, G., Dubhashi, D., Exner, P., Forsberg, M., Gossen, G., Johansson, F.D., Johansson, R., Kågebäck, M., Mogren, O., Nugues, P., Risse, T.: Visions and open challenges for a knowledge-based culturomics. International Journal on Digital Libraries. 15, 169-187 (2015).

  BibTeX EndNote Download URL

  

  
• Nejdl, W., Risse, T.: Herausforderungen für die nationale, regionale und thematische Webarchivierung und deren Nutzung. Zeitschrift für Bibliothekswesen und Bibliographie. 62, 160--171 (2015).

  BibTeX EndNote Download URL

  

  
• Nguyen, T.N., Kanhabua, N., Niederée, C., Zhu, X.: A Time-aware Random Walk Model for Finding Important Documents in Web Archives. Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval. p. 915--918. ACM, Santiago, Chile (2015).

  Abstract BibTeX EndNote Download URL

  Due to their first-hand, diverse and evolution-aware reflection of nearly all areas of life, web archives are emerging as gold-mines for content analytics of many sorts. However, supporting search, which goes beyond navigational search via URLs, is a very challenging task in these unique structures with huge, redundant and noisy temporal content. In this paper, we address the search needs of expert users such as journalists, economists or historians for discovering a topic in time: Given a query, the top-k returned results should give the best representative documents that cover most interesting time-periods for the topic. For this purpose, we propose a novel random walk-based model that integrates relevance, temporal authority, diversity and time in a unified framework. Our preliminary experimental results on the large-scale real-world web archival collection shows that our method significantly improves the state-of-the-art algorithms (i.e., PageRank) in ranking temporal web pages.

  

  
• Nguyen, T.N., Kanhabua, N., Nejdl, W., Niederée, C.: Mining Relevant Time for Query Subtopics in Web Archives. Proceedings of the 24th International Conference on World Wide Web. p. 1357--1362. ACM, Florence, Italy (2015).

  Abstract BibTeX EndNote Download URL

  With the reflection of nearly all types of social cultural, societal and everyday processes of our lives in the web, web archives from organizations such as the Internet Archive have the potential of becoming huge gold-mines for temporal content analytics of many kinds (e.g., on politics, social issues, economics or media). First hand evidences for such processes are of great benefit for expert users such as journalists, economists, historians, etc. However, searching in this unique longitudinal collection of huge redundancy (pages of near-identical content are crawled all over again) is completely different from searching over the web. In this work, we present our first study of mining the temporal dynamics of subtopics by leveraging the value of anchor text along the time dimension of the enormous web archives. This task is especially useful for one important ranking problem in the web archive context, the time-aware search result diversification. Due to the time uncertainty (the lagging nature and unpredicted behavior of the crawlers), identifying the trending periods for such temporal subtopics relying solely on the timestamp annotations of the web archive (i.e., crawling times) is extremely difficult. We introduce a brute-force approach to detect a time-reliable sub-collection and propose a method to leverage them for relevant time mining of subtopics. This is empirically found effective in solving the problem.

  

  
• Kanhabua, N., Ngoc Nguyen, T., Nejdl, W.: Learning to Detect Event-Related Queries for Web Search. Proceedings of the 24th International Conference on World Wide Web. p. 1339--1344. ACM, Florence, Italy (2015).

  Abstract BibTeX EndNote Download URL

  In many cases, a user turns to search engines to find information about real-world situations, namely, political elections, sport competitions, or natural disasters. Such temporal querying behavior can be observed through a significant number of event-related queries generated in web search. In this paper, we study the task of detecting event-related queries, which is the first step for understanding temporal query intent and enabling different temporal search applications, e.g., time-aware query auto-completion, temporal ranking, and result diversification. We propose a two-step approach to detecting events from query logs. We first identify a set of event candidates by considering both implicit and explicit temporal information needs. The next step further classifies the candidates into two main categories, namely, event or non-event. In more detail, we leverage different machine learning techniques for query classification, which are trained using the feature set composed of time series features from signal processing, along with features derived from click-through information, and standard statistical features. In order to evaluate our proposed approach, we conduct an experiment using two real-world query logs with manually annotated relevance assessments for 837 events. To this end, we provide a large set of event-related queries made available for fostering research on this challenging task.

  

  
• Souza, T., Demidova, E., Risse, T., Holzmann, H., Gossen, G., Szymanski, J.: Semantic URL Analytics to Support Efficient Annotation of Large Scale Web Archives. International KEYSTONE Conference. p. 153--166. Springer (2015).

  BibTeX EndNote Download URL

  

  
• Fetahu, B., Gadiraju, U., Dietze, S.: Improving Entity Retrieval on Structured Data. International Semantic Web Conference (1). p. 474--491. Springer (2015).

  BibTeX EndNote Download URL

  

  
• Rokicki, M., Zerr, S., Siersdorfer, S.: Groupsourcing: Team Competition Designs for Crowdsourcing. Proceedings of the 24th International Conference on World Wide Web. p. 906--915. ACM, Florence, Italy (2015).

  Abstract BibTeX EndNote Download URL

  Many data processing tasks such as semantic annotation of images, translation of texts in foreign languages, and labeling of training data for machine learning models require human input, and, on a large scale, can only be accurately solved using crowd based online work. Recent work shows that frameworks where crowd workers compete against each other can drastically reduce crowdsourcing costs, and outperform conventional reward schemes where the payment of online workers is proportional to the number of accomplished tasks ("pay-per-task"). In this paper, we investigate how team mechanisms can be leveraged to further improve the cost efficiency of crowdsourcing competitions. To this end, we introduce strategies for team based crowdsourcing, ranging from team formation processes where workers are randomly assigned to competing teams, over strategies involving self-organization where workers actively participate in team building, to combinations of team and individual competitions. Our large-scale experimental evaluation with more than 1,100 participants and overall 5,400 hours of work spent by crowd workers demonstrates that our team based crowdsourcing mechanisms are well accepted by online workers and lead to substantial performance boosts.

  

  
• Tran, T.A., Niederee, C., Kanhabua, N., Gadiraju, U., Anand, A.: Balancing Novelty and Salience: Adaptive Learning to Rank Entities for Timeline Summarization of High-impact Events. Proceedings of the 24th ACM International on Conference on Information and Knowledge Management. p. 1201--1210. ACM, Melbourne, Australia (2015).

  Abstract BibTeX EndNote Download URL

  Long-running, high-impact events such as the Boston Marathon bombing often develop through many stages and involve a large number of entities in their unfolding. Timeline summarization of an event by key sentences eases story digestion, but does not distinguish between what a user remembers and what she might want to re-check. In this work, we present a novel approach for timeline summarization of high-impact events, which uses entities instead of sentences for summarizing the event at each individual point in time. Such entity summaries can serve as both (1) important memory cues in a retrospective event consideration and (2) pointers for personalized event exploration. In order to automatically create such summaries, it is crucial to identify the "right" entities for inclusion. We propose to learn a ranking function for entities, with a dynamically adapted trade-off between the in-document salience of entities and the informativeness of entities across documents, i.e., the level of new information associated with an entity for a time point under consideration. Furthermore, for capturing collective attention for an entity we use an innovative soft labeling approach based on Wikipedia. Our experiments on a real large news datasets confirm the effectiveness of the proposed methods.

  

  
• Fetahu, B., Markert, K., Anand, A.: Automated News Suggestions for Populating Wikipedia Entity Pages. Proceedings of the 24th ACM International on Conference on Information and Knowledge Management. p. 323--332. ACM, Melbourne, Australia (2015).

  Abstract BibTeX EndNote Download URL

  Wikipedia entity pages are a valuable source of information for direct consumption and for knowledge-base construction, update and maintenance. Facts in these entity pages are typically supported by references. Recent studies show that as much as 20% of the references are from online news sources. However, many entity pages are incomplete even if relevant information is already available in existing news articles. Even for the already present references, there is often a delay between the news article publication time and the reference time. In this work, we therefore look at Wikipedia through the lens of news and propose a novel news-article suggestion task to improve news coverage in Wikipedia, and reduce the lag of newsworthy references. Our work finds direct application, as a precursor, to Wikipedia page generation and knowledge-base acceleration tasks that rely on relevant and high quality input sources. We propose a two-stage supervised approach for suggesting news articles to entity pages for a given state of Wikipedia. First, we suggest news articles to Wikipedia entities (article-entity placement) relying on a rich set of features which take into account the salience and relative authority of entities, and the novelty of news articles to entity pages. Second, we determine the exact section in the entity page for the input article (article-section placement) guided by class-based section templates. We perform an extensive evaluation of our approach based on ground-truth data that is extracted from external references in Wikipedia. We achieve a high precision value of up to 93% in the article-entity suggestion stage and upto 84% for the article-section placement. Finally, we compare our approach against competitive baselines and show significant improvements.

  

  
• Fetahu, B., Anand, A., Anand, A.: How Much is Wikipedia Lagging Behind News? Proceedings of the ACM Web Science Conference. p. 28:1--28:9. ACM, Oxford, United Kingdom (2015).

  Abstract BibTeX EndNote Download URL

  Wikipedia, rich in entities and events, is an invaluable resource for various knowledge harvesting, extraction and mining tasks. Numerous resources like DBpedia, YAGO and other knowledge bases are based on extracting entity and event based knowledge from it. Online news, on the other hand, is an authoritative and rich source for emerging entities, events and facts relating to existing entities. In this work, we study the creation of entities in Wikipedia with respect to news by studying how entity and event based information flows from news to Wikipedia. We analyze the lag of Wikipedia (based on the revision history of the English Wikipedia) with 20 years of The New York Times dataset (NYT). We model and analyze the lag of entities and events, namely their first appearance in Wikipedia and in NYT, respectively. In our extensive experimental analysis, we find that almost 20% of the external references in entity pages are news articles encoding the importance of news to Wikipedia. Second, we observe that the entity-based lag follows a normal distribution with a high standard deviation, whereas the lag for news-based events is typically very low. Finally, we find that events are responsible for creation of emergent entities with as many as 12% of the entities mentioned in the event page are created after the creation of the event page.

  

  
• Siersdorfer, S., Kemkes, P., Ackermann, H., Zerr, S.: Who With Whom And How?: Extracting Large Social Networks Using Search Engines. Proceedings of the 24th ACM International on Conference on Information and Knowledge Management. p. 1491--1500. ACM, Melbourne, Australia (2015).

  BibTeX EndNote Download URL

  

  

2014

• Tran, T., Georgescu, M., Zhu, X., Kanhabua, N.: Analysing the Duration of Trending Topics in Twitter Using Wikipedia. Proceedings of the 2014 ACM Conference on Web Science. p. 251--252. ACM, Bloomington, Indiana, USA (2014).

  Abstract BibTeX EndNote Download URL

  The analysis of trending topics in Twitter is a goldmine for a variety of studies and applications. However, the contents of topics vary greatly from daily routines to major public events, enduring from a few hours to weeks or months. It is thus helpful to distinguish trending topics related to real-world events with those originated within virtual communities. In this paper, we analyse trending topics in Twitter using Wikipedia as reference for studying the provenance of trending topics. We show that among different factors, the duration of a trending topic characterizes exogenous Twitter trending topics better than endogenous ones.

  

  
• Holzmann, H., Risse, T.: Insights into Entity Name Evolution on Wikipedia. In: Benatallah, B., Bestavros, A., Manolopoulos, Y., Vakali, A., and Zhang, Y. (eds.) Web Information Systems Engineering – WISE 2014. pp. 47-61. Springer International Publishing (2014).

  Abstract BibTeX EndNote Download URL

  Working with Web archives raises a number of issues caused by their temporal characteristics. Depending on the age of the content, additional knowledge might be needed to find and understand older texts. Especially facts about entities are subject to change. Most severe in terms of information retrieval are name changes. In order to find entities that have changed their name over time, search engines need to be aware of this evolution. We tackle this problem by analyzing Wikipedia in terms of entity evolutions mentioned in articles regardless the structural elements. We gathered statistics and automatically extracted minimum excerpts covering name changes by incorporating lists dedicated to that subject. In future work, these excerpts are going to be used to discover patterns and detect changes in other sources. In this work we investigate whether or not Wikipedia is a suitable source for extracting the required knowledge.

  

  
• Tran, T., Nguyen, T.N.: Hedera: Scalable Indexing and Exploring Entities in Wikipedia Revision History. in Poster & System Demonstration Proceedings of 13th International Semantic Web Conference (ISWC2014), Riva Del Garda, Italy, October 2014 (2014).

  BibTeX EndNote Download URL

  

  
• Kanhabua, N., Nejdl, W.: On the Value of Temporal Anchor Texts in Wikipedia. Proceedings of SIGIR 2014 Workshop on Temporal, Social and Spatially-aware Information Access (#TAIA2014) (2014).

  BibTeX EndNote Download URL

  

  
• Kanhabua, N., Nguyen, T.N., Niederee, C.: What triggers human remembering of events? A large-scale analysis of catalysts for collective memory in Wikipedia. IEEE/ACM Joint Conference on Digital Libraries. Institute of Electrical and Electronics Engineers (IEEE) (2014).

  BibTeX EndNote Download URL

  

  
• Tran, T., Ceroni, A., Georgescu, M., Djafari Naini, K., Fisichella, M.: WikipEvent: Leveraging Wikipedia Edit History for Event Detection. In: Benatallah, B., Bestavros, A., Manolopoulos, Y., Vakali, A., and Zhang, Y. (eds.) Web Information Systems Engineering – WISE 2014. pp. 90-108. Springer International Publishing (2014).

  Abstract BibTeX EndNote Download URL

  Much of existing work in information extraction assumes the static nature of relationships in fixed knowledge bases. However, in collaborative environments such as Wikipedia, information and structures are highly dynamic over time. In this work, we introduce a new method to extract complex event structures from Wikipedia. We propose a new model to represent events by engaging multiple entities, generalizable to an arbitrary language. The evolution of an event is captured effectively based on analyzing the user edits history in Wikipedia. Our work provides a foundation for a novel class of evolution-aware entity-based enrichment algorithms, and considerably increases the quality of entity accessibility and temporal retrieval for Wikipedia. We formalize this problem and introduce an efficient end-to-end platform as a solution. We conduct comprehensive experiments on a real dataset of

  

  
• Zerr, S., Papapetrou, O., Demidova, E.: HEALTH+Z: Confidential Provider Selection in Collaborative Healthcare P2P Networks. In: Proceedings, C.E.U.R.W. (ed.) Proceedings of the 1st International Workshop on Privacy-Preserving IR: When Information Retrieval Meets Privacy, (PIR 2014), co-located with SIGIR 2014, July 11 2014, Gold Coast, Australia. pp. 1-6. CEUR (2014).

  BibTeX EndNote Download URL

  

  
• Holzmann, H., Risse, T.: Extraction of evolution descriptions from the web. IEEE/ACM Joint Conference on Digital Libraries. Institute of Electrical and Electronics Engineers (IEEE) (2014).

  BibTeX EndNote Download URL

  

  
• Demidova, E., Barbieri, N., Dietze, S., Funk, A., Holzmann, H., Maynard, D., Papailiou, N., Peters, W., Risse, T., Spiliotopoulos, D.: Analysing and Enriching Focused Semantic Web Archives for Parliament Applications. Future Internet. 6, 433 (2014).

  Abstract BibTeX EndNote Download URL

  The web and the social web play an increasingly important role as an information source for Members of Parliament and their assistants, journalists, political analysts and researchers. It provides important and crucial background information, like reactions to political events and comments made by the general public. The case study presented in this paper is driven by two European parliaments (the Greek and the Austrian parliament) and targets an effective exploration of political web archives. In this paper, we describe semantic technologies deployed to ease the exploration of the archived web and social web content and present evaluation results.

  

  
• Kotsakos, D., Lappas, T., Kotzias, D., Gunopulos, D., Kanhabua, N., Nørv\aag, K.: A Burstiness-aware Approach for Document Dating. Proceedings of the 37th International ACM SIGIR Conference on Research &\#38; Development in Information Retrieval. p. 1003--1006. ACM, Gold Coast, Queensland, Australia (2014).

  Abstract BibTeX EndNote Download URL

  A large number of mainstream applications, like temporal search, event detection, and trend identification, assume knowledge of the timestamp of every document in a given textual collection. In many cases, however, the required timestamps are either unavailable or ambiguous. A charac- teristic instance of this problem emerges in the context of large repositories of old digitized documents. For such doc- uments, the timestamp may be corrupted during the digiti- zation process, or may simply be unavailable. In this paper, we study the task of approximating the timestamp of a doc- ument, so-called document dating. We propose a content- based method and use recent advances in the domain of term burstiness, which allow it to overcome the drawbacks of pre- vious document dating methods, e.g. the fix time partition strategy. We use an extensive experimental evaluation on different datasets to validate the efficacy and advantages of our methodology, showing that our method outperforms the state of the art methods on document dating.

  

  
• Yuan, W., Demidova, E., Dietze, S., Zhou, X.: Analyzing Relative Incompleteness of Movie Descriptions in the Web of Data: A Case Study. Proceedings of the ISWC 2014 Posters & Demonstrations Track, a track within the 13th International Semantic Web Conference (ISWC 2014). pp. 197-200. CEUR-WS.org (2014).

  BibTeX EndNote Download URL

  

  
• Ceroni, A., Tran, N.K., Kanhabua, N., Niederée, C.: Bridging Temporal Context Gaps Using Time-aware Re-contextualization. Proceedings of the 37th International ACM SIGIR Conference on Research &\#38; Development in Information Retrieval. p. 1127--1130. ACM, Gold Coast, Queensland, Australia (2014).

  Abstract BibTeX EndNote Download URL

  Understanding a text, which was written some time ago, can be compared to translating a text from another language. Complete interpretation requires a mapping, in this case, a kind of time-travel translation between present context knowledge and context knowledge at time of text creation. In this paper, we study time-aware re-contextualization, the challenging problem of retrieving concise and complementing information in order to bridge this temporal context gap. We propose an approach based on learning to rank techniques using sentence-level context information extracted from Wikipedia. The employed ranking combines relevance, complimentarity and time-awareness. The effectiveness of the approach is evaluated by contextualizing articles from a news archive collection using more than 7,000 manually judged relevance pairs. To this end, we show that our approach is able to retrieve a significant number of relevant context information for a given news article.

  

  
• Rokicki, M., Chelaru, S., Zerr, S., Siersdorfer, S.: Competitive Game Designs for Improving the Cost Effectiveness of Crowdsourcing. Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Management. p. 1469--1478. ACM, Shanghai, China (2014).

  Abstract BibTeX EndNote Download URL

  Crowd based online work is leveraged in a variety of applications such as semantic annotation of images, translation of texts in foreign languages, and labeling of training data for machine learning models. However, annotating large amounts of data through crowdsourcing can be slow and costly. In order to improve both cost and time efficiency of crowdsourcing we examine alternative reward mechanisms compared to the "Pay-per-HIT" scheme commonly used in platforms such as Amazon Mechanical Turk. To this end, we explore a wide range of monetary reward schemes that are inspired by the success of competitions, lotteries, and games of luck. Our large-scale experimental evaluation with an overall budget of more than 1,000 USD and with 2,700 hours of work spent by crowd workers demonstrates that our alternative reward mechanisms are well accepted by online workers and lead to substantial performance boosts.

  

  
• Holzmann, H., Risse, T.: Named Entity Evolution Analysis on Wikipedia. Proceedings of the 2014 ACM Conference on Web Science. p. 241--242. ACM, Bloomington, Indiana, USA (2014).

  Abstract BibTeX EndNote Download URL

  Accessing Web archives raises a number of issues caused by their temporal characteristics. Additional knowledge is needed to find and understand older texts. Especially entities mentioned in texts are subject to change. Most severe in terms of information retrieval are name changes. In order to find entities that have changed their name over time, search engines need to be aware of this evolution. We tackle this problem by analyzing Wikipedia in terms of entity evolutions mentioned in articles. We present statistical data on excerpts covering name changes, which will be used to discover similar text passages and extract evolution knowledge in future work.