Publications

2018

• Fafalios, P., Kasturia, V., Nejdl, W.: Ranking Archived Documents for Structured Queries on Semantic Layers. In: Chen, J., Gonçalves, M.A., Allen, J.M., Fox, E.A., Kan, M.-Y., and Petras, V. (eds.) JCDL. pp. 155-164. ACM (2018).

  BibTeX EndNote URL

  

  
• Fafalios, P., Iosifidis, V., Ntoutsi, E., Dietze, S.: TweetsKB: A Public and Large-Scale RDF Corpus of Annotated Tweets. In: Gangemi, A., Navigli, R., Vidal, M.-E., Hitzler, P., Troncy, R., Hollink, L., Tordai, A., and Alam, M. (eds.) ESWC. pp. 177-190. Springer (2018).

  BibTeX EndNote URL

  

  
• Hube, C., Fetahu, B.: Detecting Biased Statements in Wikipedia. In: Champin, P.-A., Gandon, F.L., Lalmas, M., and Ipeirotis, P.G. (eds.) WWW (Companion Volume). pp. 1779-1786. ACM (2018).

  BibTeX EndNote URL

  

  
• Lee, K.-W.R., Hoang, T.-A., Lim, E.-P.: Discovering Hidden Topical Hubs and Authorities in Online Social Networks. Proceeding of 2018 SIAM International Conference on Data Mining, SDM18, San Diego, California, USA (2018).

  BibTeX EndNote URL

  

  
• Quraishi, M., Fafalios, P., Herder, E.: Viewpoint Discovery and Understanding in Social Networks. In: Akkermans, H., Fontaine, K., Vermeulen, I., Houben, G.-J., and Weber, M.S. (eds.) WebSci. pp. 47-56. ACM (2018).

  BibTeX EndNote URL

  

  
• Gottschalk, S., Demidova, E.: EventKG: A Multilingual Event-Centric Temporal Knowledge Graph. In Proceedings of the 15th Extended Semantic Web Conference (ESWC 2018). p. 272--287. Springer (2018).

  Abstract BibTeX EndNote URL

  One of the key requirements to facilitate semantic analytics of information regarding contemporary and historical events on the Web, in the news and in social media is the availability of reference knowledge repositories containing comprehensive representations of events and temporal relations. Existing knowledge graphs, with popular examples including DBpedia, YAGO and Wikidata, focus mostly on entity-centric information and are insufficient in terms of their coverage and completeness with respect to events and temporal relations. EventKG presented in this paper is a multilingual event-centric temporal knowledge graph that aims to address this gap. EventKG incorporates over 690 thousand contemporary and historical events and over 2.3 million temporal relations extracted from several large-scale knowledge graphs and less structured sources and makes this information available through a canonical representation. In this paper we present EventKG including its data model, extraction process, and characteristics and discuss its relevance for several real-world applications including Question Answering, timeline generation and cross-cultural analytics.

  

  
• Holzmann, H., Anand, A., Khosla, M.: What the HAK? Estimating Ranking Deviations in Incomplete Graphs. Proceedings of the 14th International Workshop on Mining and Learning with Graphs (MLG) (2018).

  Abstract BibTeX EndNote URL

  Abstract: Most real-world graphs collected from the Web like Web graphs and social network graphs are incomplete. This leads to inaccurate estimates of graph properties based on link analysis such as PageRank. In this paper we focus on studying such deviations in ordering/ranking imposed by PageRank over incomplete graphs. We first show that deviations in rankings induced by PageRank are indeed possible. We measure how much a ranking, induced by PageRank, on an input graph could deviate from the original unseen graph. More importantly, we are interested in conceiving a measure that approximates the rank correlation among them without any knowledge of the original graph. To this extent we formulate the HAK measure that is based on computing the impact redistribution of PageRank according to the local graph structure. Finally, we perform extensive experiments on both real-world Web and social network graphs with more than 100M vertices and 10B edges as well as synthetic graphs to showcase the utility of HAK.

  

  
• Fafalios, P., Holzmann, H., Kasturia, V., Nejdl, W.: Building and Querying Semantic Layers for Web Archives (Extended Version). International Journal on Digital Libraries. (2018).

  Abstract BibTeX EndNote URL

  Web archiving is the process of collecting portions of the Web to ensure that the information is preserved for future exploitation. However, despite the increasing number of web archives worldwide, the absence of efficient and meaningful exploration methods still remains a major hurdle in the way of turning them into a usable and useful information source. In this paper, we focus on this problem and propose an RDF/S model and a distributed framework for building semantic profiles (“layers”) that describe semantic information about the contents of web archives. A semantic layer allows describing metadata information about the archived documents, annotating them with useful semantic information (like entities, concepts, and events), and publishing all these data on the Web as Linked Data. Such structured repositories offer advanced query and integration capabilities, and make web archives directly exploitable by other systems and tools. To demonstrate their query capabilities, we build and query semantic layers for three different types of web archives. An experimental evaluation showed that a semantic layer can answer information needs that existing keyword-based systems are not able to sufficiently satisfy.

  

  
• Singh, J., Anand, A.: Posthoc Interpretability of Learning to Rank Models using Secondary Training Data. arXiv preprint arXiv:1806.11330. (2018).

  BibTeX EndNote URL

  

  
• Yannakis, T., Fafalios, P., Tzitzikas, Y.: Heuristics-based Query Reordering for Federated Queries in SPARQL 1.1 and SPARQL-LD. 2nd Workshop on Querying the Web of Data (QuWeDa'18) - in conjunction with ESWC 2018 (2018).

  BibTeX EndNote URL

  

  
• Mohapatra, N., Iosifidis, V., Ekbal, A., Dietze, S., Fafalios, P.: Time-Aware and Corpus-Specific Entity Relatedness. Workshop on Deep Learning for Knowledge Graphs and Semantic Technologies (DL4KGS'18), in conjunction with ESWC 2018 (2018).

  Abstract BibTeX EndNote URL

  Entity relatedness has emerged as an important feature in a plethora of applications such as information retrieval, entity recommendation and entity linking. Given an entity, for instance a person or an organization, entity relatedness measures can be exploited for generating a list of highly-related entities. However, the relation of an entity to some other entity depends on several factors, with time and context being two of the most important ones (where, in our case, context is determined by a particular corpus). For example, the entities related to the International Monetary Fund are different now compared to some years ago, while these entities also may highly differ in the context of a USA news portal compared to a Greek news portal. In this paper, we propose a simple but flexible model for entity relatedness which considers time and entity aware word embeddings by exploiting the underlying corpus. The proposed model does not require external knowledge and is language independent, which makes it widely useful in a variety of applications.

  

  

2017

• Holzmann, H., Nejdl, W., Anand, A.: Exploring Web Archives Through Temporal Anchor Texts. Proceedings of the 2017 ACM on Web Science Conference, WebSci 2017, Troy, NY, USA, June 25 - 28, 2017. p. 289--298 (2017).

  BibTeX EndNote URL

  

  
• Ben Ellefi, M., Bellahsene, Z., John, B., Demidova, E., Dietze, S., Szymanski, J., Todorov, K.: RDF Dataset Profiling - a Survey of Features, Methods, Vocabularies and Applications. Semantic Web Journal. (2017).

  Abstract BibTeX EndNote URL

  The Web of Data, and in particular Linked Data, has seen tremendous growth over the past years. However, reuse and take-up of these rich data sources is often limited and focused on a few well-known and established RDF datasets. This can be partially attributed to the lack of reliable and up-to-date information about the characteristics of available datasets. While RDF datasets vary heavily with respect to the features related to quality, provenance, interlinking, licenses, statistics and dynamics, reliable information about such features is essential to enable dataset discovery and selection in tasks such as entity linking, distributed query, search or question answering. Even though there exists a wealth of works contributing to the task of dataset profiling in general, these works are spread across a wide range of communities. In this survey, we provide a first comprehensive overview of the RDF dataset profiling features, methods, tools and vocabularies. We organize these building blocks of dataset profiling in a taxonomy and illustrate the links between the dataset profiling and feature extraction approaches and several application domains. This survey is aimed towards data practitioners, data providers and scientists, spanning a large range of communities and drawing from different fields such as dataset profiling, assessment, summarization and characterization. Ultimately, this work is intended to facilitate the reader to identify the relevant features for building a dataset profile for intended applications together with the methods and tools capable of extracting these features from the datasets as well as vocabularies to describe the extracted features and make them available.

  

  
• Joao, R.S.: Time-Aware Entity Linking. (2017).

  Abstract BibTeX EndNote URL

  Entity Linking is the task of automatically identifying entity mentions in a piece of text and linking them to their corresponding entries in a reference knowledge base like Wikipedia. Although there is a plethora of works on entity linking, existing state-of-the-art approaches do not explicitly consider the time aspect and specifically the temporality of an entity’s prior probability (popularity) and embedding (semantic network). Consequently, they show limited performance in annotating old documents like news or web archives, while the problem is bigger in cases of short texts with limited context, such as archives of social media posts and query logs. This thesis focuses on this problem and proposes a modeling that leverages time-aware prior probabilities and word embeddings in the entity linking task

  

  
• Holzmann, H.: Software as a first-class citizen in web archives. (2017).

  BibTeX EndNote URL

  

  
• Holzmann, H., Goel, V., Gustainis, E.N.: Universal Distant Reading through Metadata Proxies with ArchiveSpark. Proceedings of the 2017 IEEE International Conference on Big Data (2017).

  BibTeX EndNote URL

  

  
• Singh, J., Anand, A.: Designing Search Tasks for Archive Search. Proceedings of the 2017 Conference on Conference Human Information Interaction and Retrieval. p. 361--364. ACM, Oslo, Norway (2017).

  Abstract BibTeX EndNote URL

  Longitudinal corpora like legal, corporate and newspaper archives are of immense value to a variety of users, and time as an important factor strongly influences their search behavior in these archives. While many systems have been developed to support users' temporal information needs, questions remain over how users utilize these advances to satisfy their needs. Analyzing their search behavior will provide us with novel insights into search strategy, guide better interface and system design and highlight new problems for further research. In this paper we propose a set of search tasks, with varying complexity, that IIR researchers can utilize to study user search behavior in archives. We discuss how we created and refined these tasks as the result of a pilot study using a temporal search engine. We not only propose task descriptions but also pre and post-task evaluation mechanisms that can be employed for a large-scale study (crowdsourcing). Our initial findings show the viability of such tasks for investigating search behavior in archives.

  

  
• Ernst, P., Mishra, A., Anand, A., Setty, V.: BioNex: A System For Biomedical News Event Exploration. Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval. p. 1277--1280. ACM, Shinjuku, Tokyo, Japan (2017).

  Abstract BibTeX EndNote URL

  We demonstrate BioNex, a system to mine, rank and visualize biomedical news events. BioNex takes biomedical queries such as "Ebola virus disease" and retrieves the k most relevant news events for them. To achieve this we first mine the generic news events by clustering them on a daily basis using general named entities and textual features. These clusters are also tagged with disambiguated biomedical entities which aid in biomedical news event exploration. The clusters are then used to compute the importance scores for the event clusters based on a combination of textual, semantic, popularity and historical importance features. BioNex also visualizes the retrieved event clusters to highlight the top news events and corresponding news articles for the given query. The visualization also provides the context for news events using (1) a chain of historically relevant news event clusters, and (2) other non-biomedical events from the same day.

  

  
• Fernando, Z.T., Marenzi, I., Nejdl, W.: ArchiveWeb: collaboratively extending and exploring web archive collections---How would you like to work with your collections? International Journal on Digital Libraries. 1--17 (2017).

  Abstract BibTeX EndNote Download URL

  Curated web archive collections contain focused digital content which is collected by archiving organizations, groups, and individuals to provide a representative sample covering specific topics and events to preserve them for future exploration and analysis. In this paper, we discuss how to best support collaborative construction and exploration of these collections through the ArchiveWeb system. ArchiveWeb has been developed using an iterative evaluation-driven design-based research approach, with considerable user feedback at all stages. The first part of this paper describes the important insights we gained from our initial requirements engineering phase during the first year of the project and the main functionalities of the current ArchiveWeb system for searching, constructing, exploring, and discussing web archive collections. The second part summarizes the feedback we received on this version from archiving organizations and libraries, as well as our corresponding plans for improving and extending the system for the next release.

  

  
• Setty, V., Anand, A., Mishra, A., Anand, A.: Modeling Event Importance for Ranking Daily News Events. Proceedings of the Tenth ACM International Conference on Web Search and Data Mining. p. 231--240. ACM, Cambridge, United Kingdom (2017).

  Abstract BibTeX EndNote URL

  We deal with the problem of ranking news events on a daily basis for large news corpora, an essential building block for news aggregation. News ranking has been addressed in the literature before but with individual news articles as the unit of ranking. However, estimating event importance accurately requires models to quantify current day event importance as well as its significance in the historical context. Consequently, in this paper we show that a cluster of news articles representing an event is a better unit of ranking as it provides an improved estimation of popularity, source diversity and authority cues. In addition, events facilitate quantifying their historical significance by linking them with long-running topics and recent chain of events. Our main contribution in this paper is to provide effective models for improved news event ranking. To this end, we propose novel event mining and feature generation approaches for improving estimates of event importance. Finally, we conduct extensive evaluation of our approaches on two large real-world news corpora each of which span for more than a year with a large volume of up to tens of thousands of daily news articles. Our evaluations are large-scale and based on a clean human curated ground-truth from Wikipedia Current Events Portal. Experimental comparison with a state-of-the-art news ranking technique based on language models demonstrates the effectiveness of our approach.

  

  
• Gottschalk, S., Demidova, E.: MultiWiki: Interlingual Text Passage Alignment in Wikipedia. ACM Trans. Web. 11, 6:1--6:30 (2017).

  Abstract BibTeX EndNote URL

  In this article, we address the problem of text passage alignment across interlingual article pairs in Wikipedia. We develop methods that enable the identification and interlinking of text passages written in different languages and containing overlapping information. Interlingual text passage alignment can enable Wikipedia editors and readers to better understand language-specific context of entities, provide valuable insights in cultural differences, and build a basis for qualitative analysis of the articles. An important challenge in this context is the tradeoff between the granularity of the extracted text passages and the precision of the alignment. Whereas short text passages can result in more precise alignment, longer text passages can facilitate a better overview of the differences in an article pair. To better understand these aspects from the user perspective, we conduct a user study at the example of the German, Russian, and English Wikipedia and collect a user-annotated benchmark. Then we propose MultiWiki, a method that adopts an integrated approach to the text passage alignment using semantic similarity measures and greedy algorithms and achieves precise results with respect to the user-defined alignment. The MultiWiki demonstration is publicly available and currently supports four language pairs.

  

  
• Lee, R.K.-W., Hoang, T.-A., Lim, E.-P.: On Analyzing User Topic-Specific Platform Preferences Across Multiple Social Media Sites. Proceedings of the 26th International Conference on World Wide Web. p. 1351--1359. International World Wide Web Conferences Steering Committee, Perth, Australia (2017).

  Abstract BibTeX EndNote URL

  Topic modeling has traditionally been studied for single text collections and applied to social media data represented in the form of text documents. With the emergence of many social media platforms, users find themselves using different social media for posting content and for social interaction. While many topics may be shared across social media platforms, users typically show preferences of certain social media platform(s) over others for certain topics. Such platform preferences may even be found at the individual level. To model social media topics as well as platform preferences of users, we propose a new topic model known as MultiPlatform-LDA (MultiLDA). Instead of just merging all posts from different social media platforms into a single text collection, MultiLDA keeps one text collection for each social media platform but allowing these platforms to share a common set of topics. MultiLDA further learns the user-specific platform preferences for each topic. We evaluate MultiLDA against TwitterLDA, the state-of-the-art method for social media content modeling, on two aspects: (i) the effectiveness in modeling topics across social media platforms, and (ii) the ability to predict platform choices for each post. We conduct experiments on three real-world datasets from Twitter, Instagram and Tumblr sharing a set of common users. Our experiments results show that the MultiLDA outperforms in both topic modeling and platform choice prediction tasks. We also show empirically that among the three social media platforms, "Daily matters" and "Relationship matters" are dominant topics in Twitter, "Social gathering", Öuting" and "Fashion" are dominant topics in Instagram, and "Music", "Entertainment" and "Fashion" are dominant topics in Tumblr.

  

  
• Sperber, W., Dalitz, W., Holzmann, H.: Software citation, landing pages, and the swMATH service. (2017).

  BibTeX EndNote URL

  

  
• Fetahu, B., Markert, K., Anand, A.: Fine Grained Citation Span for References in Wikipedia. Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing (2017).

  BibTeX EndNote URL

  

  
• Fafalios, P., Iosifidis, V., Stefanidis, K., Ntoutsi, E.: Multi-aspect Entity-Centric Analysis of Big Social Media Archives. In: Kamps, J., Tsakonas, G., Manolopoulos, Y., Iliadis, L., and Karydis, I. (eds.) International Conference on Theory and Practice of Digital Libraries. pp. 261-273. Springer (2017).

  Abstract BibTeX EndNote URL

  Social media archives serve as important historical information sources, and thus meaningful analysis and exploration methods are of immense value for historians, sociologists and other interested parties. In this paper, we propose an entity-centric approach to analyze social media archives and we define measures that allow studying how entities are reflected in social media in different time periods and under different aspects (like popularity, attitude, controversiality, and connectedness with other entities). A case study using a large Twitter archive of 4 years illustrates the insights that can be gained by such an entity-centric multi-aspect analysis.

  

  
• Gossen, G., Demidova, E., Risse, T.: Extracting Event-Centric Document Collections from Large-Scale Web Archives. Proceedings of the 21st International Conference on Theory and Practice of Digital Libraries. pp. 116-127. Springer (2017).

  Abstract BibTeX EndNote URL

  Web archives are typically very broad in scope and extremely large in scale. This makes data analysis appear daunting, especially for non-computer scientists. These collections constitute an increasingly important source for researchers in the social sciences, the historical sciences and journalists interested in studying past events. However, there are currently no access methods that help users to efficiently access information, in particular about specific events, beyond the retrieval of individual disconnected documents. Therefore we propose a novel method to extract event-centric document collections from large scale Web archives. This method relies on a specialized focused extraction algorithm. Our experiments on the German Web archive (covering a time period of 19 years) demonstrate that our method enables the extraction of event-centric collections for different event types.

  

  
• Elsahar, H., Demidova, E., Gottschalk, S., Gravier, C., Laforest, F.: Unsupervised Open Relation Extraction. Proceedings of the ESWC 2017 Satellite Events. Springer (2017).

  Abstract BibTeX EndNote URL

  We explore methods to extract relations between named entities from free text in an unsupervised setting. In addition to standard feature extraction, we develop a novel method to re-weight word embeddings. We alleviate the problem of features sparsity using an individual feature reduction. Our approach exhibits a significant improvement by 5.8% over the state-of-the-art relation clustering scoring a F1-score of 0.416 on the NYT-FB dataset.

  

  
• Gottschalk, S., Demidova, E., Bernacchi, V., Rogers, R.: Ongoing Events in Wikipedia: A Cross-lingual Case Study. Proceedings of the 2017 ACM Conference on Web Science (WebSci 2017). pp. 387-388. ACM (2017).

  Abstract BibTeX EndNote URL

  In order to effectively analyze information regarding ongoing events that impact local communities across language and country borders, researchers often need to perform multilingual data analysis. This analysis can be particularly challenging due to the rapidly evolving event-centric data and the language barrier. In this abstract we present preliminary results of a case study with the goa to better understand how researchers interact with multilingual event-centric information in the context of cross-cultural studies and which methods and features they use.

  

  
• Fetahu, B., Markert, K., Anand, A.: Fine Grained Citation Span for References in Wikipedia. CoRR. abs/1707.07278, (2017).

  BibTeX EndNote URL

  

  
• Hoang, T.-A., Lim, E.-P.: Modeling Topics and Behavior of Microbloggers: An Integrated Approach. ACM TIST. 8, 44:1-44:37 (2017).

  BibTeX EndNote URL

  

  
• Hoang, T.-A., Lim, E.-P.: Highly Efficient Mining of Overlapping Clusters in Signed Weighted Networks. CIKM (2017).

  BibTeX EndNote URL

  

  
• Holzmann, H., Anand, A.: Tempas: Temporal Archive Search Based on Tags. CoRR. abs/1702.01076, (2017).

  BibTeX EndNote URL

  

  
• Fafalios, P., Holzmann, H., Kasturia, V., Nejdl, W.: Building and Querying Semantic Layers for Web Archives. ACM/IEEE Joint Conference on Digital Libraries. pp. 11-20. IEEE Computer Society (2017).

  Abstract BibTeX EndNote URL

  Web archiving is the process of collecting portions of the Web to ensure that the information is preserved for future exploitation. However, despite the increasing number of web archives worldwide, the absence of efficient and meaningful exploration methods still remains a major hurdle in the way of turning them into a usable and useful information source. In this paper, we focus on this problem and propose an RDF/S model and a distributed framework for building semantic profiles ("layers") that describe semantic information about the contents of web archives. A semantic layer allows describing metadata information about the archived documents, annotating them with useful semantic information (like entities, concepts and events), and publishing all this data on the Web as Linked Data. Such structured repositories offer advanced query and integration capabilities and make web archives directly exploitable by other systems and tools. To demonstrate their query capabilities, we build and query semantic layers for three different types of web archives. An experimental evaluation showed that a semantic layer can answer information needs that existing keyword-based systems are not able to sufficiently satisfy.

  

  
• Holzmann, H., and Thomas Risse,: Accessing web archives from different perspectives with potential synergies. 2nd International Conference on Web Archives / Web Archiving Week (RESAW/IIPC) 2017 (2017).

  BibTeX EndNote URL

  

  
• Fernando, Z.T., Marenzi, I., Nejdl, W.: ArchiveWeb: collaboratively extending and exploring web archive collections - How would you like to work with your collections? International Journal on Digital Libraries. (2017).

  Abstract BibTeX EndNote URL

  Curated web archive collections contain focused digital content which is collected by archiving organizations, groups, and individuals to provide a representative sample covering specific topics and events to preserve them for future exploration and analysis. In this paper, we discuss how to best support collaborative construction and exploration of these collections through the ArchiveWeb system. ArchiveWeb has been developed using an iterative evaluation-driven design-based research approach, with considerable user feedback at all stages. The first part of this paper describes the important insights we gained from our initial requirements engineering phase during the first year of the project and the main functionalities of the current ArchiveWeb system for searching, constructing, exploring, and discussing web archive collections. The second part summarizes the feedback we received on this version from archiving organizations and libraries, as well as our corresponding plans for improving and extending the system for the next release.

  

  
• Fafalios, P., Kasturia, V., Nejdl, W.: Towards a Ranking Model for Semantic Layers over Digital Archives. ACM/IEEE-CS Joint Conference on Digital Libraries. pp. 336-337. IEEE Computer Society (2017).

  Abstract BibTeX EndNote URL

  Archived collections of documents (like newspaper archives) serve as important information sources for historians, journalists, sociologists and other interested parties. Semantic Layers over such digital archives allow describing and publishing metadata and semantic information about the archived documents in a standard format (RDF), which in turn can be queried through a structured query language (e.g., SPARQL). This enables to run advanced queries by combining metadata of the documents (like publication date) and content-based semantic information (like entities mentioned in the documents). However, the results returned by structured queries can be numerous and also they all equally match the query. Thus, there is the need to rank these results in order to promote the most important ones. In this paper, we focus on this problem and propose a ranking model that considers and combines: i) the relativeness of documents to entities, ii) the timeliness of documents, and iii) the relations among the entities.

  

  

2016

• Gossen, G., Demidova, E., Risse, T.: Analyzing Web Archives Through Topic and Event Focused Sub-collections. Proceedings of the 8th ACM Conference on Web Science. p. 291--295. ACM, Hannover, Germany (2016).

  Abstract BibTeX EndNote Download URL

  Web archives capture the history of the Web and are therefore an important source to study how societal developments have been reflected on the Web. However, the large size of Web archives and their temporal nature pose many challenges to researchers interested in working with these collections. In this work, we describe the challenges of working with Web archives and propose the research methodology of extracting and studying sub-collections of the archive focused on specific topics and events. We discuss the opportunities and challenges of this approach and suggest a framework for creating sub-collections.

  

  
• Rokicki, M., Zerr, S., Siersdorfer, S.: Just in Time: Controlling Temporal Performance in Crowdsourcing Competitions. Proceedings of the 25th International Conference on World Wide Web. p. 817--827. International World Wide Web Conferences Steering Committee, Montr&\#233;al, Qu&\#233;bec, Canada (2016).

  BibTeX EndNote Download URL

  

  
• Kanhabua, N., Kemkes, P., Nejdl, W., Nguyen, T.N., Reis, F., Tran, N.K.: How to Search the Internet Archive Without Indexing It. In: Fuhr, N., Kovács, L., Risse, T., and Nejdl, W. (eds.) Research and Advanced Technology for Digital Libraries - 20th International Conference on Theory and Practice of Digital Libraries, TPDL 2016, Hannover, Germany, September 5-9, 2016, Proceedings. p. 147--160. Springer (2016).

  BibTeX EndNote Download URL

  

  
• Holzmann, H., Nejdl, W., Anand, A.: On the Applicability of Delicious for Temporal Search on Web Archives. Proceedings of the 39th International ACM SIGIR Conference on Research and Development in Information Retrieval. p. 929--932. ACM, Pisa, Italy (2016).

  Abstract BibTeX EndNote Download URL

  Web archives are large longitudinal collections that store webpages from the past, which might be missing on the current live Web. Consequently, temporal search over such collections is essential for finding prominent missing webpages and tasks like historical analysis. However, this has been challenging due to the lack of popularity information and proper ground truth to evaluate temporal retrieval models. In this paper we investigate the applicability of external longitudinal resources to identify important and popular websites in the past and analyze the social bookmarking service Delicious for this purpose. The timestamped bookmarks on Delicious provide explicit cues about popular time periods in the past along with relevant descriptors. These are valuable to identify important documents in the past for a given temporal query. Focusing purely on recall, we analyzed more than 12,000 queries and find that using Delicious yields average recall values from 46% up to 100%, when limiting ourselves to the best represented queries in the considered dataset. This constitutes an attractive and low-overhead approach for quick access into Web archives by not dealing with the actual contents.

  

  
• Holzmann, H., Runnwerth, M., Sperber, W.: Linking Mathematical Software in Web Archives. In: Greuel, G.-M., Koch, T., Paule, P., and Sommese, A. (eds.) Mathematical Software -- ICMS 2016: 5th International Conference, Berlin, Germany, July 11-14, 2016, Proceedings. p. 419--422. Springer International Publishing, Cham (2016).

  BibTeX EndNote Download URL

  

  
• Holzmann, H., Anand, A.: Tempas: Temporal Archive Search Based on Tags. Proceedings of the 25th International Conference Companion on World Wide Web. p. 207--210. International World Wide Web Conferences Steering Committee, Montr&\#233;al, Qu&\#233;bec, Canada (2016).

  BibTeX EndNote Download URL

  

  
• Zhou, Y., Demidova, E., Cristea, A.I.: Who Likes Me More?: Analysing Entity-centric Language-specific Bias in Multilingual Wikipedia. Proceedings of the 31st Annual ACM Symposium on Applied Computing. p. 750--757. ACM, Pisa, Italy (2016).

  Abstract BibTeX EndNote Download URL

  In this paper we take an important step towards better understanding the existence and extent of entity-centric language-specific bias in multilingual Wikipedia, and any deviation from its targeted neutral point of view. We propose a methodology using sentiment analysis techniques to systematically extract the variations in sentiments associated with real-world entities in different language editions of Wikipedia, illustrated with a case study of five Wikipedia language editions and a set of target entities from four categories.

  

  
• Vo, K.D., Tran, T., Nguyen, T.N., Zhu, X., Nejdl, W.: Can We Find Documents in Web Archives Without Knowing Their Contents? Proceedings of the 8th ACM Conference on Web Science. p. 173--182. ACM, Hannover, Germany (2016).

  Abstract BibTeX EndNote Download URL

  Recent advances of preservation technologies have led to an increasing number of Web archive systems and collections. These collections are valuable to explore the past of the Web, but their value can only be uncovered with effective access and exploration mechanisms. Ideal search and ranking methods must be robust to the high redundancy and the temporal noise of contents, as well as scalable to the huge amount of data archived. Despite several attempts in Web archive search, facilitating access to Web archive still remains a challenging problem. In this work, we conduct a first analysis on different ranking strategies that exploit evidences from metadata instead of the full content of documents. We perform a first study to compare the usefulness of non-content evidences to Web archive search, where the evidences are mined from the metadata of file headers, links and URL strings only. Based on these findings, we propose a simple yet surprisingly effective learning model that combines multiple evidences to distinguish "good" from "bad" search results. We conduct empirical experiments quantitatively as well as qualitatively to confirm the validity of our proposed method, as a first step towards better ranking in Web archives taking metadata into account.

  

  
• Singh, J., Nejdl, W., Anand, A.: Expedition: A Time-Aware Exploratory Search System Designed for Scholars. Proceedings of the 39th International ACM SIGIR Conference on Research and Development in Information Retrieval. p. 1105--1108. ACM, Pisa, Italy (2016).

  Abstract BibTeX EndNote Download URL

  Archives are an important source of study for various scholars. Digitization and the web have made archives more accessible and led to the development of several time-aware exploratory search systems. However these systems have been designed for more general users rather than scholars. Scholars have more complex information needs in comparison to general users. They also require support for corpus creation during their exploration process. In this paper we present Expedition - a time-aware exploratory search system that addresses the requirements and information needs of scholars. Expedition possesses a suite of ad-hoc and diversity based retrieval models to address complex information needs; a newspaper-style user interface to allow for larger textual previews and comparisons; entity filters to more naturally refine a result list and an interactive annotated timeline which can be used to better identify periods of importance.

  

  
• Shaltev, M., Zab, J.-H., Kemkes, P., Siersdorfer, S., Zerr, S.: Cobwebs from the Past and Present: Extracting Large Social Networks Using Internet Archive Data. Proceedings of the 39th International ACM SIGIR Conference on Research and Development in Information Retrieval. p. 1093--1096. ACM, Pisa, Italy (2016).

  Abstract BibTeX EndNote Download URL

  Social graph construction from various sources has been of interest to researchers due to its application potential and the broad range of technical challenges involved. The World Wide Web provides a huge amount of continuously updated data and information on a wide range of topics created by a variety of content providers, and makes the study of extracted people networks and their temporal evolution valuable for social as well as computer scientists. In this paper we present SocGraph - an extraction and exploration system for social relations from the content of around 2 billion web pages collected by the Internet Archive over the 17 years time period between 1996 and 2013. We describe methods for constructing large social graphs from extracted relations and introduce an interface to study their temporal evolution.

  

  
• Singh, J., Nejdl, W., Anand, A.: History by Diversity: Helping Historians Search News Archives. Proceedings of the 2016 ACM on Conference on Human Information Interaction and Retrieval. p. 183--192. ACM, Carrboro, North Carolina, USA (2016).

  Abstract BibTeX EndNote Download URL

  Longitudinal corpora like newspaper archives are of immense value to historical research, and time as an important factor for historians strongly influences their search behaviour in these archives. While searching for articles published over time, a key preference is to retrieve documents which cover the important aspects from important points in time which is different from standard search behavior. To support this search strategy, we introduce the notion of a Historical Query Intent to explicitly model a historian's search task and define an aspect-time diversification problem over news archives. We present a novel algorithm, HistDiv, that explicitly models the aspects and important time windows based on a historian's information seeking behavior. By incorporating temporal priors based on publication times and temporal expressions, we diversify both on the aspect and temporal dimensions. We test our methods by constructing a test collection based on The New York Times Collection with a workload of 30 queries of historical intent assessed manually. We find that HistDiv outperforms all competitors in subtopic recall with a slight loss in precision. We also present results of a qualitative user study to determine wether this drop in precision is detrimental to user experience. Our results show that users still preferred HistDiv's ranking.

  

  
• Holzmann, H., Goel, V., Anand, A.: ArchiveSpark: Efficient Web Archive Access, Extraction and Derivation. Proceedings of the 16th ACM/IEEE-CS on Joint Conference on Digital Libraries. p. 83--92. ACM, Newark, New Jersey, USA (2016).

  Abstract BibTeX EndNote Download URL

  Web archives are a valuable resource for researchers of various disciplines. However, to use them as a scholarly source, researchers require a tool that provides efficient access to Web archive data for extraction and derivation of smaller datasets. Besides efficient access we identify five other objectives based on practical researcher needs such as ease of use, extensibility and reusability. Towards these objectives we propose ArchiveSpark, a framework for efficient, distributed Web archive processing that builds a research corpus by working on existing and standardized data formats commonly held by Web archiving institutions. Performance optimizations in ArchiveSpark, facilitated by the use of a widely available metadata index, result in significant speed-ups of data processing. Our benchmarks show that ArchiveSpark is faster than alternative approaches without depending on any additional data stores while improving usability by seamlessly integrating queries and derivations with external tools.

  

  
• Holzmann, H., Nejdl, W., Anand, A.: The Dawn of Today's Popular Domains: A Study of the Archived German Web over 18 Years. Proceedings of the 16th ACM/IEEE-CS on Joint Conference on Digital Libraries. p. 73--82. ACM, Newark, New Jersey, USA (2016).

  Abstract BibTeX EndNote Download URL

  The Web has been around and maturing for 25 years. The popular websites of today have undergone vast changes during this period, with a few being there almost since the beginning and many new ones becoming popular over the years. This makes it worthwhile to take a look at how these sites have evolved and what they might tell us about the future of the Web. We therefore embarked on a longitudinal study spanning almost the whole period of the Web, based on data collected by the Internet Archive starting in 1996, to retrospectively analyze how the popular Web as of now has evolved over the past 18 years. For our study we focused on the German Web, specifically on the top 100 most popular websites in 17 categories. This paper presents a selection of the most interesting findings in terms of volume, size as well as age of the Web. While related work in the field of Web Dynamics has mainly focused on change rates and analyzed datasets spanning less than a year, we looked at the evolution of websites over 18 years. We found that around 70% of the pages we investigated are younger than a year, with an observed exponential growth in age as well as in size up to now. If this growth rate continues, the number of pages from the popular domains will almost double in the next two years. In addition, we give insights into our data set, provided by the Internet Archive, which hosts the largest and most complete Web archive as of today.

  

  
• Fernando, Z.T., Marenzi, I., Nejdl, W., Kalyani, R.: ArchiveWeb: Collaboratively Extending and Exploring Web Archive Collections. In: Fuhr, N., Kovács, L., Risse, T., and Nejdl, W. (eds.) Research and Advanced Technology for Digital Libraries - 20th International Conference on Theory and Practice of Digital Libraries, TPDL 2016, Hannover, Germany, September 5-9, 2016, Proceedings. p. 107--118. Springer (2016).

  BibTeX EndNote Download URL

  

  
• Taibi, D., Rogers, R., Marenzi, I., Nejdl, W., Ahmad, Q.A.I., Fulantelli, G.: Search As Research Practices on the Web: The SaR-Web Platform for Cross-language Engine Results Analysis. Proceedings of the 8th ACM Conference on Web Science. p. 367--369. ACM, Hannover, Germany (2016).

  Abstract BibTeX EndNote Download URL

  Search engines are the most utilized tools to access information on the Web. The success of large companies such as Google owes to their capacity to conduct users through the vast troves of knowledge and information online. Recently, the concept of search as research has been used to shift the research focus from workings of information-seeking tools towards methods for the social study of Web and particularly the social meanings of engine results. In this paper, we present SaR-Web, a web search tool that provides an automatic means to carry out search as research on the Web. It compares the results of same (translated) queries across search engine language domains, thereby enabling cross-linguistic and cross-cultural comparisons of results. SaR-Web outputs enable the comparative study of cultural mores as well as societal associations and concerns, interpreted through search engine results.

  

  
• Fulantelli, G., Marenzi, I., Ahmad, Q.A.I., Taibi, D.: SaR-Web - A Tool to Support Search as Learning Processes. In: Gwizdka, J., Hansen, P., Hauff, C., He, J., and Kando, N. (eds.) Proceedings of the Second International Workshop on Search as Learning, SAL 2016, co-located with the 39th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2016, Pisa, Italy, July 21st, 2016. CEUR-WS.org (2016).

  BibTeX EndNote Download URL

  

  
• Müller, E., Otto, C., Ewerth, R.: Semi-supervised Identification of Rarely Appearing Persons in Video by Correcting Weak Labels. Proceedings of the 2016 ACM on International Conference on Multimedia Retrieval. p. 381--384. ACM, New York, New York, USA (2016).

  Abstract BibTeX EndNote Download URL

  Some recent approaches for character identification in movies and TV broadcasts are realized in a semi-supervised manner by assigning transcripts and/or subtitles to the speakers. However, the labels obtained in this way achieve only an accuracy of $80\% - 90\%$ and the number of training examples for the different actors is unevenly distributed. In this paper, we propose a novel approach for person identification in video by correcting and extending the training data with reliable predictions to reduce the number of annotation errors. Furthermore, the intra-class diversity of rarely speaking characters is enhanced. To address the imbalance of training data per person, we suggest two complementary prediction scores. These scores are also used to recognize whether or not a face track belongs to a (supporting) character whose identity does not appear in the transcript etc. Experimental results demonstrate the feasibility of the proposed approach, outperforming the current state of the art.

  

  
• Springstein, M., Ewerth, R.: On the Effects of Spam Filtering and Incremental Learning for Web-Supervised Visual Concept Classification. Proceedings of the 2016 ACM on International Conference on Multimedia Retrieval. p. 377--380. ACM, New York, New York, USA (2016).

  Abstract BibTeX EndNote Download URL

  Deep neural networks have been successfully applied to the task of visual concept classification. However, they require a large number of training examples for learning. Although pre-trained deep neural networks are available for some domains, they usually have to be fine-tuned for an envisaged target domain. Recently, some approaches have been suggested that are aimed at incrementally (or even endlessly) learning visual concepts based on Web data. Since tags of Web images are often noisy, normally some filtering mechanisms are employed in order to remove ``spam'' images that are not appropriate for training. In this paper, we investigate several aspects of a web-supervised system that has to be adapted to another target domain: 1.) the effect of incremental learning, 2.) the effect of spam filtering, and 3.) the behavior of particular concept classes with respect to 1.) and 2.). The experimental results provide some insights under which conditions incremental learning and spam filtering are useful.

  

  
• Gottschalk, S., Demidova, E.: Analysing Temporal Evolution of Interlingual Wikipedia Article Pairs. Proceedings of the 39th International ACM SIGIR Conference on Research and Development in Information Retrieval. p. 1089--1092. ACM, Pisa, Italy (2016).

  Abstract BibTeX EndNote Download URL

  Wikipedia articles representing an entity or a topic in different language editions evolve independently within the scope of the language-specific user communities. This can lead to different points of views reflected in the articles, as well as complementary and inconsistent information. An analysis of how the information is propagated across the Wikipedia language editions can provide important insights in the article evolution along the temporal and cultural dimensions and support quality control. To facilitate such analysis, we present MultiWiki -- a novel web-based user interface that provides an overview of the similarities and differences across the article pairs originating from different language editions on a timeline. MultiWiki enables users to observe the changes in the interlingual article similarity over time and to perform a detailed visual comparison of the article snapshots at a particular time point.

  

  
• Holzmann, H., Sperber, W., Runnwerth, M.: Archiving Software Surrogates on the Web for Future Reference. In: Fuhr, N., Kovács, L., Risse, T., and Nejdl, W. (eds.) Research and Advanced Technology for Digital Libraries: 20th International Conference on Theory and Practice of Digital Libraries, TPDL 2016, Hannover, Germany, September 5--9, 2016, Proceedings. p. 215--226. Springer International Publishing, Cham (2016).

  BibTeX EndNote URL

  

  
• Fetahu, B., Markert, K., Nejdl, W., Anand, A.: Finding News Citations for Wikipedia. Proceedings of the 25th ACM International on Conference on Information and Knowledge Management. p. 337--346. ACM, Indianapolis, Indiana, USA (2016).

  Abstract BibTeX EndNote Download URL

  An important editing policy in Wikipedia is to provide citations for added statements in Wikipedia pages, where statements can be arbitrary pieces of text, ranging from a sentence to a paragraph. In many cases citations are either outdated or missing altogether. In this work we address the problem of finding and updating news citations for statements in entity pages. We propose a two-stage supervised approach for this problem. In the first step, we construct a classifier to find out whether statements need a news citation or other kinds of citations (web, book, journal, etc.). In the second step, we develop a news citation algorithm for Wikipedia statements, which recommends appropriate citations from a given news collection. Apart from IR techniques that use the statement to query the news collection, we also formalize three properties of an appropriate citation, namely: (i) the citation should entail the Wikipedia statement, (ii) the statement should be central to the citation, and (iii) the citation should be from an authoritative source. We perform an extensive evaluation of both steps, using 20 million articles from a real-world news collection. Our results are quite promising, and show that we can perform this task with high precision and at scale.

  

  
• Singh, J., Hoffart, J., Anand, A.: Discovering Entities with Just a Little Help from You. Proceedings of the 25th ACM International on Conference on Information and Knowledge Management. p. 1331--1340. ACM, Indianapolis, Indiana, USA (2016).

  Abstract BibTeX EndNote URL

  Linking entities like people, organizations, books, music groups and their songs in text to knowledge bases (KBs) is a fundamental task for many downstream search and mining applications. Achieving high disambiguation accuracy crucially depends on a rich and holistic representation of the entities in the KB. For popular entities, such a representation can be easily mined from Wikipedia, and many current entity disambiguation and linking methods make use of this fact. However, Wikipedia does not contain long-tail entities that only few people are interested in, and also at times lags behind until newly emerging entities are added. For such entities, mining a suitable representation in a fully automated fashion is very difficult, resulting in poor linking accuracy. What can automatically be mined, though, is a high-quality representation given the context of a new entity occurring in any text. Due to the lack of knowledge about the entity, no method can retrieve these occurrences automatically with high precision, resulting in a chicken-egg problem. To address this, our approach automatically generates candidate occurrences of entities, prompting the user for feedback to decide if the occurrence refers to the actual entity in question. This feedback gradually improves the knowledge and allows our methods to provide better candidate suggestions to keep the user engaged. We propose novel human-in-the-loop retrieval methods for generating candidates based on gradient interleaving of diversification and textual relevance approaches. We conducted extensive experiments on the FACC dataset, showing that our approaches convincingly outperform carefully selected baselines in both intrinsic and extrinsic measures while keeping users engaged.

  

  

2015

• Gossen, G., Demidova, E., Risse, T.: The iCrawl Wizard – Supporting Interactive Focused Crawl Specification. Proceedings of the European Conference on Information Retrieval (ECIR) 2015 (2015).

  BibTeX EndNote Download URL

  

  
• Singh, J., Anand, A., Setty, V., Anand, A.: Exploring Long Running News Stories Using Wikipedia. Proceedings of the ACM Web Science Conference. p. 57:1--57:2. ACM, Oxford, United Kingdom (2015).

  Abstract BibTeX EndNote Download URL

  A significant portion of today's news articles are part of long running stories. To better understand the context of these stories journalists, social scientists and other scholars use news collections to find temporal and topical insights. However these insights are devoid of user impressions, derived from click-through data and query logs, and are only reliable if the collection is complete and consistent. In this work we introduce the notion of combining user impressions from Wikipedia with news collection based insights for long running news story exploration and outline promising new research directions. We also demonstrate our initial attempts with a prototype system called NewsEX.

  

  
• Tahmasebi, N., Borin, L., Capannini, G., Dubhashi, D., Exner, P., Forsberg, M., Gossen, G., Johansson, F.D., Johansson, R., Kågebäck, M., Mogren, O., Nugues, P., Risse, T.: Visions and open challenges for a knowledge-based culturomics. International Journal on Digital Libraries. 15, 169-187 (2015).

  BibTeX EndNote Download URL

  

  
• Nejdl, W., Risse, T.: Herausforderungen für die nationale, regionale und thematische Webarchivierung und deren Nutzung. Zeitschrift für Bibliothekswesen und Bibliographie. 62, 160--171 (2015).

  BibTeX EndNote Download URL

  

  
• Holzmann, H., Tahmasebi, N., Risse, T.: Named Entity Evolution Recognition on the Blogosphere. International Journal on Digital Libraries. 15, 209-235 (2015).

  BibTeX EndNote Download URL

  

  
• Nguyen, T.N., Kanhabua, N., Niederée, C., Zhu, X.: A Time-aware Random Walk Model for Finding Important Documents in Web Archives. Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval. p. 915--918. ACM, Santiago, Chile (2015).

  Abstract BibTeX EndNote Download URL

  Due to their first-hand, diverse and evolution-aware reflection of nearly all areas of life, web archives are emerging as gold-mines for content analytics of many sorts. However, supporting search, which goes beyond navigational search via URLs, is a very challenging task in these unique structures with huge, redundant and noisy temporal content. In this paper, we address the search needs of expert users such as journalists, economists or historians for discovering a topic in time: Given a query, the top-k returned results should give the best representative documents that cover most interesting time-periods for the topic. For this purpose, we propose a novel random walk-based model that integrates relevance, temporal authority, diversity and time in a unified framework. Our preliminary experimental results on the large-scale real-world web archival collection shows that our method significantly improves the state-of-the-art algorithms (i.e., PageRank) in ranking temporal web pages.

  

  
• Nguyen, T.N., Kanhabua, N., Nejdl, W., Niederée, C.: Mining Relevant Time for Query Subtopics in Web Archives. Proceedings of the 24th International Conference on World Wide Web. p. 1357--1362. ACM, Florence, Italy (2015).

  Abstract BibTeX EndNote Download URL

  With the reflection of nearly all types of social cultural, societal and everyday processes of our lives in the web, web archives from organizations such as the Internet Archive have the potential of becoming huge gold-mines for temporal content analytics of many kinds (e.g., on politics, social issues, economics or media). First hand evidences for such processes are of great benefit for expert users such as journalists, economists, historians, etc. However, searching in this unique longitudinal collection of huge redundancy (pages of near-identical content are crawled all over again) is completely different from searching over the web. In this work, we present our first study of mining the temporal dynamics of subtopics by leveraging the value of anchor text along the time dimension of the enormous web archives. This task is especially useful for one important ranking problem in the web archive context, the time-aware search result diversification. Due to the time uncertainty (the lagging nature and unpredicted behavior of the crawlers), identifying the trending periods for such temporal subtopics relying solely on the timestamp annotations of the web archive (i.e., crawling times) is extremely difficult. We introduce a brute-force approach to detect a time-reliable sub-collection and propose a method to leverage them for relevant time mining of subtopics. This is empirically found effective in solving the problem.

  

  
• Fetahu, B., Gadiraju, U., Dietze, S.: Improving Entity Retrieval on Structured Data. International Semantic Web Conference (1). p. 474--491. Springer (2015).

  BibTeX EndNote Download URL

  

  
• Souza, T., Demidova, E., Risse, T., Holzmann, H., Gossen, G., Szymanski, J.: Semantic URL Analytics to Support Efficient Annotation of Large Scale Web Archives. International KEYSTONE Conference. p. 153--166. Springer (2015).

  BibTeX EndNote Download URL

  

  
• Tran, T.A., Niederee, C., Kanhabua, N., Gadiraju, U., Anand, A.: Balancing Novelty and Salience: Adaptive Learning to Rank Entities for Timeline Summarization of High-impact Events. Proceedings of the 24th ACM International on Conference on Information and Knowledge Management. p. 1201--1210. ACM, Melbourne, Australia (2015).

  Abstract BibTeX EndNote Download URL

  Long-running, high-impact events such as the Boston Marathon bombing often develop through many stages and involve a large number of entities in their unfolding. Timeline summarization of an event by key sentences eases story digestion, but does not distinguish between what a user remembers and what she might want to re-check. In this work, we present a novel approach for timeline summarization of high-impact events, which uses entities instead of sentences for summarizing the event at each individual point in time. Such entity summaries can serve as both (1) important memory cues in a retrospective event consideration and (2) pointers for personalized event exploration. In order to automatically create such summaries, it is crucial to identify the "right" entities for inclusion. We propose to learn a ranking function for entities, with a dynamically adapted trade-off between the in-document salience of entities and the informativeness of entities across documents, i.e., the level of new information associated with an entity for a time point under consideration. Furthermore, for capturing collective attention for an entity we use an innovative soft labeling approach based on Wikipedia. Our experiments on a real large news datasets confirm the effectiveness of the proposed methods.

  

  
• Rokicki, M., Zerr, S., Siersdorfer, S.: Groupsourcing: Team Competition Designs for Crowdsourcing. Proceedings of the 24th International Conference on World Wide Web. p. 906--915. ACM, Florence, Italy (2015).

  Abstract BibTeX EndNote Download URL

  Many data processing tasks such as semantic annotation of images, translation of texts in foreign languages, and labeling of training data for machine learning models require human input, and, on a large scale, can only be accurately solved using crowd based online work. Recent work shows that frameworks where crowd workers compete against each other can drastically reduce crowdsourcing costs, and outperform conventional reward schemes where the payment of online workers is proportional to the number of accomplished tasks ("pay-per-task"). In this paper, we investigate how team mechanisms can be leveraged to further improve the cost efficiency of crowdsourcing competitions. To this end, we introduce strategies for team based crowdsourcing, ranging from team formation processes where workers are randomly assigned to competing teams, over strategies involving self-organization where workers actively participate in team building, to combinations of team and individual competitions. Our large-scale experimental evaluation with more than 1,100 participants and overall 5,400 hours of work spent by crowd workers demonstrates that our team based crowdsourcing mechanisms are well accepted by online workers and lead to substantial performance boosts.

  

  
• Siersdorfer, S., Kemkes, P., Ackermann, H., Zerr, S.: Who With Whom And How?: Extracting Large Social Networks Using Search Engines. Proceedings of the 24th ACM International on Conference on Information and Knowledge Management. p. 1491--1500. ACM, Melbourne, Australia (2015).

  BibTeX EndNote Download URL

  

  
• Fetahu, B., Markert, K., Anand, A.: Automated News Suggestions for Populating Wikipedia Entity Pages. Proceedings of the 24th ACM International on Conference on Information and Knowledge Management. p. 323--332. ACM, Melbourne, Australia (2015).

  Abstract BibTeX EndNote Download URL

  Wikipedia entity pages are a valuable source of information for direct consumption and for knowledge-base construction, update and maintenance. Facts in these entity pages are typically supported by references. Recent studies show that as much as 20% of the references are from online news sources. However, many entity pages are incomplete even if relevant information is already available in existing news articles. Even for the already present references, there is often a delay between the news article publication time and the reference time. In this work, we therefore look at Wikipedia through the lens of news and propose a novel news-article suggestion task to improve news coverage in Wikipedia, and reduce the lag of newsworthy references. Our work finds direct application, as a precursor, to Wikipedia page generation and knowledge-base acceleration tasks that rely on relevant and high quality input sources. We propose a two-stage supervised approach for suggesting news articles to entity pages for a given state of Wikipedia. First, we suggest news articles to Wikipedia entities (article-entity placement) relying on a rich set of features which take into account the salience and relative authority of entities, and the novelty of news articles to entity pages. Second, we determine the exact section in the entity page for the input article (article-section placement) guided by class-based section templates. We perform an extensive evaluation of our approach based on ground-truth data that is extracted from external references in Wikipedia. We achieve a high precision value of up to 93% in the article-entity suggestion stage and upto 84% for the article-section placement. Finally, we compare our approach against competitive baselines and show significant improvements.

  

  
• Kanhabua, N., Ngoc Nguyen, T., Nejdl, W.: Learning to Detect Event-Related Queries for Web Search. Proceedings of the 24th International Conference on World Wide Web. p. 1339--1344. ACM, Florence, Italy (2015).

  Abstract BibTeX EndNote Download URL

  In many cases, a user turns to search engines to find information about real-world situations, namely, political elections, sport competitions, or natural disasters. Such temporal querying behavior can be observed through a significant number of event-related queries generated in web search. In this paper, we study the task of detecting event-related queries, which is the first step for understanding temporal query intent and enabling different temporal search applications, e.g., time-aware query auto-completion, temporal ranking, and result diversification. We propose a two-step approach to detecting events from query logs. We first identify a set of event candidates by considering both implicit and explicit temporal information needs. The next step further classifies the candidates into two main categories, namely, event or non-event. In more detail, we leverage different machine learning techniques for query classification, which are trained using the feature set composed of time series features from signal processing, along with features derived from click-through information, and standard statistical features. In order to evaluate our proposed approach, we conduct an experiment using two real-world query logs with manually annotated relevance assessments for 837 events. To this end, we provide a large set of event-related queries made available for fostering research on this challenging task.

  

  
• Tran, A.T., Tran, N.K., Hadgu, A.T., Jäschke, R.: Semantic Annotation for Microblog Topics Using Wikipedia Temporal Information. In: Màrquez, L., Callison-Burch, C., Su, J., Pighin, D., and Marton, Y. (eds.) Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, EMNLP 2015, Lisbon, Portugal, September 17-21, 2015. p. 97--106. The Association for Computational Linguistics (2015).

  BibTeX EndNote Download URL

  

  
• Fetahu, B., Anand, A., Anand, A.: How Much is Wikipedia Lagging Behind News? Proceedings of the ACM Web Science Conference. p. 28:1--28:9. ACM, Oxford, United Kingdom (2015).

  Abstract BibTeX EndNote Download URL

  Wikipedia, rich in entities and events, is an invaluable resource for various knowledge harvesting, extraction and mining tasks. Numerous resources like DBpedia, YAGO and other knowledge bases are based on extracting entity and event based knowledge from it. Online news, on the other hand, is an authoritative and rich source for emerging entities, events and facts relating to existing entities. In this work, we study the creation of entities in Wikipedia with respect to news by studying how entity and event based information flows from news to Wikipedia. We analyze the lag of Wikipedia (based on the revision history of the English Wikipedia) with 20 years of The New York Times dataset (NYT). We model and analyze the lag of entities and events, namely their first appearance in Wikipedia and in NYT, respectively. In our extensive experimental analysis, we find that almost 20% of the external references in entity pages are news articles encoding the importance of news to Wikipedia. Second, we observe that the entity-based lag follows a normal distribution with a high standard deviation, whereas the lag for news-based events is typically very low. Finally, we find that events are responsible for creation of emergent entities with as many as 12% of the entities mentioned in the event page are created after the creation of the event page.

  

  
• Gossen, G., Demidova, E., Risse, T.: iCrawl: Improving the Freshness of Web Collections by Integrating Social Web and Focused Web Crawling. Proceedings of the 15th ACM/IEEE-CS Joint Conference on Digital Libraries. p. 75--84. ACM, Knoxville, Tennessee, USA (2015).

  Abstract BibTeX EndNote Download URL

  Researchers in the Digital Humanities and journalists need to monitor, collect and analyze fresh online content regarding current events such as the Ebola outbreak or the Ukraine crisis on demand. However, existing focused crawling approaches only consider topical aspects while ignoring temporal aspects and therefore cannot achieve thematically coherent and fresh Web collections. Especially Social Media provide a rich source of fresh content, which is not used by state-of-the-art focused crawlers. In this paper we address the issues of enabling the collection of fresh and relevant Web and Social Web content for a topic of interest through seamless integration of Web and Social Media in a novel integrated focused crawler. The crawler collects Web and Social Media content in a single system and exploits the stream of fresh Social Media content for guiding the crawler.

  

  

2014

• Kanhabua, N., Nguyen, T.N., Niederee, C.: What triggers human remembering of events? A large-scale analysis of catalysts for collective memory in Wikipedia. IEEE/ACM Joint Conference on Digital Libraries. Institute of Electrical and Electronics Engineers (IEEE) (2014).

  BibTeX EndNote Download URL

  

  
• Tran, T., Nguyen, T.N.: Hedera: Scalable Indexing and Exploring Entities in Wikipedia Revision History. in Poster & System Demonstration Proceedings of 13th International Semantic Web Conference (ISWC2014), Riva Del Garda, Italy, October 2014 (2014).

  BibTeX EndNote Download URL

  

  
• Zerr, S., Papapetrou, O., Demidova, E.: HEALTH+Z: Confidential Provider Selection in Collaborative Healthcare P2P Networks. In: Proceedings, C.E.U.R.W. (ed.) Proceedings of the 1st International Workshop on Privacy-Preserving IR: When Information Retrieval Meets Privacy, (PIR 2014), co-located with SIGIR 2014, July 11 2014, Gold Coast, Australia. pp. 1-6. CEUR (2014).

  BibTeX EndNote Download URL

  

  
• Holzmann, H., Risse, T.: Insights into Entity Name Evolution on Wikipedia. In: Benatallah, B., Bestavros, A., Manolopoulos, Y., Vakali, A., and Zhang, Y. (eds.) Web Information Systems Engineering – WISE 2014. pp. 47-61. Springer International Publishing (2014).

  Abstract BibTeX EndNote Download URL

  Working with Web archives raises a number of issues caused by their temporal characteristics. Depending on the age of the content, additional knowledge might be needed to find and understand older texts. Especially facts about entities are subject to change. Most severe in terms of information retrieval are name changes. In order to find entities that have changed their name over time, search engines need to be aware of this evolution. We tackle this problem by analyzing Wikipedia in terms of entity evolutions mentioned in articles regardless the structural elements. We gathered statistics and automatically extracted minimum excerpts covering name changes by incorporating lists dedicated to that subject. In future work, these excerpts are going to be used to discover patterns and detect changes in other sources. In this work we investigate whether or not Wikipedia is a suitable source for extracting the required knowledge.

  

  
• Kanhabua, N., Nejdl, W.: On the Value of Temporal Anchor Texts in Wikipedia. Proceedings of SIGIR 2014 Workshop on Temporal, Social and Spatially-aware Information Access (#TAIA2014) (2014).

  BibTeX EndNote Download URL

  

  
• Tran, T., Ceroni, A., Georgescu, M., Djafari Naini, K., Fisichella, M.: WikipEvent: Leveraging Wikipedia Edit History for Event Detection. In: Benatallah, B., Bestavros, A., Manolopoulos, Y., Vakali, A., and Zhang, Y. (eds.) Web Information Systems Engineering – WISE 2014. pp. 90-108. Springer International Publishing (2014).

  Abstract BibTeX EndNote Download URL

  Much of existing work in information extraction assumes the static nature of relationships in fixed knowledge bases. However, in collaborative environments such as Wikipedia, information and structures are highly dynamic over time. In this work, we introduce a new method to extract complex event structures from Wikipedia. We propose a new model to represent events by engaging multiple entities, generalizable to an arbitrary language. The evolution of an event is captured effectively based on analyzing the user edits history in Wikipedia. Our work provides a foundation for a novel class of evolution-aware entity-based enrichment algorithms, and considerably increases the quality of entity accessibility and temporal retrieval for Wikipedia. We formalize this problem and introduce an efficient end-to-end platform as a solution. We conduct comprehensive experiments on a real dataset of

  

  
• Holzmann, H., Risse, T.: Extraction of evolution descriptions from the web. IEEE/ACM Joint Conference on Digital Libraries. Institute of Electrical and Electronics Engineers (IEEE) (2014).

  BibTeX EndNote Download URL

  

  
• Rokicki, M., Chelaru, S., Zerr, S., Siersdorfer, S.: Competitive Game Designs for Improving the Cost Effectiveness of Crowdsourcing. Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Management. p. 1469--1478. ACM, Shanghai, China (2014).

  Abstract BibTeX EndNote Download URL

  Crowd based online work is leveraged in a variety of applications such as semantic annotation of images, translation of texts in foreign languages, and labeling of training data for machine learning models. However, annotating large amounts of data through crowdsourcing can be slow and costly. In order to improve both cost and time efficiency of crowdsourcing we examine alternative reward mechanisms compared to the "Pay-per-HIT" scheme commonly used in platforms such as Amazon Mechanical Turk. To this end, we explore a wide range of monetary reward schemes that are inspired by the success of competitions, lotteries, and games of luck. Our large-scale experimental evaluation with an overall budget of more than 1,000 USD and with 2,700 hours of work spent by crowd workers demonstrates that our alternative reward mechanisms are well accepted by online workers and lead to substantial performance boosts.

  

  
• Demidova, E., Barbieri, N., Dietze, S., Funk, A., Holzmann, H., Maynard, D., Papailiou, N., Peters, W., Risse, T., Spiliotopoulos, D.: Analysing and Enriching Focused Semantic Web Archives for Parliament Applications. Future Internet. 6, 433 (2014).

  Abstract BibTeX EndNote Download URL

  The web and the social web play an increasingly important role as an information source for Members of Parliament and their assistants, journalists, political analysts and researchers. It provides important and crucial background information, like reactions to political events and comments made by the general public. The case study presented in this paper is driven by two European parliaments (the Greek and the Austrian parliament) and targets an effective exploration of political web archives. In this paper, we describe semantic technologies deployed to ease the exploration of the archived web and social web content and present evaluation results.

  

  
• Kotsakos, D., Lappas, T., Kotzias, D., Gunopulos, D., Kanhabua, N., Nørv\aag, K.: A Burstiness-aware Approach for Document Dating. Proceedings of the 37th International ACM SIGIR Conference on Research &\#38; Development in Information Retrieval. p. 1003--1006. ACM, Gold Coast, Queensland, Australia (2014).

  Abstract BibTeX EndNote Download URL

  A large number of mainstream applications, like temporal search, event detection, and trend identification, assume knowledge of the timestamp of every document in a given textual collection. In many cases, however, the required timestamps are either unavailable or ambiguous. A charac- teristic instance of this problem emerges in the context of large repositories of old digitized documents. For such doc- uments, the timestamp may be corrupted during the digiti- zation process, or may simply be unavailable. In this paper, we study the task of approximating the timestamp of a doc- ument, so-called document dating. We propose a content- based method and use recent advances in the domain of term burstiness, which allow it to overcome the drawbacks of pre- vious document dating methods, e.g. the fix time partition strategy. We use an extensive experimental evaluation on different datasets to validate the efficacy and advantages of our methodology, showing that our method outperforms the state of the art methods on document dating.

  

  
• Tran, T., Georgescu, M., Zhu, X., Kanhabua, N.: Analysing the Duration of Trending Topics in Twitter Using Wikipedia. Proceedings of the 2014 ACM Conference on Web Science. p. 251--252. ACM, Bloomington, Indiana, USA (2014).

  Abstract BibTeX EndNote Download URL

  The analysis of trending topics in Twitter is a goldmine for a variety of studies and applications. However, the contents of topics vary greatly from daily routines to major public events, enduring from a few hours to weeks or months. It is thus helpful to distinguish trending topics related to real-world events with those originated within virtual communities. In this paper, we analyse trending topics in Twitter using Wikipedia as reference for studying the provenance of trending topics. We show that among different factors, the duration of a trending topic characterizes exogenous Twitter trending topics better than endogenous ones.

  

  
• Yuan, W., Demidova, E., Dietze, S., Zhou, X.: Analyzing Relative Incompleteness of Movie Descriptions in the Web of Data: A Case Study. Proceedings of the ISWC 2014 Posters & Demonstrations Track, a track within the 13th International Semantic Web Conference (ISWC 2014). pp. 197-200. CEUR-WS.org (2014).

  BibTeX EndNote Download URL

  

  
• Ceroni, A., Tran, N.K., Kanhabua, N., Niederée, C.: Bridging Temporal Context Gaps Using Time-aware Re-contextualization. Proceedings of the 37th International ACM SIGIR Conference on Research &\#38; Development in Information Retrieval. p. 1127--1130. ACM, Gold Coast, Queensland, Australia (2014).

  Abstract BibTeX EndNote Download URL

  Understanding a text, which was written some time ago, can be compared to translating a text from another language. Complete interpretation requires a mapping, in this case, a kind of time-travel translation between present context knowledge and context knowledge at time of text creation. In this paper, we study time-aware re-contextualization, the challenging problem of retrieving concise and complementing information in order to bridge this temporal context gap. We propose an approach based on learning to rank techniques using sentence-level context information extracted from Wikipedia. The employed ranking combines relevance, complimentarity and time-awareness. The effectiveness of the approach is evaluated by contextualizing articles from a news archive collection using more than 7,000 manually judged relevance pairs. To this end, we show that our approach is able to retrieve a significant number of relevant context information for a given news article.

  

  
• Holzmann, H., Risse, T.: Named Entity Evolution Analysis on Wikipedia. Proceedings of the 2014 ACM Conference on Web Science. p. 241--242. ACM, Bloomington, Indiana, USA (2014).

  Abstract BibTeX EndNote Download URL

  Accessing Web archives raises a number of issues caused by their temporal characteristics. Additional knowledge is needed to find and understand older texts. Especially entities mentioned in texts are subject to change. Most severe in terms of information retrieval are name changes. In order to find entities that have changed their name over time, search engines need to be aware of this evolution. We tackle this problem by analyzing Wikipedia in terms of entity evolutions mentioned in articles. We present statistical data on excerpts covering name changes, which will be used to discover similar text passages and extract evolution knowledge in future work.