Fast datalog evaluation for batch and stream graph processing
نویسندگان
چکیده
Abstract Implementing complex algorithms for big data, artificial intelligence, and graph processing requires enormous effort. Succinct, declarative programs to solve problems that can be efficiently executed batching streaming data are in demand. This paper presents Nexus, a distributed Datalog evaluation system. It evaluates using the semi-naive algorithm batch incremental asynchronous iteration. Furthermore, we evaluate with aggregates determine advantages of implementing iteration on its performance. Our experimental results show Nexus significantly outperforms acyclic dataflow-based systems.
منابع مشابه
Apache FlinkTM: Stream and Batch Processing in a Single Engine
Apache Flink1 is an open-source system for processing streaming and batch data. Flink is built on the philosophy that many classes of data processing applications, including real-time analytics, continuous data pipelines, historic data processing (batch), and iterative algorithms (machine learning, graph analysis) can be expressed and executed as pipelined fault-tolerant dataflows. In this pape...
متن کاملFast and Highly-Available Stream Processing
Recently, there has been significant interest in applications where high-volume, continuous data streams need to be processed with low latency. These applications include financial market monitoring, network monitoring, sensor-based environment monitoring, call analysis, battlefield monitoring, asset tracking, and Web feed analysis. To facilitate the applications, several stream-processing syst...
متن کاملPig Squeal: Bridging Batch and Stream Processing Using Incremental Updates
Title of dissertation: Pig Squeal: Bridging Batch and Stream Processing Using Incremental Updates James Holmes Lampton, Jr., Doctor of Philosophy, 2015 Dissertation directed by: Professor Ashok Agrawala Department of Computer Science As developers shift from batch MapReduce to stream processing for better latency, they are faced with the dilemma of changing tools and maintaining multiple code b...
متن کاملStream Reasoning in Temporal Datalog
In recent years, there has been an increasing interest in extending traditional stream processing engines with logical, rule-based, reasoning capabilities. This poses significant theoretical and practical challenges since rules can derive new information and propagate it both towards past and future time points; as a result, streamed query answers can depend on data that has not yet been receiv...
متن کاملAn Extension of Datalog for Graph Queries
Supporting aggregates in recursive logic rules is a crucial long-standing problem for Datalog. To solve this problem, we propose Datalog that supports queries and reasoning on the number of distinct occurrences satisfying given goals, or conjunction of goals, in rules. By using a generalized notion of multiplicity called frequency, we show that graph queries can be easily expressed in Datalog ....
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: World Wide Web
سال: 2022
ISSN: ['1573-1413', '1386-145X']
DOI: https://doi.org/10.1007/s11280-021-00960-w