City Research Online

A distributed stream library for Java 8

Chan, Y., Wellings, A., Gray, I. and Audsley, N. C. ORCID: 0000-0003-3739-6590 (2017). A distributed stream library for Java 8. IEEE Transactions on Big Data, 3(3), pp. 262-275. doi: 10.1109/TBDATA.2017.2666201

Abstract

Java 8 has introduced new capabilities such as lambda expressions and streams which simplify data-parallel computing. However, as a base language for Big Data systems, it still lacks a number of important capabilities such as processing very large datasets and distributing the computation over multiple machines. This paper gives an overview of the Java 8 Streams API and proposes extensions to allow its use in Big Data systems. It also shows how the API can be used to implement a range of standard Big Data paradigms. Finally, it compares performance with that of Hadoop and Spark. Despite being a proof-of-concept implementation, results indicate that it is a lightweight and efficient framework, comparable in performance to Hadoop and Spark, and is up to 5 times faster for the largest input sizes tested.

Publication Type: Article
Additional Information: © 2017 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.
Publisher Keywords: Big Data, Java, distributed computing, programming models
Subjects: Q Science > QA Mathematics > QA75 Electronic computers. Computer science
Departments: School of Mathematics, Computer Science & Engineering
Date available in CRO: 04 Oct 2021 12:21
Date deposited: 4 October 2021
Date of acceptance: 1 February 2017
Date of first online publication: 7 February 2017
URI: https://openaccess.city.ac.uk/id/eprint/26846
[img]
Preview
Text - Accepted Version
Download (209kB) | Preview

Export

Downloads

Downloads per month over past year

View more statistics

Actions (login required)

Admin Login Admin Login