SQPR: Stream query planning with reuse

Kalyvianaki, E., Wiesemann, W., Vu, Q. H., Kuhn, D. & Pietzuch, P. (2011). SQPR: Stream query planning with reuse. Paper presented at the 2011 IEEE 27th International Conference on Data Engineering (ICDE), 11-04-2011 - 16-04-2011, Hannover, Germany.

[img]
Preview
PDF - Accepted Version
Download (708kB) | Preview

Abstract

When users submit new queries to a distributed stream processing system (DSPS), a query planner must allocate physical resources, such as CPU cores, memory and network bandwidth, from a set of hosts to queries. Allocation decisions must provide the correct mix of resources required by queries, while achieving an efficient overall allocation to scale in the number of admitted queries. By exploiting overlap between queries and reusing partial results, a query planner can conserve resources but has to carry out more complex planning decisions. In this paper, we describe SQPR, a query planner that targets DSPSs in data centre environments with heterogeneous resources. SQPR models query admission, allocation and reuse as a single constrained optimisation problem and solves an approximate version to achieve scalability. It prevents individual resources from becoming bottlenecks by re-planning past allocation decisions and supports different allocation objectives. As our experimental evaluation in comparison with a state-of-the-art planner shows SQPR makes efficient resource allocation decisions, even with a high utilisation of resources, with acceptable overheads.

Item Type: Conference or Workshop Item (Paper)
Additional Information: © 2011 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.
Subjects: Q Science > QA Mathematics > QA75 Electronic computers. Computer science
Divisions: School of Informatics > Department of Computing
URI: http://openaccess.city.ac.uk/id/eprint/8177

Actions (login required)

View Item View Item

Downloads

Downloads per month over past year

View more statistics