Parallel Querying with Non-Dedicated Computers: An Evaluation of Adaptivity Options
- Speaker: Professor Norman Paton (University of Manchester)
- Host: Ian Pratt-Hartmann
- 8th February 2006 at 14:15 in 1.5
The increasing prevalence of networked storage and computational resources, along with grid middleware to manage access to and sharing of such resources, raises the prospect that queries can be run over resources obtained on demand, rather than on dedicated infrastructures obtained as part of a complex and costly procurement activity. However, the movement of query processing into non-dedicated environments means that it is necessary to take account of the load fluctuations that characterise autonomous distributed settings. Several proposals have been made for the application of adaptive query processing (AQP) techniques that modify the way a query is evaluated to take account of varying resource loads, but we know of no previous study that compares their performance. This seminar presents an evaluation of AQP techniques that seek to balance load in query plans in which partitioned parallelism is used to improve the scaleability of query performance. Existing adaptive load balancing strategies are compared with a new approach inspired by work on distributed hash tables. The evaluation identifies situations in which the different approaches may be used most effectively, describes contexts in which their costs outweight their benefits, and provides some general insights into the behaviour of adaptive query processors. As this seminar describes work carried out while on a sabbatical visit to IBM Almaden, there will also be some anecdotes on the sabbatical experience and life in Silicon Valley.