Project description

Technology has long had an important role in supporting human memory. However, developments in pervasive computing are beginning to offer the potential to re-think and re-define how technology can augment human memory. For example, widespread sensing and systems for quantified self are creating an environment in which it is possible to capture fine-grained traces of many aspects of human activity that may help in remembering items of interest. Likewise, ever-present display technologies such as digital signage, and mobile and wearable devices provide opportunities to ensure that memory cues are available at opportune moments.

Emerging research in pervasive computing for human memory augmentation has largely focused on a single sensory modality, predominantly using visual data for memory capture and cueing (using, for example, worn camera such as the SenseCam or Narrative Clip). However, both human perception and human memory are multi-sensory (i.e. they combine visual, audio, touch, smell, taste); future technology for digital memory augmentation should reflect this multi-sensory perspective.

This PhD project will consider how we might create a multi-sensory memory augmentation system, analogous to biological memory, that fuses data from the pervasive technology all around us. This augmentation system is unlikely to directly mirror the human senses, but will instead make novel use of new developments in non-visual capture and presentation technologies (e.g. the always-on audio of Siri, Amazon Echo etc.). The project will lay the foundation for future multi-sensory platforms by uniquely considering how multiple channels reflecting the different senses might be combined to create technology that mirrors our natural capacity for multi-sensory human memory.

Development of multi-sensory memory augmentation must address challenges in both technical and social domains, such as:

(i) What forms of multi-sensory data can be captured by (or on behalf of) users whilst engaged in their daily activities?
(ii) What social, ethical and usability issues limit this capture?
(iii) How can data streams representing multiple sensory inputs be mined and integrated to create effective human memory cues?
(iv) What are the most appropriate presentation mediums for multi-sensory cueing?
Given the broad scope of the project, we are open to and interested in applicants' ideas within the scope of the described research area.

This PhD project may examine either the human elements of multi-sensory memory augmentation, the systems aspects of implementing this, or a combination of human and systems themes. One possible roadmap for this PhD may look something like:
* Year 1: Identification of one or more target senses. A literature review that establishes the role of the target sense(s) in human memory and human memory augmentation.
* Year 2: Development and evaluation of one or more prototypes that leverage the target sense(s) to augment memory, leading to some generalisable knowledge for future multi-sensory augmentation systems.
* Year 3: Development and evaluation of a consolidated platform that builds on the knowledge acquired in Year 2. Writing of thesis (or alternate format submission) for examination.
The student should expect to publish their work in the form of academic papers at venues such as ACM CHI, PACM IMWUT and ACM Transactions on Applied Perception.

This PhD would be attractive to candidates with experience and training in human-computer interaction or other related disciplines (e.g., experimental/cognitive psychology, mobile/pervasive computing). A successful applicant is likely to have demonstrable experience of EITHER experimental design OR software development; they will need to demonstrate both capability and willingness to develop the missing skill.