OU Profiles homepage Edit my profile User guide Accessibility Statement
Picture  of Simon Holland

Dr Simon Holland

Senior Lecturer In Computing

School of Computing & Communications

simon.holland@open.ac.uk

Biography

Professional biography

Simon is a Senior Lecturer in Computing. He is Founder and Director of the Music Computing Lab, and a member of the Centre for Research in Computing and the Interaction Design Research Group. Simon’s research interests are in Music Computing, Digital Health and Human Computer Interaction. 

Research interests

Simon Holland founded and directs the Music Computing Lab, a research group in the Centre for Research in Computing.  His research focuses on the interconnected areas of Music Computing, Human Computer Interaction, and Digital Health. He has served as PI or Co-I on ten external research grants, totalling nearly £4.5 million, including Polifonia (EU, € 3,025,435.06), the HAPPIE Haptic Authoring Pipeline for Immersive Experiences (Innovate UK, £998,538), the AHRC  E-Sense Project on digitally enhanced senses, £200,000), the Haptic Bracelets (Goldcrest Trust),  the ESRC Older People and Technological Inclusion ESRC project, and the NATO Science Committee. He has published over 100 refereed research articles, and co-edited two seminal books on Music and HCI. He was co-author of Human Computer Interaction (Preece et al)— for many years the worldwide best seller in HCI. He was a founding member of the editorial Board of the Journal of Music Technology and Education. He was lead organizer of two international workshops on Music and HCI, the most recent at CHI 2016 in San Jose.  He has devised numerous human-centred computing systems and interaction techniques including Harmony Space, the Haptic Bracelets, the Haptic Drum Kit, the Audio GPS, and a new and highly expressive form of interaction, Direct Combination.  He has supervised 12 PhD students to completion, and examined 24 PhDs. He is currently co-Investigator on the Innovate-UK funded project, Happie - Creative touch, Design prototyping, and leads a research collaboration with PJ Care Neurological Care Homes.

He is lead Editor of  New directions in music and human-computer interaction. Eds. Holland S, Mudd T, Wilkie-McKenna K, McPherson A, Wanderley MM  Springer, London. ISBN 978-3-319-92069- 6 (2019).

Teaching interests

Simon has taught undergraduate and postgraduate courses in Artificial Intelligence, Intelligent Tutoring systems, Human Computer Interaction, Smalltalk, Object Oriented Programming and Design and Software Engineering. He was  been presentation chair of M868, and has taught on PMT80, PMT607, M868, M206, M255 M256, M250,  TM354, TM356  He is is currently teaching on TM255, Communication and Information Technologies, and TM356, Interaction design and the User Experience.

Projects

Music and HCI Network (CHIME)

The Music and Human-Computer Interaction Network (MIN) will connect academics, researchers, SMEs, charities and industrial partners in order to pool and expand ideas, drive new research, and develop collaborative projects around emerging issues in the field. The UK has an excellent track record of research in the two contributing areas, including a number of internationally recognised research centres and organisations around the country. The combined field is broad however, and communication and collaboration between researchers, charities and businesses is essential for both informed, relevant, high-impact research, and for charities and businesses to engage with and creatively explore the possibilities suggested by current research. Research in music and human computer interaction is useful not only to music and musicians - one of the UK's largest industries - but also to research generally into making computers easy to control and understand. Research aimed at making computers fluidly useful for music and musicians has inspired numerous innovations in human computer interaction, for example in gestural interfaces, data gloves and touch interfaces. Many musical activities involve coordinating the whole body, in real-time, often with other people, while also making significant perceptual and cognitive demands. Techniques for wearable motion capture originally developed for musicians are now being used in a range of clinical applications such as managing physical rehabilitation after stroke. Because of this long established synergy between music and HCI, a network that brings together researchers in music and HCI along with interested parties from industry and charities will have the capacity to create new theories, products, approaches and services for musicians, the music industry and computing generally. This network therefore aims to support communication and dialog between academics, charities, and SMEs working in fields such as digital music, creativity and interaction, assistive technology, accessible musical instruments, live performance, music production, haptic and tactile interactions, interactive machine learning and music technology for education.

Haptics PhD Studentship

This research will aim to design, develop and evaluate innovative digital systems to improve health and rehabilitation, with particular reference to stroke and other neurological conditions. The work will draw on leading edge research in human computer interaction and on wearable, haptic and other advanced technologies. Sources of inspiration will include recent work in musical cognition and rhythm perception and production. A key starting point for this research will be recent seminal research on the use of multi-limb haptics and motion sensing to exploit human biological entrainment mechanisms for purposes ranging from musical training to stroke and other neurological rehabilitation.

CoA: HAPPIE Audience of the Future - Creative touch, Design prototyping

For distance learning students often it is difficult to facilitate creative studio-based activities in the same format as conventional universities. This can result in a short fall of distance learning students ‘maker-process’ skills and practice. The benefits of haptic technology for applied online education, such as e-crafts and e-design practice, is that it enables and enriches the student’s hands-on approach. Currently haptic technology is able to aid students’ interactions to 3D virtual environments. However, haptics that could afford greater sensory engagements with the maker-process with a fidelity akin to real-world is still to be developed. This project aims to embed haptics within a variety of ‘hands-on’ methods, designed by Open University (Engineering and Innovation school) academics. This will enable the development of a working haptic prototype which will focus on haptics and the ‘maker-process’ aiming to add value to core learning technology resources for undergraduate and post graduate programmes. In line with the OU’s ‘access to learning policies’ there is a greater onus placed onto inclusive learning, and the provision of accessible learning technology to include a variety of learners with a range of needs e.g. sensory impairment, motor skills etc… By increasing access to virtual immersive technologies a wider student body could be given access to engage with their own version of the maker-process using multimodal interactions.

Digital Inclusion Skills for Carers bringing Opportunities, Value and Excellence. (K-11-021-VW)

There is evidence from numerous surveys in the UK that carers suffer from longstanding physical health problems and are strongly affected by stress, anxiety and depression. Research carried out by Cheffings, (2003) revealed that 50% of carers believed that their health was adversely affected by their caring role. Additionally, in an ageing population, family members are expected to take on more and more complex care tasks. In particular home carers can encounter feelings of isolation because they are less part of a professional community compared to e.g. staff in a care home. They need ways to find answers to care questions or psychological support to help them fulfil their role. Online information, advice and guidance and access to a community of peers and experts can help. The DISCOVER solution is an innovative, broadly accessible solution that is focused upon improving the quality of life of carers and the older people they care for through learning, informational access and support. DISCOVER aims to improve the digital skills of domiciliary care workers, to give them the ability to pass on these digital skills to the older people they care for and to provide opportunities for both carers and older people to learn these skills together. DISCOVER will provide access to e-Learning for digital skills (both explicit and embedded into all elements of training), access to carer skills; access to care management tools and communications, including the ability to output to telehealth systems; or provide updates to family; and act as an access point to a range of existing information and support networks that exist in each country where DISCOVER is deployed.

Older People and Technological Inclusion: multidisciplinary perspectives on contemporary realities and aspirations. (K-10-004-CH)

The proposed research is a series of 4 academic seminars to explore the concept of technological inclusion in relation to age, and to establish a new interdisciplinary network of researchers including experienced researchers, PhD students and early-career researchers, older people and service providers, to critically examine how older people are positioned in the age of digital information. The first 3 seminars will take place at the OU; the final one at Bletchley Park.

Using rhythm for rehabilitation: evaluation of a novel haptic device

Cueing using metronomic rhythmic sensory stimulation has been shown to improve gait for stroke survivors, but most versions of this approach have used auditory and visual cues. In contrast, we have developed a prototype wearable system for rhythmic cueing based on haptics, which has been shown to be effective in pilot studies (Holland et al. 2015). The next step is to refine the current experimental system to make it practical for everyday use by therapists, and to determine what refinements will be needed for self-managed use by stroke survivors in the community. As part of this process we need to identify issues and requirements by drawing on the views of physiotherapists, nurses, interaction designers and stroke survivors. To apply the knowledge gained to improved design in an effective way we will use rapid prototyping - this involves building and modifying successive prototypes for representative users to interact with. This process goes through cycles, where user reactions feed into the next iteration of the prototype.

Publications

Book

PPIG 2022: Proceedings of the 33rd Annual Workshop of the Psychology of Programming Interest Group (2023)

New Directions in Music and Human-Computer Interaction (2019)

Music and Human-Computer Interaction (2013)

Multimedia Interface Design in Education (1992)

Book Chapter

Understanding Music Interaction, and Why It Matters (2019)

TMAP Design Cards for Technology-Mediated Audience Participation in Live Music (2019)

Machine Learning, Music and Creativity: An Interview with Rebecca Fiebrink (2019)

Haptics for the development of fundamental rhythm skills, including multi-limb coordination (2018)

Harmony and Technology Enhanced Learning (2016)

Exploring New Technologies through Playful Peer-to-Peer Engagement in Informal Learning (2015)

Video analysis for evaluating music interaction: musical tabletops (2013)

Song walker harmony space: embodied interaction design for complex musical skills (2013)

The Haptic Bracelets: learning multi-limb rhythm skills from haptic stimuli while reading (2013)

Towards a participatory approach for interaction design based on conceptual metaphor theory: a case study from music interaction (2013)

Music interaction: understanding music and human-computer interaction (2013)

Scratching the scale labyrinth (2011)

Whole body interaction in abstract domains (2011)

A first empirical study of direct combination in a ubiquitous environment (2005)

A Critical Analysis of Synthesizer User Interfaces for Timbre (2004)

Mobile HCI and sound (2004)

Direct combination: a new user interaction principle for mobile and ubiquitous HCI (2002)

Artificial Intelligence in Music Education: A Critical Review (2000)

Learning about harmony with Harmony Space: an overview (1994)

MOTIVE: The development of an AI tool for beginning melody composers (1994)

Interface Design for Empowerment: a Case Study from Music (1992)

Architecture of a knowledge-based music tutor (1990)

Computers in Music Education (1987)

Journal Article

Democratizing Clinical Movement Analysis: Assessing the Versatility of MoJoXlab with Open-protocol Inertial Sensors (2024)

Using rhythm for rehabilitation: the acceptability of a novel haptic cueing device in extended stroke rehabilitation (2022)

Rhythmic Haptic Cueing Using Wearable Devices as Physiotherapy for Huntington Disease: Case Study (2020)

A Nonproprietary Movement Analysis System (MoJoXlab) Based on Wearable Inertial Measurement Units Applicable to Healthy Participants and Those With Anterior Cruciate Ligament Reconstruction Across a Range of Complex Tasks: Validation Study (2020)

Rhythmic Haptic Cueing for Gait Rehabilitation of People With Hemiparesis: Quantitative Gait Study (2020)

The role of nonlinear dynamics in interactions with digital and acoustic musical instruments (2020)

Nonlinear Dynamical Processes in Musical Interactions: investigating the role of nonlinear dynamics in supporting surprise and exploration in interactions with digital musical instruments (2019)

A small repeated measures study of haptic cueing upon gait symmetry in people after stroke (2017)

Walk to the Beat: A Case Report of the Use of a Novel Haptic Device to Improve Walking after Stroke (2017)

Empirically testing Tonnetz, voice-leading, and spectral models of perceived triadic distance (2016)

Exploring the Digital Music Instrument Trombosonic with Extreme Users and at a Participatory Performance (2014)

Testing a computational model of rhythm perception using polyrhythmic stimuli (2013)

A MIDI sequencer that widens access to the compositional possibilities of novel tunings (2012)

What can the language of musicians tell us about music interaction design? (2010)

Sense before syntax: a path to a deeper understanding of objects (2007)

An introduction to aspect-oriented music representation (2007)

AudioGPS: Spatial audio navigation with a minimal attention interface (2002)

Sound investments: some applications of computers in music teaching, and a revolutionary new program (1994)

Beauty and the beast: New approaches to teaching computing for humanities students at the University of Aberdeen (1992)

Other

Preliminary report on the design of a constraint-based musical planner (1991)

Presentation / Conference

Proceedings of the CHIME Music and HCI Workshop 2023 (2023)

An agent for creative development in drum kit playing (2023)

A principled approach to the development of drum improvisation skills through interaction with a conversational agent (2020)

Gait Rehabilitation for Neurological Conditions using Wearable Devices (2019)

Choosers: The design and evaluation of a visual algorithmic music composition language for non-programmers (2018)

Patterns of Design (2018)

Wearable Haptic Devices for Long-Term Gait Re-education for Neurological Conditions (2018)

How can rhythmic haptic cueing using wearable haptic devices help gait rehabilitation for stroke survivors: a longitudinal pilot study (2018)

Wearables for Long Term Gait Rehabilitation of Neurological Conditions (2018)

Designing for Diabetes Decision Support Systems with Fluid Contextual Reasoning (2018)

Data, Data Everywhere, and Still Too Hard to Link: Insights from User Interactions with Diabetes Apps (2018)

A longitudinal rehabilitation case study for hemiparetic gait using outdoor rhythmic haptic cueing via a wearable device (2018)

Gait rehabilitation by outdoor rhythmic haptic cueing using wearable technology for neurological conditions: a case study (2018)

Choosers: designing a highly expressive algorithmic music composition system for non-programmers (2017)

Design Implications for Technology-Mediated Audience Participation in Live Music (2017)

Two kinds of entrainment in gait rehabilitation using haptic metronomic cues (2017)

Assessing the Accuracy of an Algorithm for the Estimation of Spatial Gait Parameters Using Inertial Measurement Units: Application to Healthy Subject and Hemiparetic Stroke Survivor (2017)

Rhythm and Gait rehabilitation: a case study using rhythmic haptic cueing for stabilising gait and improving asymmetries (2017)

Characterizing the Landscape of Musical Data on the Web: State of the Art and Challenges (2017)

Designing a Highly Expressive Algorithmic Music Composition System for Non-Programmers (2016)

Questioning Classic Patient Classification Techniques in Gait Rehabilitation: Insights from Wearable Haptic Technology (2016)

Wearable Haptic Devices For Post- Stroke Gait Rehabilitation (2016)

The Haptic Bracelets: Gait rehabilitation after Stroke (2016)

Questioning the Reflection Paradigm for Diabetes Mobile Apps (2016)

Music and HCI (2016)

Walk to the beat: a single case study investigating a novel haptic device to improve walking after stroke (2016)

Rhythmic Haptic Cueing for Entrainment: Assisting post-stroke gait rehabilitation (2016)

Designing, Developing, and Evaluating the Future Internet of Personal Health (2016)

A pilot study using tactile cueing for gait rehabilitation following stroke (2015)

Prototyping a method for the assessment of real-time EEG sonifications (2015)

Investigating the effects of introducing nonlinear dynamical processes into digital musical interfaces (2015)

A blended user centred design study for wearable haptic gait rehabilitation following hemiparetic stroke (2015)

Towards bridging a gap in a musical live performance (2015)

Nonlinear Dynamical Systems as Enablers of Exploratory Engagement with Musical Instruments (2014)

Experimence: Considerations for Composing a Rock Song for Interactive Audience Participation (2014)

Eliciting Domain Knowledge Using Conceptual Metaphors: A Case Study from Music Interaction (2014)

Dynamical Interactions with Electronic Instruments (2014)

A cognitive dimensions analysis of interaction design for algorithmic composition software (2014)

Dynamical Systems in Interaction Design for Improvisation (2014)

A Gait Rehabilitation pilot study using tactile cueing following Hemiparetic Stroke (2014)

A review of real-time EEG sonification research (2013)

Kolab: appropriation & improvisation in mobile tangible collaborative interaction (2012)

We can work it out: towards a participatory approach to designing music interactions (2011)

Hex Player—a virtual musical controller (2011)

Sound Spheres: A Design Study of the Articulacy of a Non-Contact Finger Tracking Virtual Musical Instrument (2011)

Older people & technological innovations-lifelong, learning and applications for health and wellbeing (2010)

Older people and technological innovations: lifelong learning and applications for health and wellbeing (2010)

A novel user interface for musical timbre design (2010)

Asymmetrical Multi-User Co-operative Whole Body Interaction in Abstract Domains (2010)

Feeling the beat where it counts: fostering multi-limb rhythm skills with the haptic drum kit (2010)

Analysis of conceptual metaphors to improve music software: the role of prior experience in inclusive music interaction (2009)

Prototyping Whole Body Navigation of Musical Harmony (2009)

Evaluating musical software using conceptual metaphors (2009)

Running up Blueberry Hill: Prototyping whole body interaction in harmony space (2009)

Feel the force: Using tactile technologies to investigate the extended mind (2008)

Timbre space as synthesis space: towards a navigation based approach to timbre specification (2008)

Symmetric composition of musical concerns (2006)

The simplification of complex interactions for more inclusive social communication technologies (position paper) (2005)

Towards a mapping of timbral space (2005)

Reflective Composition: the declarative composition of roles to unify objects, roles, and aspects [poster session] (2004)

Applying aspect-oriented programming to music computing (2004)

Using Dynamic Aspects in Music Composition Systems (2004)

Smart homes and extended families (2004)

Applying Direct Combination to afford spontaneity in Pervasive Computing (2002)

AudioGPS: spatial audio in a minimal attention interface (2001)

Direct Combination (1999)

Employing Object Technology to Expose Fundamental Object Concepts (1999)

Separable User Interface Architectures in Teaching Object Technology (1999)

Enhancing Interaction Scenarios with a Domain-Oriented Pattern Language (1999)

Exploiting Smalltalk Modules In A Customizable Programming Environment (1999)

Avoiding Object Misconceptions (1997)

The Object Shop – Using CD-ROM Multimedia To Introduce Object Concepts (1997)

From software user to software author: an initial pedagogy for Introductory Object-Oriented Computing (1996)

The Joy Of Software – Starting With Objects (1996)

What is the Value of Virtual Reality for Conceptual Learning? Towards a Theoretical Framework (1996)

Understanding harmony and technology in music education (1995)

Pervasiveness of a Programming Paradigm: Questions Concerning an Object-oriented Approach (1994)

HCI and Music: Squaring the Circle (1993)

An AI Tool for the Analysis and Generation of Melodies (1992)

Virtual Worlds and their Role in Investigating Change in Cognitive Models of Motion (1992)

Visual Programming in Prolog (1991)

Two-dimensional Visual Programming and Three-Dimensional Execution Visualisation in Prolog (1991)

New Cognitive Theories of Harmony Applied to Direct Manipulation Tools for Novices (1987)

Report

An analysis of algorithmic composition interaction design with reference to Cognitive Dimensions (2014)

Tutorial: An Introduction to AspectMusic (2006)

Synthesizer user interface design - lessons learned from a heuristic review (2004)

Audio Navigation: Using Spatial Audio In Ubiquitous Interfaces (2004)

Using Spatial Audio in Minimal Attention Interfaces: Towards An Effective Audio GPS Navigation System (2004)

Using Aspects to Help Composers (2003)

The application of Direct Combination to Mobile and Ubiquitous Human Computer Interaction (2002)

A simple, technology-neutral lingua franca for location systems, applied to combined indoor-outdoor navigation (2002)

Meeting the software engineering challenges of interacting with dynamic and ad-hoc computing environments (2002)

A Visual Representation for Interaction Scenarios (1998)

Thesis

Artificial intelligence, education and music : the use of artificial intelligence to encourage and facilitate music composition by novices (1989)

Working Paper

Using rhythm for rehabilitation: evaluation of a novel haptic device (2017)

When is Direct Combination Useful? (2004)

A Simple Taxonomy of Search Reduction in Direct Combination (2004)

Learning about harmony with Harmony Space: an overview (Extended Technical Report) STAN-M-88 (1994)