
Dr Simon Holland
Senior Lecturer In Computing
School of Computing & Communications
Biography
Professional biography
Simon is a Senior Lecturer in Computing. He is Founder and Director of the Music Computing Lab, and a member of the Centre for Research in Computing and the Interaction Design Research Group. Simon’s research interests are in Music Computing, Digital Health and Human Computer Interaction.
Research interests
Simon Holland founded and directs the Music Computing Lab, a research group in the Centre for Research in Computing. His research focuses on the interconnected areas of Music Computing, Human Computer Interaction, and Digital Health. He has served as PI or Co-I on ten external research grants, totalling nearly £4.5 million, including Polifonia (EU, € 3,025,435.06), the HAPPIE Haptic Authoring Pipeline for Immersive Experiences (Innovate UK, £998,538), the AHRC E-Sense Project on digitally enhanced senses, £200,000), the Haptic Bracelets (Goldcrest Trust), the ESRC Older People and Technological Inclusion ESRC project, and the NATO Science Committee. He has published over 100 refereed research articles, and co-edited two seminal books on Music and HCI. He was co-author of Human Computer Interaction (Preece et al)— for many years the worldwide best seller in HCI. He was a founding member of the editorial Board of the Journal of Music Technology and Education. He was lead organizer of two international workshops on Music and HCI, the most recent at CHI 2016 in San Jose. He has devised numerous human-centred computing systems and interaction techniques including Harmony Space, the Haptic Bracelets, the Haptic Drum Kit, the Audio GPS, and a new and highly expressive form of interaction, Direct Combination. He has supervised 12 PhD students to completion, and examined 24 PhDs. He is currently co-Investigator on the Innovate-UK funded project, Happie - Creative touch, Design prototyping, and leads a research collaboration with PJ Care Neurological Care Homes.
He is lead Editor of New directions in music and human-computer interaction. Eds. Holland S, Mudd T, Wilkie-McKenna K, McPherson A, Wanderley MM Springer, London. ISBN 978-3-319-92069- 6 (2019).
Teaching interests
Simon has taught undergraduate and postgraduate courses in Artificial Intelligence, Intelligent Tutoring systems, Human Computer Interaction, Smalltalk, Object Oriented Programming and Design and Software Engineering. He was been presentation chair of M868, and has taught on PMT80, PMT607, M868, M206, M255 M256, M250, TM354, TM356 He is is currently teaching on TM255, Communication and Information Technologies, and TM356, Interaction design and the User Experience.
Projects
Music and HCI Network (CHIME)
The Music and Human-Computer Interaction Network (MIN) will connect academics, researchers, SMEs, charities and industrial partners in order to pool and expand ideas, drive new research, and develop collaborative projects around emerging issues in the field. The UK has an excellent track record of research in the two contributing areas, including a number of internationally recognised research centres and organisations around the country. The combined field is broad however, and communication and collaboration between researchers, charities and businesses is essential for both informed, relevant, high-impact research, and for charities and businesses to engage with and creatively explore the possibilities suggested by current research. Research in music and human computer interaction is useful not only to music and musicians - one of the UK's largest industries - but also to research generally into making computers easy to control and understand. Research aimed at making computers fluidly useful for music and musicians has inspired numerous innovations in human computer interaction, for example in gestural interfaces, data gloves and touch interfaces. Many musical activities involve coordinating the whole body, in real-time, often with other people, while also making significant perceptual and cognitive demands. Techniques for wearable motion capture originally developed for musicians are now being used in a range of clinical applications such as managing physical rehabilitation after stroke. Because of this long established synergy between music and HCI, a network that brings together researchers in music and HCI along with interested parties from industry and charities will have the capacity to create new theories, products, approaches and services for musicians, the music industry and computing generally. This network therefore aims to support communication and dialog between academics, charities, and SMEs working in fields such as digital music, creativity and interaction, assistive technology, accessible musical instruments, live performance, music production, haptic and tactile interactions, interactive machine learning and music technology for education.
Haptics PhD Studentship
This research will aim to design, develop and evaluate innovative digital systems to improve health and rehabilitation, with particular reference to stroke and other neurological conditions. The work will draw on leading edge research in human computer interaction and on wearable, haptic and other advanced technologies. Sources of inspiration will include recent work in musical cognition and rhythm perception and production. A key starting point for this research will be recent seminal research on the use of multi-limb haptics and motion sensing to exploit human biological entrainment mechanisms for purposes ranging from musical training to stroke and other neurological rehabilitation.
CoA: HAPPIE Audience of the Future - Creative touch, Design prototyping
For distance learning students often it is difficult to facilitate creative studio-based activities in the same format as conventional universities. This can result in a short fall of distance learning students ‘maker-process’ skills and practice. The benefits of haptic technology for applied online education, such as e-crafts and e-design practice, is that it enables and enriches the student’s hands-on approach. Currently haptic technology is able to aid students’ interactions to 3D virtual environments. However, haptics that could afford greater sensory engagements with the maker-process with a fidelity akin to real-world is still to be developed. This project aims to embed haptics within a variety of ‘hands-on’ methods, designed by Open University (Engineering and Innovation school) academics. This will enable the development of a working haptic prototype which will focus on haptics and the ‘maker-process’ aiming to add value to core learning technology resources for undergraduate and post graduate programmes. In line with the OU’s ‘access to learning policies’ there is a greater onus placed onto inclusive learning, and the provision of accessible learning technology to include a variety of learners with a range of needs e.g. sensory impairment, motor skills etc… By increasing access to virtual immersive technologies a wider student body could be given access to engage with their own version of the maker-process using multimodal interactions.
Digital Inclusion Skills for Carers bringing Opportunities, Value and Excellence. (K-11-021-VW)
There is evidence from numerous surveys in the UK that carers suffer from longstanding physical health problems and are strongly affected by stress, anxiety and depression. Research carried out by Cheffings, (2003) revealed that 50% of carers believed that their health was adversely affected by their caring role. Additionally, in an ageing population, family members are expected to take on more and more complex care tasks. In particular home carers can encounter feelings of isolation because they are less part of a professional community compared to e.g. staff in a care home. They need ways to find answers to care questions or psychological support to help them fulfil their role. Online information, advice and guidance and access to a community of peers and experts can help. The DISCOVER solution is an innovative, broadly accessible solution that is focused upon improving the quality of life of carers and the older people they care for through learning, informational access and support. DISCOVER aims to improve the digital skills of domiciliary care workers, to give them the ability to pass on these digital skills to the older people they care for and to provide opportunities for both carers and older people to learn these skills together. DISCOVER will provide access to e-Learning for digital skills (both explicit and embedded into all elements of training), access to carer skills; access to care management tools and communications, including the ability to output to telehealth systems; or provide updates to family; and act as an access point to a range of existing information and support networks that exist in each country where DISCOVER is deployed.
Older People and Technological Inclusion: multidisciplinary perspectives on contemporary realities and aspirations. (K-10-004-CH)
The proposed research is a series of 4 academic seminars to explore the concept of technological inclusion in relation to age, and to establish a new interdisciplinary network of researchers including experienced researchers, PhD students and early-career researchers, older people and service providers, to critically examine how older people are positioned in the age of digital information. The first 3 seminars will take place at the OU; the final one at Bletchley Park.
Using rhythm for rehabilitation: evaluation of a novel haptic device
Cueing using metronomic rhythmic sensory stimulation has been shown to improve gait for stroke survivors, but most versions of this approach have used auditory and visual cues. In contrast, we have developed a prototype wearable system for rhythmic cueing based on haptics, which has been shown to be effective in pilot studies (Holland et al. 2015). The next step is to refine the current experimental system to make it practical for everyday use by therapists, and to determine what refinements will be needed for self-managed use by stroke survivors in the community. As part of this process we need to identify issues and requirements by drawing on the views of physiotherapists, nurses, interaction designers and stroke survivors. To apply the knowledge gained to improved design in an effective way we will use rapid prototyping - this involves building and modifying successive prototypes for representative users to interact with. This process goes through cycles, where user reactions feed into the next iteration of the prototype.
Publications
Book
New Directions in Music and Human-Computer Interaction (2019)
Book Chapter
Understanding Music Interaction, and Why It Matters (2019)
TMAP Design Cards for Technology-Mediated Audience Participation in Live Music (2019)
Machine Learning, Music and Creativity: An Interview with Rebecca Fiebrink (2019)
Haptics for the development of fundamental rhythm skills, including multi-limb coordination (2018)
Harmony and Technology Enhanced Learning (2016)
Exploring New Technologies through Playful Peer-to-Peer Engagement in Informal Learning (2015)
Video analysis for evaluating music interaction: musical tabletops (2013)
Song walker harmony space: embodied interaction design for complex musical skills (2013)
The Haptic Bracelets: learning multi-limb rhythm skills from haptic stimuli while reading (2013)
Music interaction: understanding music and human-computer interaction (2013)
Scratching the scale labyrinth (2011)
Whole body interaction in abstract domains (2011)
A first empirical study of direct combination in a ubiquitous environment (2005)
A Critical Analysis of Synthesizer User Interfaces for Timbre (2004)
Direct combination: a new user interaction principle for mobile and ubiquitous HCI (2002)
Artificial Intelligence in Music Education: A Critical Review (2000)
Learning about harmony with Harmony Space: an overview (1994)
MOTIVE: The development of an AI tool for beginning melody composers (1994)
Interface Design for Empowerment: a Case Study from Music (1992)
Journal Article
The role of nonlinear dynamics in interactions with digital and acoustic musical instruments (2020)
A small repeated measures study of haptic cueing upon gait symmetry in people after stroke (2017)
Empirically testing Tonnetz, voice-leading, and spectral models of perceived triadic distance (2016)
Testing a computational model of rhythm perception using polyrhythmic stimuli (2013)
A MIDI sequencer that widens access to the compositional possibilities of novel tunings (2012)
What can the language of musicians tell us about music interaction design? (2010)
Sense before syntax: a path to a deeper understanding of objects (2007)
An introduction to aspect-oriented music representation (2007)
AudioGPS: Spatial audio navigation with a minimal attention interface (2002)
Other
Preliminary report on the design of a constraint-based musical planner (1991)
Presentation / Conference
Proceedings of the CHIME Music and HCI Workshop 2023 (2023)
An agent for creative development in drum kit playing (2023)
Gait Rehabilitation for Neurological Conditions using Wearable Devices (2019)
Wearable Haptic Devices for Long-Term Gait Re-education for Neurological Conditions (2018)
Wearables for Long Term Gait Rehabilitation of Neurological Conditions (2018)
Designing for Diabetes Decision Support Systems with Fluid Contextual Reasoning (2018)
Design Implications for Technology-Mediated Audience Participation in Live Music (2017)
Two kinds of entrainment in gait rehabilitation using haptic metronomic cues (2017)
Characterizing the Landscape of Musical Data on the Web: State of the Art and Challenges (2017)
Designing a Highly Expressive Algorithmic Music Composition System for Non-Programmers (2016)
Wearable Haptic Devices For Post- Stroke Gait Rehabilitation (2016)
The Haptic Bracelets: Gait rehabilitation after Stroke (2016)
Questioning the Reflection Paradigm for Diabetes Mobile Apps (2016)
Rhythmic Haptic Cueing for Entrainment: Assisting post-stroke gait rehabilitation (2016)
Designing, Developing, and Evaluating the Future Internet of Personal Health (2016)
A pilot study using tactile cueing for gait rehabilitation following stroke (2015)
Prototyping a method for the assessment of real-time EEG sonifications (2015)
Towards bridging a gap in a musical live performance (2015)
Nonlinear Dynamical Systems as Enablers of Exploratory Engagement with Musical Instruments (2014)
Experimence: Considerations for Composing a Rock Song for Interactive Audience Participation (2014)
Eliciting Domain Knowledge Using Conceptual Metaphors: A Case Study from Music Interaction (2014)
Dynamical Interactions with Electronic Instruments (2014)
A cognitive dimensions analysis of interaction design for algorithmic composition software (2014)
Dynamical Systems in Interaction Design for Improvisation (2014)
A Gait Rehabilitation pilot study using tactile cueing following Hemiparetic Stroke (2014)
A review of real-time EEG sonification research (2013)
Kolab: appropriation & improvisation in mobile tangible collaborative interaction (2012)
We can work it out: towards a participatory approach to designing music interactions (2011)
Hex Player—a virtual musical controller (2011)
A novel user interface for musical timbre design (2010)
Asymmetrical Multi-User Co-operative Whole Body Interaction in Abstract Domains (2010)
Feeling the beat where it counts: fostering multi-limb rhythm skills with the haptic drum kit (2010)
Prototyping Whole Body Navigation of Musical Harmony (2009)
Evaluating musical software using conceptual metaphors (2009)
Running up Blueberry Hill: Prototyping whole body interaction in harmony space (2009)
Feel the force: Using tactile technologies to investigate the extended mind (2008)
Timbre space as synthesis space: towards a navigation based approach to timbre specification (2008)
Symmetric composition of musical concerns (2006)
Towards a mapping of timbral space (2005)
Applying aspect-oriented programming to music computing (2004)
Using Dynamic Aspects in Music Composition Systems (2004)
Smart homes and extended families (2004)
Applying Direct Combination to afford spontaneity in Pervasive Computing (2002)
AudioGPS: spatial audio in a minimal attention interface (2001)
Employing Object Technology to Expose Fundamental Object Concepts (1999)
Separable User Interface Architectures in Teaching Object Technology (1999)
Enhancing Interaction Scenarios with a Domain-Oriented Pattern Language (1999)
Exploiting Smalltalk Modules In A Customizable Programming Environment (1999)
Avoiding Object Misconceptions (1997)
The Object Shop – Using CD-ROM Multimedia To Introduce Object Concepts (1997)
The Joy Of Software – Starting With Objects (1996)
What is the Value of Virtual Reality for Conceptual Learning? Towards a Theoretical Framework (1996)
Understanding harmony and technology in music education (1995)
Pervasiveness of a Programming Paradigm: Questions Concerning an Object-oriented Approach (1994)
HCI and Music: Squaring the Circle (1993)
An AI Tool for the Analysis and Generation of Melodies (1992)
Virtual Worlds and their Role in Investigating Change in Cognitive Models of Motion (1992)
Visual Programming in Prolog (1991)
Two-dimensional Visual Programming and Three-Dimensional Execution Visualisation in Prolog (1991)
New Cognitive Theories of Harmony Applied to Direct Manipulation Tools for Novices (1987)
Report
Tutorial: An Introduction to AspectMusic (2006)
Synthesizer user interface design - lessons learned from a heuristic review (2004)
Audio Navigation: Using Spatial Audio In Ubiquitous Interfaces (2004)
Using Aspects to Help Composers (2003)
The application of Direct Combination to Mobile and Ubiquitous Human Computer Interaction (2002)
Thesis
Working Paper
Using rhythm for rehabilitation: evaluation of a novel haptic device (2017)
When is Direct Combination Useful? (2004)
A Simple Taxonomy of Search Reduction in Direct Combination (2004)
Learning about harmony with Harmony Space: an overview (Extended Technical Report) STAN-M-88 (1994)