
Dr Paul Mulholland
Senior Research Fellow
Biography
Professional biography
Paul Mulholland's research is concerned with developing computational tools and methods to help people make sense of information, and develop and express their ideas. He has been a Principal Investigator or Co-Investigator on 12 national or European projects delivering outcomes including: technologies for the visitor use of archives and museum collections, mobile applications for formal and informal learning; automated narrative generation tools; and semantic and knowledge technologies for school and work-based learning. He has a first degree in psychology, a Masters degree in computing and PhD in cognitive science. He has published over 100 conference and journal articles. He is Deputy Editor of the International Journal of Human Computer Studies.
As part of the European Union Horizon 2020 SPICE project he is working on tools and methods to support Citizen Curation, in which citizens are supported in developing personal interpretations of cultural artefacts, sharing their own perspective and understanding the perspectives of others.
Within the European Union Horizon 2020 Polifonia project he is working on the use of visualisation, gesture and haptic interfaces to create novel ways of interacting with musical content.
Research interests
My research falls in the areas of technology enhanced learning, human computer interaction, museum informatics, digital humanities and music computing.
Projects
Polifonia: a digital harmoniser for Musical Heritage Knowledge
Polifonia implements a digital ecosystem for European Musical Heritage: music objects along with relevant related knowledge about their cultural and historical context, expressed in different languages and styles, and across centuries. The ecosystem will include methods, tools, guidelines, experiences, and creative designs, openly shared according to F.A.I.R. principles. The aim is to provoke a paradigm shift in musical heritage preservation, management, studying, interaction, and exploitation. Ten pilots, spanning from historical bells and organ heritage, classification of polyphonic notated music, to the historical role of music in children's lives, will drive the development of the ecosystem through continuous validation of its technologies. The Web, its standard formats and protocols are used as its reference architecture. Knowledge graphs are the enabling technology for integrating, representing, and interlinking music-related data with heterogeneous and distributed provenance. Dedicated research in Semantic Web, Data Science, Machine Learning, Language Technologies, and Human-Machine Interaction will enable discovery and automatic analysis of massive data, as well as their reuse for research, consumption and promotion. The project is conceived by an interdisciplinary team of passionate researchers and curators: computer scientists, anthropologists and ethnomusicologists, historians of music, linguists, musical heritage archivists, cataloguers and administrators, and creative professionals. They bring real-world use cases to define the ten pilots. The planned dissemination and exploitation actions allow the creation of a stakeholder network since the early stage of the project. Specific initiatives address societal and economic challenges: increased accessibility to musical heritage for people with disabilities; reproducible and sustainable creative designs for promoting musical heritage; increased engagement of young female students in STEM curricula.
Promoting Equity in Higher Education: A Socio-Technical Approach to AI Fairness
Our project is dedicated to advancing socio-technical approaches to fairness in Higher Education within the UK, with a specific focus on addressing bias and discrimination in AI-driven learning analytics systems. As technology increasingly shapes the student experience and educational outcomes, institutions worldwide are recalibrating their data strategies for positive impacts on student success. However, the integration of fairness, both technically and socially, remains an understudied aspect of these strategies. Leveraging the successful OUAnalyse system at the Open University, our project seeks to establish a framework of best practices, risks, and compliance with UK regulations. This initiative aims to guide Higher Education institutions in the UK as they develop their digital strategies, emphasising the core principles of fairness and responsible AI. The OU Analyse Dashboard (OUA) serves as our primary use case, a Predictive Learning Analytics System supporting over 200,000 students. Through weekly alerts to tutors, the system identifies students at risk of dropping out, contributing to a noteworthy increase in retention rates, especially for ethnic minority and socioeconomically disadvantaged students. Our project is organised around five fundamental pillars: Fairness, Explainability, Transparency, Accountability, and Social Justice. In the pursuit of fairness, we will conduct a comprehensive examination of PLA's performance across diverse student cohorts, paying special attention to intersectionality. Scrutinising prediction errors, and whether they affect differently to different student groups based on sensitive characteristics, such as gender, ethnicity, disability, Index of Multiple Deprivation, and previous qualifications, we aim to uncover biases and enhance overall system fairness. Transparency is addressed by providing students with predictions and explanations, piloting a student dashboard to make predictions visible and empowering students to take proactive measures in collaboration with tutors. In terms of accountability, we will investigate tutor reactions to predictions, ensuring equitable support for all students. The project will contribute to the broader discourse on ethical AI implementation in education, aligning with the UK government's innovation-focused approach to AI regulation. Collaboration with regulatory bodies, such as The Equality & Human Rights Commission and The Information Commissioner's Office, will ensure compliance of the provided framework with equality and human rights law, data protection, and sector-specific legislation. Beyond technical considerations, our commitment to Equality, Diversity, and Inclusion (EDI) is central. We actively address societal inequalities, striving to mitigate biases in PLA systems and reducing awarding gaps, ensuring fair and equal opportunities for all students. Our dissemination efforts encompass scientific conferences, collaboration with regulatory bodies, and engagement with higher education communities, fostering knowledge sharing and responsible AI practices. The project's potential economic impact lies in improving student investment returns, contributing to a socially inclusive educational landscape with diversified job opportunities. Success metrics include compliance with government priorities, regional impacts on other universities, and economic and social impacts on student quality of life and inclusion. Through this comprehensive approach, our project aims to reshape the AI in the education landscape, fostering fairness, transparency, and inclusivity.
Publications
Book
Book Chapter
Deep Learning to Encourage Citizen Involvement in Local Journalism (2022)
Facade-X: An Opinionated Approach to SPARQL Anything (2021)
Modelling museum narratives to support visitor interpretation (2016)
Weaving location and narrative for mobile guides (2013)
Video analysis for evaluating music interaction: musical tabletops (2013)
Song walker harmony space: embodied interaction design for complex musical skills (2013)
The Haptic Bracelets: learning multi-limb rhythm skills from haptic stimuli while reading (2013)
Music interaction: understanding music and human-computer interaction (2013)
Scripting personal inquiry (2012)
Whole body interaction in abstract domains (2011)
Pioneers, subcultures and cooperatives: The grassroots augmentation of urban places (2008)
Cognitive modelling and cognitive architectures (2005)
Ubiquity from the bottom up: Grassroots initiated networked communities (2005)
Community story exchange (2004)
A Critical Analysis of Synthesizer User Interfaces for Timbre (2004)
Supporting the exploration and use of heritage stories (2004)
Knowledge support for virtual modelling and simulation (2003)
Journal Article
Integrating Citizen Experiences in Cultural Heritage Archives with a Linked Non-Open Data Hub (2024)
Uncomfortable Revelations: can citizen curation widen access to museums? (2021)
Enabling multiple voices in the museum: Challenges and approaches (2020)
Introduction to the Special Issue on the 50th Anniversary of IJHCS (2019)
Supporting the discoverability of open educational resources (2019)
Detecting Important Life Events on Twitter Using Frequent Semantic and Syntactic Subgraphs (2016)
Personal inquiry: orchestrating science investigations within and beyond the classroom (2015)
Challenges in personalisation: supporting mobile science inquiry learning across contexts (2013)
nQuire: technological support for personal inquiry learning (2012)
Personal inquiry learning trajectories in geography: technological support across contexts (2011)
What can the language of musicians tell us about music interaction design? (2010)
Using netbooks to support mobile learners' investigations across activities and places (2010)
Personal inquiry: innovations in participatory design and models for inquiry learning (2010)
Pragmatic support for taxonomy-based annotation of structured digital documents (2009)
Using mobile phones to map online community resources to a physical museum space (2009)
Worlds and transformations: Supporting the sharing and reuse of engineering design knowledge (2007)
Re-using digital narrative content in interactive games (2007)
Introduction: contemporary perspectives on learning for work (2005)
CIPHER: Enabling Communities of Interest to Promote Heritage of European Regions (2002)
A Methodological approach to supporting organisational learning (2001)
Learning by building: A visual modelling language for psychology students (2000)
Integrating working and enrichment approach learning: a document (2000)
Supporting organisational learning: an overview of the ENRICH approach (2000)
Sharing engineering design knowledge in a distributed environment (2000)
An effective web-based software visualization learning environment (1998)
Teaching programming at a distance: the Internet software visualization laboratory (1997)
Presentation / Conference
Investigating Vividness Bias in Language Models Through Art Interpretations (2024)
Supporting the End-User Curation of Cultural Heritage Knowledge Graphs (2024)
CERSEI: Cognitive Effort Based Recommender System for Enhancing Inclusiveness (2023)
Musical Meetups: a Knowledge Graph approach for Historical Social Network Analysis (2023)
SPICE Linked Data Hub - Data infrastructure and tools to support Citizen Curation (2023)
An agent for creative development in drum kit playing (2023)
Deep Viewpoints: Scripted Support for the Citizen Curation of Museum Artworks (2023)
Edge-labelled graphs and property graphs - to the user, more similar than different (2022)
Edge-labelled graphs and property graphs - to the user, more similar than different (2022)
Introducing empathy into recommender systems as a tool for promoting social cohesion (2021)
Tourist recommendations with a touch of SPICE: A TRS with deep cultural understanding (2021)
Towards Advanced Interfaces for Citizen Curation (2020)
A comparison of the cognitive difficulties posed by SPARQL query constructs (2020)
Gait Rehabilitation for Neurological Conditions using Wearable Devices (2019)
Deriving Personas Based on Attitudes to Interruption and Information Overload (2019)
Wearable Haptic Devices for Long-Term Gait Re-education for Neurological Conditions (2018)
Wearables for Long Term Gait Rehabilitation of Neurological Conditions (2018)
Using SPARQL – the practitioners’ viewpoint (2018)
Characterizing the Landscape of Musical Data on the Web: State of the Art and Challenges (2017)
Designing a Highly Expressive Algorithmic Music Composition System for Non-Programmers (2016)
Identifying Important Life Events from Twitter Using Semantic and Syntactic Patterns (2016)
Identifying Prominent Life Events on Twitter (2015)
Making sense of description logics (2015)
Navigation strategies in the cityscape/datascape (2015)
Predicting Answering Behaviour in Online Question Answering Communities (2015)
Prototyping a method for the assessment of real-time EEG sonifications (2015)
Modelling Question Selection Behaviour in Online Communities (2015)
Automatic Identification of Personal Life Events in Twitter (2015)
Storyscope: Supporting the authoring and reading of museum stories using online data sources (2015)
Nonlinear Dynamical Systems as Enablers of Exploratory Engagement with Musical Instruments (2014)
Mobile technology to support coherent story telling across freely explored outdoor artworks (2014)
Eliciting Domain Knowledge Using Conceptual Metaphors: A Case Study from Music Interaction (2014)
Dynamical Interactions with Electronic Instruments (2014)
A cognitive dimensions analysis of interaction design for algorithmic composition software (2014)
Dynamical Systems in Interaction Design for Improvisation (2014)
Using ontologies: understanding the user experience (2014)
Cultural learning across the smart city (2014)
Modelling the meaning of museum stories (2013)
Exploring scholarly data with Rexplore. (2013)
An empirical perspective on representing time (2013)
Curation, curation, curation (2013)
Storyscope: using theme and setting to guide story enrichment from external data sources (2013)
QrAte: historical learning through a curatorial inquiry task using web resources (2012)
Storyspace: a story-driven approach for creating museum narratives (2012)
We can work it out: towards a participatory approach to designing music interactions (2011)
CHARACTER-EYES: story-driven inquiry from a character’s viewpoint (2011)
A novel approach to visualizing and navigating ontologies (2011)
An event-based approach to describing and understanding museum narratives (2011)
A “laboratory of knowledge-making” for personal inquiry learning (2011)
More notspots than hotspots: strategies for undertaking networked learning in the real world (2010)
Personal Inquiry: scripting support for inquiry learning by participatory design (2010)
A novel user interface for musical timbre design (2010)
Knowledge modelling to support inquiry learning tasks (2010)
Visual summaries of data: a spatial hypertext approach to user feedback (2010)
The challenge of supporting networked personal inquiry learning across contexts (2010)
Reasoning across multilingual learning resources in human genetics (2009)
Evaluating musical software using conceptual metaphors (2009)
Activity guide: an approach to scripted inquiry learning (2009)
Intelligent support for inquiry learning from images: a learning scenario and tool (2009)
Supporting location-based inquiry learning across school, field and home contexts (2008)
From design to narrative: the development of inquiry-based learning models (2008)
Exploring pathways across stories (2008)
Investigating the Effects of Exploratory Semantic Search on the Use of a Museum Archive (2008)
Investigating personal inquiry: An activity theory approach (2008)
Timbre space as synthesis space: towards a navigation based approach to timbre specification (2008)
Using ICT to support public and private community memories: case studies and lessons learned (2006)
Knowledge support of simulation model development by reuse (2006)
Semantic browsing of digital collections (2005)
Spotlight browsing of resource archives (2005)
Towards a mapping of timbral space (2005)
Grassroots initiated networked communities: A study of hybrid physical/virtual communities (2005)
Scene-driver: a narrative-driven game architecture reusing broadcast animation content (2004)
Story Fountain: Intelligent support for Story Research And Exploration (2004)
Knowledge support of cautious sharing of knowledge for simulation model assembly (2004)
Scene driver: reusing broadcast animation content for engaging, narratively coherent games (2004)
Supporting the sharing and reuse of modelling and simulation design knowledge (2003)
Methodology and tools to support storytelling in cultural heritage forums (2003)
Framework for Knowledge Support of Simulation in Engineering (2002)
A Cultural Heritage Forum Celebrating Technological Innovation at Station X (2002)
Using genre to support active participation in learning communities (2001)
Presentation / Conference Contribution
Report
PVC-Challenges Response: A student facing dashboard for online undergraduate students (2023)
Synthesizer user interface design - lessons learned from a heuristic review (2004)