Musical performers spend many years achieving proficiency on their instruments. Newly-created digital musical instruments (DMIs) face a significant barrier to adoption in that few performers are willing to repeat these years of training to develop expertise on an unknown instrument. Without expert players, evaluating the success of a DMI design is challenging, and establishing its place in a broader musical community is nearly impossible. As a result, while many digital instruments have been created over the past decade, few have achieved lasting impact beyond the first few performances. This fellowship proposes a new approach to DMI design which repurposes the existing skills and experience of trained musicians, providing them with a rapid path to virtuosity without years of retraining. The research programme is organised around two complementary themes: study of performer-instrument interaction and creation of new instruments which capture the full richness and subtlety of virtuosic performance. First, models will be developed of the interaction between performer and instrument. Instrumental performance can be considered a special case of human-machine interaction which is interesting both for its complexity and for the common experience that the musical instrument becomes an extension of the body: while playing, the performer is often not consciously thinking about the instrument. Controlled experiments and participatory design exercises will establish how an instrument's design affects the development of expertise, and how existing expertise can be transferred to newly-created instruments. Second, the resulting models will be applied to the DMI creation process, taking a holistic approach unifying hardware design, digital signal processing, human-computer interaction (HCI) and artistic considerations. Existing DMIs often implicitly prioritise the convenience of the computer over the experience of the human player. On acoustic instruments, the entire physical object contributes to the sound, however subtly, but the choice of sensors in a DMI typically reduces the performer's actions to just a few machine-tractable dimensions. This fellowship will create instruments which deliberately oversample the interaction, using more sensors and higher sampling rates than apparently necessary, not to create a more complicated instrument, but rather to capture the subtle nuances that experts prize. Evaluation of the new DMIs will help refine the original models of performer-instrument interaction. This project focuses on musical interaction, but the principle of repurposing expertise is widely applicable within HCI. The capabilities of human users cannot be modelled only through generic cognitive and motor processes; real people have specialist skills developed over years of practice, and new technologies which connect to those skills are far more likely to find acceptance than ones which must be learned from scratch. Music is a good test case since instrumental training is widespread and reasonably standardised, but the findings will be relevant to other expert domains. This fellowship supports the time of the PI and two postdoctoral researchers. One postdoc will focus on performer studies and interaction, the other on digital signal processing and data mapping strategies. Close collaboration with musicians throughout the research will ensure its relevance to that community. This research integrates hardware design, digital signal processing, human-computer interaction, cognitive science, musicology and arts practice. The PI, with background and professional activities in music composition, electronic engineering and HCI, is ideally placed to lead this multidisciplinary project.

The project will achieve these objectives by addressing the following research questions: - How can Interdisciplinary Music be advanced beyond its current practice by collaborating with leading technological innovators? - How can recent developments in electronics instruments, motion sensors, AI, virtual reality, web design and interactive animation and other technological innovations be used by creative musicians in new ways to create ambitious new work? - How can musicians utilise specialist technological expertise in achieving more ambitious uses of film, analogue synthesisers and other existing technologies? - Can these technological innovations be applied to other arts and other industries? Cyborg Soloists will address these questions by creating a Centre for R&D in Interdisciplinary Music at Royal Holloway. The Centre will combine the resources of the Music, Media Arts, Drama, Electrical Engineering, and Computer Science departments. Internationally renowned musicians and technological innovators will be invited to create work that expands the field of interdisciplinary music that will then be toured to major international festivals and venues. The hub will host residencies for visiting artists, giving these bespoke teams the time and resources to tackle the technical and artistic challenges limiting the expansion of the field. The applicant will lead the projects, mentor the collaborators, and be one of the primary artists in most of the projects. Ethnographic documentation of the projects will be used to research the creative processes leading to new insights into musician-technology interactions. And the new and adapted technologies will have wide applications across musical genres, other performing arts, as well as having wider applications in human-computer interactions. By creating research teams between musicians and technological innovators, a range of ambitious, innovative new projects will be explored, focused on the following topic areas: Extending the Body Creating cyborg musicians through the use of motion sensors, 3D motion capture, and brainwave scanning tech. Remaking the Old Repurposed analogue technologies, expanding the possibilities of analogue synthesisers, and of common analogue technologies such as tape machines and turntables Film-Body interaction Compositions featuring live video interaction with the performer, using new films featuring performers and composers, and re-editing existing films Hyperinstruments Creating new musical instruments that extend existing ones with electronics. These would range from expansions of new digital instruments (such as Air Sticks), new surface-based sensors, instruments featuring in-built sensors, and new acoustic instruments using 3D-printing technology AI and new types of interaction with computers Artificial Intelligence to develop computerised co-performers (acting as chamber musicians) and create new types of visual interaction. Music with Virtual Reality Virtual reality to create musical works that expand on RHUL's StoryFutures project and Jonathan Packam's VR scores. Music using the Internet and Mobile Technologies Exploring the use of mobile phones for audience interaction, as well as website-based crowdsourced compositions Hybrid Installation Works Long form performances and works created for art spaces, facilitating new types of audience interaction. Staged Hybrid Works Inviting directors and theatre practitioners to create major works with composers and performers, featuring the technological innovations developed. Research Outputs from the project will including the 25 compositions, at least 10 new technological innovations developed with industry partners, performance tours by the applicant and invited collaborators, 12 journal articles, 12 conference papers, new software and hardware, 2 conferences, 2 festivals, and an edited book about new directions in interdisciplinary music.

EPSRC Centre for Doctoral Training in Digital Entertainment University of Bath and Bournemouth University The Centre for Digital Entertainment (CDE) supports innovative research projects in digital media for the games, animation, visual effects, simulation, cultural and healthcare industries. Being an Industrial Doctorate Centre, CDE's students spend one year being trained at the university and then complete three years of research embedded in a company. To reflect the practical nature of their research they submit for an Engineering Doctorate degree. Digital media companies are major contributors to the UK economy. They are highly-respected internationally and find their services in great demand. To meet this demand they need to employ people with the highest technical skills and the imagination to use those skills to a practical end. The sector has become so successful that the shortage of such people now constrains them from expanding further. Our Doctoral Training Centre is already addressing that and has become the national focus for this kind of training. We do this by combining core taught material with an exciting and unusual range of activities designed to challenge and extend the students' knowledge beyond the usual boundaries. By working closely with companies we can offer practical challenges which really push the limits of what can be done with digital media and devices, and by the people using them. We work with many companies and 40-50 students at any one time. As a result we are able to support the group in ways which would not be possible for individual students. We can place several students in one company, we can send teams to compete in programming competitions, and we can send groups to international training sessions. This proposal is to extend and expand this successful Centre. Major enhancements will include use of internationally leading industry experts to teach Master Classes, closer cooperation between company and university researchers, business training led by businesses and options for international placements in an international industry. We will replace the entire first year teaching with a Digital Media programme specifically aimed at these students as a group. The graduates from this Centre will be the technical leaders of the next generation revolution in this fast-moving, demanding and exciting industry.

The CDT in Media and Arts Technology will train PhD students to become skilled researchers and practitioners at the intersection of science, technology, digital media and the arts. The proposed CDT builds on the outstanding success of Queen Mary's current Media and Arts Technology (MAT) programme, introducing new training elements in Design, Innovation and Materials and expanded industrial and international partnerships. It addresses all 3 of EPSRCS's Digital Economy themes, particularly Digitally Connected Citizens and many ICT themes, especially Next Generation Interaction Technologies, Data to Knowledge and ICT for Manufacturing; Digital Healthcare. MAT is firmly grounded in Britain's Digital Economy (DE), which contributes the biggest share of GDP in any g20 nation and is projected to increase by a third by 2016. The Creative Digital sector in East London, on Queen Mary's doorstep and known as Tech City, is the fastest growing DE cluster in the UK, outstripping Greater London and the UK for jobs growth since 2001. It now accounts for 48,500 jobs in over 3200 companies, ranging from micro-business and SMEs to global players like Ustwo and Last.fm, and is attracting inward investment from international players such as IBM, Facebook, and Google. The Creative Digital sector demands workers with a high degree of technical skill coupled with creative skills, able to work in multi-disciplinary teams: exactly the type of graduate MAT will produce. The MAT CDT has an established network of over 40 external partners including: large companies (BBC, IBM, Orange, Sony and Procter & Gamble) health organisations (Royal Hospital of Neurodisability) and Tech City SMEs (Cinimod, Lean Mean Fighting Machine, Ustwo, Playgen, United Visual Artists, Hide&Seek, Troika), cultural institutions (Barbican, Science Museum and V&A), and governmental bodies (UKTI, TCIO, DSTL and London & Partners). Many partners host students' Advanced Placement Project, provide data sets and technical resources, supervision and mentoring, and exposure to a wide range of markets and audiences. The CDT acts as a focus bringing together otherwise disparate external bodies who discover shared interests and values. Because DE is a key strategic area for QML, the university invests heavily in the area. The existing MAT CDT catalysed the formation of qMedia, a cross-Faculty Research Centre based in the School of Electronic Engineering and Computer Science, and continues to be at its core. qMedia includes the world leading Centre for Digital Music, the newly formed Cognitive Science Group, the Multimedia and Vision Group, and members of the Networks, Vision and Antennas Groups. In EECS alone, qMedia has >40 academics, 41 RAs, 102 PhD students and a portfolio of grants with a current value of over £21 million. The CDT led to a major expansion in Digital Media research and teaching at Queen Mary. It inspired the creation of both a MSc in Media and Arts Technology and a BSc(Eng) in Multimedia and Arts Technology. The University invested around £3M in 200m2 of facilities for the MAT CDT, including Media and Arts Technology Studios, CDT hub (work/meeting space), 'maker' workshops, and a multimedia IT suite for audio/video editing. In conclusion, the existing CDT and its proposed renewal brings value nationally, locally and to the university. It is also a major international beacon of excellence that has led to several international partnerships, particularly in USA and China.

The digital games industry has global revenues of $65bn (in 2011) predicted to grow to $82bn by 2017. The UK is a major player, whose position at third internationally (behind the US and Japan) is under threat from China, South Korea and Canada. The £3bn UK market for games far exceeds DVD and movie box office receipts and music sales. Driven by technology advances, the industry has to reinvent itself every five years with the advent of new software, interaction and device technologies. The influential 2011 Nesta "Next Gen" review of the skills needs of the UK Games and Visual Effects industry found that more than half (58%) of video games employers report difficulties in filling positions with recruits direct from education and recommended a substantial strengthening of games industry-university research collaboration. IGGI will create a sustainable centre which will provide the ideal mechanism to consolidate the scientific, technical, social, cultural and cognitive dimensions of gaming, ensuring that the industry benefits from a cohort of exceptional research-trained postgraduates and harnessing research-led innovation to ensure that the UK remains at the forefront of innovation in digital games. The injection of 55+ highly qualified PhD graduates and their associated research projects will transform the way the games industry works with the academic community in the UK. IGGI will provide students with a deep grounding in the core technical and creative skills needed to design, develop and deliver a game, as well as training in the scientific, social, therapeutic and cultural possibilities offered by the study of games and games players. Throughout their PhDs the students will participate in practical industrial workshops, intensive game development challenges and a yearly industrialy-facing symposium. All students will undertake short- and longer-term placements with companies that develop and use games. These graduates will push the frontiers of research in interaction, media, artificial intelligence (AI) and computational creativity, creating new game-themed research areas at the boundaries of computer science and economics, sociology, biology, education, robotics and other fields. The two core themes of IGGI are: Intelligent Games - increasing the flow of intelligence from research into digital games. We will use research advances to seed the creation of a new generation of more intelligent and engaging digital games, to underpin the distinctiveness and growth of the UK games industry. The study of intelligent games will be underpinned by new business models and research advances in data mining (game analytics) which can exploit vast volumes of gameplay data. Game Intelligence - increasing the use of intelligence from games to achieve scientific and social goals. Analysis of gameplay data will allow us to understand individual behaviour and preference on a hitherto impossible scale, making games into a powerful new tool to achieve scientific and societal goals. We will work with user groups and the games industry to produce new genres of games which can yield therapeutic, educational and social benefits and use games to seed a new era of scientific experimentation into human behaviour, preference and interaction, in economics, sociology, psychology and human-computer-interaction. The IGGI CDT will provide a major advance in an area of great importance to the UK economy and massive impact on society. It will provide training for the leaders of the next generation of researchers, developers and entrepreneurs in digital games, forging economic growth through a distinctly innovative and research-engaged UK games industry. IGGI will massively boost the notion of digital games as a tool for scientific research and societal good.