MULTIMEDIA WRITING PROCEDURES FOR MUSICAL COMPOSITION: SEVEN CASE STUDIES PROCEDIMIENTOS DE ESCRITURA MULTIMEDIA PARA LA COMPOSICIÓN MUSICAL: SIETE ESTUDIO S DE CASO

: The purpose of this article is to describe the multimedia writing (MW) procedures that have proven to be effective in resolving some of the difficulties faced by composers in: 1 . notation of timbre, 2 . notation of new performance modes, 3 . notation of new vocal techniques, 4. notation of a tonal language in a score, 5. notation of the instructions for the use of the computer programs contained in the work, 6 . synchronization of the oral tradition musicians with the audio recordings, and 7. time reduction during the learning of the work. To address these issues, we used MW in six "experiments" —composition projects designed to answer specific questions about notation. These experiments and other subsequent projects allowed us to confirm our findings. In our experiments, MW was a set of procedures that allowed information to be represented in different ways simultaneously. The information can be text, sound, still or moving images, among others. These procedures can be used to record a message with the aim of saving it and transferring it from the composer to the performer. The results obtained demonstrated the effectiveness of MW in overcoming the current limitations of the Western music writing system (WMWS).


ABSTRACT:
The purpose of this article is to describe the multimedia writing (MW) procedures that have proven to be effective in resolving some of the difficulties faced by composers in: 1. notation of timbre, 2.
notation of new performance modes, 3. notation of new vocal techniques, 4. notation of a tonal language in a score, 5. notation of the instructions for the use of the computer programs contained in the work, 6.
synchronization of the oral tradition musicians with the audio recordings, and 7. time reduction during the learning of the work. To address these issues, we used MW in six "experiments" -composition projects designed to answer specific questions about notation.
These experiments and other subsequent projects allowed us to confirm our findings. In our experiments, MW was a set of procedures that allowed information to be represented in different ways simultaneously.
The information can be text, sound, still or moving images, among others. These procedures can be used to record a message with the aim of saving it and transferring it from the composer to the performer.
The results obtained demonstrated the effectiveness of MW in overcoming the current limitations of the Western music writing system (WMWS).
Keywords: multimedia writing, music notation, case studies, notation limitations.

INTRODUCTION
In this section, we recall the existence of a problem: the Western musical writing system (WMWS) and the multimedia notations used for musical composition, to this day, have limitations in notating certain aspects of music with sufficient precision (Covarrubias, 2016). The problem of the WMWS limitations is discussed extensively in several articles (Chemillier, 2000), (Chemillier, 2003), (François, 2013), (Scaldaferri, 2007), , [16]. For composers, the limitations of WMWS lead to difficulties in integrating into works elements that cannot be captured by this musical notation system (Covarrubias, 2013). Indeed, these limitations can lead to ambiguities in the text or very complex or overloaded, and sometimes also ambiguous, annotation instructions; this can cause many difficulties in the interpretation and performance of the work (François, 2013). Among all the elements difficult to notate using the WMWS, we have defined seven case studies: 1. the notation of timbre, 2. the notation of new performance modes, 3. the notation of new vocal techniques, 4. the notation of a tonal language in a score, 5. the notation of the instructions for the use of the computer programs contained in the work, 6. the synchronization of the oral tradition musicians with the audio recordings, and 7. the time reduction during the learning of the work (Covarrubias, 2016). This selection was made based on the author's notation needs as a composer, however, these notation difficulties with the WMWS have also been pointed out by other composers, ethnomusicologists, and musicologists¹. Through these cases, we can see that, in the field of musical composition, the WMWS and the types of multimedia notation available to this day are limited in capturing and accurately representing various elements that we wish to include in our compositions. That is why we ask: In the context of new compositions, how can we notate the elements that are not being unambiguously captured with the musical writing system and the multimedia notations that have been available so far, and which are then transmitted to the performer quickly?
This article is structured as follows: first, we analyze the limitations of the WMWS when it comes to capturing certain musical elements. Second, we report on the experiments carried out. Finally, we describe the specific MW procedures used to solve each difficulty.
¹ As for the timbre, the difficulties have been demonstrated by François (2013) and Traube (2015). For the inclusion of new performance modes in the work and the synchronization of the performer with a recording, the inaccuracies of WMWS have been described by Covarrubias (2016). For notation of new vocal techniques, the difficulties have been described by Gangard (1987). The drawbacks regarding the transmission of information to the performer for the use of the software included in the work and the irrelevance of instructions including still images and descriptions with written text are described by Covarrubias (2016). The problem of excessive time consumption to learn a new work containing new signs is addressed by Becker (1999) and François (2013).

EXPERIMENTS
To answer our research question, we conducted 6 experiments. Each experiment is a research-composition project where we use MW to annotate and integrate accurately into the work one or more of the elements described in the introduction. Each experiment aims to answer a set of sub-questions derived from the central research question and test a set of specific hypotheses² . The experiments or musical works are:

MW WRITING PROCEDURES IN MUSICAL COMPOSITION
The results of our research (Covarrubias, 2016) show the effectiveness of a set of procedures allowing the simultaneous use of several modes of representation of information, such as texts, sounds, still or moving images, codes, etc., used to record a message to preserve it and transmit it from the composer to the performer. In this section, we describe for each case study the multimedia writing procedures that have proved most effective for their notation.

The notation of timbre
The results of the experiments 2, La voz de Nangui and 6, Cantos de viento, demonstrate that the composer can notate the timbre thanks to multimedia writing much more precisely than with paper notation. In the same way, this type of writing also allows the transmission of information from the composer to the performer considerably faster. For the question "How to write timbre in musical composition?" we find that, although timbral notation can be done by writing music on paper, the use of MW is preferable because its notation offers many advantages. Thus, to notate timbre and then convey this information quickly and accurately to the performer, we demonstrate that MW allows timbral notation using the procedures described below.
3.1.1. The notation of an execution mode Regarding the notation of timbres obtained from the new performance modes, including video tutorials in the score has proved effective. The effectiveness of this method was ² Pour une Écriture Multimédia dans la Composition Musicale describes, for each experience, the stages of the procedure of composition and writing of the work in order to allow the reader to know the general characteristics of music and the writing procedures that have allowed the notation of the elements described above. Likewise, for each experiment, we present the results. ³ These experiments were conducted between 2012 and 2016 as part of a PhD project (Covarrubias, 2016 3.1.2. The imitation of an audio sample to obtain a vocal sound. Concerning the question "How can composers write music with timbres that they invented and then quickly transmit them to the performer?" the results of experiment 2, La voz de Nangui, show that in the context of musical composition, the notation of timbre in the score can be carried out by including audio samples for the performer to imitate. More specifically, the results of this experiment showed that a professional soprano singer with Western training was able to imitate 8 vocal sounds emitted with different vocal techniques (imaginary animal sounds)⁵. Compared to paper notation, the MW used in the score in this work eliminated ambiguities about the timbre to be obtained and provided the sound to be imitated; the audio file embedded in the score was a useful aural reference for the performer. Without the presence of the audio file in the score, the production of these sounds would not have been feasible since it is impossible to describe them with words or represent them with images. This technique is only valid for the imitation of vocal sounds. It is useful when the timbres produced by the performer must rigorously correspond to the timbres conceived by the composer.

Variation of sounds from a reference sound
In relation to the question "How can composers write music that includes timbres they invented and quickly transmit them to the performer?" the results of experiment 2, La voz de Nangui, show that a soprano was able to imitate the vocal sounds in the recordings included in the score and perform variations of the imitated sounds. The most effective method was as follows: 1. The singer learns to interpret a new tone by imitating the audio file of a vocal sound; 2. The singer associates the learned sound with a new graphic sign; and 3. The singer varies the pitch and duration of the sound in relation to changes in the sign, e.g., a widening of the sign implies a longer duration of the sound relative to the reference sound. The same method was also effective in experiment 6, Cantos de viento, in which the flutist modified the sounds learned from the video tutorials according to variations in the graphic sign. The results of experiment 3, Desorden Bipolar, show that a soprano was able to perform vocal techniques of ⁵ The 8 vocal sounds to be imitated are available at: http://www.sabinacovarrubias.com/la-voz-de-nangui.
In the score La voz de Nangui, the sounds to be imitated are also made by a female voice (the composer's voice). These sounds are described by analogies and metaphors; they are also represented by new signs.
blues, bolero, Mexican ranchera and rap using recordings of pieces of these genres as reference. The results of experiment 3 show that this method is only effective if the singers have familiarized themselves beforehand with the musical genres they will be asked to sing; this assumes that they have heard these genres in a way that they can recognize them. They may be part of their culture or their social environment.

A guide for a musician from a non-European tradition
For the question "How to include timbres obtained from the performance modes of oral tradition music and instruments?" the results of experiments 4 and 5 (Poly-Musique and Les mots du Tama, respectively) show that the inclusion of the performance of a musician from the oral tradition, playing their own instrument in the work, leads to the integration of a large number of timbres present in their musical system⁷. Consequently, it is relevant to the notion of timbre to include a musician from oral tradition when the composer wants to include instruments, timbres or performance techniques that cannot be played or produced by musicians from the written Western tradition. Performances by the oral tradition musician have been made possible by multimedia writing. The multimedia score served as a guide for the musician to learn the piece. This resource allowed the musician to play in a context different from that of their tradition. The musician was able to bring the timbres of their musical system to our music. Integrating a musician of oral tradition to include the timbres of their music into Western electroacoustic music has more advantages than transcribing the music of the oral tradition into Western writing or having Western musicians play the "complicated" techniques of other traditions. As far as timbre is concerned, we can ⁶ The audio examples included in the video tutorials are available at: http://www.sabinacovarrubias.com/ cantos-de-viento. ⁷ The results of experiments 4 and 5 show that a musician of oral tradition (a Senegalese griot) was able to perform electroacoustic music. In these two works the musician used a large number of different ways of playing the tama drum. The timbres obtained in the work Les mots du tama are the result of the sum of these numerous playing modes provided by the musician, and of the treatments made by the computer, such as filter treatments, granular synthesis and delay line.
foresee that a musician in the written tradition will never be able to master as many modes of performance in a short period of time as the master musician of the oral tradition.

The notation of new performance techniques
This section describes and analyzes the solutions we have found for integrating new performance modes into a musical score so that the work can be learned correctly in a limited time. In particular, we have found answers to the difficulty that arises when composers want to integrate into the work timbres produced from new performance techniques that they have invented. In this case, the problem is that the necessary movements to produce the performance technique cannot be precisely described or demonstrated by the texts and images that accompany the score and do it in a limited time⁸.

The transmission of knowledge through video tutorials
In experiment 6, the new performance techniques were given to the performer through video tutorials⁹. These videos reproduce a "one-to-one music lesson" scenario in which a teacher demonstrates the performance modes. The composer assumes the role of Let's say we want to give the player the necessary information to apply a new technique for playing an instrument. In fact, it is a question of teaching the movements to play an instrument. For this case, the representation on paper made from images is not adequate, unless the movement to be represented is "simple". ⁹ The results of experiment 6 of this research, Cantos de viento, show that, through multimedia writing, a professional recorder player is able to correctly execute 14 new performance modes invented by the composer on the recorder. The recorder player reproduces exactly all the movements requested by the composer. In this piece only the mouthpiece of an alto recorder is used and all playing modes are applied to this part of the recorder. The movements used in the playing modes involve many very subtle finger movements and very precise hand and wrist positions. Another advantage of MW appears in the results of the same experiment: the optimization of time during the learning of the work. The musician invested a total of 5 h 20 min in learning this work to perform it in the creation concert.
of animated and sound elements (see figure 1).
The procedure used to ask the interpreter to execute the new performance techniques in the work included the following steps: 1. Presentation of a new sign to the interpreter. 2.
Requesting the performer to press a button located above or next to this sign to be taken to the video corresponding to it. The action of pressing the button results in the display of a video. 3. The interpreter watches the video demonstrating how to execute the technique. 4. Repeat the above steps until all the signs included in the score have been presented. Once all the performance modes associated with the cues have been learned, the player can perform the piece.

The notation of new vocal techniques
This section describes and analyzes the solutions we have found for composers to integrate vocal techniques and timbres that they have invented. In particular, we have found an answer to the question that arises when the texts printed in the score are insufficient to show or describe precisely the new timbre and new techniques that the composer asks the performer to produce (Masse, 2001). We present two alternatives that allow the composer to integrate these elements into the musical work employing MW. The results of these experiments show that this type of writing allows the transmission of information from the composer to the performer in a limited period of time; although this can be achieved by writing music on paper, the use of MW brings advantages over this notation. Next, we present the two alternatives for integrating non-European vocal techniques that have proved effective in our experimental context.

Imitation of new reference sounds
Although the imitation of reference sounds is a practice already in use, so far in written music, it has been limited to the imitation of sounds previously known to the performer; for example, instruments, animal sounds, electronic sounds, belching, moaning. The integration of the audio files into the score allows us to ask the performer to imitate new sounds never heard before. To do this, we ask the singer to perform the following actions in this order: 1. We present a new graphic sign. 2. We show the sound that is associated with this sign. This recording can start playing, for example, by pressing a button below the sign to be interpreted. The interpreter is asked to associate the sound with the sign. 3. The interpreter learns to imitate the sound. 4. Once the interpreter is able to imitate the sound, they read a score and perform the learned sound every time the sign appears. It seems relevant to us to analyze the fact that if the vocal sound is well imitated, the singer will also be able to master the vocal technique associated with the timbre¹⁰.

Variation of reference sounds
This section presents the method that has proven to be effective in integrating non-European vocal techniques into a musical composition. This method, which includes MW in a score, offers a solution to the problems that arise when works containing passages that require the performer to master a non-European vocal technique.

The results of Experiment 3, Desorden Bipolar, show that a professional singer with
Western training can perform vocal techniques specific to bolero, Mexican ranchera, rap, and gospel-type blues, within a new work without having any previous experience in performing these techniques. In addition, the performer is able to learn the work in a limited amount of time. The method used to integrate these techniques into the work was as follows: 1. We asked the performer to listen to audio files containing entire songs from these musical genres. 2. We showed the performer a text and a series of photographs describing a "theatrical situation" and the emotional state associated with each musical genre. These texts and photographs make it clear that each musical genre is associated with an emotional state and that it is essential to experience it to be able to sing that musical genre. 3. We present in the score the indication "musical genre X", for example, bolero, ranchera, etc., above a musical passage so that it is sung with the technique, expressiveness, and theatrical elements associated with this genre. This method was effective as long as the singer was "culturally" familiar with the musical genres in question; that is, the singer was able to recognize them (to tell ¹⁰ It seems to us relevant to analyze the fact that the soprano is able to master new vocal techniques of the imitations of a voice timbre vocal tone without receiving any instruction or indication on how to perform these new techniques. We did not give the soprano any written or video instruction on movement, tongue position, lip position, mouth opening, etc. This was proposed by Traube (2015) and confirmed by the results of experiment 6, Cantos de viento, of this research. Thus, the production of a specific timbre includes the use of a performance technique that is inseparable from it. This implies that the successful production of a timbre by imitation may involve the acquisition of a skill that is the use of a specific technique. In this case, we see that this principle applies to timbre and vocal techniques, i.e., that successfully imitating a specific vocal timbre also allows to use the vocal technique which is inseparable from that timbre.
what genre they were when learning them) because they had already heard them in their cultural environment.

The notation of a tonal language in a score
The results of Experiment 2, La voz de Nangui, show that, thanks to the MW, a Westerntrained soprano accurately performed the "melody" of a Mazatec language text, namely a poem by the poet Gloria Martínez Carrera. The singer read the text in Mazatec but retained only the tones and vowels in this reading. The composer asked the singer to do it for musical purposes.

Synchronization of text-to-audio and graphic guides
To include the tones of Mazatec in the work La voz de Nangui, we applied the following method: 1. In the score of the work, the singer was asked to listen to the audio files¹¹ and then imitate only the tones, that is, the pitch or "melody of the language", and the vowels of the text. If the singer had difficulty doing this, it was necessary to repeat this step, but using the files slowed down to half the original speed¹². 2. The singer was asked to listen to audio files slowed down to half the normal speed and then imitate only the tones and vowels. 3. The singer was asked to use graphic guides synchronized with the audio recordings of the score to guide the soprano¹³.

The notation of instructions for the use of the computer programs contained in the work
This section describes and analyzes the solutions we have found to simplify the tasks of the sound technician or the musician in charge of installing and using the software included in the musical work. It can be a MaxMSP or Pure Data patch, or another type of program. In particular, we have found answers to the difficulty that arises when there is no specialized person to carry out these tasks. In this case, the difficulty lies in the fact that the person in charge of installing and manipulating the software throughout the work may take an inordinate amount of time to understand the instructions described on paper and to learn how to use the program; as a consequence, the work may not be performed correctly or may be unplayable in places where there is no specialist in computer-assisted music.
¹¹ The audio files are recordings of the poetry. They have been integrated into the score to support the learning of the work; there is a tab for each sentence of the poetry. The recordings of these sentences are available at : https://www.sabinacovarrubias.com/la-voz-de-nangui ¹² These audio files, embedded in the score, contain recordings of the phrases of the poetry which are played at half the normal speed, but keeping the original tones. This audio transformation has been achieved by stretching the time of the original files with the AudioSculpt application. It must be taken into account that these files are a support for the learning of the work.
¹³ These graphical guides included in the score are the results of fundamental frequency analyses applied to recordings of the poet's voice in Mazatec. The analyses carried out with the AudioSculpt software show a tone-time graph on a musical staff.

Video tutorials of the "screenshot" type
When it comes to explaining the operation of a program whose user interface exists on a computer screen, video tutorials of the "screen capture" type are an optimal medium for this purpose¹⁴. These materials show how to install and use the software that is part of the work. This solution allowed the performers to install the software for the piece Inner sounds for four guitarists and a computer, and it was useful for teaching the performers how to use it while performing the work. These concerts were rehearsed and performed without the presence of the composer; the only information contained in the score was sufficient for the recreation of the work. Likewise, this type of document also allowed the transmission of information from the composer to the sound technicians in a considerably faster way than writing on paper. Thus, to the question of how to record the instructions for using the software (a patch in this case) that contains the work, we answer that it is effective to give installation instructions, as well as a manual of the software in question, through the "screen capture" type videos that are integrated into the score.

The synchronization of musicians of oral tradition with audio recordings
A large number of mixed music works containing a fixed electroacoustic part require the synchronization of the performers with previously made recordings. The scores of these works were written on paper. To ensure the precise synchronization of the musicians with the soundtrack, many composers have included in the score, as a guide, the graphic representation of the sounds reproduced by the soundtrack that is part of the work. Although this method is effective, we present another writing alternative that offers more advantages regarding the precision of the interpretation and the reduction of the learning time of the work. Therefore, for the question, How can we synchronize oral tradition musicians with previously made audio recordings? The answer is: we can do it using MW. This type of notation has an advantage. The results of experiment 4, Polymusique, show that MW supports and allows the synchronization of oral tradition musicians with previously created audio or video files.

The sound guide
As for the integration of musicians from the oral tradition into mixed music works, this is possible thanks to the "sound guide" type of score. This is an audio file created on the base of a thorough knowledge of how the musical system of the oral tradition musicians works. This audio guide contains musical references from the musical tradition in question so that when the musicians recognize these elements, they know what and how to play.
¹⁴ The results of experiment 6 provide a solution to these difficulties: the inclusion of screenshot-type video tutorials embedded in the score.

The visual guide
In order to incorporate the oral musician into the mixed music piece, animated visual guides synchronized with the soundtrack were also effective. These guides helped the musician learn the piece and later, at the time of the performance, indicated the beginning and end of his interventions (Covarrubias, 2016).

The time reduction during the learning process of the work.
This section describes and analyzes the solutions we have found to reduce the time during the learning process of the work by the performer. In particular, we have found answers to the difficulty that arises when the notation contains new performance modes and the time to learn the work is "short". In this case, the score contains many new signs and is therefore accompanied by a lengthy instruction manual that defines each sign in terms of action. Also, the time required to learn the work is too much in relation to the number of rehearsal sessions available to the performer. The inclusion of the following elements in the score has helped reduce the time the performer spends learning the work. These elements are types of notation.

Video tutorials for knowledge transfer
The video tutorials helped to optimize the learning time of the works Inner Sounds and Cantos de viento. In the first work, the video tutorials, which include the teachings of a master of the oral tradition, allowed the guitarist to learn a performance mode of music from oral tradition. In Cantos de viento, the video tutorials allowed the flutist to learn how to perform 14 new performance modes invented by the composer using the mouthpiece of a recorder. These results suggest that the benefits of teaching through multimedia materials reported by Berk, (Berk, 2009) (Carrera, 2014). (i.e., "increased memorization, understanding of content, deeper learning") are also applicable to teaching performance modes through videos. Thus, we find that learning a skill (in this case, the new performance modes) with MW is faster than with a paper document, and that teaching a skill included in the score is also part of the writing of the piece.

Demonstrations and Examples
The

Reference Sounds
The reference sounds, contained in audio files, included in the score of La voz de Nangui allowed the singer to learn to interpret the sounds requested in the score; these sounds, practically impossible to transcribe or describe, are "imitations of imaginary animal sounds".

Screenshot Type Tutorials
The "screen capture" video tutorials allowed the technician¹⁵ to learn how to handle the installation of the libraries and the software containing the work, as well as the handling of the Max MSP patch during the performance of the work. This helped clarify the technical instructions and, consequently, reduce the rehearsal time of the work; there was only one rehearsal before the day of the concert.

Auditory and Visual Guides
The audio files, whose function is to "guide", allowed the musician of the oral tradition to play a piece of mixed music: Les mots du tama. In this case, the audio file is only listened to by the oral musician and not the public. This file, containing rhythms recognizable by the musician, guided the performance in the electroacoustic work.
The visual guides, i.e. the video files, were useful in the compositions Poly-Musique and Les mots du tama to indicate to the oral musician when to start performing and the moments of silence in the pieces. The musician did not perform any rehearsals before the day of the recording of the piece. The multimedia score was sufficient to perform the piece without rehearsal. The inclusion of these multimedia elements in the score

CONCLUSION
In the context of musical composition, multimedia writing (MW), as used here, has proven to be effective in annotating certain elements that cannot be annotated accurately with the Western music writing system (WMWS), written words, or fixed diagrams that are printed on the documents that usually accompany the score. The effectiveness of MW was evaluated in terms of its ability to retain and transmit to the performer the information written by the composer. Similarly, the effectiveness of MW was assessed in terms of the time the performers invested in reading and learning the work. Thus, compared to printed scores on paper, MW allowed much greater accuracy in the transmission of information and a reduction in the time needed to read and ¹⁵ During the realization of Inner sounds in Mexico, a musician-composer was in charge of installing the patch and handling it.
learn the work. The set of writing procedures described in this article can be applied to a large number of multimedia supports software, applications on mobile devices, PDF documents with embedded hyperlinks or HTLM documents. We consider it pertinent that composers who wish to incorporate these musical aspects into their works learn how to transmit knowledge through multimedia documents; also that they acquire multimedia writing skills, for example, recording and editing audio and video, building multimedia documents, such as building a web page. Therefore, we consider it relevant to design a didactic document that allows composers to learn how to use the writing procedures described in this article in the context of their works.