Adding and updating Mutopia pieces

A rough guide which will be added to as I remember more things and as the build scripts improve.

NOTE that I would like to move the Java software to python, this will allow better integration and code-sharing with the website back end. A start has been made in mutopia.py but this is incomplete.

[NEW] - relevant to new pieces
[UPD] - relevant to updated pieces

Last update: 7 October 2011

Prerequisites

1. Checked out version of archive: Retrieve from github - see https://github.com/chrissawer/The-Mutopia-Project/ for checkout options. Ensure it is accessible under the MUTOPIA_BASE environment variable (see below).
2. Environment variables: The following environment variables need setting up.

   LILYPOND_BIN=~/lilypond/usr/bin/lilypond   [path to lilypond binary]
   MUTOPIA_BASE=~/Mutopia                     [path to checked out version of Mutopia archive]

3. Scripts: Should be on the PATH and CLASSPATH (eg. in .bashrc). I store them in MUTOPIA_BASE/UsefulScripts but this is not mandatory.
   • mutopia-clean.sh - Cleans MIDI/PDF (etc.) files out to start again
   • mutopia-combine.sh - Combines multiple PDF/MIDI files into zip files and generates RDF
   • mutopia-compile.sh - Compiles a piece from .ly file(s)
   • mutopia-history.sh - Generates .log file from git
   • Mutopia.java, MutopiaMaps.java, MutopiaPiece.java, MutopiaRDF.java, MutopiaRebuild.java - Java software to deal with .ly files, mostly header extraction and processing
   • mutopia.py - Start of a rewrite of Mutopia(etc.).java in Python

Checking the contribution or update

1. [NEW] Legal: Check that the source edition is listed and that it is not obviously a copyrighted edition. See the legal information page for more info. Modern Urtext editions are not acceptable. IMSLP references are fine as long as the piece is out-of-copyright throughout the world.
2. [UPD] Correction verification: Verify corrections with IMSLP - some we get sent are just plain wrong! Update piece as required including update to latest stable LilyPond syntax.
3. [NEW/UPD] Check for hard-coded paper/margins: Piece should not specify paper size! Unless it's landscape then the "landscape hack" comes in. If there are non-default margins check if they are necessary and that they work with both A4 and Letter paper sizes.
4. [NEW/UPD] Headers: Check for consistency of headers:

   java Mutopia -c piece-name.ly

   [UPD] Old .ly files will likely need fixing. See "RDF comparison check" below for some tips.

Compiling the contribution or update

1. [NEW] Update datafiles: If necessary, add the composer, intrument(s) or style to the relevant .dat file in MUTOPIA_BASE/datafiles
2. [NEW] Put .ly file in the right place: Under the composer directory, create the appropriate opus directory (if opus numbers exist), matching the format of other opus directories for that composer. Within this directory, create a subdirectory for the piece called piece-name, where the .ly file is piece-name.ly. If there are several .ly files then create another sub-directory within this called piece-name-lys, and put all the .ly, .ily (etc.) files in there, under the supplied directory structure if necessary.
3. [NEW/UPD] LilyPond version: The LilyPond \version should match exactly the version installed on your machine to compile the piece, as this field may be used to rebuild the piece in the future with the same LilyPond version.
4. [NEW] Allocate piece number: These are currently allocated by Chris for each piece. TODO: improve this.
5. [NEW/UPD] Tagline/footer: The tagline and the footer should be deleted and automatically updated as follows. The piece number is taken from the old footer. The tagline is completely rebuilt based on the piece's license and contributor details.

   java Mutopia -f piece-name.ly

6. [UPD] Clean directory: Clean out old PDF/MIDI files. Note this does NOT keep a backup. You should do this manually for checking the final result.

   mutopia-clean.sh

7. [NEW/UPD] Ready to compile: (re)build files:

   mutopia-compile.sh piece-name.ly

   For multiple files, change to the piece-name directory (not piece-name-lys) and run mutopia-compile.sh for each appropriate .ly file. You will need to decide which -preview.png file is the best one to use and rename it as piece-name-preview.png.
8. [NEW/UPD] Check PDF output: Check that the PDF output is sane, and that the paper sizes are correct.
   I use zathura and alt-tab switching between two PDFs to compare. Other PDF readers are available. :-)
9. [NEW/UPD] Build metadata: (re)build the metadata as follows. This is stored in RDF format which is perfectly human readable:

   mutopia-combine.sh piece-name        OR
   java Mutopia -r piece-name.ly        [simple files with single PDF/MIDI output only]

   NB. This parses the output of 'file piece-name-preview.png' to determine the preview PNG size. Unfortunately different versions of file give different output so if this fails you will need to change MutopiaPiece.java, ~line 126. TODO: Is there a better way of doing this?
10. [NEW] RDF sanity check: Have a quick look at the RDF to check nothing is obviously wrong.
11. [UPD] RDF comparison check: Diff RDF with previous one and check for differences.
    Obviously date and often PNG height/width will be different. Check other differences. Some old files put 'n/a' for lyricist and arranger, this is no longer done so leave them blank. If fields need adjusting please add or alter the 'mutopia' field in the .ly file (eg. mutopiatitle) rather than changing the original.
    We aim to be consistent with opus references for a given composer, this is usually "Op. 123" or "BWV 123". Also titles within a series, eg. WTK.

Adding the contribution or update

1. [NEW/UPD] Check in: If you have read/write git access, push to github. If not, send Chris a patch or contact him for read/write access.
   The commit message should be sensible, and include the name of the person who contributed the piece or correction. For new pieces I usually use:

   Initial version of piece_name
   (Contributor Name)

   For updates the message should be something along the lines of:

   Correct G->Gb in bar 10
   (Joe Bloggs)
   Update to LilyPond 2.14 syntax
   (Max Mustermann)

   We are not too fussy about the exact format of this message but it is best to put people's names in the commit itself to give credit for who has spotted the mistake.
2. [NEW/UPD] Update main server: Currently done by e-mailing Chris. TODO: Improve this.

TODO: Instructions for collections

TODO: Multi-LilyPond capable scripts need finishing

TODO: Improve support for automating compilation of multi-ly pieces