One of music21’s important features is its capability to help users examine large bodies of musical works, or corpora. Music21 provides tools for extracting useful metadata - the key signatures used in each piece, their durations, ambitus and so forth - into collections called metadata bundles. It also allows users to search those bundles, and persist them disk for later research.
Music21 works with three categories of corpora, made explicit via the corpus.Corpus abstract class.
The first category is the core corpus, a large collection of musical works packaged with most music21 installations, including many works from the common practice era, and inumerable folk songs, in a variety of formats:
>>> coreCorpus = corpus.CoreCorpus()
>>> len(coreCorpus.getPaths())
3045
Note
If you’ve installed a “no corpus” version of music21, you can still access the core corpus with a little work. Download the core corpus from music21’s website, and install it on your system somewhere. Then, teach music21 where you installed it like this:
>>> coreCorpus.manualCoreCorpusPath = 'path/to/core/corpus'
Music21 also has the notion of a virtual corpus: a collection of musical works to be found at various locations online which, for reasons of licensing, haven’t been included in the core corpus:
>>> virtualCorpus = corpus.VirtualCorpus()
>>> virtualCorpus.getPaths()[0]
'http://kern.ccarh.org/cgi-bin/ksdata?l=cc/bach/cello&file=bwv1007-01.krn&f=xml'
Finally, music21 allows for local corpora: bodies of works provided and configured by individual music21 users for their own research. Local corpora behave identically to the core and virtual corpora, and can be searched and cached in the same manner:
>>> localCorpus = corpus.LocalCorpus()
You can add and remove paths from a local corpus with the addPath() and removePath() methods:
>>> localCorpus.addPath('~/Desktop')
>>> localCorpus.directoryPaths
('/Users/josiah/Desktop',)
>>> localCorpus.removePath('~/Desktop')
In addition to the default local corpus, music21 allows users to create and save as many named local corpora as they like, which will persist from session to session.
Let’s create a new local corpus, give it a directory to find music files in, and then save it:
>>> aNewLocalCorpus = corpus.LocalCorpus('A new corpus')
>>> aNewLocalCorpus.existsInSettings
False
>>> aNewLocalCorpus.addPath('~/Desktop')
>>> aNewLocalCorpus.directoryPaths
('/Users/josiah/Desktop',)
>>> aNewLocalCorpus.save()
>>> aNewLocalCorpus.existsInSettings
True
We can see that our new local corpus is saved by checking for the names of all saved local corpora:
>>> corpus.LocalCorpus.listLocalCorporaNames()
[None, u'trecento', 'A new corpus', u'bach', u'fake']
Note
When running listLocalCorporaNames(), you will see None - indicating the default local corpus - along with the names of any non-default local corpora you’ve manually created yourself. In the above example, a number of other corpora have already been created.
Finally, we can delete the local corpus we previously created like this:
>>> aNewLocalCorpus.delete()
>>> aNewLocalCorpus.existsInSettings
False
In music21, metadata is information about a score, such as its composer, title, initial key signature or ambitus. A metadata bundle is a collection of metadata pulled from an arbitrarily large group of different scores. Users can search through metadata bundles to find scores with certain qualities, such as all scores in a given corpus with a time signature of 6/8, or all scores composed by Monteverdi.
There are a number of different ways to acquire a metadata bundle. One is to access the metadataBundle attribute of any Corpus instance to get its corresponding metadata bundle:
>>> coreCorpus.metadataBundle
<music21.metadata.bundles.MetadataBundle 'core': {14956 entries}>
Music21 also provides a handful of convenience methods for getting metadata bundles associated with the virtual, local or core corpora:
>>> coreBundle = metadata.MetadataBundle.fromCoreCorpus()
>>> localBundle = metadata.MetadataBundle.fromLocalCorpus()
>>> otherLocalBundle = metadata.MetadataBundle.fromLocalCorpus('blah')
>>> virtualBundle = metadata.MetadataBundle.fromVirtualCorpus()
We strongly recommend using the above from*() methods. Some of these metadata bundles can become quite large, and methods like fromCoreCorpus() will cache the metadata bundle in memory once it has been read from disk, potentially saving you a lot of time.
You can also make metadata bundles manually, by passing in the name of the corpus you want the bundle to refer to, or an actual Corpus instance itself:
>>> coreBundle = metadata.MetadataBundle('core')
>>> coreBundle = metadata.MetadataBundle(corpus.CoreCorpus())
However, you’ll need to read the bundle’s saved data from disk before you can do anything useful with the bundle. Bundles don’t read their associated JSON files automatically when they’re manually instantiated.
>>> coreBundle
<music21.metadata.bundles.MetadataBundle 'core': {0 entries}>
>>> coreBundle.read()
<music21.metadata.bundles.MetadataBundle 'core': {14956 entries}>
That’s a lot of information! Now let’s see what we can do with it ...
When you search metadata bundles, music21 examines each metadata object in the entire bundle and attempts to match your search string against the contents of the various search fields saved in that metadata object. Just as with creating metadata bundles, there are few different ways to search them.
You can use corpus.search() to search the metadata associated with all known corpora, core, virtual and even each local corpus:
>>> corpus.search('6/8')
<music21.metadata.bundles.MetadataBundle {2211 entries}>
You can also search against a single Corpus instance, like this:
>>> corpus.CoreCorpus().search('6/8')
<music21.metadata.bundles.MetadataBundle {2211 entries}>
Finally, if you already have a reference to a metadata bundle, you can search there too:
>>> bachBundle = coreBundle.search('bach', 'composer')
>>> bachBundle
<music21.metadata.bundles.MetadataBundle {21 entries}>
Because the result of every metadata search is also a metadata bundle, you can search your search results!
>>> bachBundle.search('3/4')
<music21.metadata.bundles.MetadataBundle {4 entries}>
When you search metadata bundles, you can search either through every search field in every metadata instance, or through a single, specific search field. For example, searching for “bach” as a composer renders different results from searching for the word “bach” in general:
>>> coreBundle.search('bach', 'composer')
<music21.metadata.bundles.MetadataBundle {21 entries}>
>>> coreBundle.search('bach', 'title')
<music21.metadata.bundles.MetadataBundle {20 entries}>
>>> coreBundle.search('bach')
<music21.metadata.bundles.MetadataBundle {150 entries}>
So what fields can we actually search through? You can find out like this:
>>> for field in corpus.Corpus.listSearchFields():
... field
...
'alternativeTitle'
'composer'
'date'
'keySignatureFirst'
'keySignatures'
'localeOfComposition'
'movementName'
'movementNumber'
'noteCount'
'number'
'opusNumber'
'pitchHighest'
'pitchLowest'
'quarterLength'
'tempoFirst'
'tempos'
'timeSignatureFirst'
'timeSignatures'
'title'
Now that we know what all the search fields are, we can search through some of the more obscure corners of the core corpus:
>>> corpus.CoreCorpus().search('taiwan', 'locale')
<music21.metadata.bundles.MetadataBundle {27 entries}>
You can use a predicate callable as a query too, although make sure to construct your predicates with care:
>>> predicate = lambda x: 5000 < x
>>> corpus.CoreCorpus().search(predicate, 'noteCount')
<music21.metadata.bundles.MetadataBundle {23 entries}>
Now let’s take a closer look at some search results:
>>> bachBundle = corpus.CoreCorpus().search('bach', 'composer')
>>> bachBundle[0]
<music21.metadata.bundles.MetadataEntry: bach_choraleAnalyses_riemenschneider014_rntxt>
Metadata bundles are composed of metadata entries. These entries allow us to associate a given RichMetadata object with a file name, and also allow us to parse the associated file into a music21 score:
>>> bachBundle[0].sourcePath
u'bach/choraleAnalyses/riemenschneider014.rntxt'
>>> bachBundle[0].metadataPayload
<music21.metadata.RichMetadata object at 0x112f54250>
>>> bachBundle[0].parse()
<music21.stream.Score 4421475216>
Another useful feature of music21’s metadata bundles is that they can be operated on as though they were sets, allowing you to union, intersect and difference multiple metadata bundles, thereby creating more complex search results:
>>> beethovenBundle = corpus.search('beethoven', field='composer')
>>> beethovenBundle
<music21.metadata.bundles.MetadataBundle {16 entries}>
>>> bachBundle.union(beethovenBundle)
<music21.metadata.bundles.MetadataBundle {37 entries}>
Consult MetadataBundle‘s API for a more in depth look at how this works.
Metadata bundles can be written to and read from disk. They can also be completely rebuilt:
>>> coreBundle = metadata.MetadataBundle('core')
>>> coreBundle.read()
>>> coreBundle.write()
To add information to a bundle, use the addFromPaths() method:
>>> newBundle = metadata.MetadataBundle()
>>> paths = corpus.CoreCorpus().getBachChorales()
>>> failedPaths = newBundle.addFromPaths(paths)
>>> failedPaths
[]
>>> newBundle
<music21.metadata.bundles.MetadataBundle {402 entries}>
Note
Building metadata information can be an incredibly intensive process. For example, building the core metadata bundle can easily take as long as an hour! Please use caution, and be patient, when building metadata bundles from large corpora. To monitor the corpus-building progress, make sure to set ‘debug’ to True in your user settings:
>>> environment.UserSettings()['debug'] = True
You can delete, rebuild and save a metadata bundle in one go with the rebuild() method:
>>> virtualBundle = metadata.MetadataBundle.fromVirtualCorpus()
>>> virtualBundle.rebuild()
To delete a metadata bundle’s cached-to-disk JSON file, use the delete() method:
>>> virtualBundle.delete()
Deleting a metadata bundle’s JSON file won’t empty the in-memory contents of that bundle. For that, use clear():
>>> virtualBundle.clear()