.. _usersGuide_26_iterators:

.. WARNING: DO NOT EDIT THIS FILE:
   AUTOMATICALLY GENERATED.
   PLEASE EDIT THE .py FILE DIRECTLY.


User's Guide, Chapter 26: Stream Iteration and Filtering
========================================================

We learned enough about streams in
:ref:`Chapter 6 <usersGuide_06_stream2>` to be able to get started,
but you've preservered and hopefully are ready to learn more about how
to get the most out of getting through a score. So this chapter will
delve deeper into the concept of iteration, that is, going through an
object one step at a time, and filtering out elements so only those in
classes or areas you want are found. Let's review and describe the
concept of iteration in Python (or most programming languages) for a
second.

Suppose you had a list like this:

.. code:: ipython3

    letterList = ['a', 'b', 'c']

Now you could get your ABCs out of it in this way:

.. code:: ipython3

    alphabet = ''
    alphabet += letterList[0]
    alphabet += letterList[1]
    alphabet += letterList[2]
    alphabet




.. parsed-literal::
   :class: ipython-result

    'abc'



But it's far easier, especially for a big list, to *iterate* over it
using a ``for`` loop:

.. code:: ipython3

    alphabet = ''
    for letter in letterList:
        alphabet += letter
    
    alphabet




.. parsed-literal::
   :class: ipython-result

    'abc'



We can *iterate* over a list because lists are *iterable* (or,
conversely, for the tautology department, because we can *iterate* over
a list, we call it *iterable*) and there are some functions and methods
that do great things on iterable objects, such as join them:

.. code:: ipython3

    ''.join(letterList)




.. parsed-literal::
   :class: ipython-result

    'abc'



Or give the minimum value from a numeric list:

.. code:: ipython3

    min([10, 20, 30, -3423, 40])




.. parsed-literal::
   :class: ipython-result

    -3423



Or give the length of an iterable:

.. code:: ipython3

    len(letterList)




.. parsed-literal::
   :class: ipython-result

    3



In Python, there's a special type of *iterable* object called a
*generator* which gives out objects as they are needed. One generator
that we have seen already is the ``range()`` function (called ``xrange``
in Python 2):

.. code:: ipython3

    zeroToFifty = range(51)
    zeroToFifty




.. parsed-literal::
   :class: ipython-result

    range(0, 51)



We can find the first number in that range that is divisible by 5:

.. code:: ipython3

    for n in zeroToFifty:
        print(n)
        if n != 0 and n % 5 == 0:
            break


.. parsed-literal::
   :class: ipython-result

    0
    1
    2
    3
    4
    5


At this point we've stopped going through the ``range`` object and no
more numbers are ever made or stored in memory -- this point doesn't
matter to much for a set of numbers up to 50, but for numbers up to
millions, or, as we will see, a repertory of scores of hundreds of
thousands of notes, saving a few seconds here and there really adds up.

Streams, as we have seen, are iterable:

.. code:: ipython3

    s = stream.Part(id='restyStream')
    s.append(note.Note('C#'))
    s.append(note.Rest(quarterLength=2.0))
    s.append(note.Note('D', quarterLength=1.5))
    s.append(note.Rest(quarterLength=1.0))
    
    for thing in s:
        print(thing, thing.quarterLength)


.. parsed-literal::
   :class: ipython-result

    <music21.note.Note C#> 1.0
    <music21.note.Rest rest> 2.0
    <music21.note.Note D> 1.5
    <music21.note.Rest rest> 1.0


When you iterate over a Stream, it is actually creating a lightweight
object called a ``StreamIterator`` to help make things easier. We can
create one directly by calling ``.iter`` on any stream:

.. code:: ipython3

    sIter = s.iter
    sIter




.. parsed-literal::
   :class: ipython-result

    <music21.stream.iterator.StreamIterator for Part:restyStream @:0>



This information tells us that ``sIter`` is an iterator going over the
``Part`` object with id ``restyStream`` and it is currently ready to
give out the first object, number 0. We can get the next thing in the
Stream by calling ``next()`` on the Stream. (In Python 2, this is a
little different, but you're using Python 3, right?)

.. code:: ipython3

    next(sIter)




.. parsed-literal::
   :class: ipython-result

    <music21.note.Note C#>



.. code:: ipython3

    next(sIter)




.. parsed-literal::
   :class: ipython-result

    <music21.note.Rest rest>



.. code:: ipython3

    sIter




.. parsed-literal::
   :class: ipython-result

    <music21.stream.iterator.StreamIterator for Part:restyStream @:2>



But for the most part, you'll want to use the built in way of going
through an iterable, that is, with a ``for`` loop:

.. code:: ipython3

    for el in sIter:
        print(el, el.quarterLength)


.. parsed-literal::
   :class: ipython-result

    <music21.note.Note C#> 1.0
    <music21.note.Rest rest> 2.0
    <music21.note.Note D> 1.5
    <music21.note.Rest rest> 1.0


Filtering elements in iteration
-------------------------------

So this does exactly what iterating directly on the Stream does -- but
it's good to know that a ``StreamIterator`` is silently being generated
so that you can see what else these Iterators do. Most importantly, a
``StreamIterator`` can add filters to it. Let's add a ``ClassFilter``
from the :ref:`moduleStreamFilters` module:

.. code:: ipython3

    restFilter = stream.filters.ClassFilter('Rest')
    sIter.addFilter(restFilter)
    for el in sIter:
        print(el, el.quarterLength)


.. parsed-literal::
   :class: ipython-result

    <music21.note.Rest rest> 2.0
    <music21.note.Rest rest> 1.0


Now when we go through sIter, we are only getting those objects that
match all of the filters on it. We can also filter by offset. Let's
create a new iterator and add an
:class:`~music21.stream.filters.OffsetFilter` to it.

.. code:: ipython3

    sIter2 = s.iter
    offsetFilter = stream.filters.OffsetFilter(offsetStart=0.5, offsetEnd=4.0)
    sIter2.addFilter(offsetFilter)
    for el in sIter2:
        print(el, el.offset)


.. parsed-literal::
   :class: ipython-result

    <music21.note.Rest rest> 1.0
    <music21.note.Note D> 3.0


And multiple filters can be chained together to get something more
powerful:

.. code:: ipython3

    sIter3 = s.iter
    sIter3.addFilter(restFilter)
    sIter3.addFilter(offsetFilter)
    for el in sIter3:
        print(el, el.offset)


.. parsed-literal::
   :class: ipython-result

    <music21.note.Rest rest> 1.0


Other filters that ``music21`` has in the :ref:`moduleStreamFilters`
include:

-  :class:`~music21.stream.filters.IsFilter` which returns elements
   that are exactly the same as the objects passed in (useful for
   getting the context of an object in a stream)

-  :class:`~music21.stream.filters.IsNotFilter`, even more useful, for
   getting everything but an object or list of objects

-  :class:`~music21.stream.filters.IdFilter` for finding items by Id.

-  :class:`~music21.stream.filters.ClassNotFilter` for finding items
   other than a list of classes.

-  and :class:`~music21.stream.filters.GroupFilter` for finding
   elements which have a particular group name.

Filter Shortcuts
~~~~~~~~~~~~~~~~

Filtering elements by offset or by class is so common, that ``music21``
has some shortcuts for adding filters to it, like this:

.. code:: ipython3

    sIter4 = s.iter
    sIter4.getElementsByClass('Rest')
    sIter4.getElementsByOffset(0.5, 4.0)
    for el in sIter4:
        print(el, el.offset)


.. parsed-literal::
   :class: ipython-result

    <music21.note.Rest rest> 1.0


Easier still, since each of these methods returns the original filter
object, you can chain them right in the for loop:

.. code:: ipython3

    sIter5 = s.iter
    for el in sIter5.getElementsByClass('Rest').getElementsByOffset(0.5, 4.0):
        print(el, el.offset)


.. parsed-literal::
   :class: ipython-result

    <music21.note.Rest rest> 1.0


And you can even skip the iterator step for the most common of these
filters, and ``music21`` will recognize what you want to do and create
the iterator for you:

.. code:: ipython3

    for el in s.getElementsByClass('Rest').getElementsByOffset(0.5, 4.0):
        print(el, el.offset)


.. parsed-literal::
   :class: ipython-result

    <music21.note.Rest rest> 1.0


The shortcut methods that ``music21`` exposes on Iterators include:

-  :meth:`~music21.stream.iterator.StreamIterator.getElementById`
   which adds an ``IdFilter``

-  :meth:`~music21.stream.iterator.StreamIterator.getElementsByClass`
   which adds a ``ClassFilter``

-  :meth:`~music21.stream.iterator.StreamIterator.getElementsByGroup`
   which adds a ``GroupFilter``

-  :meth:`~music21.stream.iterator.StreamIterator.getElementsByOffset`
   which adds an ``OffsetFilter``

And there are also properties (that is, written without parentheses)
which add certain filters:

-  :attr:`~music21.stream.iterator.StreamIterator.notes` which filters
   out everything but ``Note`` and ``Chord`` objects

-  :attr:`~music21.stream.iterator.StreamIterator.notesAndRests` which
   filters out everything except ``GeneralNote`` objects

-  :attr:`~music21.stream.iterator.StreamIterator.parts` which returns
   all the ``Part`` objects

-  :attr:`~music21.stream.iterator.StreamIterator.voices`
-  :attr:`~music21.stream.iterator.StreamIterator.voices` which
   returns all the ``Voice`` objects

-  :attr:`~music21.stream.iterator.StreamIterator.spanners` which
   returns all the ``Spanner`` objects

Custom Filters
~~~~~~~~~~~~~~

Creating your own filter is pretty easy too. The easiest way is to
create a function that takes in an element and returns True or False
depending on whether the object matches the filter.

We will create a filter to see if the element has a ``.pitch`` attribute
and then if that pitch attribute has a sharp on it:

.. code:: ipython3

    def sharpFilter(el):
        if (hasattr(el, 'pitch') 
                and el.pitch.accidental is not None
                and el.pitch.accidental.alter > 0):
            return True
        else:
            return False
        
    sIter6 = s.iter
    sIter6.addFilter(sharpFilter)
    
    for el in sIter6:
        print(el)


.. parsed-literal::
   :class: ipython-result

    <music21.note.Note C#>


.. note::
   Before `music21` v.4.0.6, filters must take in two arguments, the second one, being the iterator object itself,
   can be ignored.

Recursive and Offset Iterators
------------------------------

``Music21`` comes with two other iterators that let you do powerful
operations. The most commonly used is the
:class:`~music21.stream.iterators.RecursiveIterator` which burrows
down into nested Streams to get whatever you want. Let's load in a
nested stream:

.. code:: ipython3

    bach = corpus.parse('bwv66.6')
    for thing in bach:
        print(thing)


.. parsed-literal::
   :class: ipython-result

    <music21.metadata.Metadata object at 0x119b0e7f0>
    <music21.stream.Part Soprano>
    <music21.stream.Part Alto>
    <music21.stream.Part Tenor>
    <music21.stream.Part Bass>
    <music21.layout.StaffGroup <music21.stream.Part Soprano><music21.stream.Part Alto><music21.stream.Part Tenor><music21.stream.Part Bass>>


Right, we remember that often the actual notes of a piece can be hidden
inside Parts, Measures, and Voices. A recursive iterator gets to them,
and they're created by calling ``recurse()`` on a stream.

.. code:: ipython3

    recurseIter = bach.recurse()
    recurseIter




.. parsed-literal::
   :class: ipython-result

    <music21.stream.iterator.RecursiveIterator for Score:0x119b0e6d8 @:0>



Let's add a filter for only E#s to it, and look into it. Instead of
checking to see if each element has a ``.name`` attribut we'll put a
``try...except`` clause around it, and if it does not have the ``.name``
attribute (and thus raises and ``AttributeError`` we will return False.

.. code:: ipython3

    def eSharpFilter(el):
        try:
            if el.name == 'E#':
                return True
            else:
                return False
        except AttributeError:
            return False
    
    recurseIter.addFilter(eSharpFilter)
    
    for el in recurseIter:
        print(el, el.measureNumber)


.. parsed-literal::
   :class: ipython-result

    <music21.note.Note E#> 9
    <music21.note.Note E#> 3
    <music21.note.Note E#> 7
    <music21.note.Note E#> 7
    <music21.note.Note E#> 2
    <music21.note.Note E#> 6


Note that the measure numbers don't keep increasing. That's because the
recurse iterator finishes one part before returning to the next. We can
use the fancy ``.getContextByClass`` to figure out what part it is in:

.. code:: ipython3

    for el in recurseIter:
        pId = el.getContextByClass('Part').id
        print(el, el.measureNumber, pId)


.. parsed-literal::
   :class: ipython-result

    <music21.note.Note E#> 9 Soprano
    <music21.note.Note E#> 3 Alto
    <music21.note.Note E#> 7 Alto
    <music21.note.Note E#> 7 Tenor
    <music21.note.Note E#> 2 Bass
    <music21.note.Note E#> 6 Bass


(as an aside, ``.measureNumber`` is just a shortcut for
``.getContextByClass('Measure').number``, so we are actually looking up
two contexts)

Another useful iterator is the OffsetIterator, which returns lists of
elements grouped by offset. Let's add some more things to our Stream
before we see how it works.

.. code:: ipython3

    s.insert(0, clef.TrebleClef())
    s.insert(0, key.KeySignature(3))
    s.insert(1, instrument.Trumpet())
    
    # normal iterator
    for el in s:
        print(el, el.offset)


.. parsed-literal::
   :class: ipython-result

    <music21.clef.TrebleClef> 0.0
    <music21.key.KeySignature of 3 sharps> 0.0
    <music21.note.Note C#> 0.0
    Trumpet 1.0
    <music21.note.Rest rest> 1.0
    <music21.note.Note D> 3.0
    <music21.note.Rest rest> 4.5


Unlike with the normal ``StreamIterator`` or the ``RecursiveIterator``,
there is no method on ``Stream`` to create an offset iterator, so we
will create one directly:

.. code:: ipython3

    oIter = stream.iterator.OffsetIterator(s)
    for elementGroup in oIter:
        print(elementGroup[0].offset, elementGroup)


.. parsed-literal::
   :class: ipython-result

    0.0 [<music21.clef.TrebleClef>, <music21.key.KeySignature of 3 sharps>, <music21.note.Note C#>]
    1.0 [<music21.instrument.Trumpet 'Trumpet'>, <music21.note.Rest rest>]
    3.0 [<music21.note.Note D>]
    4.5 [<music21.note.Rest rest>]


From Iterator to Stream
-----------------------

From either a ``StreamIterator`` or a ``RecursiveIterator`` a new
``Stream`` object can be generated by calling ``.stream()`` on it. On a
``RecursiveIterator``, this does not put the elements into substreams.

.. code:: ipython3

    onlyESharps = bach.recurse().addFilter(eSharpFilter)
    esharpStream = onlyESharps.stream()
    esharpStream.show('text')


.. parsed-literal::
   :class: ipython-result

    {8.0} <music21.note.Note E#>
    {10.0} <music21.note.Note E#>
    {23.0} <music21.note.Note E#>
    {25.5} <music21.note.Note E#>
    {27.0} <music21.note.Note E#>
    {34.5} <music21.note.Note E#>


.. code:: ipython3

    esharpStream.derivation




.. parsed-literal::
   :class: ipython-result

    <Derivation of <music21.stream.Score 0x119de7a58> from <music21.stream.Score 0x119b0e6d8> via "eSharpFilter">



This can be useful if you'd like to do plots on the resulting stream,
though this one is a bit too obvious...

.. code:: ipython3

    esharpStream.plot('pitchclass')



.. image:: usersGuide_26_iterators_62_0.png
   :width: 363px
   :height: 382px


But maybe this one could tell someone something:

.. code:: ipython3

    esharpStream.plot('pianoroll')



.. image:: usersGuide_26_iterators_64_0.png
   :width: 587px
   :height: 382px


Perhaps not. But iterators are not the main point -- what you can do
with them is more important, so we will return to working with actual
musical objects in :ref:`Chapter 27 <usersGuide_27_graceNotes>` when
we talk about Grace Notes.