Melodyne, the music editor that knows notes

Celemony's Melodyne recognises the notes in audio data and displays them on pitch and time grids. It allows you to make notes longer or shorter by dragging, change the pitch without changing the formant, duplicate them, etc, etc. You can even export audio to a MIDI file.

If you listen to a music piece that's been altered with Melodyne, it's very hard to tell the difference between the original sound and the one that's been changed. You can correct audio — especially lead vocals, but if my tests are anything to go on, everything else too — without people hearing it's been digitally modified in the end-result.

Melodyne music editor interface

Hence, Melodyne can be used to create second voices or entire ensembles from copies of the original track. You can also use it to create variation in repeated phrases, tidy up rhythms or use quantisation techniques to change them. Finally, you can pitch-shift and time-shift audio without the dreaded "Donald Duck" syndrome sounding through.

I tested Melodyne with spoken voice, cantatas and organ music from Bach, a part of the lamentations of the Virgin Mary by Caldara, one of Bach's Goldberg Variations (piano) and a part from the requiem by Fauré. I also tried it with modern lounge music and Kodo percussion. Instruments included the human voice, choirs, piano, pipe organ, symphonic orchestras and harp.

Celemony let me play with Melodyne Editor, the version that has polyphonic capabilities, and which comes both as a stand-alone app and DAW plug-in. Thanks to ReWire support, I could use Melodyne as a rack instrument in Reason. In Logic Pro, Melodyne works just like any other plug-in, except perhaps for the need to "Transfer" the audio from the DAW into the plug-in and then back again.

Melodyne music editor as Logic Pro plugin

I did most of my testing in the stand-alone version. The first thing you'll notice about Melodyne is that it's very simple to select and manipulate the notes it recognises. Polyphonic material is recognised using a unique algorithm, which goes under the acronym NDA. The notes can be split, stretched, their pitch can be changed, you can duplicate them, replace one note by another, and more.

You can load existing audio files in Melodyne or record new material directly to the editor. The recognition algorithm is automatically set to "percussion", "polyphonic" or "melodic", but any choice the app makes can be easily overruled. Sometimes the automatic recognition feature can't determine clearly between these modes and you might have to step in and select one yourself.

That's not really a problem if you're used to dealing with sound recordings. Of course, there's nothing stopping you from selecting an algorithm that doesn't match, just to be creative in what you're going to do with the notes afterwards. However, as notes are recognised based on the selected algorithm, the results may not be what you expected, unless you know darn well what you're doing.

As most sound, even when it sounds simple to our ears, is quite complex in its construction. Melodyne therefore can't be and is not perfect in its recognition of notes. It may "think" an overtone (click the link for a Wikipedia entry on overtones) is a note and vice versa. In order to correct for these 'errors', the app offers a working mode called Note Assignment. Note Assignment mode is meant to fine-tune Melodyne's analysis results. If you enter that mode notes of which the app wasn't sure during analysis show up as hollow notes. In this mode, you can assign wrongly recognised overtones to notes, separate notes where separations are due, change the range Melodyne will try to recognise and pull open note ends where individual notes may 'blend' into each other.

To help you with this task, you can have Melodyne play back the notes exclusively through its synthesizer — playing what it thinks it has 'heard' — rather than the original sound. Melodyne will also let you control the potential notes it sees using a simple slider. Finally, by broadening the range that Melodyne recognises to its maximum, you can even get a visual feedback on notes that aren't recognised but which you do hear.

Erroneous recognition of notes/overtones happened more frequently as the polyphonic material became more complex. Melodyne didn't have much problems with recognising percussion.

Note Assignment mode offers a very user-friendly correction paradigm: you simply double-click the hollow notes that you can hear have been recognised wrongly. The app will immediately show you your correction by switching the hollow note (wrongly seen as overtone) with the filled note (wrongly seen as note). Furthermore, as notes blend into each other, Melodyne on occasion will fail to see two different notes as individuals. You can separate them by just clicking over the note.

When all notes have been assigned properly, you can start working with pitch shifts, time stretching, etc. Here as well, Melodyne offers you more control than other editors would. For example, the pitch shift tool has pitch modulation and pitch drift subtools. The timing tool has an attack speed subtool, etc. With all of these tools you change the notes' characteristics by clicking and dragging. You get immediate feedback on what you did, both visually and sound-wise. An individual note can even be played by click-holding it with the mouse!

The visual feedback Melodyne offers is stunning. You can not just see notes, but also pitch curves on the note itself, its position on the scale, the musical notation, the fading of the sound as 'note tails', and more. During playback you can have Melodyne highlight each note that's being played in real-time.

Selecting notes is equally flexible. From the standard OS X methods to the Snake selection (where you paint over notes you want to select), it's all there to help you edit your sound.

Because of Melodyne's unique approach, you can easily copy and paste notes. One test I performed was to copy all notes in a short sequence and then paste all of them offset by a quarter note and lower pitch to quickly create a second voice. The creative possibilities are endless.

One of the more esoteric (to me, anyway) features of Melodyne is scale editing — indeed, this application allows you to change the musical scale you're working with. You can even launch an Open Scale window where all scales, from all continents are listed. You can select any of them for your audio, but also create your own. As I am not that well-versed into music and sound engineering, I didn't try any of these features.

Melodyne music editor scale detective

Even beyond that, there's till more to discover about Melodyne. Of all those features, I tried one that fascinated me: the ability to turn your audio into a MIDI file. This worked well provided all notes were in their right places. With some of the complex polyphonic material I tried, it was beyond my skills to get a correct MIDI file out of the audio, but given the skill and the time, you can create a MIDI file out of any audio with Melodyne.

<

p>Melodyne is a unique sound editor with unique capabilities. It requires a good deal of knowledge of sound and music to get the most out of it. For simple sound editing, I regard Melodyne as overkill, but if you want to work with notes that you can actually see as well as hear, there's nothing like it. My test version costs 399 EUR.

Comments

  1. Knute5 says

    I think of Melodyne (and Autotune) as Photoshop for audio. Pretty amazing technology, but if not used artfully it’s painfully obvious. Still learning to master it myself.