Background

It was back in 2005 that Cakewalk first added Roland V-Vocal to SONAR 5 for integrated audio stretching and pitch correction. While VVocal was an exceptional tool in its time and is still used by many for its ability to do fine editing of a vocal performance, the industry has advanced since then and companies like Celemony with their dedicated focus in this area have made great strides. A couple of years ago Celemony released their ARA SDK to integrate Melodyne integration into a DAW host application. Since VVocal’s integration in SONAR is similar in principle to what ARA does, it was natural for us to consider integrating ARA technology into SONAR.

 

What is ARA?

ARA (Audio Random Access) is a general protocol that allows host integration with any audio edit capable plugin such as Melodyne. Technically speaking ARA itself is not a plugin protocol. It is a protocol that allows the host and ARA capable plugin to transfer parameters and audio region data back and forth either offline or in realtime.

ARA support is exposed by the plugin as an extension to the VST3 interface. However very little is done at the VST level besides streaming the plugin audio output back into the host engine signal flow graph. The bulk of the work is handled by the custom ARA interfaces. This is because unlike normal VST plugins, ARA requires random access lookahead to audio on the timeline, in response to the user re-sequencing, time stretch or otherwise mangling the audio data in the editor user interface. In a sense each ARA plugin works like a dedicated sequencer for its audio region!

 

Region FX

SONAR provides several ways to add effects to audio data. You can add plug-in effects to a track effects bins to process the entire track. You can also add plug-ins to clip effects bins to only process individual clips. For even greater precision, you can apply special processing to specific audio regions. These effects are called Region FX.

Any ARA-compatible VST3 audio plug-in will be available as a Region FX in SONAR, and will be listed in the Track view Region FX menu.

To create a Region FX simply select a region of audio and press CTRL-M or choose the specific region effect from the region FX menu. SONAR converts the selected region to a new special Region FX clip.

 

Both ARA capable plugins as well as V-Vocal can be region based effects. If you have V-Vocal installed on your system, it will additionally also be listed in the region FX menu and can be used as before.

NOTE: As of X3, VVocal is no longer distributed so if you need it in an older project you will have to install it via your SONAR X2 installation disks.

 

You can additionally have normal clip effects on a Region FX clip. A Region FX appears right before the clip effects bin in the signal chain.

 

 

NOTE:  Although the signal flowchart above shows clip level operations prior to the RegionFX, all of these are committed at the time of creation of the ARA region FX. Once the ARA region has been created you cannot add clip gain envelopes or manipulate any Pre-ARA audio clip data, and the user interface will block operations like clip gain and clip fades. The reason for this is the ARA data is committed at creation time and all ARA level edits are relative to the committed region data. You can however, make post clip FX bin changes such like clip FX and clip FX envelopes.

 

Melodyne ARA Capabilities

 

Melodyne ARA supports the following core features:

• DNA engine – pitch detection and editing. (polyphonic pitch detection only in editor version, mono in essential)

• Pitch correction

• Time Stretching

• Audio editing and re-sequencing. Blobs in Melodyne can be copied and pasted or reordered arbitrarily in time allowing creative recycling of audio material

• MIDI extraction from audio

Below is a video showing the basic workflow with Melodyne in SONAR X3.

 

Benefits of ARA

Users of Melodyne will be familiar with the tedium of having to manually transfer the audio for the region in real time into Melodyne. Not only do you have to do this for every clip that you wish to process in Melodyne, but doing operations like time stretching in Melodyne is very non intuitive, since you cannot use the hosts standard editing tools. Another downside is that since the plugin has does not have access to the host, it must manage the audio data recorded independently from the host.

The main benefit of ARA is that it allows SONAR to integrate with plug-ins that can edit and analyze audio, such as Melodyne.  ARA-compatible plug-ins have random access to the audio data since they can also re-sequence the audio. Without ARA, a normal plug-in only receives audio data from the host program in real-time during playback, and knows nothing about the musical context of the audio data. This may be sufficient for some types of plug-ins like reverb and EQ, but not for plug-ins that need to analyze the entire musical context (notes, rhythm, tempo, key, etc.) in order to produce optimal results.

ARA greatly improves the workflow when using plugins like Melodyne that do MIDI analysis and pitch and timing correction. Compatible plug-ins can exchange information about the project’s timeline data, including audio clips, tempo, pitch, rhythm and much more, which allows them to work behave as if the plug-in was part of SONAR.

You get the following integration advantages:

• Manual audio transfers in advance are no longer necessary

• Create the RegionFX and open the UI with a simple keyboard shortcut (CTRL-R).

• Copy and paste RegionFX clips in SONAR to make copies of edits that can then be edited independently in Melodyne. Very useful for making variations from a single audio clip.

• Slip editing a clip in SONAR automatically trims the region displayed in Melodyne so you can see exactly what will be rendered.

• Integrated high quality ARA time stretching. CTRL-Drag to slip stretch any ARA clip to time stretch it using ARA time stretching. This is the same gesture used by audio snap stretching but the time stretching is done by ARA in this case.

• Melodyne and SONAR transports are synchronized so you can work exclusively in the Melodyne UI without switching back and forth to the host UI

• Click in Melodyne’s timeline area to set the now time in SONAR

• Set loop points directly from Melodyne’s UI by dragging on the timeline

• Start or stop SONAR’s transport from Melodyne by double clicking on the timeline

• Undo/Redo integration with Melodyne and SONAR

• Use Melodynes high quality pitch stretching for scrubbing audio

• Drag and drop audio-to-MIDI conversion. Pitch to MIDI is as simple as dragging and dropping audio to a MIDI/Instrument track

 

Audio-To-MIDI conversion

Besides its well known pitch correction abilities, one of the most powerful features of Melodyne ARA is its MIDI conversion. Melodynes outstanding pitch extraction technology opens up a exciting new possibilities for using virtual instruments in X3. To create a layered arrangements from an audio track, simply ARAfy the clips and drag them to a MIDI or instrument track to extract the MIDI from Melodyne’s DNA engine.

This opens up software synths to all instrumentalists and vocalists since a MIDI controller or tedious step entry of MIDI is not required for input. A vocalist can easily sing in multiple parts, ARAfy them and create an entire MIDI arrangement with just voice.

Melodyne also does a stellar job of analyzing polyphonic data. If you own the editor version of Melodyne it will extract MIDI from a polyphonic audio clip. I have tested it with piano, guitar and trumpet parts and had success getting accurate transcriptions of the audio. For guitar. in some ways its better than having a guitar controller – since the plugin doesn’t do realtime pitch to MIDI, the analysis and tracking is more accurate. Keep in mind though that you will need a clean and relatively dry audio source. It won’t track distorted guitars or heavily processed sounds well.

To use ARA audio-to-MIDI conversion in SONAR, all you need to do is one of the following:

• drag an audio clip from a audio track to a MIDI track

• drag a wave file from the browser to a MIDI track

• drag an RegionFX clip to a MIDI track (fastest since no analysis is required)

Melodyne ARA Tips And Tricks for SONAR X3

Here is an informative article from Celemony on getting the most out of Melodyne and ARA in SONAR X3.

ARA components

(supplemental info only – end users need not know these details to use ARA )

As explained above, ARA is a host protocol to access timeline audio data, tempo map and transport control. Here is a brief technical description of the ARA components associated with a Region FX in SONAR.

There are 3 components to an ARA region effect in SONAR:

VST3 plug-in extension

A VST3 instance of the ARA plugin is associated with every Region FX region. The function of the plugin is quite different from a normal clip effect however, since the primary purpose of the ARA plugin is to act as an audio output tap point. No realtime DSP processing as such is done by the VST plugin itself. Instead it functions as as a companion to ARA, streaming the output and providing access to other VST parameters.

ARA model graph

All Region FX region have an associated ARA model graph.

The model graph is a set of components that fully describe the region of audio to be processed by ARA. It consists of an audio source, its associated modifications (edits made to the audio via the plugin editor), a playback region (describes the duration and placement of the audio region on SONAR’s timeline), and a musical context (timing info)

ARA document model

All ARA plugins of the same type in the project share a single ARA document model. The document model is analogous to the SONAR project file and saves the state for all ARA model graphs for related ARA plugins in the project. All document models for ARA plugins are saved within the project file in SONAR and is transparent to the end user.

Summary 

ARA integration opens up an arsenal of advanced clip level editing tools to the SONAR user. Whether you are interested in pitch correction, editing, clip level audio re-sequencing, or MIDI extraction, you now have a full ARA-fication toolkit at your disposal!