Pitch Shifting in Real Time

[ExamplePrograms]

Description

Description-Separate-1

Overview

Postprocessing instrumental, musical, and vocal sounds widely use pitch shifting techniques. You can use the LabVIEW Advanced Signal Processing Toolkit to design offline or online pitch shifting applications. This article describes a practical example of an online pitch shifting application by using the Advanced Signal Processing Toolkit.

Pitch Shifting Fundamentals

Pitch, known as the fundamental frequency, is one of the main characteristics of sound. In human voices, the pitch is the frequency of the glottal pulse. In music, the pitch is the frequency of fundamental, instrumental vibration. When modifying the pitch, the audio texture can change substantially. For example, researchers and musicians shift the pitch of sound to disguise a speaker's gender, to correct instruments' frequency, and to make unperceptive sounds audible.

One way to change the pitch is to play a sound at normal speed while recording it at a different speed. However, because this process also changes the time duration, pitch shifting is not suitable in real-time.

There are two general means of shifting pitch. James Flanagan and R. M. Golden invented the Phase Vocoder ^[1] which is a time-frequency domain method. Eric Moulines and Francis Charpentier created the Pitch Synchronous Overlap Add (PSOLA) ^[2] which is purely a time domain method.

This article describes how to implement an online pitch shifting algorithm by using the National Instruments LabVIEW graphical development environment. This article includes the Pitch Shifting in Real Time demonstration VI, which is a real-time pitch shifting application built with LabVIEW and the Advanced Signal Processing Toolkit.

Online Pitch Shifting Applications

Researchers use pitch shifting algorithms in various applications. The following studies are examples of applications that use pitch shifting algorithms.

• In karaoke, Matti Ryynanen uses pitch shifting to tune the user's singing so that it reaches the original melody in real-time. 
• In music composition, Emilia Gomez shows that pitch shifting can help composers ease their creative process.

An application made in early 2009 at the Copenhagen Zoo in Denmark changes the pitch of the elephants’ rumble, so that visitors are able to hear the rumble in real-time. Although some of the harmonic components lie within the human audible range, in most circumstances the elephant rumble is infrasonic, as its pitch is less than 20 Hz which is not perceptive by the human ear. Shifting up the rumble pitch by one or two octaves in real-time thus results in a much clearer sound.

Originalelephantrumble.wav is a segment recorded from an elephant at the Copenhagen Zoo. Figure 1 shows the Gabor spectrogram of the Original Elephant Rumble Signal.

Figure 1: Gabor Spectrogram of the Original Elephant Rumble Signal

Speedupelephantrumble.wav is the result after speeding up the original elephant rumble signal four times. Figure 2(a) shows its Gabor spectrogram. Pitchshiftedelephantrumble.wav is the result after the pitch of the original rumble has been shifted up 4 times (two octaves). Figure 2(b) shows its Gabor spectrogram. From Figure 1 and Figure 2, we can conclude that changing pitch should have the same frequency domain impact as changing speed, while keeping the same time duration as the original signal. 

             

Figure 2: Gabor Spectrograms

Using LabVIEW for Online Pitch Shifting

There are several methods to perform online pitch shifting. The most popular method is the Phase Vocoder. This method first stretches a time domain signal by interpolating the signal in the time-frequency domain. Then the method resamples the signal to shift the pitch while restoring the time length. Figure 3 shows the flow chart of online pitch shifting using the Phase Vocoder method.

Figure 3: Online Pitch Shifting Flow Chart Using the Phase Vocoder

In Figure 3, applying the window w(n) breaks the source signal s(n) into signal blocks x(n). Each signal block x(n) is represented by Gabor coefficients in the time-frequency domain using the Gabor transform. After interpolating the Gabor coefficients in the time-frequency domain, these coefficients form the time-stretched signal y(n). Resampling y(n) stretches back the time domain in order to get the pitch-shifted signal z(n) while keeping the same time length.

The Advanced Signal Processing Toolkit includes the Online JTFA VIs, that you can use to transform a signal between time domain and time-frequency domain in real-time as shown in Figure 4. This figure also shows how to wire the Graphics & Sound VIs to control the sound device for input and output.

Figure 4: Shifting Pitch in Real Time Using LabVIEW

The Advanced Signal Processing Toolkit also includes the TFA Dual Window VI and the Dual Window Express VI. You can use this VI and Express VI to compute the dual window for the Gabor transform and Gabor expansion. Refer to the LabVIEW Help for more information on the Online JFTA VIs, the TFA Dual Window VI, and the Dual Window Express VI.

Many audio processing such as speaker gender changing, music processing, and sound modifying utilize pitch shifting techniques. By using LabVIEW and the Advanced Signal Processing Toolkit, you can change pitch offline as well as online.

 Pitch Shifting in Real Time VI

The Pitch Shifting in Real Time VI requires the LabVIEW Run-Time Engine 8.6.1. You can download a free copy of this software.

The LabVIEW graphical development environment and the Advanced Signal Processing Toolkit include many analytical VIs which you can use for pitch shifting analysis. For more information about these VIs, refer to the LabVIEW Help. 

 References

[1] J. L. Flanagan, R. M. Golden, Phase Vocoder, Bell Systems Technical Journal, 1966 (45), pp. 1493-1509.

[2] E. Moulines, F. Charpentier, Pitch Synchronous Waveform Processing Technique for Text to Speech Synthesis Using Diphones, Speech Communication, 1995(16), pp. 175-205.

[3] M. Ryynanen, T. Virtanen, J. Paulus, A. Klapuri, Accompaniment Separation and Karaoke Application Based on Automatic Melody Transcription, IEEE International Conference on Multimedia and Expo, 2008, pp. 1147-1420.

[4] E. Gomez, G. Peterschmitt, X. Amatriain, P. Herrera, Content-Based Melodic Transformations of Audio Material for a Music Processing Application, Proc of 6^th International Conference on Digital Audio Effects, London, UK, Sept 2003.

Description-Separate-2