So far we’ve been generating a steady waveform that keeps playing. Let’s see how we can react to MIDI and start/stop the waveform at the right pitch according to the notes we’re receiving.
MIDI Handling Basics
When our plugin is loaded into a host, it receives all MIDI data on the track it is on. When a note is played or released, the plugin’s ProcessMidiMsg member function is called. It doesn’t matter if the note was played on a MIDI keyboard or it came from a piano roll. And it’s not just for key presses, but also for things like Pitch Bend or Control Changes (CC) (e.g. when there’s automation for a plugin parameter). The ProcessMidiMsg function is passed an IMidiMsg, which describes the MIDI event in a normalized, format-independent way. It has member functions like NoteNumber and Velocity. We’re going to use these to find out at what pitch and volume our oscillator will play.
Whenever a MIDI message is received, the system is already playing back an audio buffer that was generated previously. There’s no way to push some audio to the system just at the moment when MIDI data arrives. We have to remember what happened until ProcessDoubleReplacing is called again. We also need the time when each message arrived, so that we can keep the timing intact when we’re generating the next audio buffer.
The IMidiQueue is the right tool for this.
Creating a MIDI Receiver
We’re going to reuse our Synthesis project. If you’re using version control, this may be a good time to commit. Create a new class MIDIReceiver and make sure the .cpp file is compiled with each target. Put the interface between #define and #endif in MIDIReceiver.h:
#pragma clang diagnostic push
#pragma clang diagnostic ignored "-Wextra-tokens"
#include "IPlug_include_in_plug_hdr.h"
#pragma clang diagnostic pop
#include "IMidiQueue.h"
class MIDIReceiver {
private:
IMidiQueue mMidiQueue;
static const int keyCount = 128;
int mNumKeys; // how many keys are being played at the moment (via midi)
bool mKeyStatus[keyCount]; // array of on/off for each key (index is note number)
int mLastNoteNumber;
double mLastFrequency;
int mLastVelocity;
int mOffset;
inline double noteNumberToFrequency(int noteNumber) { return 440.0 * pow(2.0, (noteNumber - 69.0) / 12.0); }
public:
MIDIReceiver() :
mNumKeys(0),
mLastNoteNumber(-1),
mLastFrequency(-1.0),
mLastVelocity(0),
mOffset(0) {
for (int i = 0; i < keyCount; i++) {
mKeyStatus[i] = false;
}
};
// Returns true if the key with a given index is currently pressed
inline bool getKeyStatus(int keyIndex) const { return mKeyStatus[keyIndex]; }
// Returns the number of keys currently pressed
inline int getNumKeys() const { return mNumKeys; }
// Returns the last pressed note number
inline int getLastNoteNumber() const { return mLastNoteNumber; }
inline double getLastFrequency() const { return mLastFrequency; }
inline int getLastVelocity() const { return mLastVelocity; }
void advance();
void onMessageReceived(IMidiMsg* midiMessage);
inline void Flush(int nFrames) { mMidiQueue.Flush(nFrames); mOffset = 0; }
inline void Resize(int blockSize) { mMidiQueue.Resize(blockSize); }
};
We have to include IPlug_include_in_plug_hdr.h here because otherwise IMidiQueue.h will generate errors.
As you can see, we’re keeping a private IMidiQueue member to store MIDI messages. We’re also storing some information about which notes are being played and how many in total. The three mLast... members are needed because our plugin will be monophonic: A note played later will mute any previous one (“last note priority“). The function noteNumberToFrequency converts from a MIDI note number to a frequency in Hz. We need it because our Oscillator class deals with frequencies, not note numbers.
The public section defines some inline getters and passes Flush and Resize through to mMidiQueue. In Flush, we’re also setting mOffset to zero: Calling mMidiQueue.Flush(nFrames) means that we discard nFrames from the queue’s front. We have already processed that length in the last call to the advance function. Resetting mOffset ensures that inside the next advance, we’re starting from the queue’s front again. (Thanks to Tale for help with this.)
The const behind the parentheses means that the function doesn’t modify non-mutable members of its class.
Let’s add the implementation for onMessageReceived to MIDIReceiver.cpp:
void MIDIReceiver::onMessageReceived(IMidiMsg* midiMessage) {
IMidiMsg::EStatusMsg status = midiMessage->StatusMsg();
// We're only interested in Note On/Off messages (not CC, pitch, etc.)
if(status == IMidiMsg::kNoteOn || status == IMidiMsg::kNoteOff) {
mMidiQueue.Add(midiMessage);
}
}
This function will be called whenever the plugin receives a MIDI message. We select only note on/off messages and Add them to our mMidiQueue.
The interesting function is advance:
void MIDIReceiver::advance() {
while (!mMidiQueue.Empty()) {
IMidiMsg* midiMessage = mMidiQueue.Peek();
if (midiMessage->mOffset > mOffset) break;
IMidiMsg::EStatusMsg status = midiMessage->StatusMsg();
int noteNumber = midiMessage->NoteNumber();
int velocity = midiMessage->Velocity();
// There are only note on/off messages in the queue, see ::OnMessageReceived
if (status == IMidiMsg::kNoteOn && velocity) {
if(mKeyStatus[noteNumber] == false) {
mKeyStatus[noteNumber] = true;
mNumKeys += 1;
}
// A key pressed later overrides any previously pressed key:
if (noteNumber != mLastNoteNumber) {
mLastNoteNumber = noteNumber;
mLastFrequency = noteNumberToFrequency(mLastNoteNumber);
mLastVelocity = velocity;
}
} else {
if(mKeyStatus[noteNumber] == true) {
mKeyStatus[noteNumber] = false;
mNumKeys -= 1;
}
// If the last note was released, nothing should play:
if (noteNumber == mLastNoteNumber) {
mLastNoteNumber = -1;
mLastFrequency = -1;
mLastVelocity = 0;
}
}
mMidiQueue.Remove();
}
mOffset++;
}
This is called on every sample while we’re generating an audio buffer. As long as there are messages in the queue, we’re processing and removing them from the front (using Peek and Remove). But we only do this for MIDI messages whose mOffset isn’t greater than the current offset into the buffer. This means that we process every message at the right sample, keeping the relative timing intact.
After reading noteNumber and velocity, the if statement distinguishes note on and off messages (no velocity is interpreted as note off). In both cases, we’re keeping track of which notes are being played, as well as how many of them. We also update the mLast... members so the already mentioned last note priority happens. This is the place where we know the frequency has to change, so we’re updating it here. Finally, the mOffset is incremented so that the receiver knows how far into the buffer it currently is. An alternative would be to pass the offset in as an argument. So we now have a class that can receive all incoming MIDI note on/off messages. It always keeps track of which notes are being played and what the last played note (and frequency) was. Let’s make use of it!
Using the MIDI Receiver
First, go into resource.h and make the following changes:
// #define PLUG_CHANNEL_IO "1-1 2-2"
#if (defined(AAX_API) || defined(RTAS_API))
#define PLUG_CHANNEL_IO "1-1 2-2"
#else
// no audio input. mono or stereo output
#define PLUG_CHANNEL_IO "0-1 0-2"
#endif
// ...
#define PLUG_IS_INST 1
// ...
#define EFFECT_TYPE_VST3 "Instrument|Synth"
// ...
#define PLUG_DOES_MIDI 1
This tells the host that our plugin is an instrument that does MIDI. The 0-1 0-2 means that there’s either no input and one output (mono) (0-1) or no input and two outputs (stereo) (0-2).
Next, go into Synthesis.h and #include "MIDIReceiver.h" below Oscillator.h. In the public section, add the following member function declaration:
// to receive MIDI messages:
void ProcessMidiMsg(IMidiMsg* pMsg);
Add a new MIDIReceiver instance to the private section:
private:
// ...
MIDIReceiver mMIDIReceiver;
Go into Synthesis.cpp and add the (quite simple) implementation:
void Synthesis::ProcessMidiMsg(IMidiMsg* pMsg) {
mMIDIReceiver.onMessageReceived(pMsg);
}
This function will be called whenever the application receives a MIDI message. We’re passing the messages through to our MIDI receiver.
Let’s clean up a bit. Change the two enums at the top:
enum EParams
{
kNumParams
};
enum ELayout
{
kWidth = GUI_WIDTH,
kHeight = GUI_HEIGHT
};
Create just one default preset:
void Synthesis::CreatePresets() {
MakeDefaultPreset((char *) "-", kNumPrograms);
}
Don’t do anything when a parameter is changed:
void Synthesis::OnParamChange(int paramIdx)
{
IMutexLock lock(this);
}
We don’t need a knob in our user interface. Let’s change the constructor to something really minimal:
Synthesis::Synthesis(IPlugInstanceInfo instanceInfo)
: IPLUG_CTOR(kNumParams, kNumPrograms, instanceInfo) {
TRACE;
IGraphics* pGraphics = MakeGraphics(this, kWidth, kHeight);
pGraphics->AttachPanelBackground(&COLOR_RED);
AttachGraphics(pGraphics);
CreatePresets();
}
When some audio preference is changed, we have to update our oscillator with the new sample rate:
void Synthesis::Reset()
{
TRACE;
IMutexLock lock(this);
mOscillator.setSampleRate(GetSampleRate());
}
The only thing left is ProcessDoubleReplacing. Thinking about it, we have to call mMIDIReceiver.advance() on every sample. After that, we’ll getLastVelocity and getLastFrequency from the MIDI receiver. Then we’ll call mOscillator.setFrequency() and mOscillator.generate() to fill the audio buffer with a tone at the right frequency.
We have designed generate to operate on a complete buffer and fill it. The MIDI receiver works on a sample level: MIDI messages can have any offset into a buffer, so mLastFrequency can change at any sample. We have to change our Oscillator class so it can work on a sample level as well.
First, move the twoPI out of generate and into the private section in Oscillator.h. Make sure you put it below the declaration of mPI, because variables are initialized in the order in which they are declared. While we’re there, let’s also add a bool to indicate if the oscillator is currently muted (when no note is playing):
const double twoPI;
bool isMuted;
Initialize them by modifying the constructor’s initializer list. It should now look like this:
Oscillator() :
mOscillatorMode(OSCILLATOR_MODE_SINE),
mPI(2*acos(0.0)),
twoPI(2 * mPI), // This line is new
isMuted(true), // And this line
mFrequency(440.0),
mPhase(0.0),
mSampleRate(44100.0) { updateIncrement(); };
Add an inline setter to the public section:
inline void setMuted(bool muted) { isMuted = muted; }
Right below that, add the following declaration:
double nextSample();
We’ll call this function on every sample to get audio data from the oscillator. Put the following implementation in Oscillator.cpp:
double Oscillator::nextSample() {
double value = 0.0;
if(isMuted) return value;
switch (mOscillatorMode) {
case OSCILLATOR_MODE_SINE:
value = sin(mPhase);
break;
case OSCILLATOR_MODE_SAW:
value = 1.0 - (2.0 * mPhase / twoPI);
break;
case OSCILLATOR_MODE_SQUARE:
if (mPhase <= mPI) {
value = 1.0;
} else {
value = -1.0;
}
break;
case OSCILLATOR_MODE_TRIANGLE:
value = -1.0 + (2.0 * mPhase / twoPI);
value = 2.0 * (fabs(value) - 0.5);
break;
}
mPhase += mPhaseIncrement;
while (mPhase >= twoPI) {
mPhase -= twoPI;
}
return value;
}
As you can see, we’re using twoPI here, and it would be wasteful to keep calculating it every sample. That’s why we turned it into a constant member variable.
Whenever the oscillator is muted, we just return zero. The switch should look very familiar, although we’re not using a for loop anymore. We’re just generating a single value instead of filling an entire buffer. This structure also allows us to move the phase incrementing part our, avoiding duplication.
This was a good example of refactoring existing code because it’s no longer flexible enough. Of course, we could have spent an hour or two thinking about what we need from our oscillator before writing the buffer-based generate function. But actually implementing it took less than an hour. In simple applications like this one, it’s sometimes more efficient to just implement one approach and see how it works in practice. Most of the time (like in this case), you’ll find that the overall idea was right (e.g. how to calculate different waveforms), but maybe you forgot one facet of the problem. If, on the other hand, you’re designing a public API, changing something later may be very inconvenient, so you better think thoroughly in advance. It depends.
We will call setFrequency on every sample. This means that updateIncrement will also be called very often, and it’s not very optimized:
void Oscillator::updateIncrement() {
mPhaseIncrement = mFrequency * 2 * mPI / mSampleRate;
}
2 * mPI * mSampleRate only changes when the sample rate changes. So you could cache this calculation and only recalculate it inside Oscillator::setSampleRate. But overzealous optimization like this can make code ugly. In practice, I didn’t notice a performance issue here. After all, we’re only playing one voice at the time, so when we get polyphonic, things will be different and we will definitely improve this. Now we’re ready to implement ProcessDoubleReplacing in Synthesis.cpp:
void Synthesis::ProcessDoubleReplacing(
double** inputs,
double** outputs,
int nFrames)
{
// Mutex is already locked for us.
double *leftOutput = outputs[0];
double *rightOutput = outputs[1];
for (int i = 0; i < nFrames; ++i) {
mMIDIReceiver.advance();
int velocity = mMIDIReceiver.getLastVelocity();
if (velocity > 0) {
mOscillator.setFrequency(mMIDIReceiver.getLastFrequency());
mOscillator.setMuted(false);
} else {
mOscillator.setMuted(true);
}
leftOutput[i] = rightOutput[i] = mOscillator.nextSample() * velocity / 127.0;
}
mMIDIReceiver.Flush(nFrames);
}
In the for loop, we’re first letting the MIDI receiver update its values (by calling advance). If there’s a note playing (i.e. velocity > 0), we update the oscillator’s frequency and unmute it. Otherwise, we mute the oscillator (meaning that nextSample will return zeros).
After that, it’s simply about calling nextSample to get a value, changing its volume (velocity is an integer between 0 and 127), and assigning the result to both output buffers. Finally we call Flush to move the MIDI queue’s front.
Try it!
Run as VST2 or AU. If the AudioUnit doesn’t appear in your host, you may have to change the PLUG_UNIQUE_ID in resource.h. If two plugins have the same ID, your host may ignore all but one of them.
You will have to input some MIDI data into your plugin. The easiest way to do that is to use REAPER’s virtual keyboard. Click View → Virtual MIDI Keyboard to show it. On the track with your plugin, there’s a round red record button. Right-click it and configure the track to receive MIDI from the virtual keyboard:
In the same menu, make sure Monitor Input is enabled. Now, with the virtual keyboard’s window focussed, you can play your plugin using your computer keyboard. Press the QWERTY keys and your should hear sound from your plugin.
If you have a MIDI keyboard connected, you can also try the standalone version. Make you you select the right MIDI input in the preferences. If you don’t hear any audio, you may have to delete ~/Library/Application Support/Synthesis/settings.ini.
You can download the current project here. Next time, we’ll add a nice virtual keyboard to our plugin’s GUI!