I have summer camp with several successive week long classes coming up and am trying to put together some fun programming materials to engage the kids and hopefully encourage them to explore more.

Yesterday I created the start of my next project for teaching kids on the subject of computer programming using Lazarus and Free Pascal as the teaching tools. Here is a brief video demonstrating what I have so far along with a github page containing the sources.

https://cache.getlazarus.org/videos/sound-shop-example.mp4 https://github.com/sysrpl/SoundShop/

What the Sound Shop program does:

Sound Shop let's kids write functions and hear the results of those functions. Right now it's dead simple to write wave form functions, but I am going to enhance the program to allow for effect pipelines which map unto keys. These pipelines will be a lot like the node in my Image Shop program but will map to piano keys instead of the display.

I need to take a break from these example projects for a few days and work on helping the school with schedules, but I thought I'd ask anyone who find this interesting for some help.

Here are some of the things anyone can do to help:

Write a routine to load a music or song that will play on the keyboard using the currently assigned piano keys. You can turn a key on or off by simply typing Piano.Key[Index] := True/False. This will light up or dim the key and play the corresponding mapped audio channel. Users can play already along with with music or songs, so there's that feature. Obviously this feature will also need a few demo music or song files. You can choose however you want to store and read a music or song file.

Write an MP3 streamer that will convert in memory portions of an MP3 file to audio samples. This will allow piano key channels to be mapped to sounds other than computer generated waves. Also, I will eventually hook effects up to both wave and MP3 samples. Please no third part libraries to convert MP3 data to samples, only Pascal code. I will do this if no one takes up the task.

Fix the issues with 32 bpp PNG images on Linux.

Some of the things I am going to do:

I am going to add a sound pipeline to the program to allow for effects and other things to modify sounds before they are routed from the piano keys to the underlying audio system. Each piano key will have its own pipeline.

I am going to add an oscilloscope to visualize the all the collectively playing samples as they form waves. The scope will have a few controls to adjust the phase, stretch, and amplitude.

I might add a bar staff allowing rectangular notes to be places on it and stretched. Moving notes up or down on the staff changes the mapped piano key, and stretching alters the note duration.

If anyone want to contribute it would be much appreciated.

Thanks

Anthony
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lazarus-ide.org/pipermail/lazarus/attachments/20190515/54d5bce2/attachment.html>

Ported from PDMP3 Public domain MP3 decoder
https://github.com/serbod/acs/blob/master/src/formats/general/mp3.pas but I don't tested it properly.

Also, whole ACS components can be useful to build interactive pipeline for sound processing
https://github.com/serbod/acs/

ср, 15 мая 2019 г. в 20:45, Anthony Walter via lazarus <

Thank you so much for a link to your MP3 decoder! I will take a look at integrating it this coming weekend.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lazarus-ide.org/pipermail/lazarus/attachments/20190515/f799b197/attachment.html>

I just looked at you MP3 decoder source. It looks like it mimics the mpg123 API which is perfect. The only question I have is what is the best way to seek to a time in an MP3 memory buffer for use with your pdmp3_feed and pdmp3_read functions? I don't want to decode whole MP3 streams into PCM wave data, rather I want to be able to do random moves using pointer given a time offset from the start of the MP3 sample data and decode a small portion.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lazarus-ide.org/pipermail/lazarus/attachments/20190515/b6273f44/attachment-0001.html>

On Wed, May 15, 2019 at 2:45 PM Anthony Walter via lazarus <

Hi Anthony,

thanks for sharing this incredible work.

Could you attach some license to this stuff? The question is just because you wrote nice classes like TPianoKeyboard which could be used in a final application. (for example, I could use it to produce alarm songs in my Raspberry board dynamically instead of playing them from files)

best,

I usually add GPL licences to my repositories but I forgot this time. Thanks for reminding me. It's now attached.

Regarding the TPianoKeyboard class here is some helpful information.

It's a TGraphicControl.
It relies on the Music unit, also in the same project, to convert keys to frequencies and also to format note names as strings. You can define the number of keys on the piano by setting the KeyCount property. KeyCount ignores attempts to set its value to less than 1. You can change the on screen size of the piano by setting the ScaleFactor property.
Setting Key[Index] := False/True turns the piano key from an released state to a pressed state (when true).
You can listen to the OnKeyToggle event to take action when a piano key is pressed or released.
The KeyToFrequency method gives you the frequency in hertz of the piano key by index.
The piano does its best to keep middle C in the middle of its key layout when you change KeyCount.

If you (or anyone really) have any improvement ideas for TPianoKeyboard or any other code in the project, let me know and I'll do my best to accommodate.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lazarus-ide.org/pipermail/lazarus/attachments/20190515/7ca32f76/attachment.html>

On Wed, May 15, 2019 at 4:51 PM Anthony Walter via lazarus <

Two useful features (suggestions :-D):

* Split the "sound logic" from TPianoKeyboard to own class (TPianoSynthesizer?) allowing to use it in non-LCL based applications;
* Supposing the feature above is done, the synthesizer could allow to record key sequences by typing or by string. (it would be useful to study applications like DefleMask project <http://www.delek.net> and play songs (strings only, without drums, bass etc.) like "Sonic - Green Hill Zone" https://www.youtube.com/watch?v=0YY13qh7X4A, "Spring Yard Zone" https://www.youtube.com/watch?v=_8LUfR1iH_0. (Castlevania, Zelda and so on <3)

Silvio,

Actually I am to add a node based system for mapping each key of the keyboard similar to this project
<https://www.getlazarus.org/learn/tutorials/examples/imageshop/>. Instead of source images, like in the Image Shop program, there will be wave source nodes and file source nodes. The node between the source and the key will be effect nodes. Effects can be anything you program, such as delay, echo, pan, chorus, and anything you can imagine. Effects, like wave source, will be simple functions you write and register before running the program.

I will also be adding an oscilloscope control to graph the source and effect in real time.

If anyone wants to contribute an midi file read that converts midi files into time / frequencies, I'd love to add that.

Thanks again.

Anthony
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lazarus-ide.org/pipermail/lazarus/attachments/20190515/c5033ffa/attachment-0001.html>

Hello,

To perform random seek in mp3 streams you must process each header in the mp3 file and take note where each header start, so create some kind of index file. For mp3 in CBR mode you can calculate skips as each frame is constant in size, but in VBR you must create the index, the big problem is that to know if an mp3 is VBR or CBR you must read all headers :-( Some extensions in ID3 tags store the index for headers, others in ancillary data.

To decode after seek you must skip back 9 frames (if available), start to decode and discard the sound associated with that 9 frames, and the continue decoding and using data. The reason is that mp3 frames can share data using "byte reservoir" fields across up to 9 frames and if you do not discard that sound you are in risk of start the play with some nice bleeps, blurps, shrrrss, or any other non expected sound.

Got it. Thanks.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lazarus-ide.org/pipermail/lazarus/attachments/20190515/baed980e/attachment.html>

Hi,

I your video you talked about popping and clicking of the tones. You partly solved this by adding an Attack and a Release (you use attack here in in fact these are 2 different values, and while you're at it you can add a Decay too :))

Anyway, I haven't looked at the code itself (only what was shown in the video), but most likely those pops and clicks are caused by phase jumps when you switch from one frequency to another.

Imagine your playing A -> B and you start generating the sine of B at phase=0, there is a big chance that A isn't at phase=0. So you get a sudden jump in your output. your ear is very sensitive for this and results in a click.

What I've done in this case is based on the frequency/samplerate to play, is calculate the delta phase for each sample (for 220Hz/48k -> delta=220*2Pi/48000) and increase a global phase for each sample you calculate with this global phase.
Therefore if if you change frequency, the phase of the new tone continues where the previous ended.

To prevent your global phase to grow, substract 2Pi when it is larger than 2Pi. Or even better, leave out 2Pi in the delta phase, and use 2Pi*globalphase to calculate and globalphase=fraq(globalphase + deltaphase)

Hope this solves your issue,
Marc

Marc,

That is interesting and you've given me something to consider and perhaps adjust. Thank you.

Also with regards to attack and release, I've opted to call it just attack. That is when the note is starting to play, or ending playing, I perform a short fade of the wave form on both ends using the same value. I also mix in a sustain which I am using to fade the note out over time after the initial attack.

So the formula I am using is assuming a note is playing (StartTime >= CurrentTime):

if StartTime - CurrentTime < Attack then
  Mix := StartTime - CurrentTime / Attack
else
  Mix := 1;

Then I used the Mix blend variable to adjust the wave amplitude (Volume) by multiplying it against the left and right channels.

// Sample Left and Right are a 16 bit signed integers
Sample.Left := Trunc(AudioWave(F) * Volume * Mix);
Sample.Right := -Sample.Left;

Where:

F is the a value between 0.0..1.0 representing the current wave cycle AudioWave is the user defined function to convert F into a wave shape returning a value between -1.0..1.0
Volume is the peak we allow for the Left or Right on Sample (in my case the constant value 2048)
Mix is between 0.0..1.0 and is used to fade in for attack, fade out for release, and fade out using sustain

So I should fix this by adding a tiny bit of phase change to F based on which keyboard key is being processed.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lazarus-ide.org/pipermail/lazarus/attachments/20190516/2dead9c1/attachment.html>

As long as you don't reset F to 0 when playing a new note this should work.

To verify, you can use Audacity to record the generated audio (or write it to file as raw pcm). If you switch to spectogram view, you see those phase errors as a spike in the diagram

Marc

Just an update. I've added the ability to program music with it. You have the ability to either load music from a file or create music by adding notes to the piano Music property.

http://cache.getlazarus.org/videos/sound-shop-music.mp4

Marc,

The phase of each note is now set to a completely completely random value right before it a note is pressed, but I still get the pops and clicking. I am nearly 100% sure the problem is with Linux, SDL 2, or my hardware. I say this because I am experiencing multiple clicks or pops when within the holding of one constant tone.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lazarus-ide.org/pipermail/lazarus/attachments/20190516/db681b7f/attachment.html>

Just to verify that it isn't caused by your program, just open a temp filestream and write every sample you write to the sdl library also to that stream. When the pops happen. Stop running, close the stream and import the raw file into audacity. If the spectral view is clean, it is not your generation of tones. One other thing what can happen is that your buffer underflows. Ie there are no samples to play anymore

Marc

I will take you advice and write the samples out to a PCM wave file then examine them in audacity. Regarding the underflow, I always wipe the buffer with zeros before I play a note and the popping occurs sometimes when only one note is playing. That is when there is the least amount of processing, but you might be onto something. I am using the smallest buffer size of 512 samples and the threading system might not invoke my audio callback in time to fill the buffer before the sound should be playing. I will try to increase the buffer size which should relieve the race condition, but increasing the buffer size also increases the resolution where notes can be played making them less accurate.

And right now I just tested the buffer size and simplifying the computation of Left and Right samples. This greatly reduced the popping, so I am thinking the problem is related to my callback is taking too long to compute. That or SDL and Linux are not scheduling the audio callback as aggressively as they should. I will test on Windows with a lower end CPU (a different machine) and see if the problem presents itself there as well.

If the problem lays in my computation taking too long, perhaps I will try compiling with different optimization levels or simplifying my audio callback. I am using doubles with division and multiplication (and the Sin function with some waveforms) quite a bit during my callback. Perhaps FPC generates less than ideal instructions to perform math on the double type. -------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lazarus-ide.org/pipermail/lazarus/attachments/20190516/837da865/attachment-0001.html>

AW:
I will take you advice and write the samples out to a PCM wave file then examine them in audacity. Regarding the underflow, I always wipe the buffer with zeros before I play a note and the popping occurs sometimes when only one note is playing. That is when there is the least amount of processing, but you might be onto something. I am using the smallest buffer size of 512 samples and the threading system might not invoke my audio callback in time to fill the buffer before the sound should be playing. I will try to increase the buffer size which should relieve the race condition, but increasing the buffer size also increases the resolution where notes can be played making them less accurate.

MW:
512 samples, thats about 10 msec @48k, thats pretty short. At work I work with streaming voice and we try to keep atleast 30ms of audio in the soundbuffers. And this is lowlevel winapi. I don’t know the overhead on Linux (it’s one of my todos)

AW:
And right now I just tested the buffer size and simplifying the computation of Left and Right samples. This greatly reduced the popping, so I am thinking the problem is related to my callback is taking too long to compute. That or SDL and Linux are not scheduling the audio callback as aggressively as they should. I will test on Windows with a lower end CPU (a different machine) and see if the problem presents itself there as well.

MW:
Maybe ofload the sample calculation in another thread. So you only have to feed the samples at the time ther are needed, while in the mean time you can continue to generate them

AW:
If the problem lays in my computation taking too long, perhaps I will try compiling with different optimization levels or simplifying my audio callback. I am using doubles with division and multiplication (and the Sin function with some waveforms) quite a bit during my callback. Perhaps FPC generates less than ideal instructions to perform math on the double type.

Marc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lazarus-ide.org/pipermail/lazarus/attachments/20190516/00ab26b8/attachment.html>

So progressing with the program I've added the ability to both play songs and record audio. Up to date source are in the github repository.

But I still have audio clicks and the waveform looks fine in Audacity. Here are two links demonstrating the clicking or pops:

View and sound from Audacity:
https://cache.getlazarus.org/videos/sound-clicks.mp4

Pure wave file:
https://cache.getlazarus.org/videos/sound-clicks.wav

When I zoom up on the samples in Audacity really close they look fine. They are smooth with no sharp jumps, but I still hear the clicking when either playing the wave. I am at a loss. Help!

By the way, I don't hear these pops or click on Windows. Can some of you try the wave file on your Windows computers and tell me if you hear the pops and clicks? If you feel like it can you look at the wave file in Audacity and isolate the samples causing the problem? I don't see any problems with the wave samples.

Thanks.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lazarus-ide.org/pipermail/lazarus/attachments/20190517/f3b4fa9c/attachment.html>

I think I isolated some wave samples that don't look smooth. I will try fix my code.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lazarus-ide.org/pipermail/lazarus/attachments/20190517/144835c7/attachment.html>

Hi,
You have still some (possible) clicks in your source. Have a look at its spectrogram: https://paste.pics/e2acc70acd050d843565b21e4fed82b1

Almost the same you see in the output:
https://paste.pics/0b5b5d895fbab3729888df8f2500f374

When looking at the output waveform you see missing audio: https://paste.pics/a5259cfe447e4cd2a1fd559cc89d7360

Marc