Play WAV files in Codea

[JockM]

Play WAV files in Codea

Codea has this lovely soundbuffer()function, but to date there has been no good way to load PCM audio into your app. AND you had to get the raw data, and not something more convenient… like a .WAV file.

Until now. For a personal project I needed the ability to play WAV files, and I figured I would share it with you all.

How do I get it?

You can download a sample project here which will give you the SoundData class, and a sample audio file*.

I will be creating a github project to hold this and other utility classes I will be creating.

How Do I Use It?

Assuming there is a variable named helloSound that holds the WAV file. All you have to do is:

-- decode the WAV file
soundData = SoundData()
soundData:decodeWavData(helloSound)

-- create a soundbuffer
helloBuffer = soundData:getSoundBuffer()

-- play it
sound(helloBuffer)

You should do the first two steps in yoursetup()function so the data is ready to play when it is needed. Once you have made a sound buffer there is no need to keep the SoundData class around. So either store it in a local variable, or remember to assign it tonil so it will be garbage collected.

How Do I Get WAV Files Into My Project?

You can do this one of two ways:

1. Usinghttp.get()to download the file from the 'net
2. Embedded in your Codea project using my BString utility (or the like)

Be ware that binary encoded WAV files can be quite large an make your project slow to load. So use lower bit rates and sample sizes if you can.

What Can I Play?

Currently this class supports PCM (aka uncompressed) .WAV files, either mono or stereo, unsigned 8bit or signed 16bit formats. At least that is the theory, I have only tested 8 bit files so far.

Eventually I want to support ADPCM compression as well as straight PCM. It will make the WAV data smaller in your project as well.

I might be convinced to support AIF PCM/ADPCM files, but don’t expect much more than that. I will leave MP3 decoding as an intellectual exercise for the reader.

Resources

For those interested in the WAV file format I recommend this page, it has proved to me more than useful.

*: In case you have ever wondered what that jerk JockM sounds like…

[Vega]

Wow, @JockM, this is a pretty good hack. Thanks for sharing this. Obviously it will be good to have the ability to load and play .wav or .mp3 on the fly in the future, but this seems like a reasonable stop gap until then. I am downloading to give it a shot now.

[sim]

Thanks for the great post, @JockM. Very instructive.

[JockM]

@Vega I am not quite sure what you mean by on the fly, but I do want to support a kind of crude streaming mode. You would pass in a URL an the class would use range headers to download the file in chunks, filling the soundbuffer as it went.

Oh and I forgot to mention, you could always load WAV files that have been loaded into the system using the storage APIs. And I have been making notes this morning for an AudioPlayer class that could provide play/pause and seek. It will be tricky to implement since there are no callbacks when the soundbuffer has run out, but it can be done.

MP3 Decoding is covered by software patents, so it becomes a bit of a sticky legal issue — the issue of unlicensed decoders on a system that is licensed is untested and I don’t want to risk it.

I could write a MP3 decoder in Lua, call it a reference implementation for educational purposes and be legally OK myself, but all that does is pass the liability down the line to whomever uses it; but I am not keen to do that either.

Sooner or later we will get better Audio APIs— which ideally will use the iOS APIs for decoding audio files — and this will all become moot.

[Vega]

@JockM what I was referring to by “on the fly” is only loading the audio files when needed rather than processing them all into memory when the game loads. I actually think my fileio project might solve this issue. Saving audio files as images, sounds abstract

[JockM]

@Vega The one big downside right now is that playback control is limited to starting — no pause or seek is possible. One of my plans is to enhance the SoundData class so you can get a sound buffer from an arbitrary location and for an arbitrary length.

Then I will create a AudioPlayer class that takes a SoundData object and breaks it up into, say, quarter second chunks. It will then use a thread to start the next chunk as the previous one is ending. Its an imperfect approach but it should work.

[FabulamGames]

This is really helpful, thanks!

[Jmv38]

Great work JockM

[juaxix]

What about ogg?

[aciolino]

Someone has to find and write a reference loader for ogg, or MP3 or any of those formats. I don’t think I could have written that WAV player myself without a LOT of free time; I can’t imagine OGG would be any easier!

[tnlogy]

Nice! But if you want to play MP3 or similar I guess it’s best to modify the Codea runtime, until support for it arrives. It would be nice with support for saving binary data and playing MP3s though.

For the runtime I’ve tried adding this to SoundCommands.mm which seems to work fine. Need to save a reference to the audio player though, and make sure it is garbage collected.

int mp3sound(struct lua_State* L) {
    NSURL *url = [NSURL fileURLWithPath:[NSString stringWithFormat:@"%@/audio.m4a", [[NSBundle mainBundle] resourcePath]]];
    
    NSError *error;
    AVAudioPlayer *audioPlayer = [[AVAudioPlayer alloc] initWithContentsOfURL:url error:&error];
    audioPlayer.numberOfLoops = -1;
    
    [audioPlayer play];
    return 0;
}

And then added this function to SoundCommands.h and LuaState.m. Then I’ve just added the audio file I’ve tested with to the Xcode project.

[FabulamGames]

So, I’ve been trying this out with a 16 bit .wav file, and it’s not playing anything. I put debug messages everywhere, so it is receiving the file, it’s just not playing it. I tried loading Cargo-Bot via Codea, and I noticed that the music isn’t working in there either… it’s playing the sound effects but not the music track. It works properly when I load the app itself, but in Codea it isn’t working. I’m on an iPad 1, with iOS 5. Maybe it’s too outdated?

[Jordan]

@FabulamGames, there is a variable called NO_SOUND at the beginning of the main file, change that to true.

[Jordan]

False*

[aciolino]

I noticed that if the wav file has header info, the wav file is not decoded. I see my wave files with no header chunk.

[Neztec]

I think I’ve followed these instructions correctly, but I instantly get "error: [string “SoundData = class()…”]:158: attempt to index local ‘data’ (a nil value)

Any thoughts? I’m using the sample example with the Walrus sound.

[SkyTheCoder]

@Neztec Do you have the “Walrus” tab?

It worked for me, I imported the .codea folder using iExplorer.

[Neztec]

Well I did, but I had started by transplanting the files into my project… your response made me try just moving the entire sample project into Codea and that worked (walrus wav).

But every wav I create doesn’t work. I tried making an unsigned 16 bit and unsigned 8 bit in Audicity but neither of them made any sound (no error, though). Any thoughts?

[aciolino]

No headers! Remove the header chunk.

[SkyTheCoder]

@Neztec Hmm… I tried to make my own custom sound too, but it was too big and crashed/froze and crashed when I tried to load it (20s). I’ll try a smaller file tomorrow. I have to get of for the night…