What is Oboe ?
Oboe is a high-performance audio library developed by Google specifically for Android applications, leveraging the capabilities of C++ to achieve low latency and efficient audio processing. The library is designed to interact closely with the native audio capabilities of Android devices, providing developers with direct access to device-level features for optimal performance.
Advantages of using Oboe over other alternatives
To quote directly from the official Oboe documentation:
use Oboe to gain the following benefits:
Achieve the lowest latency. Oboe helps your application achieve the lowest-possible audio latency for a given device and Android version combination.
Use the best available native library. On devices running Android API 8.1 (API level 27) and higher, Oboe uses AAudio. For devices running lower versions, Oboe uses OpenSL ES.
Avoid audio bugs. Oboe includes workarounds for some known audio issues that manifest on specific devices or versions of Android. Using Oboe helps your application avoid these issues without having to implement or test your own solutions.
What are we doing in this post?
In this post, let’s see how we can utilize Oboe within a Rust environment for a react native android application to generate simple sine waves. While Rust and C++ are interoperable through FFI (Foreign Function Interface), integrating Oboe with Rust involves managing Rust’s build system to properly link and utilize the C++ components of Oboe.
This typically includes configuring the build process to compile Oboe’s C++ code into a shared library (*.so on Android), and then interfacing with it from Rust using appropriate FFI bindings or wrappers. This approach allows us to harness Oboe’s powerful audio capabilities while leveraging Rust’s safety guarantees and expressive features for our application development needs.
For a seamless integration of Rust code into your React Native CLI Android app, please follow the setup outlined in my previous post on how to call Rust code from a React Native CLI Android app.
This post builds upon that foundation, ensuring you can effectively implement and leverage the capabilities discussed here.
To set up your Rust project as per the Cargo.toml configuration, ensure you have the following dependencies installed:
Install Rust dependencies
Oboe
Add Oboe to your project for optimized audio capabilities:
## run in the backend directory where rust code lives
cargo install oboe --version 0.4Atomic Float
Install Atomic Float for managing atomic floating-point operations:
## run in the backend directory where rust code lives
cargo install atomic_float --version 1.0These packages are essential for integrating advanced audio functionalities and precise floating-point operations into your Rust backend. Once installed, your project will be equipped to compile and run smoothly as configured in your cargo.toml.
This is how our cargo.toml file look like after installing oboe and atomic_float.
[package]
name = "backend"
version = "0.1.0"
edition = "2021"
[lib]
crate-type = ["cdylib"]
name = "backend"
[dependencies]
log = "0.4"
oboe = { version = "0.4", features = ["java-interface"] }
android_logger = "0.13.1"
uniffi = { version = "0.28.0", features = ["cli"] }
atomic_float = "1.0.0"
[build-dependencies]
uniffi = { version = "0.28.0", features = ["build"]}
uniffi_build = { version = "0.28.0", features = ["builtin-bindgen"] }
[[bin]]
name = "uniffi-bindgen"
path = "uniffi-bindgen.rs"Update rust build process
fn main() {
uniffi_build::generate_scaffolding("src/backend.udl").unwrap();
println!("cargo:rustc-link-lib=c++abi");
}In the main function of this Rust script, we utilize uniffi_build to generate scaffolding based on the Unified Definition Language (UDL) file located at src/backend.udl. This scaffolding facilitates seamless integration between Rust and external libraries like Oboe, ensuring smooth interoperability and function invocation.
Additionally, the line println!("cargo:rustc-link-lib=c++abi"); is crucial for linking the C++ ABI (Application Binary Interface). This step is essential for successfully compiling and linking Oboe, which is written in C++, into our Rust project. By specifying c++abi, we ensure that the Rust compiler includes the necessary C++ ABI support during the build process, enabling Oboe’s functionalities to be accessed and utilized within our Rust application.
This setup allows us to harness Oboe’s powerful audio capabilities while leveraging Rust’s safety and performance benefits, ensuring robust and efficient integration for our backend application.
Update the UDL file
namespace backend {
boolean start_sine_wave();
};The Unified Definition Language (UDL) snippet defines a namespace backend with a single function start_sine_wave(), which returns a boolean value. This UDL specification outlines the interface for initiating a sine wave audio generation process within the backend module of the application for this demo application we are building.
Update the lib.rs file
#![allow(unused)]
#[macro_use]
extern crate log;
extern crate android_logger;
use android_logger::Config;
use log::LevelFilter;
uniffi::include_scaffolding!("backend");
mod audio;
use audio::*;
pub fn start_sine_wave() -> bool {
let mut sine: SineGen = SineGen::default();
sine.try_start()
}In this Rust file (lib.rs), the module audio encapsulates functionalities related to audio processing, including the SineGen structure for sine wave generation. The start_sine_wave() function initializes and attempts to start the sine wave generator, leveraging the SineGen module to manage and initiate the audio stream.
The inclusion of uniffi::include_scaffolding!("backend"); ensures integration with external components defined in the backend module, facilitating interoperability and function invocation across Rust and other languages or frameworks.
Setup audio playback using Oboe
This is how audio.rs looks like:
use atomic_float::AtomicF32;
use std::{
f32::consts::PI,
marker::PhantomData,
sync::{atomic::Ordering, Arc},
thread,
time::Duration,
};
use oboe::{
AudioDeviceDirection, AudioDeviceInfo, AudioFeature, AudioOutputCallback, AudioOutputStream,
AudioOutputStreamSafe, AudioStream, AudioStreamAsync, AudioStreamBase, AudioStreamBuilder,
DataCallbackResult, DefaultStreamValues, Mono, Output, PerformanceMode, SharingMode, Stereo,
};
/// Sine-wave generator stream
#[derive(Default)]
pub struct SineGen {
stream: Option<AudioStreamAsync<Output, SineWave<f32, Mono>>>,
}
impl SineGen {
/// Create and start audio stream
pub fn try_start(&mut self) -> bool {
if self.stream.is_none() {
let param = Arc::new(SineParam::default());
let mut stream = AudioStreamBuilder::default()
.set_performance_mode(PerformanceMode::LowLatency)
.set_sharing_mode(SharingMode::Shared)
.set_format::<f32>()
.set_channel_count::<Mono>()
.set_callback(SineWave::<f32, Mono>::new(¶m))
.open_stream()
.unwrap();
log::debug!("start stream: {:?}", stream);
param.set_sample_rate(stream.get_sample_rate() as _);
stream.start().unwrap();
self.stream = Some(stream);
thread::sleep(Duration::from_millis(5000));
self.try_stop();
}
return false;
}
/// Pause audio stream
#[allow(dead_code)]
pub fn try_pause(&mut self) {
if let Some(stream) = &mut self.stream {
log::debug!("pause stream: {:?}", stream);
stream.pause().unwrap();
}
}
/// Stop and remove audio stream
pub fn try_stop(&mut self) {
if let Some(stream) = &mut self.stream {
log::debug!("stop stream: {:?}", stream);
stream.stop().unwrap();
self.stream = None;
}
}
}
pub struct SineParam {
frequency: AtomicF32,
gain: AtomicF32,
sample_rate: AtomicF32,
delta: AtomicF32,
}
impl Default for SineParam {
fn default() -> Self {
Self {
frequency: AtomicF32::new(220.0),
gain: AtomicF32::new(0.5),
sample_rate: AtomicF32::new(0.0),
delta: AtomicF32::new(0.0),
}
}
}
impl SineParam {
fn set_sample_rate(&self, sample_rate: f32) {
let frequency = self.frequency.load(Ordering::Acquire);
let delta = frequency * 2.0 * PI / sample_rate;
self.delta.store(delta, Ordering::Release);
self.sample_rate.store(sample_rate, Ordering::Relaxed);
println!(
"Prepare sine wave generator: samplerate={}, time delta={}",
sample_rate, delta
);
}
#[allow(dead_code)]
fn set_frequency(&self, frequency: f32) {
let sample_rate = self.sample_rate.load(Ordering::Relaxed);
let delta = frequency * 2.0 * PI / sample_rate;
self.delta.store(delta, Ordering::Relaxed);
self.frequency.store(frequency, Ordering::Relaxed);
}
#[allow(dead_code)]
fn set_gain(&self, gain: f32) {
self.gain.store(gain, Ordering::Relaxed);
}
}
pub struct SineWave<F, C> {
param: Arc<SineParam>,
phase: f32,
marker: PhantomData<(F, C)>,
}
impl<F, C> Drop for SineWave<F, C> {
fn drop(&mut self) {
println!("drop SineWave generator");
}
}
impl<F, C> SineWave<F, C> {
pub fn new(param: &Arc<SineParam>) -> Self {
println!("init SineWave generator");
Self {
param: param.clone(),
phase: 0.0,
marker: PhantomData,
}
}
}
impl<F, C> Iterator for SineWave<F, C> {
type Item = f32;
fn next(&mut self) -> Option<Self::Item> {
let delta = self.param.delta.load(Ordering::Relaxed);
let gain = self.param.gain.load(Ordering::Relaxed);
let frame = gain * self.phase.sin();
self.phase += delta;
while self.phase > 2.0 * PI {
self.phase -= 2.0 * PI;
}
Some(frame)
}
}
impl AudioOutputCallback for SineWave<f32, Mono> {
type FrameType = (f32, Mono);
fn on_audio_ready(
&mut self,
_stream: &mut dyn AudioOutputStreamSafe,
frames: &mut [f32],
) -> DataCallbackResult {
for frame in frames {
*frame = self.next().unwrap();
}
DataCallbackResult::Continue
}
}
impl AudioOutputCallback for SineWave<f32, Stereo> {
type FrameType = (f32, Stereo);
fn on_audio_ready(
&mut self,
_stream: &mut dyn AudioOutputStreamSafe,
frames: &mut [(f32, f32)],
) -> DataCallbackResult {
for frame in frames {
frame.0 = self.next().unwrap();
frame.1 = frame.0;
}
DataCallbackResult::Continue
}
}You can go through the code to understand how we are generating the sine wave audio. The highlighted code below is responsible for using the oboe API to start an output stream to which we are writing the sine wave audio frames for a specific frequency.
impl SineGen {
/// Create and start audio stream
pub fn try_start(&mut self) -> bool {
if self.stream.is_none() {
let param = Arc::new(SineParam::default());
let mut stream = AudioStreamBuilder::default()
.set_performance_mode(PerformanceMode::LowLatency)
.set_sharing_mode(SharingMode::Shared)
.set_format::<f32>()
.set_channel_count::<Mono>()
.set_callback(SineWave::<f32, Mono>::new(¶m))
.open_stream()
.unwrap();
log::debug!("start stream: {:?}", stream);
param.set_sample_rate(stream.get_sample_rate() as _);
stream.start().unwrap();
self.stream = Some(stream);
thread::sleep(Duration::from_millis(5000));
self.try_stop();
}
return false;
}
}
In the audio.rs file, Oboe is utilized to manage audio playback, specifically through the SineGen structure. This structure encapsulates functionality to generate and control sine wave audio streams using Oboe’s AudioStreamBuilder and related configurations. The module also includes SineParam, which adjusts parameters like sample rate and frequency, ensuring precise control over the generated audio. Additionally, the file features utilities to probe and display device audio capabilities and configurations using Oboe’s querying functionalities.
This setup leverages Oboe’s robust features for low-latency audio output, making it ideal for applications requiring high-performance audio playback and real-time processing capabilities.
Integrate kotlin bindings
In the BackendModule.kt file, Kotlin bindings generated by Uniffi enable seamless integration with Rust functionalities within a React Native application. The play method invokes the startSineWave() function defined in the Rust backend, facilitating direct control over audio generation and playback operations. This integration leverages Uniffi’s capabilities to bridge Rust and Kotlin, ensuring efficient communication between different parts of the application stack.
package com.sampleapp
import com.facebook.react.bridge.Promise
import android.util.Log
import com.sampleapp.uniffi.startSineWave
import com.facebook.react.bridge.ReactApplicationContext
import com.facebook.react.bridge.ReactContextBaseJavaModule
import com.facebook.react.bridge.ReactMethod
class BackendModule internal constructor(context: ReactApplicationContext?) : ReactContextBaseJavaModule(context) {
override fun getName(): String {
return "BackendModule"
}
@ReactMethod
fun play(promise: Promise) {
var isPlaying = startSineWave()
promise.resolve(isPlaying)
}
}React Native Frontend
Finally, let’s use the above created kotlin bindings to trigger the sine wave generation using a simple TailwindCSS UI.
import React, {useEffect, useState} from 'react';
import {Button, NativeModules, SafeAreaView, View} from 'react-native';
const {BackendModule} = NativeModules;
const SineWave = () => {
const [isPlaying, setIsPlaying] = useState(false);
const play = () => {
setIsPlaying(true);
setTimeout(async () => {
const isPlaying = await BackendModule.play();
setIsPlaying(isPlaying);
}, 100);
};
return (
<View className="w-screen h-screen flex items-center justify-center bg-white">
{!isPlaying ? (
<Button title="Play Sine Wave" color="#241584" onPress={play} />
) : (
<Button title="Playing" color="#24DD84" />
)}
</View>
);
};
function App(): JSX.Element {
return (
<SafeAreaView>
<SineWave />
</SafeAreaView>
);
}
export default App;This post took a trivial example of playing sine waves through Oboe to demonstrate how to integrate Oboe with Rust for React Native Android apps. However, with the same foundation you can have rust with Oboe manage your app’s entire Audio with high performance.
If you want to see how to integrate TailwindCSS into your react native app, follow this short tutorial I’ve written on it.