|
| 1 | +# Custom Audio Source |
| 2 | + |
| 3 | +This guide explains how to use the `CustomAudioSource` class to push audio data from external sources directly to the WebRTC audio pipeline. |
| 4 | + |
| 5 | +## Overview |
| 6 | + |
| 7 | +The `CustomAudioSource` allows you to provide audio data from custom sources such as: |
| 8 | +- Audio files |
| 9 | +- Network streams |
| 10 | +- Generated audio (tones, noise, etc.) |
| 11 | +- Audio processing libraries |
| 12 | +- Any other source of raw audio data |
| 13 | + |
| 14 | +This is particularly useful when you need to: |
| 15 | +- Stream pre-recorded audio |
| 16 | +- Process audio before sending it |
| 17 | +- Generate synthetic audio |
| 18 | +- Integrate with external audio APIs |
| 19 | + |
| 20 | +## Basic Usage |
| 21 | + |
| 22 | +### Creating a Custom Audio Source |
| 23 | + |
| 24 | +To use a custom audio source, you first need to create an instance: |
| 25 | + |
| 26 | +```java |
| 27 | +import dev.onvoid.webrtc.media.audio.CustomAudioSource; |
| 28 | + |
| 29 | +// Create a new CustomAudioSource instance |
| 30 | +CustomAudioSource audioSource = new CustomAudioSource(); |
| 31 | +``` |
| 32 | + |
| 33 | +### Creating an Audio Track |
| 34 | + |
| 35 | +Once you have a custom audio source, you can create an audio track with it: |
| 36 | + |
| 37 | +```java |
| 38 | +import dev.onvoid.webrtc.PeerConnectionFactory; |
| 39 | +import dev.onvoid.webrtc.media.audio.AudioTrack; |
| 40 | + |
| 41 | +// Create a PeerConnectionFactory (you should already have this in your WebRTC setup) |
| 42 | +PeerConnectionFactory factory = new PeerConnectionFactory(); |
| 43 | + |
| 44 | +// Create an audio track using the custom audio source |
| 45 | +AudioTrack audioTrack = factory.createAudioTrack("audio-track-id", audioSource); |
| 46 | +``` |
| 47 | + |
| 48 | +### Pushing Audio Data |
| 49 | + |
| 50 | +The key feature of `CustomAudioSource` is the ability to push audio data directly to the WebRTC pipeline: |
| 51 | + |
| 52 | +```java |
| 53 | +// Parameters for the audio data |
| 54 | +int bitsPerSample = 16; // Common values: 8, 16, 32 |
| 55 | +int sampleRate = 48000; // Common values: 8000, 16000, 44100, 48000 |
| 56 | +int channels = 2; // 1 for mono, 2 for stereo |
| 57 | +int frameCount = 480; // For 10ms of audio at 48kHz |
| 58 | + |
| 59 | +// Create a buffer for the audio data |
| 60 | +// Size = frameCount * channels * (bitsPerSample / 8) |
| 61 | +int bytesPerSample = bitsPerSample / 8; |
| 62 | +byte[] audioData = new byte[frameCount * channels * bytesPerSample]; |
| 63 | + |
| 64 | +// Fill the buffer with your audio data |
| 65 | +// ... |
| 66 | + |
| 67 | +// Push the audio data to the WebRTC pipeline |
| 68 | +audioSource.pushAudio(audioData, bitsPerSample, sampleRate, channels, frameCount); |
| 69 | +``` |
| 70 | + |
| 71 | +## Audio Format Considerations |
| 72 | + |
| 73 | +When pushing audio data, you need to consider the following parameters: |
| 74 | + |
| 75 | +### Bits Per Sample |
| 76 | +- **8-bit**: Lower quality, smaller data size |
| 77 | +- **16-bit**: Standard quality for most applications |
| 78 | +- **32-bit**: Higher quality, larger data size |
| 79 | + |
| 80 | +### Sample Rate |
| 81 | +- **8000 Hz**: Telephone quality |
| 82 | +- **16000 Hz**: Good for speech |
| 83 | +- **44100 Hz**: CD quality |
| 84 | +- **48000 Hz**: Standard for digital audio workstations and professional audio |
| 85 | + |
| 86 | +### Channels |
| 87 | +- **1 (Mono)**: Single channel audio |
| 88 | +- **2 (Stereo)**: Dual channel audio |
| 89 | + |
| 90 | +### Frame Count |
| 91 | +The number of frames depends on the desired buffer size and sample rate. For a 10ms buffer: |
| 92 | +- At 8000 Hz: 80 frames |
| 93 | +- At 16000 Hz: 160 frames |
| 94 | +- At 44100 Hz: 441 frames |
| 95 | +- At 48000 Hz: 480 frames |
| 96 | + |
| 97 | +## Advanced Usage |
| 98 | + |
| 99 | +### Continuous Audio Streaming |
| 100 | + |
| 101 | +For continuous streaming, you'll typically push audio data in a separate thread: |
| 102 | + |
| 103 | +```java |
| 104 | +import java.util.concurrent.Executors; |
| 105 | +import java.util.concurrent.ScheduledExecutorService; |
| 106 | +import java.util.concurrent.TimeUnit; |
| 107 | + |
| 108 | +public class AudioStreamer { |
| 109 | + private final CustomAudioSource audioSource; |
| 110 | + private final ScheduledExecutorService executor; |
| 111 | + private final int bitsPerSample = 16; |
| 112 | + private final int sampleRate = 48000; |
| 113 | + private final int channels = 2; |
| 114 | + private final int frameCount = 480; // 10ms at 48kHz |
| 115 | + |
| 116 | + public AudioStreamer(CustomAudioSource audioSource) { |
| 117 | + this.audioSource = audioSource; |
| 118 | + this.executor = Executors.newSingleThreadScheduledExecutor(); |
| 119 | + } |
| 120 | + |
| 121 | + public void start() { |
| 122 | + // Schedule task to run every 10ms |
| 123 | + executor.scheduleAtFixedRate(this::pushNextAudioBuffer, 0, 10, TimeUnit.MILLISECONDS); |
| 124 | + } |
| 125 | + |
| 126 | + public void stop() { |
| 127 | + executor.shutdown(); |
| 128 | + } |
| 129 | + |
| 130 | + private void pushNextAudioBuffer() { |
| 131 | + // Create and fill audio buffer |
| 132 | + int bytesPerSample = bitsPerSample / 8; |
| 133 | + byte[] audioData = new byte[frameCount * channels * bytesPerSample]; |
| 134 | + |
| 135 | + // Fill audioData with your audio samples |
| 136 | + // ... |
| 137 | + |
| 138 | + // Push to WebRTC |
| 139 | + audioSource.pushAudio(audioData, bitsPerSample, sampleRate, channels, frameCount); |
| 140 | + } |
| 141 | +} |
| 142 | +``` |
| 143 | + |
| 144 | +## Integration with Audio Tracks |
| 145 | + |
| 146 | +### Adding Sinks to Monitor Audio |
| 147 | + |
| 148 | +You can add sinks to the audio track to monitor the audio data: |
| 149 | + |
| 150 | +```java |
| 151 | +import dev.onvoid.webrtc.media.audio.AudioTrackSink; |
| 152 | + |
| 153 | +// Create a sink to monitor the audio data |
| 154 | +AudioTrackSink monitorSink = (data, bitsPerSample, sampleRate, channels, frames) -> { |
| 155 | + System.out.println("Received audio: " + |
| 156 | + bitsPerSample + " bits, " + |
| 157 | + sampleRate + " Hz, " + |
| 158 | + channels + " channels, " + |
| 159 | + frames + " frames"); |
| 160 | + |
| 161 | + // You can process or analyze the audio data here |
| 162 | +}; |
| 163 | + |
| 164 | +// Add the sink to the audio track |
| 165 | +audioTrack.addSink(monitorSink); |
| 166 | + |
| 167 | +// When done, remove the sink |
| 168 | +audioTrack.removeSink(monitorSink); |
| 169 | +``` |
| 170 | + |
| 171 | +## Cleanup |
| 172 | + |
| 173 | +When you're done with the custom audio source, make sure to clean up resources: |
| 174 | + |
| 175 | +```java |
| 176 | +// Dispose of the audio track |
| 177 | +audioTrack.dispose(); |
| 178 | + |
| 179 | +// If you're using a scheduled executor for pushing audio |
| 180 | +audioStreamer.stop(); |
| 181 | +``` |
| 182 | + |
| 183 | +--- |
| 184 | + |
| 185 | +## Conclusion |
| 186 | + |
| 187 | +The `CustomAudioSource` provides a flexible way to integrate external audio sources with WebRTC. By understanding the audio format parameters and properly managing the audio data flow, you can create applications that use custom audio from virtually any source. |
| 188 | + |
| 189 | +For more advanced audio processing options, consider exploring the audio processing APIs available in this documentation. |
0 commit comments