Silent Audio Failure When Receiving Mono Audio in Unity SDK

[FushimiYuki]

Problem

When a Unity client receives mono (1 channel) audio from remote participants, the audio silently fails to play with no errors or warnings. The issue is hard to debug because:

• ✅ AudioStream creates successfully
• ✅ AudioSource.isPlaying = true
• ✅ All logs show "success"
• ❌ But no audio is heard

Reproduction Steps

1. Remote participant publishes audio using a mono microphone (most built-in laptop mics are mono)
2. Unity client subscribes to the audio track:

   var audioSource = gameObject.AddComponent<AudioSource>();
   var audioStream = new AudioStream(remoteAudioTrack, audioSource);

3. Result: Complete silence, no error messages
4. Other clients (Web, mobile) hear the audio fine

Environment

• Unity 2022.3+ with default audio settings (stereo output)
• Remote participant with mono microphone
• LiveKit Unity SDK (latest)

Root Cause

Location: client-sdk-rust~/webrtc-sys/src/audio_resampler.cpp

WebRTC's RemixAndResample() returns 0 bytes when converting mono→stereo with same sample rate:

Input:  1 channel, 480 samples, 48kHz
Output: 2 channels, 0 samples, 48kHz  ← samples_per_channel is 0
Result: Empty data, no audio

The function correctly sets num_channels=2 but fails to set samples_per_channel, resulting in empty output.

Impact

• Severity: High - Complete audio loss
• Silent failure: No errors in logs, very hard to debug
• Common scenario: Many users have mono microphones
• All platforms affected: macOS, Windows, Linux, iOS, Android

Workaround

Add this to Runtime/Scripts/AudioStream.cs in OnAudioStreamEvent():

var uFrame = _resampler.RemixAndResample(frame, _numChannels, _sampleRate);

// Workaround for mono→stereo bug
if ((uFrame == null || uFrame.Length == 0) && 
    frame.NumChannels == 1 && _numChannels == 2 && 
    frame.SampleRate == _sampleRate)
{
    // Manual mono to stereo conversion
    int samplesPerChannel = (int)frame.SamplesPerChannel;
    short[] monoData = new short[samplesPerChannel];
    short[] stereoData = new short[samplesPerChannel * 2];
    
    var monoSpan = new Span<byte>(frame.Data.ToPointer(), frame.Length);
    MemoryMarshal.Cast<byte, short>(monoSpan).CopyTo(monoData);
    
    for (int i = 0; i < samplesPerChannel; i++)
    {
        stereoData[i * 2] = monoData[i];     // Left
        stereoData[i * 2 + 1] = monoData[i]; // Right
    }
    
    var stereoBytes = MemoryMarshal.Cast<short, byte>(stereoData.AsSpan());
    _buffer?.Write(stereoBytes);
    return; // Skip normal path
}

// Normal path...
if (uFrame != null && uFrame.Length > 0)
{
    var data = new Span<byte>(uFrame.Data.ToPointer(), uFrame.Length);
    _buffer?.Write(data);
}

This workaround successfully restores audio playback.

Why This Matters

This is a critical usability bug because:

1. Users expect audio to "just work"
2. No error messages make debugging extremely difficult
3. Affects real-world usage (mono mics are common)
4. Works fine in Web SDK, so users won't expect Unity to fail

Note

This workaround is a patch that allows my project to run properly for now. However, I'm unsure if there's a more elegant solution. Would appreciate guidance on whether:

1. This is the recommended approach, or
2. There's a better way to handle this in the SDK

Thanks for looking into this!

[sarwar-hsn]

Thank you

[ladvoc]

@FushimiYuki, thank you for reporting this issue—we will investigate. As for logging, that will be added soon by #167.

[michael811125]

Problem Description

When receiving remote audio streams with mono (1 channel) audio that needs to be played on a stereo (2 channel) output device, the AudioResampler.RemixAndResample() method returns null or an empty frame, resulting in no audio playback.

This issue is particularly common when:

• Remote participant sends mono audio (e.g., from mobile devices or certain microphones)
• Local device expects stereo output (most Unity configurations default to stereo)
• Sample rate conversion is also required (e.g., 16000Hz → 48000Hz)

Environment

• Unity 6000.0.60f1
• LiveKit Unity SDK
• Affected platforms: Android, iOS, Windows, macOS

Symptoms

• AudioResampler.RemixAndResample() returns null when input is mono and output is stereo
• No audio heard from remote participants
• Debug logs show frames are being received but buffer remains empty

---

Workaround Solution

I've implemented an enhanced AudioStream.cs with manual mono-to-stereo conversion and resampling as a fallback when the native resampler fails.

Complete AudioStream.cs Implementation

using LiveKit.Internal;
using LiveKit.Internal.FFIClients.Requests;
using LiveKit.Proto;
using System;
using System.Runtime.InteropServices;
using UnityEngine;

namespace LiveKit
{
    public sealed class AudioStream : IDisposable
    {
        internal readonly FfiHandle Handle;
        private readonly AudioSource _audioSource;
        private RingBuffer _buffer;
        private short[] _tempBuffer;
        private uint _numChannels;
        private uint _sampleRate;
        private AudioResampler _resampler = new AudioResampler();
        private object _lock = new object();
        private bool _disposed = false;

        // Buffers for mono to stereo conversion workaround
        private short[] _conversionBuffer = null;
        private short[] _resampleBuffer = null;

        public AudioStream(RemoteAudioTrack audioTrack, AudioSource source)
        {
            if (!audioTrack.Room.TryGetTarget(out var room))
                throw new InvalidOperationException("audiotrack's room is invalid");

            if (!audioTrack.Participant.TryGetTarget(out var participant))
                throw new InvalidOperationException("audiotrack's participant is invalid");

            using var request = FFIBridge.Instance.NewRequest<NewAudioStreamRequest>();
            var newAudioStream = request.request;
            newAudioStream.TrackHandle = (ulong)(audioTrack as ITrack).TrackHandle.DangerousGetHandle();
            newAudioStream.Type = AudioStreamType.AudioStreamNative;

            using var response = request.Send();
            FfiResponse res = response;
            Handle = FfiHandle.FromOwnedHandle(res.NewAudioStream.Stream.Handle);
            FfiClient.Instance.AudioStreamEventReceived += OnAudioStreamEvent;

            _audioSource = source;
            ConfigureAudioSource(_audioSource);

            var probe = _audioSource.gameObject.AddComponent<AudioProbe>();
            probe.AudioRead += OnAudioRead;

            if (!_audioSource.isPlaying)
            {
                _audioSource.Play();
            }
        }

        private void ConfigureAudioSource(AudioSource source)
        {
            source.loop = true;
            source.playOnAwake = false;
            source.spatialBlend = 0f;
            source.volume = 1f;
            source.mute = false;

            // ============================================================
            // Android-specific AudioSource configuration
            // ============================================================
            // priority = 0:
            //   - Highest priority (0 = highest, 256 = lowest/default)
            //   - Unity has limited audio channels on Android (~32)
            //   - When exceeded, lower priority sources are muted
            //   - Remote audio streams must never be muted by the system
            //
            // outputAudioMixerGroup = null:
            //   - Bypass AudioMixer, output directly to system
            //   - Reduces audio processing latency
            //   - Avoids AudioMixer effects (compression, EQ) affecting voice
            //   - Prevents AudioMixer-related issues on some Android devices
            // ============================================================
#if UNITY_ANDROID && !UNITY_EDITOR
            source.priority = 0;
            source.outputAudioMixerGroup = null;
#endif

            int frequency = AudioSettings.outputSampleRate;
            if (frequency == 0) frequency = 48000;

            var dummyClip = AudioClip.Create(
                "AudioStreamClip",
                frequency * 2,
                2,
                frequency,
                false
            );

            source.clip = dummyClip;
        }

        private void OnAudioRead(float[] data, int channels, int sampleRate)
        {
            lock (_lock)
            {
                Array.Clear(data, 0, data.Length);

                if (_buffer == null || channels != _numChannels || sampleRate != _sampleRate)
                {
                    // ============================================================
                    // Platform-specific buffer size configuration
                    // ============================================================
                    // Android: 500ms buffer - AudioTrack callback timing is unstable,
                    //          hardware varies across manufacturers
                    // iOS/Desktop: 200ms buffer - Core Audio and WASAPI provide 
                    //              stable, low-latency callbacks
                    // ============================================================
#if UNITY_ANDROID && !UNITY_EDITOR
                    int size = (int)(channels * sampleRate * 0.5);
#else
                    int size = (int)(channels * sampleRate * 0.2);
#endif

                    _buffer?.Dispose();
                    _buffer = new RingBuffer(size * sizeof(short));
                    _numChannels = (uint)channels;
                    _sampleRate = (uint)sampleRate;
                    _tempBuffer = null;
                }

                if (_tempBuffer == null || data.Length != _tempBuffer.Length)
                {
                    _tempBuffer = new short[data.Length];
                }

                Array.Clear(_tempBuffer, 0, _tempBuffer.Length);

                static float S16ToFloat(short v)
                {
                    return v / 32768f;
                }

                var temp = MemoryMarshal.Cast<short, byte>(_tempBuffer.AsSpan());
                int read = _buffer.Read(temp);
                int samplesRead = read / sizeof(short);

                if (samplesRead > 0)
                {
                    for (int i = 0; i < samplesRead && i < data.Length; i++)
                    {
                        data[i] = S16ToFloat(_tempBuffer[i]);
                    }
                }
            }
        }

        private void OnAudioStreamEvent(AudioStreamEvent e)
        {
            if ((ulong)Handle.DangerousGetHandle() != e.StreamHandle)
                return;

            if (e.MessageCase != AudioStreamEvent.MessageOneofCase.FrameReceived)
                return;

            var frame = new AudioFrame(e.FrameReceived.Frame);

            lock (_lock)
            {
                // Force-initialize buffer if not ready when frame arrives
                if (_buffer == null || _numChannels == 0)
                {
                    int channels = AudioSettings.GetConfiguration().speakerMode == AudioSpeakerMode.Mono ? 1 : 2;
                    int sampleRate = AudioSettings.outputSampleRate;
                    if (sampleRate == 0) sampleRate = 48000;

#if UNITY_ANDROID && !UNITY_EDITOR
                    int size = (int)(channels * sampleRate * 0.5);
#else
                    int size = (int)(channels * sampleRate * 0.2);
#endif

                    _buffer?.Dispose();
                    _buffer = new RingBuffer(size * sizeof(short));
                    _numChannels = (uint)channels;
                    _sampleRate = (uint)sampleRate;
                }

                unsafe
                {
                    var uFrame = _resampler.RemixAndResample(frame, _numChannels, _sampleRate);

                    // ============================================================
                    // WORKAROUND: Enhanced Mono→Stereo conversion with resampling
                    // ============================================================
                    // When native resampler fails for mono→stereo conversion,
                    // perform manual conversion as fallback.
                    // ============================================================
                    if ((uFrame == null || uFrame.Length == 0) && frame.NumChannels == 1 && _numChannels == 2)
                    {
                        int inputSamplesPerChannel = (int)frame.SamplesPerChannel;

                        // Validate frame data
                        if (inputSamplesPerChannel <= 0 || frame.Length != inputSamplesPerChannel * sizeof(short))
                        {
                            return;
                        }

                        short* monoPtr = (short*)frame.Data.ToPointer();

                        // ---------------------------------------------------------
                        // Step 1: Resample mono data if sample rates differ
                        // ---------------------------------------------------------
                        short[] monoData;
                        int monoSampleCount;

                        if (frame.SampleRate != _sampleRate)
                        {
                            int outputSampleCount = (int)(inputSamplesPerChannel * _sampleRate / frame.SampleRate);

                            if (_resampleBuffer == null || _resampleBuffer.Length < outputSampleCount)
                            {
                                _resampleBuffer = new short[outputSampleCount];
                            }

                            // Linear interpolation resampling
                            float ratio = (float)frame.SampleRate / _sampleRate;
                            for (int i = 0; i < outputSampleCount; i++)
                            {
                                float srcIndex = i * ratio;
                                int srcIndexInt = (int)srcIndex;
                                float frac = srcIndex - srcIndexInt;

                                if (srcIndexInt + 1 < inputSamplesPerChannel)
                                {
                                    short sample1 = monoPtr[srcIndexInt];
                                    short sample2 = monoPtr[srcIndexInt + 1];
                                    _resampleBuffer[i] = (short)(sample1 + frac * (sample2 - sample1));
                                }
                                else
                                {
                                    _resampleBuffer[i] = monoPtr[srcIndexInt];
                                }
                            }

                            monoData = _resampleBuffer;
                            monoSampleCount = outputSampleCount;
                        }
                        else
                        {
                            // No resampling needed, just copy
                            if (_resampleBuffer == null || _resampleBuffer.Length < inputSamplesPerChannel)
                            {
                                _resampleBuffer = new short[inputSamplesPerChannel];
                            }

                            for (int i = 0; i < inputSamplesPerChannel; i++)
                            {
                                _resampleBuffer[i] = monoPtr[i];
                            }

                            monoData = _resampleBuffer;
                            monoSampleCount = inputSamplesPerChannel;
                        }

                        // ---------------------------------------------------------
                        // Step 2: Convert mono to stereo (duplicate to both channels)
                        // ---------------------------------------------------------
                        int stereoSampleCount = monoSampleCount * 2;

                        if (_conversionBuffer == null || _conversionBuffer.Length < stereoSampleCount)
                        {
                            _conversionBuffer = new short[stereoSampleCount];
                        }

                        for (int i = 0; i < monoSampleCount; i++)
                        {
                            short sample = monoData[i];
                            _conversionBuffer[i * 2] = sample;       // Left channel
                            _conversionBuffer[i * 2 + 1] = sample;   // Right channel
                        }

                        // ---------------------------------------------------------
                        // Step 3: Write to ring buffer
                        // ---------------------------------------------------------
                        var stereoBytes = MemoryMarshal.Cast<short, byte>(_conversionBuffer.AsSpan(0, stereoSampleCount));
                        _buffer.Write(stereoBytes);

                        return;
                    }

                    // Normal path: native resampler succeeded
                    if (uFrame != null && uFrame.Length > 0)
                    {
                        var data = new Span<byte>(uFrame.Data.ToPointer(), uFrame.Length);
                        _buffer.Write(data);
                    }
                }
            }
        }

        public void Dispose()
        {
            Dispose(true);
            GC.SuppressFinalize(this);
        }

        private void Dispose(bool disposing)
        {
            if (!_disposed && disposing)
            {
                lock (_lock)
                {
                    FfiClient.Instance.AudioStreamEventReceived -= OnAudioStreamEvent;
                    _audioSource.Stop();

                    if (_audioSource.clip != null)
                    {
                        UnityEngine.Object.Destroy(_audioSource.clip);
                        _audioSource.clip = null;
                    }

                    var probe = _audioSource.GetComponent<AudioProbe>();
                    if (probe != null)
                    {
                        probe.AudioRead -= OnAudioRead;
                        UnityEngine.Object.Destroy(probe);
                    }

                    _buffer?.Dispose();
                    _buffer = null;
                    _conversionBuffer = null;
                    _resampleBuffer = null;
                }
            }
            _disposed = true;
        }

        ~AudioStream()
        {
            Dispose(false);
        }
    }
}

---

How It Works

Processing Flow Diagram

┌────────────────────────────────────────────────────────────────┐
│  Incoming Frame: Mono @ 16000Hz                                │
└────────────────────────────────────────────────────────────────┘
                              │
                              ▼
┌────────────────────────────────────────────────────────────────┐
│  Try: _resampler.RemixAndResample(frame, 2ch, 48000Hz)         │
└────────────────────────────────────────────────────────────────┘
                              │
                    ┌─────────┴─────────┐
                    │                   │
               Returns data        Returns null
                    │                   │
                    ▼                   ▼
            ┌──────────────┐   ┌─────────────────────────┐
            │ Normal Path  │   │   Manual Workaround     │
            │ Write to buf │   ├─────────────────────────┤
            └──────────────┘   │ 1. Resample 16k → 48k   │
                               │    (linear interpolation)│
                               │ 2. Mono → Stereo        │
                               │    (duplicate samples)   │
                               │ 3. Write to buffer      │
                               └─────────────────────────┘

Key Features

Feature	Description
Automatic fallback	Only activates when native resampler fails
Linear interpolation	Efficient resampling with acceptable quality
Platform-specific buffers	Android: 500ms, iOS/Desktop: 200ms
Android audio priority	Ensures remote audio is never muted by system
Proper cleanup	Disposes all buffers and unsubscribes events

Platform-Specific Configuration

Platform	Buffer Duration	AudioSource Priority	Reason
Android	500ms	0 (highest)	Unstable callback timing; limited audio channels; hardware fragmentation
iOS	200ms	Default	Core Audio provides stable, low-latency callbacks
Desktop	200ms	Default	WASAPI (Windows) / Core Audio (macOS) are reliable

Android-Specific AudioSource Settings

Setting	Value	Purpose
priority	0	Highest priority (0-256). Ensures remote audio is never muted when Unity's audio channel limit is reached
outputAudioMixerGroup	null	Bypass AudioMixer to reduce latency and avoid processing artifacts on voice audio

---

Notes

• This workaround uses linear interpolation for resampling, which is computationally efficient but may not provide the highest audio quality. For most real-time communication scenarios, this is acceptable.
• The workaround only activates when the native resampler fails, so there's no performance impact when the native path works correctly.
• Tested with various sample rate combinations: 8000→48000, 16000→44100, 24000→48000, etc.
• Android devices require special handling due to hardware fragmentation and inconsistent audio behavior across manufacturers.

---

Hope this helps others facing the same issue! 🎧

[michael811125]

MicrophoneSource & RtcAudioSource - Complete Implementation

This comment provides the complete implementation for MicrophoneSource and RtcAudioSource with automatic audio format conversion (channels and sample rate).

Key Features

• Automatic channel conversion: Mono ↔ Stereo
• Automatic sample rate conversion: Linear interpolation resampling
• Platform-specific defaults: Optimized sample rates for iOS/Android/Desktop
• Custom parameter support: Specify channels and sample rate explicitly

---

Architecture Overview

┌─────────────────────────────────────────────────────────────────────────────┐
│                         Audio Capture Pipeline                              │
└─────────────────────────────────────────────────────────────────────────────┘

┌──────────────┐     ┌──────────────┐     ┌──────────────┐     ┌─────────────┐
│  Microphone  │     │  AudioProbe  │     │RtcAudioSource│     │   LiveKit   │
│  Hardware    │ ──► │   (Unity)    │ ──► │ (Conversion) │ ──► │   Server    │
└──────────────┘     └──────────────┘     └──────────────┘     └─────────────┘
     
  Actual output:      Callback provides:   Expected format:     Sent format:
  - May be 1ch        - channels           - _expectedChannels  - Unified
  - May be 24kHz      - sampleRate         - _expectedSampleRate  format
  - Device-dependent  - float[] data       - Set in constructor

---

Platform-Specific Configuration

Platform	Default Mic Sample Rate	Reason
iOS	24000Hz	iOS microphones typically output 24kHz natively
Android	24000Hz	Good balance for mobile devices
Desktop	48000Hz	Standard professional audio rate

---

Usage Examples

Complete Example with Parameter Detection

// ========================================
// Check Microphone Device Availability
// ========================================
if (Microphone.devices == null || Microphone.devices.Length == 0)
{
    Debug.LogError("No microphone devices available!");
    yield break;
}

string micDevice = Microphone.devices[0];
Debug.Log($"Using microphone: {micDevice}");

// Get microphone capabilities
int minFreq, maxFreq;
Microphone.GetDeviceCaps(micDevice, out minFreq, out maxFreq);
Debug.Log($"Mic capabilities - Min: {minFreq}Hz, Max: {maxFreq}Hz");

// ========================================
// Key: Start microphone first to detect actual parameters
// ========================================
// The actual output format may differ from what we request,
// depending on hardware capabilities.
Debug.Log("Testing microphone to detect actual parameters...");

// Try to start at 48000Hz (LiveKit's default)
var testClip = Microphone.Start(micDevice, false, 1, 48000);

// Wait for microphone to actually start
float waitTime = 0f;
while (Microphone.GetPosition(micDevice) <= 0 && waitTime < 5f)
{
    yield return new WaitForSeconds(0.1f);
    waitTime += 0.1f;
}

if (Microphone.GetPosition(micDevice) <= 0)
{
    Debug.LogError("Microphone failed to start!");
    Microphone.End(micDevice);
    yield break;
}

// Get actual parameters from the AudioClip
int actualChannels = testClip.channels;
int actualSampleRate = testClip.frequency;

Debug.Log($"✓ Microphone actual parameters: {actualChannels} channels @ {actualSampleRate}Hz");

// Stop test recording
Microphone.End(micDevice);
yield return new WaitForSeconds(0.5f); // Wait for microphone to fully release

// ========================================
// Create audio source with detected parameters
// ========================================
var localSid = "my-audio-source";
GameObject audObject = new GameObject(localSid);
_audioObjects[localSid] = audObject;

// Create MicrophoneSource with actual hardware parameters
// RtcAudioSource will handle any necessary format conversion internally
var rtcSource = new MicrophoneSource(micDevice, audObject, actualChannels, actualSampleRate);
rtcSource.Start();
Debug.Log("Microphone started with correct parameters");

---

MicrophoneSource.cs

using System;
using System.Collections;
using UnityEngine;

namespace LiveKit
{
    /// <summary>
    /// An audio source which captures from the device's microphone.
    /// </summary>
    /// <remarks>
    /// Ensure microphone permissions are granted before calling <see cref="Start"/>.
    /// </remarks>
    sealed public class MicrophoneSource : RtcAudioSource
    {
        private readonly GameObject _sourceObject;
        private readonly string _deviceName;
        private readonly int _targetSampleRate;

        public override event Action<float[], int, int> AudioRead;

        private bool _disposed = false;
        private bool _started = false;

        /// <summary>
        /// Creates a new microphone source with specified audio parameters.
        /// </summary>
        /// <param name="deviceName">
        /// The microphone device name. Use <see cref="Microphone.devices"/> to get available devices.
        /// Pass null or empty string for the default device.
        /// </param>
        /// <param name="sourceObject">
        /// The GameObject to attach AudioSource and AudioProbe components.
        /// Must remain active in the scene during capture.
        /// </param>
        /// <param name="channels">
        /// Target number of audio channels (1=mono, 2=stereo).
        /// Note: Actual microphone output may differ; RtcAudioSource will convert if needed.
        /// </param>
        /// <param name="sampleRate">
        /// Target sample rate in Hz (e.g., 24000, 44100, 48000).
        /// 
        /// Platform recommendations:
        /// - iOS: 24000Hz (native mic output, avoids resampling)
        /// - Android: 24000Hz or 48000Hz
        /// - Desktop: 48000Hz
        /// </param>
        public MicrophoneSource(string deviceName, GameObject sourceObject, int channels, int sampleRate) 
            : base(channels, RtcAudioSourceType.AudioSourceMicrophone, sampleRate)
        {
            _deviceName = deviceName;
            _sourceObject = sourceObject;
            _targetSampleRate = sampleRate;
            UnityEngine.Debug.Log($"MicrophoneSource created: {channels}ch @ {sampleRate}Hz");
        }

        /// <summary>
        /// Creates a new microphone source with default audio parameters.
        /// Uses 2 channels and platform-specific default sample rate.
        /// </summary>
        /// <remarks>
        /// Default sample rates (defined in RtcAudioSource):
        /// - iOS/Android: 24000Hz (optimized for mobile microphones)
        /// - Desktop: 48000Hz (standard professional audio rate)
        /// </remarks>
        public MicrophoneSource(string deviceName, GameObject sourceObject) 
            : base(2, RtcAudioSourceType.AudioSourceMicrophone)
        {
            _deviceName = deviceName;
            _sourceObject = sourceObject;
        }

        /// <summary>
        /// Begins capturing audio from the microphone.
        /// </summary>
        /// <exception cref="InvalidOperationException">
        /// Thrown when the microphone is not available or unauthorized.
        /// </exception>
        /// <remarks>
        /// Ensure microphone permissions are granted before calling this method
        /// by calling <see cref="Application.RequestUserAuthorization"/>.
        /// </remarks>
        public override void Start()
        {
            base.Start();
            if (_started) return;

            if (!Application.HasUserAuthorization(mode: UserAuthorization.Microphone))
                throw new InvalidOperationException("Microphone access not authorized");

            MonoBehaviourContext.OnApplicationPauseEvent += OnApplicationPause;
            MonoBehaviourContext.RunCoroutine(StartMicrophone());

            _started = true;
        }

        private IEnumerator StartMicrophone()
        {
            UnityEngine.Debug.Log($"Starting microphone: {_deviceName} @ {_targetSampleRate}Hz");

            // Note: The actual AudioClip parameters may differ from what was requested,
            // depending on hardware capabilities. Unity will use the closest supported configuration.
            var clip = Microphone.Start(
               _deviceName,
               loop: true,
               lengthSec: 1,
               frequency: _targetSampleRate
            );

            if (clip == null)
            {
                UnityEngine.Debug.LogError("Microphone.Start returned null!");
                yield break;
            }

            var source = _sourceObject.AddComponent<AudioSource>();
            source.clip = clip;
            source.loop = true;

            var probe = _sourceObject.AddComponent<AudioProbe>();
            probe.ClearAfterInvocation();
            probe.AudioRead += OnAudioRead;

            // Wait until the microphone has started recording
            float waitTime = 0f;
            while (Microphone.GetPosition(_deviceName) <= 0 && waitTime < 5f)
            {
                yield return new WaitForSeconds(0.1f);
                waitTime += 0.1f;
            }

            if (Microphone.GetPosition(_deviceName) <= 0)
            {
                UnityEngine.Debug.LogError("Microphone failed to start within timeout!");
                yield break;
            }

            // Log actual vs requested parameters
            // The AudioClip's actual channels and frequency may differ from what was requested.
            // RtcAudioSource.OnAudioRead will handle any necessary conversion automatically.
            UnityEngine.Debug.Log($"✓ Microphone ready: {clip.channels}ch @ {clip.frequency}Hz (requested: {_targetSampleRate}Hz)");

            source.Play();

            UnityEngine.Debug.Log("✓ Microphone started and playing");
        }

        /// <summary>
        /// Stops capturing audio from the microphone.
        /// </summary>
        public override void Stop()
        {
            base.Stop();
            MonoBehaviourContext.RunCoroutine(StopMicrophone());
            MonoBehaviourContext.OnApplicationPauseEvent -= OnApplicationPause;
            _started = false;
        }

        private IEnumerator StopMicrophone()
        {
            if (Microphone.IsRecording(_deviceName))
                Microphone.End(_deviceName);

            var probe = _sourceObject.GetComponent<AudioProbe>();
            if (probe != null)
            {
                probe.AudioRead -= OnAudioRead;
                UnityEngine.Object.Destroy(probe);
            }

            var source = _sourceObject.GetComponent<AudioSource>();
            if (source != null)
            {
                UnityEngine.Object.Destroy(source);
            }
            yield return null;
        }

        /// <summary>
        /// Receives audio data from Unity's audio system and forwards it to RtcAudioSource.
        /// The callback provides ACTUAL audio data from the microphone hardware,
        /// which may differ from the requested format.
        /// RtcAudioSource.OnAudioRead will automatically convert if needed.
        /// </summary>
        private void OnAudioRead(float[] data, int channels, int sampleRate)
        {
            AudioRead?.Invoke(data, channels, sampleRate);
        }

        private void OnApplicationPause(bool pause)
        {
            if (!pause && _started)
                MonoBehaviourContext.RunCoroutine(RestartMicrophone());
        }

        private IEnumerator RestartMicrophone()
        {
            yield return StopMicrophone();
            yield return StartMicrophone();
        }

        protected override void Dispose(bool disposing)
        {
            if (!_disposed && disposing) Stop();
            _disposed = true;
            base.Dispose(disposing);
        }

        ~MicrophoneSource()
        {
            Dispose(false);
        }
    }
}

---

RtcAudioSource.cs

using LiveKit.Internal;
using LiveKit.Internal.FFIClients.Requests;
using LiveKit.Proto;
using System;
using System.Buffers;
using System.Collections;
using Unity.Collections;
using Unity.Collections.LowLevel.Unsafe;

namespace LiveKit
{
    /// <summary>
    /// Defines the type of audio source, influencing processing behavior.
    /// </summary>
    public enum RtcAudioSourceType
    {
        AudioSourceCustom = 0,
        AudioSourceMicrophone = 1
    }

    /// <summary>
    /// Capture source for a local audio track.
    /// Handles audio format conversion (channels and sample rate) automatically.
    /// </summary>
    public abstract class RtcAudioSource : IRtcSource, IDisposable
    {
        /// <summary>
        /// Event triggered when audio samples are captured from the underlying source.
        /// Provides the audio data, channel count, and sample rate.
        /// </summary>
        /// <remarks>
        /// This event is not guaranteed to be called on the main thread.
        /// </remarks>
        public abstract event Action<float[], int, int> AudioRead;

        // ============================================================
        // Default Sample Rate Configuration
        // ============================================================
        // Platform-specific defaults based on typical hardware capabilities:
        //
        // | Platform | Default    | Reason                                    |
        // |----------|------------|-------------------------------------------|
        // | iOS      | 24000Hz    | iOS mics typically output 24kHz natively  |
        // | Android  | 24000Hz    | Good balance for mobile devices           |
        // | Desktop  | 48000Hz    | Standard professional audio rate          |
        // ============================================================
#if UNITY_IOS && !UNITY_EDITOR || UNITY_ANDROID && !UNITY_EDITOR
        public static uint DefaultSampleRate = 48000;
        public static uint DefaultMicrophoneSampleRate = 24000;  // Optimized for mobile mics
#else
        public static uint DefaultSampleRate = 48000;
        public static uint DefaultMicrophoneSampleRate = DefaultSampleRate;
#endif
        public static uint DefaultChannels = 2;

        private readonly RtcAudioSourceType _sourceType;
        public RtcAudioSourceType SourceType => _sourceType;

        internal readonly FfiHandle Handle;
        protected AudioSourceInfo _info;

        // Expected audio format - incoming audio will be converted to match these values
        private readonly uint _expectedChannels;
        private readonly uint _expectedSampleRate;

        /// <summary>
        /// Temporary frame buffer for invoking the FFI capture method.
        /// </summary>
        private NativeArray<short> _frameData;

        /// <summary>
        /// Cache for resampling to reduce allocations.
        /// </summary>
        private float[] _resampleBuffer = null;

        private bool _muted = false;
        public override bool Muted => _muted;

        private bool _started = false;
        private bool _disposed = false;

        private readonly object _lock = new object();

        /// <summary>
        /// Creates a new RTC audio source with specified parameters.
        /// </summary>
        /// <param name="channels">
        /// Number of audio channels to send to LiveKit (1=mono, 2=stereo).
        /// Incoming audio will be converted to this channel count if different.
        /// </param>
        /// <param name="audioSourceType">
        /// Type of audio source. Affects default sample rate selection:
        /// - AudioSourceMicrophone: Uses DefaultMicrophoneSampleRate
        /// - AudioSourceCustom: Uses DefaultSampleRate
        /// </param>
        /// <param name="customSampleRate">
        /// Custom sample rate in Hz. If null or &lt;= 0, uses platform-specific default.
        /// Common values: 16000 (speech), 24000 (mobile), 44100 (CD), 48000 (professional)
        /// </param>
        protected RtcAudioSource(int channels = 2, RtcAudioSourceType audioSourceType = RtcAudioSourceType.AudioSourceCustom, int? customSampleRate = null)
        {
            _sourceType = audioSourceType;

            try
            {
                using var request = FFIBridge.Instance.NewRequest<NewAudioSourceRequest>();
                var newAudioSource = request.request;
                newAudioSource.Type = AudioSourceType.AudioSourceNative;
                newAudioSource.NumChannels = (uint)channels;

                // Sample rate selection: custom value takes priority, otherwise use platform default
                if (customSampleRate.HasValue && customSampleRate.Value > 0)
                {
                    newAudioSource.SampleRate = (uint)customSampleRate.Value;
                    UnityEngine.Debug.Log($"Using custom sample rate: {customSampleRate.Value}Hz");
                }
                else
                {
                    newAudioSource.SampleRate = _sourceType == RtcAudioSourceType.AudioSourceMicrophone 
                        ? DefaultMicrophoneSampleRate 
                        : DefaultSampleRate;
                    UnityEngine.Debug.Log($"Using default sample rate: {newAudioSource.SampleRate}Hz");
                }

                // Store expected values for format conversion in OnAudioRead
                _expectedChannels = newAudioSource.NumChannels;
                _expectedSampleRate = newAudioSource.SampleRate;

                UnityEngine.Debug.Log($"NewAudioSource: {newAudioSource.NumChannels}ch @ {newAudioSource.SampleRate}Hz");

                newAudioSource.Options = request.TempResource<AudioSourceOptions>();
                newAudioSource.Options.EchoCancellation = true;
                newAudioSource.Options.AutoGainControl = true;
                newAudioSource.Options.NoiseSuppression = true;

                using var response = request.Send();
                FfiResponse res = response;
                _info = res.NewAudioSource.Source.Info;
                Handle = FfiHandle.FromOwnedHandle(res.NewAudioSource.Source.Handle);

                if (Handle == null || Handle.IsInvalid)
                {
                    throw new InvalidOperationException("Failed to create audio source handle");
                }

                UnityEngine.Debug.Log($"✓ Audio source created: Handle={Handle.DangerousGetHandle()}, Channels={_expectedChannels}, SampleRate={_expectedSampleRate}");
            }
            catch (Exception ex)
            {
                UnityEngine.Debug.LogError($"RtcAudioSource constructor failed: {ex.Message}\n{ex.StackTrace}");
                throw;
            }
        }

        /// <summary>
        /// Begin capturing audio samples from the underlying source.
        /// </summary>
        public virtual void Start()
        {
            if (_started) return;
            AudioRead += OnAudioRead;
            _started = true;
        }

        /// <summary>
        /// Stop capturing audio samples from the underlying source.
        /// </summary>
        public virtual void Stop()
        {
            if (!_started) return;
            AudioRead -= OnAudioRead;
            _started = false;
        }

        /// <summary>
        /// Processes incoming audio data and sends it to LiveKit.
        /// Automatically handles format conversion if needed.
        /// </summary>
        /// <remarks>
        /// Processing flow:
        /// 1. Check if channel conversion needed (e.g., mono mic → stereo expected)
        /// 2. Check if sample rate conversion needed (e.g., 44100Hz → 48000Hz)
        /// 3. Convert float to S16 and send to LiveKit
        /// </remarks>
        private void OnAudioRead(float[] data, int channels, int sampleRate)
        {
            if (_muted) return;

            lock (_lock)
            {
                try
                {
                    bool needsChannelConversion = channels != _expectedChannels;
                    bool needsSampleRateConversion = sampleRate != _expectedSampleRate;

                    float[] processedData = data;
                    int processedChannels = channels;
                    int processedSampleRate = sampleRate;

                    // === Channel Conversion ===
                    if (needsChannelConversion)
                    {
                        processedData = ConvertChannels(processedData, channels, (int)_expectedChannels);
                        processedChannels = (int)_expectedChannels;
                    }

                    // === Sample Rate Conversion ===
                    if (needsSampleRateConversion)
                    {
                        processedData = ResampleAudio(processedData, processedChannels, sampleRate, (int)_expectedSampleRate);
                        processedSampleRate = (int)_expectedSampleRate;
                    }

                    // === Convert to S16 and Send ===
                    SendAudioFrame(processedData, processedChannels, processedSampleRate);
                }
                catch (Exception ex)
                {
                    UnityEngine.Debug.LogError($"OnAudioRead failed: {ex.Message}\n{ex.StackTrace}");
                }
            }
        }

        /// <summary>
        /// Resamples audio using linear interpolation.
        /// </summary>
        /// <remarks>
        /// Formula: output[i] = sample1 + (sample2 - sample1) × fraction
        /// 
        /// Linear interpolation is computationally efficient and acceptable
        /// for real-time voice communication.
        /// </remarks>
        private float[] ResampleAudio(float[] inputData, int channels, int inputSampleRate, int outputSampleRate)
        {
            if (inputSampleRate == outputSampleRate)
                return inputData;

            int inputSamplesPerChannel = inputData.Length / channels;
            double ratio = (double)outputSampleRate / inputSampleRate;
            int outputSamplesPerChannel = (int)Math.Ceiling(inputSamplesPerChannel * ratio);
            int outputLength = outputSamplesPerChannel * channels;

            float[] result = ArrayPool<float>.Shared.Rent(outputLength);

            try
            {
                var inputSpan = inputData.AsSpan();
                var outputSpan = result.AsSpan(0, outputLength);

                for (int ch = 0; ch < channels; ch++)
                {
                    for (int i = 0; i < outputSamplesPerChannel; i++)
                    {
                        double srcPos = i / ratio;
                        int srcIndex = (int)srcPos;
                        double frac = srcPos - srcIndex;

                        int idx1 = Math.Min(srcIndex * channels + ch, inputData.Length - 1);
                        int idx2 = Math.Min((srcIndex + 1) * channels + ch, inputData.Length - 1);

                        float sample1 = inputSpan[idx1];
                        float sample2 = inputSpan[idx2];

                        outputSpan[i * channels + ch] = (float)(sample1 + (sample2 - sample1) * frac);
                    }
                }

                float[] finalResult = new float[outputLength];
                outputSpan.CopyTo(finalResult);
                return finalResult;
            }
            finally
            {
                ArrayPool<float>.Shared.Return(result);
            }
        }

        /// <summary>
        /// Converts audio between different channel configurations.
        /// </summary>
        /// <remarks>
        /// Mono → Stereo: Duplicate to both channels
        ///   [M0][M1] → [L0][R0][L1][R1] where L=R=M
        /// 
        /// Stereo → Mono: Average both channels
        ///   [L0][R0][L1][R1] → [M0][M1] where M=(L+R)/2
        /// </remarks>
        private float[] ConvertChannels(float[] data, int sourceChannels, int targetChannels)
        {
            if (sourceChannels == targetChannels)
                return data;

            int samplesPerChannel = data.Length / sourceChannels;
            float[] converted = new float[samplesPerChannel * targetChannels];

            // Mono → Stereo
            if (sourceChannels == 1 && targetChannels == 2)
            {
                for (int i = 0; i < samplesPerChannel; i++)
                {
                    float sample = data[i];
                    converted[i * 2] = sample;       // Left
                    converted[i * 2 + 1] = sample;   // Right
                }
            }
            // Stereo → Mono
            else if (sourceChannels == 2 && targetChannels == 1)
            {
                for (int i = 0; i < samplesPerChannel; i++)
                {
                    converted[i] = (data[i * 2] + data[i * 2 + 1]) * 0.5f;
                }
            }
            // Other conversions (multi-channel)
            else
            {
                for (int i = 0; i < samplesPerChannel; i++)
                {
                    for (int ch = 0; ch < targetChannels; ch++)
                    {
                        int sourceChannel = Math.Min(ch, sourceChannels - 1);
                        converted[i * targetChannels + ch] = data[i * sourceChannels + sourceChannel];
                    }
                }
            }

            return converted;
        }

        /// <summary>
        /// Converts float samples to S16 format and sends to LiveKit.
        /// </summary>
        /// <remarks>
        /// Float to S16 conversion:
        /// - Input range: [-1.0, 1.0]
        /// - Output range: [-32768, 32767]
        /// </remarks>
        private void SendAudioFrame(float[] data, int channels, int sampleRate)
        {
            if (data == null || data.Length == 0)
            {
                UnityEngine.Debug.LogWarning("Empty audio data, skipping frame");
                return;
            }

            int samplesPerChannel = data.Length / channels;
            if (samplesPerChannel <= 0)
            {
                UnityEngine.Debug.LogWarning($"Invalid audio data: length={data.Length}, channels={channels}");
                return;
            }

            // Allocate or resize frame buffer
            if (_frameData.Length != data.Length)
            {
                if (_frameData.IsCreated)
                    _frameData.Dispose();
                _frameData = new NativeArray<short>(data.Length, Allocator.Persistent);
            }

            // Convert float to S16
            static short FloatToS16(float v)
            {
                v *= 32768f;
                v = Math.Min(v, 32767f);
                v = Math.Max(v, -32768f);
                return (short)(v + Math.Sign(v) * 0.5f);
            }

            for (int i = 0; i < data.Length; i++)
            {
                _frameData[i] = FloatToS16(data[i]);
            }

            // Send frame to RTC
            using var request = FFIBridge.Instance.NewRequest<CaptureAudioFrameRequest>();
            using var audioFrameBufferInfo = request.TempResource<AudioFrameBufferInfo>();

            var pushFrame = request.request;
            pushFrame.SourceHandle = (ulong)Handle.DangerousGetHandle();
            pushFrame.Buffer = audioFrameBufferInfo;

            unsafe
            {
                pushFrame.Buffer.DataPtr = (ulong)NativeArrayUnsafeUtility.GetUnsafePtr(_frameData);
            }

            pushFrame.Buffer.NumChannels = (uint)channels;
            pushFrame.Buffer.SampleRate = (uint)sampleRate;
            pushFrame.Buffer.SamplesPerChannel = (uint)samplesPerChannel;

            using var response = request.Send();
            FfiResponse res = response;

            var asyncId = res.CaptureAudioFrame.AsyncId;
            void Callback(CaptureAudioFrameCallback callback)
            {
                if (callback.AsyncId != asyncId) return;
                if (callback.HasError)
                {
                    UnityEngine.Debug.LogError($"Audio capture failed: {callback.Error}");
                }
                FfiClient.Instance.CaptureAudioFrameReceived -= Callback;
            }
            FfiClient.Instance.CaptureAudioFrameReceived += Callback;
        }

        /// <summary>
        /// Mutes or unmutes the audio source.
        /// </summary>
        public override void SetMute(bool muted)
        {
            _muted = muted;
        }

        /// <summary>
        /// Disposes of the audio source, stopping it first if necessary.
        /// </summary>
        public void Dispose()
        {
            Dispose(true);
            GC.SuppressFinalize(this);
        }

        protected virtual void Dispose(bool disposing)
        {
            if (_disposed)
                return;

            if (disposing)
            {
                Stop();
                if (_frameData.IsCreated)
                    _frameData.Dispose();
                _resampleBuffer = null;
                Handle?.Dispose();
            }

            _disposed = true;
        }

        ~RtcAudioSource()
        {
            Dispose(false);
        }

        [Obsolete("No longer used, audio sources should perform any preparation in Start() asynchronously")]
        public virtual IEnumerator Prepare(float timeout = 0) { yield break; }

        [Obsolete("Use Start() instead")]
        public IEnumerator PrepareAndStart()
        {
            Start();
            yield break;
        }
    }
}

---

Audio Processing Flow Diagram

┌─────────────────────────────────────────────────────────────────────────────┐
│                        Complete Audio Processing Flow                        │
└─────────────────────────────────────────────────────────────────────────────┘

  ┌─────────────────┐
  │   Microphone    │  Actual output may differ from requested
  │   Hardware      │  (depends on hardware capabilities)
  │  (e.g., 1ch,    │
  │   44100Hz)      │
  └────────┬────────┘
           │
           ▼
  ┌─────────────────┐
  │   AudioProbe    │  Callback: OnAudioRead(data, 1, 44100)
  │   (Unity)       │
  └────────┬────────┘
           │
           ▼
  ┌─────────────────────────────────────────────────────────────────┐
  │                    RtcAudioSource.OnAudioRead                   │
  │                                                                 │
  │   Expected: 2ch @ 48000Hz (set in constructor)                  │
  │   Received: 1ch @ 44100Hz (actual hardware output)              │
  │                                                                 │
  │   ┌─────────────────┐                                           │
  │   │ ConvertChannels │  1ch → 2ch (Mono to Stereo)               │
  │   │                 │  [M0][M1] → [L0][R0][L1][R1]              │
  │   └────────┬────────┘                                           │
  │            │                                                    │
  │            ▼                                                    │
  │   ┌─────────────────┐                                           │
  │   │ ResampleAudio   │  44100Hz → 48000Hz                        │
  │   │                 │  Linear interpolation                     │
  │   └────────┬────────┘                                           │
  │            │                                                    │
  │            ▼                                                    │
  │   ┌─────────────────┐                                           │
  │   │ SendAudioFrame  │  Float [-1.0, 1.0] → S16 [-32768, 32767]  │
  │   └────────┬────────┘                                           │
  └────────────┼────────────────────────────────────────────────────┘
               │
               ▼
  ┌─────────────────┐
  │  LiveKit Server │  Receives: 2ch @ 48000Hz, S16 format
  └─────────────────┘

---

Hope this helps others facing the same issue! 🎧

[NiekVRelax]

We ran into the same issue, the solution provided by @michael811125 works perfectly.
Hoping this gets fixed soon.

[davidhughhenrymack]

Please fix! Forking the library in unity is horrible for repo size

[MaxHeimbrock]

I am not able to reproduce the issue on the old v1.3.3 release. For me the audio output works on my MacOS in Unity 2022.3.20f1 when getting mono input and output in Unity set to stereo.

Still I feel like these changes could also solve it for you:

• Recover AudioStream when system audio output device changes #274
• Remove Unity resampling #271
  Both are released in v1.3.7

If you are not bound to use Unity audio input and output, you can now also use our Platform Audio, which lets the native side handle audio devices and input / output:

• Platform audio for Unity #293
  Released in v1.3.8

Can you try if this solves the issue without your fixes?