The main new feature for V1.0.4 is a new option to send raw audio and select the audio input device. For now Native(Win, Mac, iOS, Android) uses a different system than WebGL but this will be merged in the future.

Please help testing the new system via the scenes extra/AudioInput for Native platforms and extra/AudioWeb for WebGL.

Changelog:

• WebGL: Added new extra/AudioWeb example to show selection of audio input devices (will be merged with AudioInput scene later)
• Native: AudioInput C# calls added to allow sending of raw audio
• Native: extra/AudioInput example added. This demonstrates sending audio from an AudioSource or using Unity Microphone for audio input device selection
• Native: Added wrapper methods to control hardware audio effects
• Native: Turned off hardware audio effects by default. Echo cancellation will be handed via software now
• Native: AudioInput now supports multiple parallel audio input devices
• Windows: AudioOptions.echo_cancellation=true should work correctly now even if no incoming audio is received
• Fixed issues that could cause the video codec to automatically fallback to VP8
• Added readme files and a header to example apps
• Windows: Added workaround for playback issues via the Experimental audio layer if Dolby surround is used. For microphone access issues AudioInput can be used as a workaround.
• Signaling Server: Added support for tokens to allow only registered users to access the server
• Signaling Server: Removed the packaged zip file. Use https://github.com/because-why-not/awrtc_signaling instead.

We are looking for testers for the experimental Vision OS versions of WebRTC and WebRTC Video Chat.

For access to the vision pro build of WebRTC Video Chat please send your Unity Asset Store Invoice ID via e-mail to contact@because-why-not.com or contact @Christoph via Discord

You can also test the ports of libwebrtc and com.unity.webrtc:

• libwebrtc builds: webrtc visionpro releases
• libwebrtc source code and build scripts: webrtc visionpro github
• com.unity.webrtc fork test project with binaries: uwebrtc_visionpro_testproj releases
• com.unity.webrtc fork source code and build scripts: uwebrtc_visionpro github

• Update to WebRTC 120
• Mac: MacOSX min version is now 10.15
• iOS: Rebuild with iOS SDK 17.2
• Android: Rebuild with Android NDK 25 and SDK 34
• Android: minSdkVersion is now 21 for all architectures
• Changed setup of the native logger to ensure any errors during init are printed by default
• WebGL: Added BrowserMediaConfig to give access to more WebGL specific features. See folder scripts/browser
• WebGL: Selection of an input device is now in testing. Set it by using BrowserMediaConfig.AudioInputDevice and
  (UnityCallFactory.Instance.InternalFactory as Byn.Awrtc.Browser.BrowserCallFactory).GetAudioInputDevices().\
•  WebGL: Audio is now processed via the Web Audio API allowing audio processing within the Java Script plugin
• WebGL: Basic support for spatial audio via panning added via new method BrowserWebRtcCall.SetVolumePan
• WebGL: Improved autoplay block resolution. If a browser blocks audio play it will be resolved once the user interacts
  with the unity Canvas. iOS Safari might block autoplay until several touch interactions are received.

WebRTC Video Chat:

• WebGL: Fixed an issue with incorrectly setting the bitrate in Safari
• Android: Fixed a memory leak when shutting down the local camera
• Minor changes in preparation for the next HoloLens 2 update

Holo Video Chat:

• Added support for I420p frames output, skipping a copy and moving image processing to the GPU
• Output of ARGB frames is sped up
• VideoInput will move images to C++ and convert them in a single step and speed up image processing
• Reduced memory usage and reliance on C# garbage collectio
• Added a very high quality setting but with 15 FPS
• Improved documentation and made sure the xml IDE docs are included

WebRtc Video Chat:

• Updated and Tested with unity 2021 and 2022 LTS. The Unity C# projects will now require the newer Unity versions but the C# API continues to work with Unity 2020
• MediaConfig.VideoBitrateKbits added to allow setting a target birate (appears to default to 2500 kbits which is too low for HD and higher)
• MediaConfig.VideoContentHint added. This can be set to motion, detail or text to optimize the video stream for specific content
• MAINTAIN_RESOLUTION flag is removed. Set VideoContentHint to detail instead
• MediaConfig.VideoCodecs added. This allows to set a list of codecs to priotize (VP8, VP9, AV1 are usually supported. H264 on iOS and some browsers)
• RequestStats and RtcStatsEvent added to BrowserWebRtcCall (webgl) and NativeWebRtcCall(all other except UWP). This can be used to check on the current bitrate, used codecs and more
• Native: NativeMediaConfig.FrameBuffer = true is now the default to help with fluent video playback at low render FPS
• The VideoInput example scene now shows an animated panorama image to allow testing with different birate requrements
• WebGL: Calling Unity_Media_EnableScreenCapture(“screen”, true) allows capturing the screen with system audio if supported by the browser.

Holo Video Chat:

• Fixed several issues within the wrapper that could cause stalls (deadlock in the SDK) and crashes (leaking of the video tracks that resulted in the camera to remain active)
• fixed crash causing issue when deactivating EyeTracking. This resulted in the system to restart accessing the camera directly while the VideoTransformer might still access the camera
• camera access is now delayed by 2 seconds or so
• Made the log more chatty to hopefully better be able to reproduce any crashes
• Added different VideoQuality structs to make changing it easier

The new version is now available in the asset store.

Changelog:

• Updated to WebRTC 110
• From now on, example apps will be saved in Unity 2020. The C# API will continue to work in Unity 2019.
• Android: min API level should be set to 21 now
• Logging enhancements: Most logs now include the ownership hierarchy to manage multiple parallel calls
• Updated all third party dependencies including websocket-sharp and all WebGL npm dependencies
• awrtc_signaling: updated dependencies and changed the default config to support IPv6
• Fixed some incorrect namespaces in Byn.Awrtc.Native.dll
• Native: Major changes in how the audio functions. Let me know if any new bugs show up
• Native: Added first experimental support for custom AudioInput via the C# wrapper WebRtcCSharp.AudioInput. Example apps will be made available later.
• Native: Added UnityCallFactory.Config for testing purposes. It will allow turning on global experimental changes such as the new windows audio module
• Native: Added NativeMediaConfig.FrameBuffer. If set to true this will add a 1 frame long frame buffer to smooth the video frame rate.
• Native: Added NativeMediaConfig.ReturnLocalVideo. This is true by default and can be set to false to increase performance if access to the local video feed is not required within C#
• Native: Added PeerFactoryConfig to the native API to allow changing global settings and use multiple factories in parallel in the future
• Windows: Added support for the newer WebRTC audio module. This can be set via the UnityCallFactory.Config (must be set before the factory is initialized)

From now on a docker based setup is supported for the signaling server. With this setting up a new signaling server should only take a few minutes including getting a SSL certificate from letsencrypt.

Requirements:

• basic understanding of the linux command line, ssh and docker
• a linux server with docker and docker-compose installed (tested with ubuntu 22)
• a public domain that points to your servers IP address
• an open TCP port 80 and 443

For more info please check out the awrtc_signaling_docker github repository.

You can find it here: Holo Video Chat

Holo Video Chat contains a port of the WebRTC Video Chat plugin for UWP x86, x64 and ARM 32 for the HoloLens 2. It includes the same ICall, IMediaNetwork and IVideoInput C# interfaces that are available for the other platforms.

Holo Video Chat is compatible to WebRTC Video Chat on a network level and it can be used to connect the HoloLens 2 to all other platforms including browser based apps developed on top of awrtc_browser.

For more information check out the Holo Video Chat guide!

• Deprecated Unity 2018 LTS support. Lowest version is now Unity 2019 LTS for Unity examples. The C# API itself should continue to work with older Unity versions
• Mac: Update to MacOSX SDK 12.3
• iOS: Update to iOS SDK 15.5
• Windows: Switched to Visual Studio Clang build pipeline for x86 and x64 builds. This might change some performance characteristics.
• Android : Update to NDK 23 and SDK API 31
• Android: Updated AndroidManifest.xml & build pipeline to work with latest Android versions without generating warnings
• Android: Due to a security issue in older WebRTC versions the play store with show a warning using the older versions. Please update as soon as possible.
• Native: Updated native WebRTC library to WebRTC104.
• Native: Native log level is now correctly set if the asset runs in the Unity Editor
• Native: Fixed flag MAINTAIN_RESOLUTION in UnityCallFactory. Setting this to true should stop automatic downscaling on all native platforms by setting the content hint to Detail
• Native: The VideoInput API can now force & change resolutions on the fly via the UpdateFrame method. This can be used to dynamically scale down the video to reduce memory and CPU usage. The Width/Height values in MediaConfig are ignored.
• Native: The VideoInput API should now support a wider range of framerates and allow changing framerates on the fly
• Native: Added CommonNetworkConfig.PeerConnectionTimeout. This value allows setting a timeout for establishing the direct peer to peer connection
• WebGL: Fixed log levels being set incorrectly for the JavaScript side backend
• WebGL: TextureFrame and FrameProcessor have been updated to reduce the risk of memory leaks if a frame is not used e.g. due to Exceptions in user code. IFrame.Dispose must still be called though!
• WebGL: updated third party dependencies
• WebGL: Using the webrtc-adapter to help with cross browser compatibility can now be deactivated.
• WebGL: Errors during video/audio device configuration are now correctly moved from JavaScript to C#
• UnityCallFactory will now warn and automatically set Application.runInBackground=true if not done so by the app developer

• Examples: Signaling & TURN server urls have been updated
• CallApp: CallApp will now always use secure websockets by default
• CallApp: The app now uses the new FrameProcessor to convert platform specific formats into Unity Texture2D. Using I420p can reduced CPU overhead by moving some conversion to the GPU.
• New MediaConfig default values: sending Video and Audio is now off by default
• Improved logging. ICall/IMediaNetwork and peers now have a global ID to allow associating log messages with individual instances. Log messages are prefixed with “awrtc:”
• log level can now be set via SetLogLevel/SetNativeLogLevel
• Fixed misleading warning log & issue with dropping signaling messages that arrive after a call already successfully connects
• Removed various obsolete classes & method
• Improved error messages given by UnityCallFactory & websockets
• Removed .NET 3.5 backwards compatibility
• Added NetworkConfig.KeepSignalingAlive to keep the signaling connection active
• KeepSignalingAlive and IsConference=true now allows custom server side code to start & shutdown client side peers by connecting/disconnecting a signaling connection.
• ConnectionId’s for signaling and MediaNetwork connectionIds are the same now
• VideoInput: Shared API added for all supported platforms including WebGL
• VideoInput: Added information for use with HDR
• Native: CommonNetworkConfig can now be used to create Calls/MediaNetwork instances with custom signaling network / custom websocket timeouts
• Native: Several old create methods removed from the factory classes. Use the create methods that get NetworkConfig as argument instead.
• Native: Added codec settings in NativeMediaConfig(experimental) to priotize codecs or change other codec specific settings
• Native: NativeMediaConfig.AudioAccess (experimental): Can be used to request emiting AudioFrame events with raw PCM for each incoming connection
• Native: Proxy support added (experimental)
• WebGL: Added support for Screen Capture API. Enabled via Unity_Media_EnableScreenCapture(“Screen capture”)
• WebGL: Fixed memory leak if multiple streams are active in parallel that use the “Native” pixel format.
• WebGL: updated npm dependencies
• Android x64 support added
• Android: Removed all WebCamTexture references to ensure Unity does not automatically add the Camera permission if it is removed from the AndroidManifest.xml
• Android: Fixed a bug causing log message to be truncated
• Android: added proguard-user.txt with rules to ensure proguard doesn’t strip required java components
• iOS: Added workaround for hardware decoding issues with resolutions greater than 720p
• awrtc_borwser (experimental): Added an alternative peer configuration enabled via SINGLE_NEGOTIATION=true. It will allow setting up the peer for video & audio but allows enabling/disabling these without renegotiation
• ConferenceApp: Added randomly generated usernames to show how ICall.Send can be used to tell different users apart and keep a reference between username & ConnectionId
• Ice Restart (experimental) NetworkConfig.MaxIceRestart can be set to allow several reconnection attempts before a connection is treated as disconnected.
• Reconfigure (experimental): ICall/IMediaNetwork.Configure can now be called at run-time to change settings without reconnecting. This requires an active signaling connection (KeepSignalingAlive=true).