Banuba AR Face Filter

This guide is provided by Banuba. Agora is planning a documentation upgrade program for all extensions on the marketplace. Please stay tuned.

By integrating Banuba Face AR SDK and Agora.io SDK you can enable augmented reality features in video calls such as face filters, face touch up filters and virtual backgrounds. You can find the integration examples below.

Prerequisites

The latest Banuba SDK archive

Banuba Extension files for Android

Banuba trial client token.

To receive a trial token or a full commercial licence from Banuba - please fill in our form on form on banuba.com website, or contact us via info@banuba.com.

Integrate Banuba Face AR extension

Get Started

Make sure that you have Android NDK and CMake are installed! Project contains C++ sources, so this is required! Recommended NDK versions is 21.1.6352462, required CMake version is 3.9, 3.14 or above is recommended (Android SDK manager provides 3.18.1, it should be installed).
Get the latest Banuba SDK archive, Banuba trial client token. To receive full commercial license from Banuba - please fill in our form on form on banuba.com website, or contact us via info@banuba.com.
Copy .aar file from the Banuba SDK archive into libs dir:

banuba_effect_player-release.aar => libs/
Copy include/bnb directory into libs/bnb_sdk/:

include/bnb => libs/bnb_sdk/bnb
Get the latest Agora Video SDK archive.
Copy jar files from the Agora Video SDK archive into libs dir:

agora-rtc-sdk.jar => libs/
Copy architecture folders from the Agora Video SDK archive into jniLibs dir:

arm64-v8a, armeabi-v7a and x86_64 => libs/jniLibs/
Copy Agora SDK headers (*.h files) into libs/agora_rtc_sdk/include:

low_level_api/include => libs/agora_rtc_sdk/include
Copy and Paste your Banuba client token into appropriate section of app/src/main/java/com/banuba/sdk/agorapluginexample/ClientToken.kt with " " symbols. For example: BANUBA_CLIENT_TOKEN = "place_your_token_here"
Visit agora.io to sign up and get token, app ID and channel ID.
Copy and Paste your Agora token, app ID and channel ID into appropriate section of app/src/main/java/com/banuba/sdk/agorapluginexample/ClientToken.kt with " " symbols. For example: AGORA_APP_ID = "place_your_token_here"
Open the project in Android Studio and run the necessary target using the usual steps.

How to use `BanubaFiltersAgoraExtension`

Add following imports:

import com.banuba.agora.plugin.BanubaEffectsLoader
import com.banuba.agora.plugin.BanubaExtensionManager
import com.banuba.agora.plugin.BanubaResourceManager
import com.banuba.agora.plugin.model.ArEffect
import com.banuba.sdk.utils.ContextProvider
Copy

Add extension to RtcEngineConfig:

RtcEngineConfig().apply {
    ...
    System.loadLibrary("banuba")
    addExtension(BanubaExtensionManager.EXTENSION_NAME)
    ContextProvider.setContext(mContext)
    ...
}
Copy

Create and initialize BanubaResourceManager:

private val banubaResourceManager by lazy(LazyThreadSafetyMode.NONE) {
    BanubaResourceManager(this)
}
Copy

override fun onCreate(savedInstanceState: Bundle?) {
    ...
    banubaResourceManager.prepare()
}
Copy

After those steps enable and initialize extension:

agoraRtc.enableExtension(
    BanubaExtensionManager.VENDOR_NAME,
    BanubaExtensionManager.VIDEO_FILTER_NAME,
    true
)
Copy

private fun initBanubaPlugin() {
    agoraRtc.setExtensionProperty(
        BanubaExtensionManager.VENDOR_NAME,
        BanubaExtensionManager.VIDEO_FILTER_NAME,
        BanubaExtensionManager.KEY_SET_RESOURCES_PATH,
        banubaResourceManager.resourcesPath
    )
    agoraRtc.setExtensionProperty(
        BanubaExtensionManager.VENDOR_NAME,
        BanubaExtensionManager.VIDEO_FILTER_NAME,
        BanubaExtensionManager.KEY_SET_EFFECTS_PATH,
        banubaResourceManager.effectsPath
    )
    agoraRtc.setExtensionProperty(
        BanubaExtensionManager.VENDOR_NAME,
        BanubaExtensionManager.VIDEO_FILTER_NAME,
        BanubaExtensionManager.KEY_SET_TOKEN,
        BANUBA_CLIENT_TOKEN
    )
}
Copy

To enable/disable effects use the following code:

private val onEffectPrepared = object : BanubaResourceManager.EffectPreparedCallback {
    override fun onPrepared(effectName: String) {
        sendEffectToFilter(effectName)
    }
}

private fun sendEffectToFilter(effect: String) {
    agoraRtc.setExtensionProperty(
        BanubaExtensionManager.VENDOR_NAME,
        BanubaExtensionManager.VIDEO_FILTER_NAME,
        BanubaExtensionManager.KEY_LOAD_EFFECT,
        effect
    )
}
Copy

banubaResourceManager.prepareEffect(Effect name, onEffectPrepared)
Copy

Effects managing

To retrieve current effects use the following code:

val effects: List<ArEffect> = BanubaEffectsLoader(this).loadEffects()
Copy

ArEffect contains following information:
name: String - pass to banubaResourceManager.prepareEffect(Effect name, onEffectPrepared). Also can be used to display label on the UI
preview: Bitmap - can be used as preview image

To modify effects, add or remove effect folder in app/src/main/assets/effects directory. By default sample contains the following effects:

ElvisUnleashed
EnglandEightPieceCap
FashionHistory1940_male
MorphingHatBow
Nerd
SnapBubblesGlasses
Space
StarGlow
TitanicJack

Prerequisites​

Integrate Banuba Face AR extension​

Get Started​

How to use BanubaFiltersAgoraExtension​

Effects managing​

Run the demo​

Prerequisites

Integrate Banuba Face AR extension

Get Started

How to use `BanubaFiltersAgoraExtension`

Effects managing

Run the demo