Bluetooth LE for modern Android Development - part 3

This is the final part of my new series on Bluetooth Low Energy for Android. It took much longer to complete this post than I expected, much because I rewrote parts of the code I show here when I found a better solution.

In my last two posts, I introduced the new Android APIs for BLE that came with Android 8.0. I gave a closer look at the CompanionDeviceManager API that makes discovery easier as well as allows your app to stay alive in the background.

In this post, we will look at the APIs for connecting and communicating with a BLE peripheral. Most of these have been around since the first introduction of BLE APIs on Android. In this post I will share my learnings over the years and what I consider the best practices, especially when combined with Kotlin and Coroutines.

Connecting

One of the most misunderstood operations in the Android BLE APIs is the one we use for initiating a connection to a peripheral. We will start with a focus on the connectGatt() functions in the BluetoothDevice class. As with the previous posts, some of this information is only available from Android 8.0, but I consider that a reasonable minimum target today, especially if you need to support Bluetooth Low Energy.

Once you know the hardware address and have an instance of BluetoothDevice for the peripheral, we can initiate a connection attempt. This is done by calling connectGatt(), which comes in four different versions (depending on your API level). Let's look at the one that takes the most parameters, as the others are simply calling that one under the hood.

public BluetoothGatt connectGatt(Context context, 
                boolean autoConnect, 
                BluetoothGattCallback callback, 
                int transport, 
                int phy, 
                Handler handler)

The first parameter is a regular Android Context instance. I suggest that you always try to use the application Context for this to avoid leaking an Activity or other instances that have a shorter lifecycle.

The second parameter, named autoConnect, is the one that causes the most confusion. Setting this to true means your app will keep trying to connect even if the device isn't immediately available, while setting it to false means it will only try to connect once. In practice, you should never set this to false as it will likely fail sometimes, even when the device is nearby and available. However, you should be aware that when set to true, your app will keep trying to connect until you call disconnect() on the returned BluetoothGatt instance.

The third parameter is the callback instance for all GATT operations, which I will cover later in this post.

The fourth parameter, named transport, is only used for devices that support dual-mode communication (both the older BR/EDR and the "regular" LE). I have yet to find a device that supports this, so I believe this is quite rare.

The fifth parameter allows you to set the preferred PHY options for the connection, but only if autoConnect is false, which makes this parameter fairly useless. I will cover PHY options later in this post and how to set them on an existing connection instead.

The sixth and final parameter was introduced in Android 8.0 and lets you pass a Handler instance that will be used for invoking the BluetoothGattCallback functions. I strongly recommend you use this as the default behavior otherwise seems to be undefined. You can use any Handler, including one that wraps the main thread. Just make sure you do pass one and if you pass one with a dedicated HandlerThread, make sure you shut it down once you call close() on the BluetoothGatt instance.

When connectGatt() is called, it immediately returns a BluetoothGatt object. However, it doesn't mean that your app is connected to the device. The result of the actual connection attempt is notified to the app by calling onConnectionChanged() on the BluetoothGattCallback instance. This is what makes the BLE APIs in Android so complicated, as it ends up becoming a classical "callback hell" unless you find a more convenient way for wrapping this API. Before we look at how to wrap this API there is another important detail to under, how to shut down a BluetoothGatt instance in a clean way.

Disconnect and Close

When you want to disconnect and cleanly shut down a connection to a device, you have to do it in two steps. First, you need to call disconnect() to shut down the BLE connection to the peripheral, then you need to call close() in order to "unregister" the BluetoothGatt instance from the system. Both are needed or you might be leaking resources.

Wrapping BluetoothGatt and BluetoothGattCallback

Anyone developing BLE applications on Android will soon notice that the framework APIs, most specifically BluetoothGatt and BluetoothGattCallback, are very cumbersome to work with. While there are third-party libraries available that make your life easier, I want to introduce a way to write a minimal wrapper "library" yourself using Kotlin and Coroutines. At the end of this section, I will briefly cover the existing BLE wrappers that I am aware of.

Our goal here is to wrap the existing BluetoothGatt and BluetoothGattCallback classes to make the API easier to use. We start with the callback and we do this by converting each callback function to an event that we can emit using a Flow from Kotlin Coroutines.

sealed class GattEvent

data class ConnectionStateChanged(val status: Int, val newState: Int) : GattEvent()

data class CharacteristicRead(val characteristic: BluetoothGattCharacteristic, val status: Int) :
    GattEvent()

data class CharacteristicWritten(val characteristic: BluetoothGattCharacteristic, val status: Int) :
    GattEvent()

// Remaining events omitted for brevity

The code above shows the implementation for each event. Note that the values in each data class are the same as the callback function they represent.

Next, we implement our own BluetoothGattCallback that will do the actual conversion.

class BluetoothGattEvents(private val events: MutableSharedFlow<GattEvent>) : BluetoothGattCallback() {

    override fun onCharacteristicWrite(
        gatt: BluetoothGatt,
        characteristic: BluetoothGattCharacteristic,
        status: Int
    ) { 
        events.tryEmit(CharacteristicWritten(characteristic, status)) 
    }
    
    override fun onConnectionStateChange(gatt: BluetoothGatt?, 
                                         status: Int, 
                                         newState: Int) {
        events.tryEmit(ConnectionStateChanged(status, newState))
    }
    

    // Remaining functions omitted for breivity...
}

The code above is the wrapper implementation for BluetoothGattCallback. It takes a MutableSharedFlow<GattEvent> as a constructor parameter, and this will be used for emitting the events from each callback function. Note how we construct a new data class for each event emitted and simply pass the parameters from the callback into the values.

We now have a callback class we can use that lets us consume the different GATT callbacks as events from a Flow instead of having to wait for a callback. Next, we need to implement a function that lets us safely queue GATT operations so that we don't start another one before we've gotten the callback of the first. The reason for this is that the BLE framework requires each operation to be complete before we initiate another, or we might end up in an inconsistent state. That means we have to wait for the callback related to the GATT operation we started before performing the next one.

The easiest way to do this is by using a Mutex from Kotlin Coroutines. Imagine we want to call readCharacteristic() using this method. We would then create a function like this:

private suspend fun queueReadChar(
    bluetoothGatt: BluetoothGatt,
    characteristic: BluetoothGattCharacteristic,
    gattEvents: SharedFlow<GattEvent>,
    mutex: Mutex
): CharacteristicRead {
    return mutex.withLock {
        gattEvents
            .onSubscription { 
                if(!bluetoothGatt.readCharacteristic(characteristic)) {
                   emit(CharacteristicRead(characteristic, GATT_FAILURE)
                }
            }
            .firstOrNull { it is CharacteristicRead } as CharacteristicRead?
            ?: CharacteristicRead(characteristic, BluetoothGatt.GATT_FAILURE)
    }
}

The code above will use the Mutex as a way to queue our GATT operations (in this case, readCharacteristic). The tricky part here is how we use the Flow of GattEvents to "wait" for the right event to be returned. Before we can call firstOrNull() on this Flow, we need to make the actual call to readCharacteristic(), but if we do it before we start collecting events (using firstOrNull()) we risk missing the event.

The way to solve that is by using onSubscription() on the Flow to make the readCharacteristic() call. This will ensure we start collecting events and call the GATT operation before the first event is emitted. This also allows us to catch any immediate errors and emit that as a failure event.

However, a Mutex is tricky to use and an observant reader might already see that we could run into a deadlock with this code. If we don't get a CharacteristicRead event back, perhaps because the device disconnected unexpectedly, we will be stuck waiting for an event forever.

The way to solve that is by adding withTimeout() to this function, which ensures that we never block the queue of GATT operations for too long. We also make the whole function an extension on Mutex to make it more readable.

private suspend fun Mutex.queueReadChar(
    gatt: BluetoothGatt,
    characteristic: BluetoothGattCharacteristic,
    gattEvents: SharedFlow<GattEvent>
): CharacteristicRead {
    return withLock {
        withTimeout(DEFAULT_GATT_TIMEOUT) {
            gattEvents
                .onSubscription {
                    val ok = gatt.readCharacteristic(characteristic)
                    if(!ok) {
                       emit(
                           CharacteristicRead(
                               characteristic, 
                               GATT_FAILURE
                       )
                    }
                }
                .firstOrNull { 
                    it is CharacteristicRead 
                } as CharacteristicRead?
                ?: CharacteristicRead(
                    characteristic, 
                    BluetoothGatt.GATT_FAILURE
                )
        }
    }
}

All the GATT operations we need can now be written in the same way and share the same Mutex. This means we can make the function above even more generic.

private suspend fun <T> Mutex.queueWithTimeout(
    timeout: Long = DEFAULT_GATT_TIMEOUT,
    block: suspend CoroutineScope.() -> T
): T {
    return try {
        withLock { withTimeout(timeout, block) }
    } catch (e: Exception) {
        throw e
    }
}

suspend fun readCharacteristic(
    gatt: BluetoothGatt,
    characteristic: BluetoothGattCharacteristic,
    events: SharedFlow<GattEvent>
): CharacteristicRead = mutex
    .queueWithTimeout {
        events
            .onSubscription {
                if (gatt.readCharacteristic(characteristic)) {
                    emit(CharacteristicRead(characteristic, 
                                            BluetoothGatt.GATT_FAILURE))
                }
            }
            .firstOrNull {
                it is CharacteristicRead &&
                    it.characteristic.uuid == characteristic.uuid
            } as CharacteristicRead?
            ?: CharacteristicRead(characteristic, 
                                  BluetoothGatt.GATT_FAILURE)
    }

The code above shows our complete  function for safely queuing GATT operations  using Mutex.withLock() and avoiding deadlocks by using withTimeout(). The block passed to withTimeout() is the actual GATT operation using the onSubscription() on the SharedFlow with the GattEvents.

The second function is an example of how to use it for reading a GATT Characteristic. This approach works for all the GATT operations and allows you to write your BLE code in Kotlin coroutines that are much easier to read. For instance, with the above solution you could write your BLE code like this:

suspend fun sayHello(characteristic: BluetoothGattCharacteristic) {
    val read = readCharacteristic(characteristic)
    if (read.status == BluetoothGatt.GATT_SUCCESS) {
        if (read.characteristic.getStringValue(0) == "Hello!") {
            characteristic.setValue("Hi there!")
            val written = writeCharacteristic(characteristic)
        }
    }
} 

While this function is fairly trivial and naive, it shows how you easily can write two consecutive GATT operations without worrying about callbacks, deadlocks, or timeouts yourself. The wrapper code is fairly small and if we decide to create wrappers for the Android BLE APIs classes, we could even build a library that is easy to write unit tests for.

The wrapper function explained above was not something I came up with entirely by myself. I got a lot of help and tips from Adam Powell and Hugo Visser. Many thanks to these two who helped me figure out the final details of this solution.

Existing wrapper libraries

There already exists a couple of libraries that wrap the Android BLE APIs. The two most interesting are listed below:

• RxAndroidBle - A wrapper based on RxJava. If Rx is something you're already using in your app, this could be a good option.
• kable - A Kotlin Multiplatform library for working with Bluetooth Low Energy. While fairly new, it shows a lot of promise. Works for both Android, iOS, and JavaScript. It also uses Kotlin Coroutines which makes it more suitable for new projects or applications already using Coroutines.

I do recommend using one of the wrapper libraries above, but if you only need a subset of the functionality or for some reason can't rely on these, I recommend building your own wrapper as described above. Also, the wrapper I introduced above is a good practice to implement in order to get a better understanding of the Android BLE APIs.

PHY options

One feature that came in Bluetooth 5.0 and was introduced in Android 8.0 is the PHY options. PHY stands for PHYisical layer and refers to the actual radio signaling for a Bluetooth Low Energy connection. More specifically, it lets you tweak how much error correction you want in the signaling which affects range and throughput. More error correction means lower throughput (because more of the signals are used for error correction) but a longer range, while less error correction means higher throughput but shorter range.

The Android API lets you set the PHY options on an established connection using BluetoothGatt.setPreferredPhy() and takes three parameters. txPhy and rxPhy controls the transmitter and receiver PHY options. These are bitwise OR of any of BluetoothDevice.PHY_LE_1M_MASK, BluetoothDevice#PHY_LE_2M_MASK, and BluetoothDevice.PHY_LE_CODED_MASK. 1M is the default for a Bluetooth Low Energy connection and means a theoretical throughput of 1 MBit/second, 2M means 2 MBit/second which gives a higher throughput but lower range, and CODED is the long-range but low throughput option. The third parameter, phyOptions, is only applicable when using CODED for the transmitter or receiver. This can be either PHY_OPTION_NO_PREFERRED, PHY_OPTION_S2 or PHY_OPTION_S8. S8 gives the longest possible range but also the lowest throughput.

If possible, I highly recommend using the PHY options in your application depending on the use case. If you need a long-range connection, like a companion app for a device that might not always be very close to the phone (for instance, a BLE-connected smartwatch), I recommend setting the CODED option with S8 as the PHY option. However, this will greatly reduce how fast you can transmit and receive data. If the range is not important I recommend setting the 2M option, but note that it will severely reduce the possible range of the connection.

It's important to know that you can switch between the different options on an existing connection. This means that you can use CODED for regular use, but switch to 2M when you need to transfer a lot of data (like when doing a firmware update) without having to reconnect to the peripheral first. Also, note that setting the PHY options will not affect energy consumption. The energy consumption for 2M is the same as for 1M and CODED because the difference is in how much error correction is used.

Finally, setting the PHY options on Android is only a recommendation to the system and remote device, and you'll get the actual PHY options from BluetothGattCallback.onPhyUpdate() or by calling BluetoothGatt.readPhy() and receiving the result in BluetoothGattCallback.onPhyRead().

For a more thorough explanation of PHY options in Bluetooth Low Energy, I highly recommend reading "Bluetooth PHY – How it Works and How to Leverage it" by Henry Anfang.

Conclusions

In this post, I covered three topics on Bluetooth Low Energy for Android. The first was how the connectGatt() function works and what the different parameters control. If these, it is most important that you have a thorough understanding of what autoConnect means.

The second part was how to use Kotlin Coroutines to make a simple wrapper for the callback-based Android framework APIs for GATT communication. I do recommend using one of the existing wrapping libraries, but the method for wrapping I introduced here might be worth looking at depending on your use case. The important lesson from this part is that you cannot initiate another GATT operation until you receive the callback for the previous one, with the exception of the peripheral disconnecting.

The final part introduced the PHY option and how it can be used for controlling the range and throughput. I highly recommend using this if possible, since it means either better throughput or longer range, depending on your use case. Since it also can be changed on an existing connection, you can easily switch between different options depending on your current needs.

Bluetooth Low Energy is a complex technology, especially when building Android applications. My hope is that this post together with the two previous ones will make your work easier and that you've gained a better understanding of how to efficiently BLE code in your Android apps.