GATT Client MQTT Gateway
With GATT Server Having ADC Sensing
The Espressif ESP32-WROOM device has both WIFI and Bluetooth Low Energy (BLE) capability and the ESP-SDK is rich with example code and contains all the libraries necessary to fully use this capability. This project resulted after cloning, building, and testing a project by pcbreflux. Interest was first generated after watching his video. The project was built with the GATT server sending data through the WIFI gateway using TLS security to a cloud account at cloudmqtt.com, an AWS service. nRF Connect on an Android phone was used as a client to send messages to the GATT server.
However, this topology is somewhat restrictive in that more uses result when the peripheral devices are GATT servers and the central device is a GATT client with WIFI gateway. Therefore, the pcbreflux project was modified to become a GATT client MQTT gateway and the example code in the ESP-SDK entitled “gatt_server” was modified to send gas sensor readings to the client. All of this is explained in the following. For anyone who may want reproduce this project it is highly recommended to first build and implement the pcbreflux project.
Software Development
With respect to Espressif devices such as the ESP32 there seems to be a lot of Arduino stuff out there, including an IDE that will run on your favorite machine, Windows or IOS. However, using the ESP-SDK on an Ubuntu machine is quite simple and straight-forward. This development used Ubuntu 16.04 running in Oracle VM VirtualBox on a Win7 PC. ‘make’ is used to configure, build, and flash.
Hardware
For all of my Espressif projects I develop my own device, schematic as well as PCB. The primary reason is the need to eliminate power supply wires by plugging the device directly into any low cost, USB 5 volt wall charger as shown in the image. The 3D printed enclosure with cover drawings are available at thingiverse.
The essential circuit features are a 5 to 3.3 volt regulator, reset button, removable jumper for flashing, pin header for UART and some GPIO, and USB male plug. Below is a close up of the device used as the server. The wire seen in the photo runs the ADC port pin over to one of the header pins, which was not done in the original design.
Here is the schematic for the circuit. The priniciple component is the LM3940 5 volt to 3.3 volt one amp regulator.
The sensor sending data to the cloud is a gas sensor, specifically the HiLetgo MQ-135 MQ135 Air Quality Sensor Hazardous Gas Detection Module. It plugs into the pin header on the server device and gets 5 volts from the device PCB. In the image is seen that the sensor itself plugs into a daughter board adapted with some pin headers. Pins for the UART are brought out to one side. The ADC on the ESP32 handles 0 to 1.0 volts, and the sensor board has a resistor network that can be revised if necessary to provide the desired voltage range.
Software Description
Install ESP-SDK
For this project the esp-sdk was cloned into directory ~/esp. It was version v3.0-dev-1690-ge676676. However, when building the GATTs-MQTT Gateway project by pcbreflux, it was found there were missing components. This was solved by cloning into a separate directory v3.0-rc1.
~esp/esp-idf/
git clone https://github.com/espressif/esp-idf.git esp-idf-v3.0-rc1
cd esp-idf-v3.0-rc1/
git checkout v3.0-rc1
git submodule update --init –recursive
Add to ~/.profile with
export IDF_PATH=~/esp/esp-idf/esp-idf-v3.0-rc1/
export PATH="$PATH:$HOME/esp/esp-idf/xtensa-esp32-elf/bin"The working directory then appeared as follows:
ESP32_gatts_mqtt_gateway
Since pcbreflux’s project serves as the framework for this project, it was cloned into a separate directory, built, and then tested to send data from nRF Connect on an Android phone to my account at cloudmqtt.com using TLS security. This work effort is a topic in itself and will not be described here. Only how it is used to create a BLE client WIFI gateway is described. Its directories appear as follows:
Later, it is seen that gateway_main.c, gatts_profile.c, and gatts_profile.h will be replaced with client code that makes this project into a GATT client.
Gatt Server with ADC and GATT Client
Since the goal is to send gas sensor data to the cloud, attention is now turned toward developing the GATT server to which is attached the gas sensor. However, it must be developed in conjunction with a GATT client that will poll the server to test it. The client code will then be used in the final project. In the ESP-SDK example/bluetooth directory the two projects indicated in the screenshot were built and modified accordingly. Note also that the final project directory is created here.
In the next screenshot one can see the source files for both server and client. In the server project the code to sense the voltage from the ADC is added, and in the client project the code to poll the server is added.
Final Project Directory
To create the final project directory one copies the content of the pcbreflux project to the directory gattc_mqtt_gateway and then removes gateway_main.c, gatts_profile.c, and gatts_profile.h. One then adds the source files from the gatt_client project. What is left is to modify gattc_demo.c to add WIFI and MQTT to finish the gateway
Detail of All Code Modifications and Additions
We are now ready to describe in detail the code changes and additions. Only the relevant portions of the code are shown below, not the complete files. We start with the gatt_server code in gatts_demo.c. A client request triggers an ESP_GATTS_READ_EVT event. See the section of added code. An ADC task is created that measures the gas sensor value and populates the voltage memory space for a 32 bit integer. The integer is then converted to four separate bytes that are used in the response to the client request.
case ESP_GATTS_READ_EVT: {
ESP_LOGI(GATTS_TAG, "GATT_READ_EVT, conn_id %d, trans_id %d, handle %d\n", param->read.conn_id, param->read.trans_id, param->read.handle);
/*begin added code*/
ESP_LOGI(GATTS_TAG, "received a read request and sending a response");
xTaskCreate(&adc1_task, "adc1_task", 10240, &voltage, 5, &xHandle);
ESP_LOGI(GATTS_TAG, "voltage measured = %d mv", voltage);
uint8_t a[4];
a[0] = voltage;
a[1] = voltage>>8;
a[2] = voltage>>16;
a[3] = voltage>>24;
/*end added code*/
esp_gatt_rsp_t rsp;
memset(&rsp, 0, sizeof(esp_gatt_rsp_t));
rsp.attr_value.handle = param->read.handle;
rsp.attr_value.len = 4;
rsp.attr_value.value[0] = a[0];
rsp.attr_value.value[1] = a[1];
rsp.attr_value.value[2] = a[2];
rsp.attr_value.value[3] = a[3];
esp_ble_gatts_send_response(gatts_if, param->read.conn_id, param->read.trans_id,
ESP_GATT_OK, &rsp);
break;
}At the beginning of the file is added:
#include "adc1_task.h"
and
static uint32_t voltage;
static TaskHandle_t xHandle = NULL;To complete the gatt_server project the code for adc1_task.c is presented in its entirety. It has been modified from examples/peripherals/adc/main/adc1_example_main.c. It runs as a task, averages the adc value over 64 samples, and then exits.
/* ADC1 Example
This example code is in the Public Domain (or CC0 licensed, at your option.)
Unless required by applicable law or agreed to in writing, this
software is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR
CONDITIONS OF ANY KIND, either express or implied.
*/
#include <stdio.h>
#include <stdlib.h>
#include "freertos/FreeRTOS.h"
#include "freertos/task.h"
#include "driver/gpio.h"
#include "driver/adc.h"
#include "esp_adc_cal.h"
#define DEFAULT_VREF 1124 //Use adc2_vref_to_gpio() to obtain a better estimate
#define NO_OF_SAMPLES 64 //Multisampling
static esp_adc_cal_characteristics_t *adc_chars;
static const adc_channel_t channel = ADC_CHANNEL_6; //GPIO34 if ADC1, GPIO14 if ADC2
static const adc_atten_t atten = ADC_ATTEN_DB_0;
static const adc_unit_t unit = ADC_UNIT_1;
static void check_efuse()
{
//Check TP is burned into eFuse
if (esp_adc_cal_check_efuse(ESP_ADC_CAL_VAL_EFUSE_TP) == ESP_OK) {
printf("eFuse Two Point: Supported\n");
} else {
printf("eFuse Two Point: NOT supported\n");
}
//Check Vref is burned into eFuse
if (esp_adc_cal_check_efuse(ESP_ADC_CAL_VAL_EFUSE_VREF) == ESP_OK) {
printf("eFuse Vref: Supported\n");
} else {
printf("eFuse Vref: NOT supported\n");
}
}
static void print_char_val_type(esp_adc_cal_value_t val_type)
{
if (val_type == ESP_ADC_CAL_VAL_EFUSE_TP) {
printf("Characterized using Two Point Value\n");
} else if (val_type == ESP_ADC_CAL_VAL_EFUSE_VREF) {
printf("Characterized using eFuse Vref\n");
} else {
printf("Characterized using Default Vref\n");
esp_err_t status = adc2_vref_to_gpio(GPIO_NUM_25);
if (status == ESP_OK) {
printf("v_ref routed to GPIO\n");
} else {
printf("failed to route v_ref\n");
}
}
}
void adc1_task(void *pvParameters)
{
uint32_t *volt = (uint32_t *) pvParameters;
//Check if Two Point or Vref are burned into eFuse
check_efuse();
//Configure ADC
if (unit == ADC_UNIT_1) {
adc1_config_width(ADC_WIDTH_BIT_12);
adc1_config_channel_atten(channel, atten);
} else {
adc2_config_channel_atten((adc2_channel_t)channel, atten);
}
//Characterize ADC
adc_chars = calloc(1, sizeof(esp_adc_cal_characteristics_t));
esp_adc_cal_value_t val_type = esp_adc_cal_characterize(unit, atten, ADC_WIDTH_BIT_12, DEFAULT_VREF, adc_chars);
print_char_val_type(val_type);
//Continuously sample ADC1
while (1) {
uint32_t adc_reading = 0;
//Multisampling
for (int i = 0; i < NO_OF_SAMPLES; i++) {
if (unit == ADC_UNIT_1) {
adc_reading += adc1_get_raw((adc1_channel_t)channel);
} else {
int raw;
adc2_get_raw((adc2_channel_t)channel, ADC_WIDTH_BIT_12, &raw);
adc_reading += raw;
}
}
adc_reading /= NO_OF_SAMPLES;
//Convert adc_reading to voltage in mV
uint32_t voltage = esp_adc_cal_raw_to_voltage(adc_reading, adc_chars);
printf("Raw: %d\tVoltage: %dmV\n", adc_reading, voltage);
*volt = voltage;
// vTaskDelay(pdMS_TO_TICKS(1000));
break;
}
vTaskDelete(xTaskGetCurrentTaskHandle());
}This concludes code changes and additions for the gatt_server project, and now attention is directed to the final project, gattc_mqtt_gateway. There are notably more changes and additions. To begin, add initializing WIFI as seen in the code snippet below.
void app_main()
{
// Initialize NVS.
esp_err_t ret = nvs_flash_init();
if (ret == ESP_ERR_NVS_NO_FREE_PAGES) {
ESP_ERROR_CHECK(nvs_flash_erase());
ret = nvs_flash_init();
}
ESP_ERROR_CHECK( ret );
initialise_wifi();
ESP_LOGI(TAG, "Test number : %d", (int) 5);Then add the initialization function to the file.
static void initialise_wifi(void) {
tcpip_adapter_init();
wifi_event_group = xEventGroupCreate();
ESP_ERROR_CHECK( esp_event_loop_init(event_handler, NULL) );
wifi_init_config_t cfg = WIFI_INIT_CONFIG_DEFAULT();
ESP_ERROR_CHECK( esp_wifi_init(&cfg) );
ESP_ERROR_CHECK( esp_wifi_set_storage(WIFI_STORAGE_RAM) );
wifi_config_t wifi_config = {
.sta = {
.ssid = EXAMPLE_WIFI_SSID,
.password = EXAMPLE_WIFI_PASS,
},
};
ESP_LOGI(MAIN_TAG, "Setting WiFi configuration SSID %s...", wifi_config.sta.ssid);
ESP_ERROR_CHECK( esp_wifi_set_mode(WIFI_MODE_STA) );
ESP_ERROR_CHECK( esp_wifi_set_config(ESP_IF_WIFI_STA, &wifi_config) );
ESP_ERROR_CHECK( esp_wifi_start() );
}Next add to the file this function. When the server disconnects it scans for a server for 30 seconds and then disconnects.
static void start_scan(void)
{
stop_scan_done = false;
Isconnecting = false;
uint32_t duration = 30;
esp_ble_gap_start_scanning(duration);
}Now make sure all of the includes match this list.
#include "freertos/FreeRTOS.h"
#include "freertos/task.h"
#include "freertos/event_groups.h"
//#include "freertos/heap_regions.h"
#include <stdint.h>
#include <string.h>
#include <stdbool.h>
#include <stdio.h>
#include "nvs.h"
#include "nvs_flash.h"
#include "controller.h"
#include "esp_wifi.h"
#include "esp_event_loop.h"
#include "esp_log.h"
#include "esp_system.h"
//#include "esp_heap_alloc_caps.h"
#include "esp_bt.h"
#include "esp_gap_ble_api.h"
#include "esp_gattc_api.h"
#include "esp_gatt_defs.h"
#include "esp_bt_main.h"
#include "esp_gatt_common_api.h"
#include "poll_task.h"
#include "mqtt_task.c"To finish setting up WIFI, add this code brought over from the pcbreflux project.
#undef ESP_ERROR_CHECK
#define ESP_ERROR_CHECK(x) do { esp_err_t rc = (x); if (rc != ESP_OK) { ESP_LOGE("err", "esp_err_t = %d", rc); assert(0 && #x);} } while(0);
/* The examples use simple WiFi configuration that you can set via
'make menuconfig'.
If you'd rather not, just change the below entries to strings with
the config you want - ie #define EXAMPLE_WIFI_SSID "mywifissid"
*/
#define EXAMPLE_WIFI_SSID "your_ssid"
#define EXAMPLE_WIFI_PASS "your_password"
/* The event group allows multiple bits for each event,
but we only care about one event - are we connected
to the AP with an IP? */
const int CONNECTED_BIT = BIT0;
static char buf[128];
#define MAIN_TAG "MAIN"
/* FreeRTOS event group to signal when we are connected & ready to make a request */
EventGroupHandle_t wifi_event_group;Some more adds at the beginning of the file are necessary:
static TaskHandle_t xHandle = NULL;
static bool stop_scan_done = false;
static bool Isconnecting = false;Now for call back function gattc_profile_event_handler. After connecting to the server client program flow proceeds until the client writes its characteristic data to the server. Successful write triggers a ESP_GATTC_WRITE_CHAR_EVT. At this point the polling task is created which then runs continuously so long as the server is connected.
case ESP_GATTC_WRITE_CHAR_EVT:
if (p_data->write.status != ESP_GATT_OK){
ESP_LOGE(GATTC_TAG, "write char failed, error status = %x", p_data->write.status);
break;
}
ESP_LOGI(GATTC_TAG, "write char success, now starting poll_task ");
/* here is created the task that will poll the server. It will delete and return upon disconnect.*/
/*The task needs passed to it the address of the app profile so that it can send requests*/
xTaskCreate(&poll_task, "poll_task", 10240, &gl_profile_tab[PROFILE_A_APP_ID], 5, &xHandle);
break;Note in the code above that the created polling task has passed to it the address of the gattc_profile_inst struct. In this way the polling task has access to the callback gattc_profile_event_handler function as well as its BLE interface. Now the polling task has a loop that regularly sends a read request to the server that in turn sends a response. In gattc_profile_event_handler responses sent from a server trigger a ESP_GATTC_READ_CHAR_EVT. The entire code for this switch is presented below.
case ESP_GATTC_READ_CHAR_EVT: {
// esp_gatt_srvc_id_t *srvc_id = &p_data->read.srvc_id;
//esp_gatt_id_t *char_id = &p_data->read.char_id;
//conn_id = p_data->open.conn_id;
//ESP_LOGI(GATTC_TAG, "READ CHAR: open.conn_id = %x search_res.conn_id = %x read.conn_id = %x", conn_id,p_data->search_res.conn_id,p_data->read.conn_id);
ESP_LOGI(GATTC_TAG, "READ CHAR: read.status = %x", p_data->read.status);
if (p_data->read.status==0) {
esp_gattc_char_elem_t char_result;
uint16_t char_count;
esp_gatt_status_t ret_status = esp_ble_gattc_get_all_char(gattc_if,p_data->read.conn_id,p_data->read.handle,
p_data->read.handle,&char_result,&char_count,0);
ESP_LOGI(GATTC_TAG, "characteristic count = %d", char_count);
if (char_result.uuid.len == ESP_UUID_LEN_16) {
ESP_LOGI(GATTC_TAG, "Char UUID16: %x", char_result.uuid.uuid.uuid16);
} else if (char_result.uuid.len == ESP_UUID_LEN_32) {
ESP_LOGI(GATTC_TAG, "Char UUID32: %x", char_result.uuid.uuid.uuid32);
} else if (char_result.uuid.len == ESP_UUID_LEN_128) {
ESP_LOGI(GATTC_TAG, "Char UUID128: %x,%x,%x,%x,%x,%x,%x,%x,%x,%x,%x,%x,%x,%x,%x,%x", char_result.uuid.uuid.uuid128[0],
char_result.uuid.uuid.uuid128[1], char_result.uuid.uuid.uuid128[2], char_result.uuid.uuid.uuid128[3],
char_result.uuid.uuid.uuid128[4], char_result.uuid.uuid.uuid128[5], char_result.uuid.uuid.uuid128[6],
char_result.uuid.uuid.uuid128[7], char_result.uuid.uuid.uuid128[8], char_result.uuid.uuid.uuid128[9],
char_result.uuid.uuid.uuid128[10], char_result.uuid.uuid.uuid128[11], char_result.uuid.uuid.uuid128[12],
char_result.uuid.uuid.uuid128[13], char_result.uuid.uuid.uuid128[14], char_result.uuid.uuid.uuid128[15]);
} else {
ESP_LOGE(GATTC_TAG, "Char UNKNOWN LEN %d\n", char_result.uuid.len);
}
for (int i = 0; i < p_data->read.value_len; i++) {
ESP_LOGI(GATTC_TAG, "%x:", p_data->read.value[i]);
}
/*The four bytes read must be assembled into a 32 bit integer with little endianess*/
uint32_t v[4] = { 0, 0, 0, 0};
char buff[30];
v[3]=p_data->read.value[0];
v[2]=p_data->read.value[1];
v[2]=v[2]<<8;
v[1]=p_data->read.value[2];
v[1]=v[1]<<16;
v[0]=p_data->read.value[3];
v[0]=v[0]<<24;
for(int i=1; i<4; i++){
v[0]=v[0] | v[i];
}
ESP_LOGI(GATTC_TAG, " v = %d", v[0]);
/*each byte received requires two chars*/
/* sprintf(&buff[0], "%x", p_data->read.value[0]);
sprintf(&buff[2], "%x", p_data->read.value[1]);
sprintf(&buff[4], "%x", p_data->read.value[2]);
sprintf(&buff[6], "%x", p_data->read.value[3]);*/
/*better way to do this*/
/* int j,i;
j=sprintf(buff, "%x ", p_data->read.value[0]);
for(i=1;i<4;i++){
j+=sprintf(buff+j, "%x ",p_data->read.value[i]);
}*/
sprintf(buff, "%d mv", v[0]);
memcpy(buf, buff, sizeof(buff));
xTaskCreate(&mqtt_task, "mqtt_task", 10240, buf, 5, NULL);
}
break;
}In the above code some of it is not used since the response received is simply four bytes. Scanning down one sees where these four bytes are cast back to a 32 bit integer and then sent on to the mqtt_task. Note also that there are two ways shown how to do the casting to a 32 bit integer.
Lastly, upon server disconnect a ESP_GATTC_DISCONNECT_EVT is triggered. The polling task (xHandle) is sent a value of 10 to indicate the disconnect. Then scanning for a server begins.
case ESP_GATTC_DISCONNECT_EVT:
connectt = false;
get_server = false;
ESP_LOGI(GATTC_TAG, "ESP_GATTC_DISCONNECT_EVT, reason = %d", p_data->disconnect.reason);
ESP_LOGI(GATTC_TAG, "sending notification to poll_task");
if( xTaskNotify(xHandle, (uint32_t) 10, eSetValueWithoutOverwrite)==pdPASS){
ESP_LOGI(GATTC_TAG,"notify has been sent");
}else{
ESP_LOGI(GATTC_TAG,"notify sending has failed");
}
ESP_LOGI(GATTC_TAG,"start scanning again for 30 seconds");
start_scan();
/*instead of sending a notification that is used by the task to end and delete itself, one can instead
delete the task here:
xTaskDelete(xHandle);*/
break;
default:
break;Finally, one needs to understant the file poll_task.c. Take particular note that struct gattc_profile_inst must be defined here as a typedef. Then the memory space passed to this function, pvParameters, can then be properly cast as seen in the code. The while loop can now send a read request on the gatt client interface every one second until it receives an notify to disconnect.
#include "sdkconfig.h"
#include "freertos/FreeRTOS.h"
#include "freertos/task.h"
#include "freertos/event_groups.h"
//#include "freertos/heap_regions.h"
#include "esp_wifi.h"
#include "esp_event_loop.h"
#include "esp_log.h"
#include "esp_system.h"
//#include "esp_heap_alloc_caps.h"
#include "esp_bt.h"
#include "bta_api.h"
#include "esp_gap_ble_api.h"
#include "esp_gattc_api.h"
#include "esp_gatt_defs.h"
#include "esp_bt_defs.h"
#include "esp_bt_main.h"
#include "nvs_flash.h"
#define POLL_TAG "POLL"
/*must create a typedef so the pointer can be cast*/
typedef struct gattc_profile_inst {
esp_gattc_cb_t gattc_cb;
uint16_t gattc_if;
uint16_t app_id;
uint16_t conn_id;
uint16_t service_start_handle;
uint16_t service_end_handle;
uint16_t char_handle;
esp_bd_addr_t remote_bda;
} gattc_profile_inst;
void poll_task(void *pvParameters) {
//block milliseconds
const TickType_t xDelay = 1000 / portTICK_PERIOD_MS;
//int i=0;
uint32_t ulEventsToProcess;
const TickType_t xMaxExpectedBlockTime = pdMS_TO_TICKS(10);
/*the device connection profile must be passed to this thread-there is only one */
gattc_profile_inst *dev = (gattc_profile_inst*) pvParameters;
// ESP_LOGI(POLL_TAG, "param: [%s]", param);
while(1){
ulEventsToProcess = ulTaskNotifyTake(pdTRUE, xMaxExpectedBlockTime);
if (ulEventsToProcess > 0){
ESP_LOGI(POLL_TAG, "received a notify event = %d", ulEventsToProcess);
ESP_LOGI(POLL_TAG,"now exiting poll_task");
vTaskDelete(xTaskGetCurrentTaskHandle());
}
ESP_LOGI(POLL_TAG,"in poll task, now sending a read_char request");
esp_ble_gattc_read_char(dev->gattc_if,
dev->conn_id,
dev->char_handle,
ESP_GATT_AUTH_REQ_NONE);
vTaskDelay(xDelay);
}
}Client Gateway and Server Tested Together
Now is shown this system working. Note that in the code there were put lots of additional ESP_LOGI() statements that help greatly in following program flow when monitoring the device via UART. The following short video shows the GATT client operating:
Lastly, the following video shows data being received at the cloud broker and also by a subscriber.
This concludes the description of this project. One of the future changes that one can do is use the “notify” capability where the server notifies the client when the voltage has changed and then sends an update. Then, polling is unecessary.
