Firmware Over The Air (FOTA) for ESP8266 SoC

With the IoT booming nowadays, the number of connected devices grows exponentially and so does the related software that drives them. There is no doubt that Firmware Over The Air (FOTA) is a highly desirable – if not required – feature for any embedded project/product both DIY or commercial. Being able to provide a remote firmware update is obviously very beneficial. The opportunity here is to enhance product functionality, operational features and provide fixes for particular issues.  Updating the firmware OTA may eliminate the need to bring a product into a service center for a repair. Although not every issue can be resolved with a firmware update, if one is available for a particular issue, it can save a lot of time and money.
I already have a large number of connected devices running at home and in case I need to do a firmware update, it is a nightmare. I have to disconnect the respective device from whatever project setup they are attached to re-program and then re-install.  Not all embedded systems have the possibility to allow FOTA due to the complexity and required connectivity, for example none of my RFM12B/RFM69CW related projects have that feature. While possible by installing additional SPI flash (see as example Moteino wireless upgrade), it isn’t a real self-updating node and still requires quite a lot of skill to achieve, and only within the range of the transceivers (not over the Internet).

The ESP8266 SoC with its WiFi connectivity is well positioned to meet that need (FOTA), my WiFi relay project would greatly benefit from it. To achieve this, the available 512KB (on most ESP8266 breakouts) on the SPI flash memory is partitioned into two, and we run code from one or the other partition. Upon FOTA, the currently not used partition gets flashed and system restarted into it:

ESP8266 FOTA memory layout 512KB SPI flash size

This solution limits the available user code to 236KB (256KB-4KB-16KB); For ESP-HTTPD based projects this would be just about the unmodified basic application. Obviously if you need to squeeze in more functionality, you’d need a larger SPI flash than 512KB. There are few off-the shelf ESP8266 breakout boards with > 512KB flash size; alternatively you could desolder and replace the SPI flash with a larger one.

Firmware update modes
I can identify two separate firmware update modes – externally pushed firmware file and another one that is initiated by the remote device on periodical or manual start basis.

Push firmware update

Push firmware update

Pull firmware update

Pull firmware update

Externally pushed firmware update would be when the device receives the firmware file over a HTTP POST request and flashes it, whereas in the self-initiated firmware update mode the device would periodically check central firmware repository for firmware update, pull the firmware and reflash itself. The later can be initiated on schedule, or manually by user (just like your computer checks for updates vs how you can manually force-check). The first method is more convenient for development mode, where the device is connected to the local network. I find it quite useful to focus on developing the code without having to manually place the node in firmware update mode, connect FTDI programmer, bring it out of firmware update mode etc. Also the speed when flashing over network connection is quite good, job gets done in roughly 10 seconds. @TVE has done great work on developing this approach with his esp-link project. Note the Makefile magic that makes all that possible.

For devices in production it makes more sense to run in the second firmware update mode, where we have them check for new firmware proactively. That also addresses connectivity issues compared to previous method as the device initiates the process from behind the firewall (no need for fixed IP address, or port forwarding for outside visibility).

That’s pretty easy to perform with SDK’s in-built functions, the following can get you going in this direction:

static void ICACHE_FLASH_ATTR ota_finished_callback(void *arg)
{
 struct upgrade_server_info *update = arg;
 if (update->upgrade_flag == true)
 {
 os_printf("[OTA]success; rebooting!\n");
 system_upgrade_reboot();
 }
 else
 {
 os_printf("[OTA]failed!\n");
 }

 os_free(update->pespconn);
 os_free(update->url);
 os_free(update);
}

static void ICACHE_FLASH_ATTR handleUpgrade(uint8_t serverVersion, const char *server_ip, uint16_t port, const char *path)
{
 const char* file;
 uint8_t userBin = system_upgrade_userbin_check();
 switch (userBin)
 {
 case UPGRADE_FW_BIN1: file = "user2.bin"; break;
 case UPGRADE_FW_BIN2: file = "user1.bin"; break;
 default:
 os_printf("[OTA]Invalid userbin number!\n");
 return;
 }

 uint16_t version=1;
 if (serverVersion <= version)
 {
 os_printf("[OTA]No update. Server version:%d, local version %d\n", serverVersion, version);
 return;
 }

 os_printf("[OTA]Upgrade available version: %d\n", serverVersion);

 struct upgrade_server_info* update = (struct upgrade_server_info *)os_zalloc(sizeof(struct upgrade_server_info));
 update->pespconn = (struct espconn *)os_zalloc(sizeof(struct espconn));

 os_memcpy(update->ip, server_ip, 4);
 update->port = port;

 os_printf("[OTA]Server "IPSTR":%d. Path: %s%s\n", IP2STR(update->ip), update->port, path, file);

 update->check_cb = ota_finished_callback;
 update->check_times = 10000;
 update->url = (uint8 *)os_zalloc(512);

 os_sprintf((char*)update->url,
 "GET %s%s HTTP/1.1\r\n"
 "Host: "IPSTR":%d\r\n"
 "Connection: close\r\n"
 "\r\n",
 path, file, IP2STR(update->ip), update->port);

 if (system_upgrade_start(update) == false)
 {
 os_printf("[OTA]Could not start upgrade\n");

 os_free(update->pespconn);
 os_free(update->url);
 os_free(update);
 }
 else
 {
 os_printf("[OTA]Upgrading...\n");
 }
}

Here are the the two methods in action:

Firmware update over the air

Security
In terms of security, well, it isn’t the most secure thing. Firmware gets downloaded/uploaded by unsecured HTTP channel and not much validation is performed to ensure the received file is what we think it is. I’m not sure what the overhead in terms of FLASH/RAM would be to enable SSL channel firmware upgrade. These concerns could be overcome by manually initiating an update (push or pull) and disabling the FOTA interfaces afterwards.

What could go wrong?
What can go wrong will go wrong eventually they say, so things must be handled properly. Interrupted download/upload, failed validation result in falling back to the previous firmware. The goal is obvious – to prevent a bricked device. While there is petty good protection against these, there is no protection against uploading buggy new firmware that bricks the device. Doing some testing, then some more testing and again testing before releasing new firmware should reduce the risk for that happening, and rolling it out in small portions also helps identifying a problem before many devices are affected.

Conclusion

I’m convinced the FOTA feature is a “must have” for any serious IoT project. I’m developing a FOTA capable (FOTA+ESPHTTPD+MQTT+SSL+FLASH CONFIG) platform that I will use for my future ESP8266 related projects.