Martin's corner on the web

RFM69 to MQTT gateway using ESP8266

I’ve mentioned my plans for this project during the IoT hangout session few months ago, it has finally materialized as working prototype. The blocker so far was the lack of ESP8266 RFM69 driver and free time on my end, so I’ve teamed up with Andrey Balarev to solve this. Andrey is an IoT enthusiast + embedded systems developer and has done excellent job in porting LowPowerLab’s RFM69 library for the ESP8266. He spent few weekends staring at the oscilloscope, re-writing SPI library code and managed to solve a number of challenges to get this library going. I’ve bundled his driver code with MQTT to get a beautifully working prototype that forwards RFM69 traffic to MQTT and vice versa. Few screenshots of the setup:

Some folks probably ask why the RFM69 and not RFM12B and why LowPoweLab’s library and not Jeelib, I’ve outlined my view on that before here. Basically with Jeelib we are forcing the RFM69 in artificial compatible mode and not taking fill advantage of the chip.

What I plan to add as functionality is payload serialization configuration, payload field name configuration and RPN based signal processing configuration so that JSON could be constructed/deconstructed from payloads in both directions.

It will be a replacement for the RFM2Pi project, a much more advanced standalone M2M gateway for the IoT.

I’ll build few PCBs and put some for sale in my shop as well. RFM69 library code will be shared at that point.

 

 

Tagged on: , , , ,
By Martin | November 7, 2015 | ESP8266 |

17 thoughts on “RFM69 to MQTT gateway using ESP8266”

  1. Bill Thomson

    Hi Martin,

    I’d like to make a suggestion based on some user feedback I’ve seen on the OpenEnergyMonitor forum. When an OEM user runs more than one RF link, sometimes they lose data because of packet collisions. The idea I had in mind was to have the RFM69 receiver listen before transmitting. Something similar to the way Ethernet collision detection works. From as simple as detecting a carrier, to as sophisticated as detecting data. Is this a function that’s already present? If not, is it possible to incorporate it into the RFM69 code?

    1. Martin Post author

      Interesting, can you point me to that discussion? Generally it shouldn’t be a problem if the remote node requires ACK, as it will resend in case of non receipt. The RFM69 is capable of monitoring RSSI, in theory it could wait for the air to clear before transmission, but that would include buffering the packets and more complex handling. Also, what are the chances that a 30ms transmission that happens every 5min for example collides with another one?If packet loss is a problem, a dual gateway could be considered, one for incoming and the other for outgoing transmissions.

  2. Johan Kanflo

    The LowPowerLab driver implements CSMA (Carrier Sense Multiple Access) [https://github.com/LowPowerLab/RFM69/blob/master/RFM69.cpp#L210]. It does not however implement Collision Avoidance (CSMA/CA) needed to prevent nodes B and C from colliding once they both detect node A has stopped transmitting. Even if you only have a few nodes in your OEM system your home security system will cause disturbance at 868MHz, as well as your neighbors and so on. CSMA/CA and ACKs are crucial for a reliable RF-link.

      1. Martin Post author

        Here is a CSMA/CA implementation for the RFM69 [https://github.com/ahessling/RFM69-STM32/blob/master/rfm69.cpp#L451-L486] , might be worth incorporating it

  3. Johan Kanflo

    That seems spot on, a must have. Just out of curiosity, what is the difference between your and Andrey Balarev’s driver and ahessling’s RFM69-STM32?

    1. Martin Post author

      I haven’t deep-dived, but it seems not to be fully compatible with LowPowerLib’s; I’m more interested in the CSMA/CA part, however that also requires modification to the remote sender side; I’m sure Felix will gladly accept a pull request once this is incorporated, I agree with you that it is a very important feature.

  4. Pingback: RFM69 to MQTT Gateway on the Super-Cheap | Hackaday

  5. Teodor V.

    Have you considered using UBJSON as the serialization format? That would be a good choice if you are intending to construct/deconstruct JSON from your packets.

    1. Martin Post author

      Sounds like a good option,I’ll dig further. Thanks for the tip! One of the things I wanted t implement is that the remote nodes announce the payload structure they will send and the gateway accepts that configuration for the particular node ID. I still have to work out how that will happen..

      1. Teodor V.

        There’s a number of binary formats for JSON, see the “See Also” section of the wiki page for pointers, as well as SMILE. There ought to be something embedded-friendly among those :3

      2. Johan Kanflo

        I have been working on an RF serialization protocol for some time and just finished a writeup of my
        solution        .

        The node payload structure is built into the protocol and the python receiver can use this meta data to construct Emoncms inputs and feeds on the fly when new nodes connect. I have some ideas about how to bridge RF to MQTT but have not looked into it in detail.

  6. Pingback: Serializing data from IoT nodes | Johan Kanflo

  7. Pingback: Spectrum analysis with RFM12B | Martin's corner on the web

  8. customdev

    Any progress updates on the gateway software? Eliminating several Raspberry Pi computers and various 5V adapters would free up some of the outlets around my home… And having to rely on network boot or SD cards.

    1. Martin Post author

      I haven’t had time to further develop the project, there will be a blog post when I get there. Indeed the purpose is to eliminate all the other hardware and so cut on the cost.. if only I had time 🙂