Martin's corner on the web

Rock-solid RFM2Pi gateway solution

I have had a good share of frustration with damaging Raspberry Pi SD cards due to wearing out/improper shut downs as a result from power failures and was looking for solution to this sort of problems for a while.

At first I thought I had the wrong SD cards, then that my power supply wasn’t good, then that I wasn’t running the latest kernel version, tried to reduce writes to the card by fiddling with swap files, using RAM disks, moving the root fs to external drive and so forth. Finally I came to the conclusion that you can’t expect from SD card based OS running in R/W mode to be reliable in the long term.

SD cards do wear out when written to and are quite sensitive to powering down while a write operation is in progress. Chances are that sooner or later you will end up with a bad SD card. In my case it was always sooner.  As I have mentioned in my post yesterday, I have found a ready to use SD card image that runs the root file system in read-only mode. That means you never have to worry about SD card wear or power failures. The down side is that you can’t run applications that require R/W disk access, like MySQL database for example. You can switch temporary between R/W and R/O modes to perform updates, new software installation.

In my case I’d like to have the RFM2Pi board forward the wireless transmissions to external and more reliable emoncms installation. Here are my steps:

  • I grabbed the IPE – Industrial Perennial Environment is a special blackout-proof flavour of Raspbian. Used root/root as username/password
  • I ran “firstboot” as recommended to expand the SD card to its full size.
  • Swithed the root fs to R/W mode so that the following steps can happen by running
ipe-rw
  • I configured the networking for my LAN
nano /etc/network/interfaces
  • I had to re-create the SSH keys so that I am able to SSH into the Raspberry Pi
sudo rm /etc/ssh/ssh_host_* && sudo dpkg-reconfigure openssh-server
  • I had to make sure that Raspberry Pi’s UART is disconnected from the console and available for programs to use. The problem here is that /boot/cmdline.txt is mounded on a R/O partition, easiest way is to insert the SD card in a computer and edit that file there. Remove the text that make reference to the UART i.e.
console=ttyAMA0,115200 kgdboc=ttyAMA0,115200
  • Run
nano /etc/inittab

At the bottom of the file comment out the line (by adding a ‘#’ at beginning)
# T0:23:respawn:/sbin/getty -L ttyAMA0 115200 vt100
[Ctrl+X] then [y] then [Enter] to save and exit

Now that the UART is free to use, I need to set up the RFM2Pi software.

I’d like to thank Jerome Lafréchoux for his great work on creating a python gateway script  that will interface the RFM12B to emoncms. Setting it up is easy:

sudo apt-get update
sudo apt-get install git-core
sudo aptitude install python-serial python-configobj
git clone http://github.com/Jerome-github/oem_gateway.git
cd oem_gateway
cp oemgateway.conf.dist oemgateway.conf
nano oemgateway.conf

The configuration file oemgateway.conf contains the configuration settings and is pretty straight forward to understand. I needed to forward RFM2Pi readings to a remote emoncms, so my configuration looks like this:

# OemGateway Configuration file example
# Copy this as oemgateway.conf (or any custom name) and edit settings

# Each listener and each buffer has
# - a [[name]]: a unique string
# - a type: the name of the class it instantiates
# - a set of init_settings (depends on the type)
# - a set of runtime_settings (depends on the type)
# Both init_settings and runtime_settings sections must be defined,
# even if empty. Init settings are used at initialization, and runtime
# settings are refreshed on a regular basis.

# All lines beginning with a '#' are comments and can be safely removed.

####################
# Gateway settings #
####################
[gateway]
# loglevel must be one of DEBUG, INFO, WARNING, ERROR, and CRITICAL
# see here : http://docs.python.org/2/library/logging.html
loglevel = DEBUG

#############
# Listeners #
#############
[listeners]

# This listener manages the RFM2Pi module
[[RFM2Pi]]
type = OemGatewayRFM2PiListener
[[[init_settings]]]
com_port = /dev/ttyAMA0
[[[runtime_settings]]]
sgroup = 210
frequency = 8
baseid = 15
sendtimeinterval = 60

# This listener gets data from a socket
[[Socket]]
type = OemGatewaySocketListener
[[[init_settings]]]
port_nb = 50011
[[[runtime_settings]]]

###########
# Buffers #
###########
[buffers]

# The two following buffers instantiate the same class,
# that formats the data for an emoncms instance.
# One sends the data to a local instance, the other one
# to a distant one.
# If active is set to False, the buffer neither records nor sends any data,
# but it holds unsent data until active becomes True.
[[emoncms_local]]
type = OemGatewayEmoncmsBuffer
[[[init_settings]]]
[[[runtime_settings]]]
domain = localhost
apikey = xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
protocol = http://
active = False
path = /emoncms

[[emoncms_remote]]
type = OemGatewayEmoncmsBuffer
[[[init_settings]]]
[[[runtime_settings]]]
domain = emoncms.org
apikey = ************MY API KEY *******************
protocol = http://
active = True
path =

Finally I had to set that the script runs at each boot by adding the following line to /etc/rc.local, just before the “exit 0” statements:

(sleep 10; python /root/oem_gateway/oemgateway.py --config-file /root/oem_gateway/oemgateway.conf)&

 

A reboot and we are in business. Finally a Raspberry Pi RFM2Pi gateway that isn’t afraid of power failure or SD card wear.

 

 

By Martin | August 20, 2013 | Raspberry Pi, RFM2Pi v2 |

27 thoughts on “Rock-solid RFM2Pi gateway solution”

  1. Nate

    Thanks! That is exactly what I had been hoping to do. I didn’t know about that OEM gateway script – I had been working on modifying the official one myself… heh.

    Keep up the good work Martin 🙂

    Nate

  2. Martins

    Dear Martin,

    thank you very much for your excellent work and sharing of it. Very nice and straight forward how-to.
    Two empiric observations: first – should line ” cp oemgateway.conf.dist oem_gateway.conf ” be ” cp oemgateway.conf.dist oemgateway.conf ” and second – my installation only came alive after running ” sudo aptitude install python-serial python-configobj ” .

    Thank you very much again –
    Martins

  3. Glyn Hudson

    Hi Martin,

    Another great post. I’ve just implemented this myself after having an SD card fail on me. I had to run sudo apt-get update before i could get the git core package to inatall. Might be worth adding this instruction to your post.

    Cheers,

  4. Frans

    Hello,
    I’m new with this but put the image on a SD cart and its running on the Pi.
    I also configure the oemgateway.conf
    I get a picture on the Pi whit the regular line from the EmonTX. (Every 5 minutes.)
    How can I login on the pi, I tried several logins and passwords but no success.
    How can I see the Pi on my PC and become an input on the Emoncms
    Thanks very much for your help.
    Regards Frans

  6. Abdullah

    Did you connect your Raspberry Pi to a display for the initial setup of the IPE image, or were you able to set it up headless using SSH?

      1. Abdullah

        Thanks for the reply. I have one more question; Are you sure this IPE image is ABSOLUTELY resistant to power outages? Have you tested it?

        1. Martin Post author

          yes, I now don’t even bother to properly shut my Pi down, I just plug the cord out. Did so many, many times over since

      2. Karoly

        Hi,

        sorry, I think I forgot to mention that telnet is enabled by default on IPE R1. It will allow root to login without a password after a successful DHCP IP-address acquisition.
        I should clarify this on IPE R1 update.

        BR
        KK

  7. Nick

    After such a success on this, I am looking to move my boiler controller pi to IPE. Currently, it runs Apache locally which is called with a simple GET request to http://192.168.1.x/heating/turnon.php .. however, Apache is being a PITA with switching off logs. I’ve commented all log stuff related in apache2.conf and 000-default.
    Just wondering whether I should be looking at Cherokee etc instead? Any thoughts?
    Oh, and apt-get upgrade with IPE wasn’t best pleased 🙁
    Definitely the way forward with SD card failures, could do with a few more basics installed. Or consider a different way of calling from one node to another.

    Just wondering if you have any thoughts on this?

    1. Nick

      In answer to my own question. I decided nginx was the way forward … re-compiled with /dev/null as the log paths. A bit of messing about with my USB permissions and I now have a working USB relay board controlling my boiler with IPE. Nice.

  8. Antoine

    Hi there,

    I followed this article and it works really well. But I’m wondering about running the script on each boot.
    I changed the properties of the script with “chmod 755 oemgateway.py” and added the line in the “rc.local” file, but the script doesn’t run at boot…
    Do you have any idea why that doesn’t work?

    I’m sure that the rc.local run at boot because I run 2 services with this file and it works…
    Thanks for the help!

  9. Alex

    Hi,

    Tnx for this guide, it works flawless 😉

    But I have one question, I would like to run the php script to gather weather info but I haven’t php installed. I have to install nginx and follow the read once Fs guide? There sin’ an option to have php and not a web server?

    Tnx

  10. Kevin

    Hi Martin,

    The gateway works flawlessly as advertised.

    In another of your posts (http://harizanov.com/2012/04/software-only-solution-for-feeding-weather-data-to-emoncms/) you talk about setting up a cron job to retrieve weather data. I would like to follow in your footsteps and get this to work using this gateway. So far I have tried installing LAMP following Bruce Fulton’s video here: http://www.youtube.com/watch?v=CEji-qN-TEE. After some error checking I’ve been able to get PHP to work (i.e. I can load index.html in the browser). Part of the process to get this working required creating a directory for error logging (/var/log/apache2/). However I find that this directory does not persist when I reboot the raspi … its as if I hadn’t made any changes. Also, when logging into the raspi through the SSH, I have had some issues consistently seeing the /var (and other) folders instead of just the oem_gateway folder.

    I am beginning to wonder if I’ve taken the right approach or not. I only started using raspis and emoncms last week Linux I just touched for the first time tonight!

    Thanks for your help.

    Kevin

  11. Nick

    Quick note that with the latest git pull, the file oemgateway.py has been renamed oemgateway … The rc.local stuff needs a slight edit.

  12. Pingback: Raspberry Pi & SD card reliability | Interacting Objects

  13. Vince

    Hi Martin,

    Thanks for this solution, this helps allot especially that i am about 200KM away from the point that we would like to measure and readout over the internet.
    I do however have a question and i guess it is still possible as you describe “You can switch temporary between R/W and R/O modes to perform updates, new software installation.”

    The ISP we have in that building changes quite allot from IP address, and we would like to be able to switch a relay or 2 aswell with the raspberry PI, but we do need the correct IP to logon the webserver of course to switch the relay.
    So my question is, is it possible to install a script that every X minute/hour uploads the ip to a DNS website suchs as Dyndns for example.
    And run a script next to it that would enable relay switching ?

    This is the first project for us with a PI and programming so i have little to no knowledge about this yet.
    Hope you can help us further with the answer.

    thanks and regards

    Vincent