It started accidentally: I took a course on Connected embedded systems architecture at CMU - something that I, having experience mostly with web and high-level systems, have feared. Next thing I know is I’m doing a research project on Lora - long range wide area network, a standard gainining popularity in IoT. While I’m not allowed to tell much about our project itself, I might try to explain the technology from a perspective of IoT application developer.
There is a great introductory article - LoRaWAN simply explained - that gives a general understanding of lora and how it’s different from “regular” networks like LTE. I’m not going to repeat it all, but here’s the list of features and peculiarities of Lora that I find important:
- It is very energy efficient, allowing device to run for years on battery. It comes with a cost of very low data rate and small payloads (it depends on specific settings, but we really have to pack payloads tightly to fit into around 10-20 bytes).
- Only a device can initiate data transmission. You can’t just connect to device, command it or ask for some data directly (well, technically you can with class-C devices, but as far as I know nobody does that, as it cancels all the energy-saving benefits). Instead, when you (a server) want to communicate, you put a message to downlink queue an wait patiently until a device connects and grabs this message. Of course, you can program a device to initiate connections often - but then again, it affects the battery.
- Lora itself is quite a low-level protocol, especially if you’re used to working with HTTP. It can’t promise much about time and order of messages exchanged on network. If you enable
acknowledgements(which is not advised to do in a production environment, as it slows down the network), the devices and network server will try to monitor how successfully the messages are sent and received, and will even perform a retry if acknowledge was not received. Aside from that, you are on your own: need to send a large payload? Develop a mechanism for cutting it in parts and then merging. Need to send multiple messages at once? A device can grab only one packet from server during the connection, but you’re welcome to add a multiple transmissions logic above it.
- The network consists of
nodeswith modules like these that you can put on arduino or any similar board. They only communicate via Lora, and are supposed to be numerous (like, thousands). Then there are
gateways- an intermediate device that communicates with nodes via Lora, and with network server via regular ethernet or wifi. There may be many of those, depending on the number of nodes and an area that needs to be covered, but still much less than nodes themselves. Finally, there is one ore more
network server, that holds information about all the devices, manages gateways and sends data to your application.
- From application side, communication with nodes and gateways works over MQTT - a messaging protocol, that for all practical purposes looks similarly to, say, rabbitmq. In the simplest case, you just subscribe to messages from devices and put them into database, visualize, or do whatever.
- On a physical level, it works using ALOHA principle - I’m not going to go into details, but it’s much simpler than you might think. The general, simplified idea is that nodes send data at arbitrary time on the same frequency, and thus interference may occur, making signal from one node indistinguishable from the other. There’re several ways to avoid it - but only statistically speaking (e.g. given that nodes communicate rarely, the chance of interference is low), without any guarantees that we’re used to have with Internet or mobile phones.
There are several open-source network servers that facilitate communication with lora and provide API for node management and gathering data. I’m working with loraserver - which actually consists of 3 components (one is an actual network server for handling data transmission to and from gateways and nodes, another enhances gateway capabilities for more simple and transparent data transmission, and the last one provides higher-level API for node management and access control).
As I mostly work with gathering data from lora network, I communicate with lora application server API a lot and had been exploring its implementation quite deeply.
This lora server is written in
Go - which I think is a great choice for the task. It is a pleasure to see a well-implemented project that follows languages’ best practices and utilizes its power fully.
Naturally, for our project that I’m building on top of lora-server I have also chosen
Go - and this is where my existing experience with this language and with web came handy.
In general, what I’m doing is moving data between systems: from message queue to database, from big Internet to small IoT device, from incoming HTTP API requests to lora-server.
Go feels perfect for gluing them all together, and building a stable long-living service.
From a practical side, Lora is just one of several protocols that are competing for title of a standard IoT protocol. There are others, like zigbee - I’m not familiar with them yet. I know there are certain tradeoffs in every approach and protocol; Lora is not ideal, and there are numerous questions about how it may scale, especially taking in account that ALOHA principle does not allow network to grow without too much interference. There probably will be some improvements and more competition among IoT protocols, and it’s not clear who would win; but for now Lora looks like a major competitor.
I learned a lot during working with lora - and will be learning more, extending my relatively narrow background of a web-developer. Moving down through the OSI stack I’m getting a better understanding of how the Internet works, and, in general, how systems are organized from separate and what it seems primitive particles. I am glad I had a chance to work with it, and although I felt uncertain first, now I believe that changing the focus in software engineering from one field to another makes me a better developer.
On a side note, during this project I’ve found myself doing a lot of conversions between
base64 (which is used in MQTT to transfer data) and raw binary representation, so I made this little chrome extension for that. Most similar extensions and online tools only try to work with ASCII and display some gibberish when non-printable characters are encountered.