This article, the 11th in a series on home automation documents how to create and integrate an IoT Desktop Device into an existing home automation system including all the necessary software functionality to enable the successful deployment within a domestic environment.
Picture 1 shows the completed IoT desktop device and Picture 2 shows all the component parts used in the prototype which were ‘shoe horned’ into the finished article.
As mentioned above this Instructable details how to make an IoT Desktop Device whose primary purpose is to present the user with an ‘at a glance’ summary of temperature and humidity levels published by distributed sensors within an IoT network accompanied by local barometric pressure along with time and date.
The design seamlessly hooks into the MQTT/OpenHAB based IoT network detailed in this series on home automation (HA), building on reused code taken from here. Alternatively as the MQTT interface is fully documented throughout it can be simply be adapted to an existing HA system via MQTT publications/subscriptions.
At it’s heart is an ESP8266-12E which is responsible for MQTT communications and controlling all functionality excepting LCD display back-lighting control and gesture sensing which use an ATTiny85 and ATMega328P respectively. The device is fully configurable via text files stored on a local SD card, though calibration and network security parameters can also be programmed via remote MQTT publications.
What parts do I need?
See the bill of materials attached below
What software do I need?
- Arduino IDE 1.6.9,
- Arduino IDE configured to programme the ESP8266-12E (same as this). Then configure the IDE as indicated in the detailed description provided in the software sketch,
- Microsoft Excel if you want to make your own icons for the LCD,
- Python v3.5.2 if you wish to use the automated test capability,
- Sigrok PulseView 0.5.0-git-b732b48 if you want to capture the gesture sensor control logic in action.
- Kicad v4.0.7 if you want to have the PCB manufactured.
What tools do I need?
- Microscope at least x3 (for SMT soldering),
- Molex connector crimping tool,
- SMD soldering Iron (with liquid flux pen and flux cored solder),
- Screwdrivers (various),
- Heat gun,
- Drills (various),
- Countersink handtool,
- Files (various),
- Dremel (various bits),
- Sturdy vice (small and large, like a black and decker work mate),
- Vernier calipers (used to measure fabrication and useful for sizing PCB components),
- Spanners and Nut drivers (various),
- Strong tweezers (for SMT soldering),
- Junior Hacksaw,
- Drill (with various drill bits),
- Fine pliers (point and snub nosed),
- Flush cutters,
- DMM with audible continuity check,
- 24MHz 8 Channel USB Logic Analyzer Saleae 8 CH Logic Analyzer for MCU ARM FPGA (for use with Sigrok PulseView).
What skills do I need?
- A lot of patience,
- A great deal of manual dexterity and excellent hand/eye coordination,
- Excellent soldering skills,
- Excellent fabrication skills,
- The ability to visualise in 3 dimensions,
- Some knowledge of software development with ‘C’ (if you want to understand the source code),
- The project contains some interesting code using; variable function arguments (kind of implementation of fscanf, missing from ESP8266 library), linked lists, pointers to pointers for generic file reading, multiple timers with call back routines via function pointers, multiple state machines and a very simple cooperative RTOS, direct display memory reads from LCD over I2C, gradual fading of LCD back-light using easing function controlled PWM, integration of non-blocking implementation of gesture sensor.
- Some knowledge of Python (how to install and run scripts, if you want to use the automated testing),
- Knowledge of Arduino and it’s IDE,
- Good knowledge of electronics,
- Knowledge of how to layout a PCB using a cad tool (Kicad), if you want to have the PCB manufactured,
- Some understanding of your home network.
- User Manual
- Circuit Overview
- PCB Manufacture
- Software System Overview
- Software Overview
- Sensor Calibration
- Gesture Control
- MQTT Topic Naming Convention
- OpenHAB Configuration
- Testing the Design
- References Used
To Part 12 : Retro Speech Synthesis Part 12 IoT, Home-Automation
Step 1: User Manual
This section describes the functionality of the IoT Desktop Device in detail from a users perspective. The video above shows the system in use and demonstrates the various functionality in action.
Pictures 1 through 3 name the main component parts of the IoT Desktop Device
From left to right.
- 1st button ‘Open Logging’ initiates a logging session. This functionality can be controlled remotely by issuing the following MQTT publication; ‘WFD/THBSen/7/Logging/Command/1’, Payload ‘1’
- 2nd button ‘Close Logging’ closes off the current logging session. This functionality can be controlled remotely by issuing the following MQTT publication; ‘WFD/THBSen/7/Logging/Command/1’, Payload ‘0’
Logging is made to the integral SD card in csv format. The name of the logging file is derived by concatenating the Day=DD, Month=MM, Hour=hh,Minute=mm of the local real time clock. ie DDMMhhmm.csv.
The following local parameters are logged; Date, Time, Temp Local, Humidity Local, Barometric Pressure, Ambient Light Level.
- 3rd button ‘Button 0’ is user programmable and has been designed to act in a bi-stable manner. ie. Press once for ‘on’ and press again for ‘off’. For each change of state a user defined MQTT topic and payload is published.
- 4th button ‘Button 1’ is also user programmable however in this instance the associated user defined MQTT topic/payload is published as a one shot each time the button is depressed. No action results when the button is released.
Configuration of these buttons is achieved via editing a text file named ‘conbutin.txt’. This file is resident on the integral SD card and must contain 6 entries as shown below;
- 1st : Button0 Pressed Topic (pressed). ie. /OHB/Req/StudyLights/Control
- 2nd : Button0 Pressed Payload. ie. 1
- 3rd : Button0 Released Topic (released). ie. /OHB/Req/StudyLights/Control
- 4th : Button0 Released Payload. ie. 0
- 5th : Button1 Pressed Topic (pressed). ie. /OHB/Req/StudyLights/Control
- 6th : Button1 Pressed Payload. ie. 0
Note : The entries are in the form of MQTT Topics and their respective payloads. If an entry is not required then the key word ‘null’ must be placed in both Topic and Payload positions
- Ambient Light Sensor
Located on the top of the enclosure in the middle of the buttons, this device monitors local ambient light levels and publishes them via the topic ‘WFD/THBSen/7/ALSStatus/1’. Units are in Lux.
The ambient light sensor is also used to trigger a change in System LCD illumination. As the local light levels increase above a user definable threshold the brightness of the display will also increase to give better contrast.
The converse is also true for decreasing light levels.
Configuration of these parameters is achieved via editing a text file named ‘confvals.txt’. This file is resident on the integral SD card and must contain 7 entries as shown below;
- 1st Display Ambient Light Sensor Upper Threshold Value. In Unsigned Long form.
- 2nd Display Ambient Light Sensor Lower Threshold Value. In Unsigned Long form.
- 3rd Display Delay Before Change Value. In Unsigned Long form.
- 4th Display Back-light Upper Value. In Unsigned Long form.
- 5th Display Back-light Lower Value. In Unsigned Long form.
- 6th Display Delay Before Scroll Value. In Unsigned Long form.
- 7th Enable gesture control. In boolean form 0 = GC off, 1 = GC on
Note : Entries 6 and 7 are general configuration items used to set the delay before an enumerated sensor value will scroll and to enable gesture control respectively.
- Gesture Sensor
The IoT Desktop Device is also fitted with a simple gesture detection sensor on the front of the enclosure which is capable of reconising the following; Left, Right, Up, Down and Near.
- Left forces a left scoll of the list enumerated sensor values
- Right forces a right scoll of the list enumerated sensor values
- Up triggers a user configurable MQTT topic publication
- Down triggers a user configurable MQTT topic publication
- Near forces the display of the local temperature and humidity levels.
Configuration of these parameters is achieved via editing a text file named ‘congesud.txt’. This file is resident on the integral SD card and must contain 4 entries such as shown below by way of example;
- 1st : Up Gesture Topic. ie. /OHB/Req/StudyLights/Control
- 2nd : Up Gesture Payload. ie. 1
- 3rd : Down Gesture Topic. ie. /OHB/Req/StudyLights/Control
- 4th : Down Gesture Payload. ie. 0
- System Display
This display presents the user with 9 items of information as shown in picture 4 above;
When not logging the display shows the words ‘NO LOGGING’. Once logging has been initiated the display generates file name as in the description for the 2nd button and starts logging local ambient values to a csv file on the SD card.
These icons indicate the various states of IoT network connection. See picture 5 above. Here you can observe there are 5 icons ascociated with 3 connection states.
AP mode : If the IoT Desktop device fails to connect to your WiFi network it will become and access point on it’s own network. By doing so it will allow you to redefine the parameters necessary to connect.
MQTT Broker : This shows two states, connected and not connected. Helpful in determining if there is an issue with the MQTT Broker on your IoT network
WiFi Network : This also shows two states, connected and not connected.
This is the time as read from the on-board real-time clock presented in 24hr format
This is the local barometric pressure reading as taken from the internal BMP085 barometric pressure sensor in millibars (mB). The value of barometric pressure is published by the device via the topic ‘WFD/THBSen/7/BarometricStatus/1’.
This gives a simple indication of the barometric trend; rising, stable or falling. See picture 5 above for the actual icons.
This is the date as read from the on-board battery backed real-time clock.
Enumerated sensor name
This is the user assigned name for a given temperature or humidity subscribed topic. Here the user makes the association with say, a temperature and the Hallway, or humidity and Master Bedroom. In each case, once the display cycles through it’s list of subscribed temperature and humidity topics, the enumerated sensor name is also displayed via association.
By doing so it makes the sensor information more readable to the user. A typical entry in the ‘consenin.txt’ configuration file would be;
- 1st : Enumerated sensor name. ie Garage
- 2nd : Temperature Topic. ie. WFD/THSen/1/TempStatus/1
- 3rd : Humidity Topic. ie. WFD/THSen/1/HumdStatus/1
The temperature of the associated subscribed topic paired with an enumerated sensor name. In degrees Celsius.
The humidity of the associated subscribed topic paired with an enumerated sensor name. In %age of relative humidity.
- SD Card
This holds all configuration data and web pages when in AP mode. Configurations files are as follows;
- calvals.txt : This file contains the calibration entries for the temperature, humidity and barometric sensors.
- secvals.txt : This file contains the parameters necessary to connect to your WiFi network and MQTT Broker.
- confvals.txt : This file contains the configuration parameters for sensing of ambient light and triggering the resulting display illumination levels.
- consenin.txt : This file contains configurable entries which enable the unit to subscribe to two topics Temperature and Humidity and link these topics to an enumerated sensor name.
- congesud.txt : This file contains configurable gesture entries which are matched to Up and Down gestures respectively.
- conbutin.txt : This file contains configurable button entries for Button 1 and 0 respectively.
- Leds 1 and 0
These are controlled remotely and can be used as general indication.
Led 1 : This led is controlled by publishing a payload of ‘0’ or ‘1’ to the following topic ‘WFD/THBSen/7/Led/Command/1’
Led 0 : This led is controlled by publishing a payload of ‘0’ or ‘1’ to the following topic ‘WFD/THBSen/7/Led/Command/2’
Led 0 has special functionality. During the initialisation phase at start up, the led flashes to indicate the state of the connection or the failure code if a connection can’t be made.
Note : Local Temperature, Humidity and Heat Index values are published via the following topics respectively;
Step 2: Circuit Overview
The desktop device contains three processors;
IC4 : ESP8266-12E
As mentioned above this processor is responsible for controlling the main functionality of the desktop device along with the WiFi back-link into the network and onward connection to the MQTT Broker.
This processor is directly connected to the integral SD card reader and externally mounted DHT22 temperature/humidity sensor for ease and speed of operation. As I/O on the ESP8266 is limited, extra I/O is provisioned via the on-board I2C interface on GPIO Pins 5 (14) SCL and 4 (13) SDA.
This I2C bus connects the following devices;
- IC1 : PCF8574A (address 0x3E), Generic I/O which connects to the 4 button inputs and two led outputs. This device is 3v3 compatible and as a consequence uses T5 and T6 to interface to the two blue led outputs which require 4.5v to illuminate off the 5v rail. Note T5 and 6 are FETs in order to allow for a ballast resistor to be used (R15, R16). A bipolar transistor would not work as there would be a Vbe of around 0.6v in saturation.
- IC6 : PCF8574A (address 0x3F), Generic I/O which ultimately connects to U3 to form the non-blocking gesture detection system. Described in detail below.
- IC2 : DS1307Z (address 0x68), This is a battery backed real-time clock, which is 5v compatible.
- Barometric Pressure Sensor Module, J6 : BMP085 (address 0x77, 0x1E). Supplied with 5v, but is 3v3 compatible via an on-board 3v3 regulator.
- LCD Module, J8 : PCF8574A (address 0x27), This generic I/O device converts serial (I2C) to parallel (4 bit parallel operation of the HD44780 LCD) which is 5v compatible. See here for further information.
- Ambient Light Sensor Module, J9 : BH1750 (address 0x23). Supplied with 5v, but is 3v3 compatible via an on-board 3v3 regulator.
- U1 : ATtiny85, PWM controller for LCD back-light control. Described in detail below.
- U3 : ATMega328P, Non-blocking implementation of gesture sensor. Described in detail below.
U1 : ATTiny85
This processor forms an I2C interface (address 0x08) to the system LCD which allows for gradual fading of the back-light and is based on the two following instructables, here and here. I non-destructively reverse engineered the PCF8574A Serial to Parallel I2C Module design and discovered it was possible to control LCD back-light intensity by connecting directly to the two LED pins on the module (marked here as X-Y connector in picture 2 above, or J3 in pic 3). Transistors T1, T2 and resistors R18, R19 form the control circuitry which enable PWM of the LCD back-light.
Given smooth fading with easing functions is processor intensive I ported the control of fading to an ATtiny85.
U3 : ATMega328P
This combination of generic I/O IC6 connected to U3 forms the non-blocking gesture detection system (see here for further details of implementation) and is 3v3 compatible. Connection to the APDS9960 is via J14 which is supplied with 5v, but is 3v3 compatible via an on-board 3v3 regulator. This device also has an I2C adress of 0x39, though this is not exposed to the ESP8266, being directly connected to U3. Picture 4 above shows a logic analyser plot of the gesture sensor in action.
The capability to reset the gesture sensor electronics was included in hardware (R37, R42, R36, D2, C18 and T4), but has not been implemented in software as the subsystem has remained reliable since operation over some months.
Mixed supply rail interface (3v3, 5v)
FETs Q1 and Q2 provide the 3v3 – 5v interface for the I2C communications. Pull up resistors ROpt1…4 were included just in case, but not used in the final design as the modules provisioned sufficient coupling to the respective supplies.
The electronics was designed to allow for the in-circuit re-programming of the ESP8266 and ATmega328P for easier debugging;
- IC4 ESP8266 : In-circuit programming is provisioned by J5, S5 and S6.
- U3 ATMega328P : In-circuit programming is provisioned by J13, with automatic reset provided by R42, C18 and D2 which enables the Arduino boot loader to function directly from the Arduino IDE.
Mixed supply rails
The desktop device requires the use of both 5v and 3v3 supply rails.
Supply to the device is first conditioned via an SMS buck converter which efficiently drops a wide range of supply input (9v – 15v) to 7.6v, decoupled via C2, this is fed directly to;
- IC3 an LD1117S50TR a 5v LDO linear voltage regulator, mounted on-board the main PCB. Supply rail decoupling is provided by C4, C6 and C20.
- J7 an LDV1117v33 a 3v3 LDO linear voltage regulator, mounted externally to a 21.6 °C/W heat sink to ensure no heating effects are felt by the locally mounted sensors. Supply rail decoupling is provided by C1, C5 and C21.
Step 3: PCB Manufacture
It’s always been my intention to commit my designs to PCB but the block to this was lack of good CAD tools (by good I mean cheap) and the prohibitively expensive cost of PCB manufacture in the UK.
I had initially purchased a perpetual ‘EAGLE HOBBYIST’ user licence for around £100 with the intention to use this toolchain for both schematic capture and PCB design. Things were looking good, I noticed Adafruit and Farnell were creating footprints for all their parts so in theory it would be relatively easy to put together a design sourcing the components from Farnell and using a local (but expensive) PCB manufacture house I’d located.
Before writing this Instructable, all my schematics were done with Eagles, though I hadn’t as yet gone so far as to create a PCB partly down to not fully understanding the design flow and there not being a vast amount of training collateral available. I did go so far as to purchase Mitchell Duncan’s ‘Learning to fly with Eagle’ (despite it taking forever to get published) and Simon Monk’s ‘Make your own PCBs with Eagle’, but to-date they remain unused, in pristine condition.
However, with the recent takeover of CadSoft by Autodesk all this changed, despite all the usual dishonest reassurances big business offer they reneged and moved over to a subscription based model, something I really dislike. Rightly or wrongly my view is when I purchase software I own it in perpetuity. It’s up to the software company to come up with a model that fits, or I simply won’t purchase your product.
Consequently I was back to square one.
However all this recently changed after I noted a great deal of interest in Kicad. After much ‘Googling’ I found a plethora of information, User Forums, YouTube videos (specifically the truly outstanding work by Chris Gammell at ‘Contextual Electronics‘) etc. All my questions were answered and designing a PCB was actually quite easy.
I now use Kicad for both my schematic diagrams and PCB design. See Pics 1 and 2 above.
Ok, so now I had a good tool chain, plenty of instruction and training but I still needed to use an expensive UK manufacturer to get the PCB fab’ed.
That is until JLCPCB arrived on the scene…
This company seems to have come from ‘left field’, offering an insanely cheap price for a dual sided PTH PCB for $2 and is now within easy reach of the hobbyist.
Ok the $2 thing is for your first order and there are some restrictions on size, but still they offer outstanding value for money.
Pics 3…6 show my JLCPCB delivery. The box was a bit worse for wear but the packing protected the contents. As you can see I received a pack of 5 PCBs, double sided, plated through hole with silk screen and green solder mask. The thinnest trace I am using is approx 8mil to get the density I needed (this was my first board and it made routing easy).
I chose the basic board package and followed the instuctions and (video) on how to generate and upload your Kicad Gerber files. Once uploaded I was presented with a ‘zoomable’ image of my board which gave me a ‘warm feeling’ everything was ok. It was almost like they wanted my business (strange concept in the UK). I paid with paypal and watched how my boards were progressing through the production cycle via my account login.
Just over a week later my boards arrived through the post and the rest is history.
Pic 7 shows my set up for populating the PCB. I used a microscope as my eyes aren’t as good as they used to be, and for the most part I used SMT components which are tiny.
Pics 8 to 10 show the board fully populated and the DIL components inserted.
Finally Pic 11 shows the board fully installed and wired up in it’s enclosure.
All connections to the board are made via 0.1″ Molex connectors to facilitate assembly/disassembly. The RTC and Barometric sensor are located away from the ESP8266 RF source to minimise the risk of interference. The on-board +5v regulator is soldered to a large thermal relief to give better dissipation under operation. I used oversized pads for the 16MHz Xtal on the ATMega328P to allow me to flow solder to all four corners of this SMT device.
Step 4: Software System Overview
This is the external SD SPI Flash Filing System and is used to hold the following information (see pic 2 above);
- Icons and ‘IoT Desktop Device Home Page’ html: Served up by the IoT device when it is unable to connect to your IoT WiFi network (usually due to incorrect security information, or first time use) and provides the user with a means of remotely configuring the sensors without the need to re-flash new SD content.
- Security Information: This holds the information used at power up by the IoT device to connect to your IoT WiFi network and MQTT Broker. Information submitted via the ‘IoT Desktop Device Home Page’ is written to this file (‘secvals.txt’).
- Calibration Information: The information contained within this file (‘calvals.txt’) is used to calibrate the on-board temperature/humidity and barometric pressure sensors should it be necessary. Calibration constants can be written to the IoT device via MQTT commands from an MQTT broker or by re-flashing the SD card.
- User programmable gesture detection aliases: The information contained within this file (‘congesud.txt’) allows the user to associate gestures with the publication of IoT payloads to a given topic.
- User programmable button press aliases: The information contained within this file (‘conbutin.txt’) allows the user to associate system button presses with the publication of IoT payloads to a given topic.
- User configurable system values: The information contained within this file (‘confvals.txt’), chosen by the user, controls certain system responses, such as ambient light levels that cause the display back-light illumination to adapt to it’s environment or display sensor scroll rate etc.
- User configurable enumerated sensor entries: The information contained within this file (‘consenin.txt’), is an association between a logical name and topics to subscribe to, to receive temperature and humidity publications for this logical device.
This functionality is invoked when the IoT device has failed to connect to your WiFi network as a WiFi station and instead has become a WiFi access point something akin to a domestic WiFi router. In the case of such a router you would typically connect to it by entering the IP Address of something like 192.168.1.1 (usually printed on a label affixed to the box) directly into your browser URL bar whereupon you would receive a login page to enter the username and password to allow you to configure the device. For the ESP8266-12E in AP mode (Access Point mode) the device defaults to the IP address 192.168.4.1, however with the mDNS server running you only have to enter the human friendly name ‘DSKTOPSRV.local’ into the browser URL bar to see the ‘IoT Desktop Device Home Page’.
The MQTT client provides all the necessary functionality to; connect to your IoT network MQTT broker, subscribe to the topics of your choice and publish payloads to a given topic. In short it provisions IoT core functionality.
HTTP Web Server
As mentioned above, if the IoT device is unable to connect to the WiFi network whose SSID, P/W etc. is defined in the Security Information file held on the SD Card the device will become an Access Point. Once connected to the WiFi network provided by the Access Point, the presence of an HTTP Web Server allows you to directly connect to the device and change it’s configuration via the use of an HTTP Web Browser it’s purpose being to serve up the ‘IoT Desktop Device Home Page’ web page which is also held on the SD Card.
This functionality gives the IoT device the capability to connect to a domestic WiFi network using the parameters in the Security Information file, without this your IoT device will not be able to subscribe/publish to the MQTT Broker.
WiFi Access Point
The ability to become a WiFi Access Point is a means by which the IoT device allows you to connect to it and make configuration changes via a WiFi station and a browser (such as Safari on the Apple iPad). This access point broadcasts an SSID = “DSKTPDEV” + the last 6 digits of the MAC address of the IoT device. The password for this closed network is imaginatively named ‘PASSWORD’
Step 5: Software Overview
To successfully compile this source code you will need a local copy of the code and libraries outlined below in Step 12,References Used. If you are not sure how to install an Arduino library go here.
The software makes use of the state-machine as shown in pic 1 above (full copy of source here). There are 5 main states as outlined below;
- This initialisation state is the first state entered after power up. During this state the enumerated sensor name file (‘consenin.txt’) is read with a call to ‘readConfigurableSensorInput();’. This enumerated sensor name versus temperature/humidity topic is loaded into a linked list, a dynamic data structure for use when connected/active. See Linked list below.
- This state is entered if after power up an invalid or missing secvals.txt file is detected
- PENDING NW
- This state is transitory, entered whilst there exists no WiFi network connection
- PENDING MQTT
- This state is transitory, entered after a WiFi network connection has been made and whilst there exists no connection to an MQTT broker on that network.
- This is the normal operational state entered once both a WiFi network connection and an MQTT Broker connection has been established. It is during this state the ambient light level, temperature, barometric pressure, heat index and humidity at the IoT Desktop Device is regularly published to the MQTT Broker.