Category Archives: Z-Wave Controllers

Z-Wave Summit 2017 at Jasco in Oklahoma City

“IoT Device Testing Best Practices” by Eric Ryherd

Summit1Click HERE to see the entire presentation including my notes. If you are a Z-Wave Alliance member a video of the presentation is usually posted on the members only section of their web site. The main takeaways from my presentation are:

  • Have a written test plan
  • Use the Compliance Test Tool (CTT) as the START of your test plan
  • Vary the environmental conditions during testing
  • Test using real world applications
  • Test using complex Z-Wave networks with routing and marginal RF links
  • Test with other hubs and devices
  • Automate testing using tools like the ZWP500
  • Code firmware with failure in mind
  • Utilize the WatchDog timer built into the Z-Wave chip

ZWaveSummit2017aThe presentation goes into detail on each of these topics so I won’t duplicate the information here. I also go thru several failures of devices I’ve been working with. You learn more from failures than you do when everything just works. Feel free to comment and let me know what topic you’d like to see for next years summit.

Z-Wave Summit Notes

One of the main purposes of the summit is to learn what’s new in Z-Wave and what Sigma is planning for the future. The most important news at this year’s summit is SmartStart. The goal for SmartStart is to simplify the user experience of installing a new device on a Z-Wave network. The concept is that a customer will open the package for a device, plug it in, the hub is already waiting for the device to be joined and the device just shows up on their phone without having to press a button or enter the 5 digit pin code. This is a “game changer” as Sigma pointed out many times during the summit. Typically a user has to put their hub into inclusion mode, read the product manual to determine the proper button press sequence to put the device into inclusion mode, wait for the inclusion to go thru, write down the NodeID number, with an S2 device they have to read the teeny-tiny 5 digit PIN code printed on the product (or scan the QR code) and then MAYBE the device is properly included. Or more often, they have to exclude and retry the process all over again a couple of times. SmartStart as you can see will make the user experience much easier to get started with Z-Wave.

SmartStart enables “pre-kitting” where a customer buys a hub and several devices as a kit. The hub and the devices in the kit are all scanned at the distribution warehouse and are all white listed on the hub web site. When the customer plugs all the devices in, they automatically join and all just magically show up ready to be used without the frustration of trying to get all the devices connected together. Unfortunately there are no devices that support SmartStart and there are no hubs that support it either – yet. We’ll get over that eventually but I suspect it’ll take a year before any significant numbers of SmartStart supported devices show up on Amazon.

SmartStart is enabled in the SDK release 6.81 which occurred during the summit. There are some other handy features in this release. The main new feature (after SmartStart) is the ability to send a multi-cast FLiR beam. One problem with FLiR devices is that they are all sleeping devices and briefly wake up once per second to see if someone wants to talk to them. Prior to 6.81 you had to wake up the devices one at a time and each one would take more than one second to wake up. If you have battery powered window shades like I do, there is a noticeable delay as the shades start moving one at a time instead of all together. Both the shades and the remote (or hub) will need to be upgraded to 6.81 before we can use this new feature. That means it’ll be again probably another year before this feature is widely available, but it’ll get there eventually.

At CES in January Sigma will be announcing the newest generation of the Z-Wave transceiver chip, the 700 series. The long awaited upgrade from an 8-bit 8051 CPU to a more capable 32-bit ARM CPU will make it much easier for developers like Express Controls to hire engineers who can code firmware and thus we’ll be able to bring more Z-Wave products to market in less time. The chip won’t be available until mid-2018 so we have to stick with our little 8-bit CPU for now.

A new web site, Z-WavePublic.com, has been populated with the Z-Wave documentation as well as images for the Beagle Bone Black and Raspberry Pi loaded with Sigmas Z/IP and Z-Ware. With one of these boards and a USB Z-Stick anyone can start developing with Z-Wave without having to sign a license agreement. Nice way to get started with Z-Wave for you DIY nerds out there. There were many other presentations on Security S2, Certification, The CIT, Z/IP, HomeKit and many other topics on the technical track of the summit. The marketing track had a different set of presentations so I recommend sending both a technical person and a marketing person to the summit.

Summit isn’t all work, work, work

IMG_20170927_090355The Summit isn’t all work all day though the days are long and tiring. Tuesday evening was a reception at Coles Garden which is a beautiful event venue. Unfortunately it was raining so we couldn’t wander thru the gardens much but Mitch, the Alliance Chairman, kept us entertained.

Wednesday evening was the Members Night at the Cowboy museum. Oil profits made a lot of wealthy Oklahomans who were able to make sizable donations to this huge museum. There is a lot more to see than we had time to explore so I’d recommend spending more time here if anyone is visiting Oklahoma City. Lots of food and drink made for an ideal networking environment with your fellow Z-Wave developers.

 

Eric Ryherd Presenting “IoT Device Testing Best Practices” at Z-Wave Summit in Oklahoma City September 26-28, 2017

Z-Wave Developers and Marketers will come together at the Z-Wave Summit at the Jasco facility in Oklahoma City September 26-28, 2017. You have to be a member of the Z-Wave Alliance to attend. I highly recommend attending if you are developing Z-Wave devices . Networking with other Z-Wave developers and having intimate access to the Sigma Design and Alliance engineers and marketing folks is invaluable. To attend, register via the Alliance member-only web site. The Alliance always has some fun in the evenings too so it’s not all work!

Eric Ryherd Presenting

Express Controls founder and Z-Wave expert Eric Ryherd (aka DrZWave) will be presenting at the summit for the 4th consecutive year. Last years presentation was “Seven Things you probably don’t know about Z-Wave” and was well received. I was surprised how many engineers were completely unaware of the many new features and command classes that have been added to Z-Wave in the past couple of years. This year’s topic is “IoT Device Testing Best Practices“. I’ll go over some of the failures I’ve found over the last several years in both my products and other products I’ve tested.

Abstract

Z-Wave wireless Internet of Things (IoT) devices are hard to test! There are countless devices already in customers hands with bugs in them that make Z-Wave seem unreliable when in fact many of the issues are bugs in the device firmware.  Eric Ryherd, Z-Wave expert and consultant, describes some of the  failures that are still shipping today and best practices when testing your IoT device to reduce the chance your device fails in your customers hands. Simple command sequences sent one at a time by a test engineer is not representative of the real world packet storms that occur in an apartment building with complex RF reflections and multiple interfering RF networks. Your device has to work in the real world and to do that you need to simulate those terrible conditions that do not happen on the engineers desk.

Author Bio

Eric Ryherd licensed Z-Wave in 2003 to develop IoT devices before the term IoT even existed. Light switches, motion & temperature sensors, water valves and meters, hubs, window shades, remote controls are just a sample of the Z-Wave IoT devices developed and tested by Express Controls. Eric applies his Z-Wave expertise in consulting, training and assisting with Z-Wave Certification to companies of all sizes. Read more by Eric at his blog – DrZWave.blog.

10 Questions when Reviewing Embedded Code

Design News posted a great article “10 Questions to Consider When Reviewing Code” and I’m just posting the list here. Follow the link for the full article with the details behind each question.

  1. Does the Program build without warnings?
  2. Are there any blocking functions?
  3. Are there any potential infinite loops?
  4. Should this function parameter be a const?
  5. Is the code’s cyclomatic complexity less than 10?
  6. Has extern been limited with a liberal use of static?
  7. Do all if…else if… conditionals end with an else? And all switch statements have a default?
  8. Are assertions and/or input/output checks present?
  9. Are header guards present?
  10. Is floating point mathematics being used?

My personal pet peeve is #3 – I am constantly reviewing that uses WHILE loops waiting for a hardware bit to change state. But what if the hardware bit is broken? Then the device is DEAD. Always have some sort of timeout and use a FOR loop instead of a WHILE loop. At least the code will move on and won’t be dead. Maybe it won’t work properly because of the broken hardware but at least the device can limp along.

How to support Over-The-Air (OTA) firmware update now that Micron has End-Of-Lifed the NVM Z-Wave needs

Sigma Designs Z-Wave chips rely on an external non-volatile memory (NVM) to store the routing tables and other data that needs to survive a power-loss. Unfortunately, Sigma chose an NVM that is single-sourced and what is even more unfortunate is that the source, Micron, has decided to End-Of-Life (EOL) the part in March of 2018. Semiconductor companies like Micron regularly EOL parts as technology moves on so I can’t fault them for that. Micron  gave plenty of warning as the EOL notice went out in 2013 giving you 5 years to prepare for the end. I and many other Z-Wave manufacturers didn’t realize the EOL until a couple of months ago when the NVM became out-of-stock at virtually all distributors! This shut down production lines and we were unable to build product! A potentially company killing event. Fortunately Micron was able to restock and we were able to ship product with only a slight delay. This temporary unavailability raises the critical nature of finding a replacement soon.

What are the Z-Wave NVM options?

Before deciding which NVM to use, you have to decide if you want to support Over-The-Air (OTA) firmware update or not. I highly recommend OTA because Z-Wave continues to evolve and being able to update the firmware in the field is a huge advantage. OTA requires enough NVM storage to hold a complete firmware image of the Z-Wave module which is 128K bytes AND the routing tables which is roughly another 32K bytes. The firmware image can be compressed which allows you to use a 1Mbit (128K byte) NVM but the price for a 2Mbit vs. 1Mbit is tiny and in some cases the larger ones are cheaper than the smaller ones because they are on a newer silicon technology.

Without OTA use commodity EEPROM

If you decide to not have OTA, the NVM options are not affected by the EOL because for slave devices you don’t even need an NVM. Simple slaves can store small amounts of application data in the on-chip FLASH of the Z-Wave chip. This saves the cost, size and power of the external NVM so in some cases this is the best way to go. For very small battery-powered devices like a key-fob, this is the way to go.

Routing slaves or controllers require an external NVM such as the  Atmel AT25256 32Kbit EEPROM. The EEPROM is a commodity device supported by several manufacturers so there is no EOL issue with these devices. Controllers and routing slaves need to store  the routing tables in NVM which are many K bytes of data. The EEPROM devices are well suited to this task because you can write to single bytes of the EEPROM at a time. The serial FLASH chips are not able to write to single bytes which is the major problem. Gateways or other controllers with another micro on-board can update the Z-Wave firmware using either the SPI pins or the UART/USB. The code to burn the firmware into the Z-Wave chip has to be written by you or contact Express Controls and we can sell you a small package to do it.

With OTA use a single source serial FLASH

If you decide to include OTA in your product (and I highly recommend it), then your only solution is to use the Adesto Tech AT45DB0xx family of serial FLASH chips. The Sigma 500 Series Integration Guide (document number INS12213) lists only the Adesto part as the sole supported option at the moment. Sigma is actively looking for alternatives but there don’t appear to be any at the moment. The problem I have with the Adesto part is that it is only available from Adesto, there is no second source. This is a unique part and worse, it has a completely unique pinout as shown below. Using the Adesto part REQUIRES PCB RE-LAYOUT! Existing designs that have already passed FCC regulatory and Z-Wave compliance may not be able to switch to the Adesto part because of the pinout differences. The pinout of each part is shown below. The pins are completely scrambled. The signals are basically the same function but their placement is scrambled.

Z-Wave Experts consulting and IoT product development

The only solution is to put both footprints on the PCB to enable the option of purchasing whichever part is available and cost-effective.

Why doesn’t Sigma support commodity serial FLASH chips? The problem with commodity serial FLASH chips is that they require you to erase an entire sector (4096 bytes) any time you want to write even one byte of data. The Micron and Adesto parts both have Page Write capabilities where a page is only 256 bytes. Every time you write a single byte of data to the NVM the Sigma code will READ 256 bytes, erase the entire 256 bytes (which is a single SPI command), and the write the 256 bytes with the updated byte of data. Any time you write more than one byte to the EEPROM you should be using the MemoryPutBuffer command and not the MemoryPutByte. Supporting commodity FLASH chips would require reading/storing/writing 16 times more data with each write. That would slow many operations down to the point of unusability.

Comparing the NVM Parts

The Adesto part is specifically designed for IoT devices and has a number of really handy features. Specifically it has an Ultra Deep Sleep mode of only 1uA. The Adesto NVM has a wide voltage range which ensures it will work in battery-powered devices even as the battery approaches its end of life. The Sigma Z-Wave 500 series chips are able to operate down to 2.3V and the Adesto part is well below that.

The table below gives a breakdown of the options for NVM with Z-Wave. Using the on-chip FLASH for a few dozen bytes of application storage is of course the lowest cost option but limits the routing capabilities of the node. This option is ideal for Key-fobs or other very space/cost constrained devices that don’t need OTA. The EEPROM is fine for controllers that can be reprogrammed using a second micro. The Micron part is not recommended since it will soon be unavailable. The Adesto part is expensive and is single sourced and has a unique pinout but it is the only solution at the moment.

Spec None EEPROM M25PE AT45DB
PinOut NA Std Std Non-Std
OTA No No Yes Yes
Size on-chip 32Kb 2Mb 2Mb
Write 1 byte 1 byte 256 bytes 256 bytes
Vcc 2.3-3.6V 1.8-5.5V 2.7-3.6V 1.65-3.6V
Sleep 0 10uA 10uA 1uA
Cost1 $0.00 $0.32 $0.42 $0.75
Use Case Small battery powered devices like Key Fobs Controllers with second micro for firmware update OBSOLETE! Recommended

If Sigma can figure out how to work with the larger 4Kbyte sector sizes of commodity serial FLASH chips then the cost of a Z-Wave device can drop by over 50 cents resulting in a couple of dollar cheaper retail price. All Z-Wave manufacturers should be pressuring Sigma to come up with a cheaper solution.

Notes:

  1. NVM costs are 1K pricing from online distributors Q1 2017 and should be used for comparison purposes only – your results will vary

Seven Habits of Highly Effective Z-Wave Networks for Consumers

You have a Smart Home using Z-Wave as a wireless technology for all these Internet of Things (IoT) devices to communicate with each other. But maybe things are not working quite as well as you expect. You press a button on your phone and 1… 2… 3… and then finally a light comes on or maybe it doesn’t come on at all! Another common problem is when a battery powered sensor was updating the temperature last week and this week it just doesn’t seem to be sending updates anymore or at best sporadically. As a Z-Wave expert I’ve built and rebuilt hundreds of Z-Wave networks and have come up with a few habits to make Z-Wave networks more reliable.

1. Minimize Polling

This is probably THE number one mistake new users of Z-Wave make. They figure Z-Wave is a high speed network so they can just poll a light switch every 3 seconds and then react to any change in the switch. Z-Wave and most other wireless networks work best when the network is highly available. If the network is busy, every device that needs to send a message has to wait its turn and then compete (and often collide) with all that polling traffic. Collisions slow everything down just like rubber-necking on the highway.

Polling used to be the only way to get around a patent that fortunately expired in February 2016. The patent forced many light switch manufacturers to not send a message when you flipped the switch. Several manufacturers found ways to get around this or they licensed the patent. But now that the patent has expired, you can get light switches that do send a report immediately when their state has changed.

So the primary way to minimize polling is to replace the few devices in your Smart Home that trigger an event  (or SmartApp or Magic or whatever your hub calls it) with one that will instantly send an update. If you have some older switches but they’re not that important to instantly know their state has changed, you can still poll them but no more than once every few minutes. Remember that if you have 60 Z-Wave devices and you poll each one once/min then you are polling once/second and the network is hammered! So only poll a couple of nodes!

2. Have enough devices to create a mesh

I can’t tell you how many people I’ve worked with that had a door lock and a hub and nothing else, maybe a battery powered thermostat. And they wondered why the connection to the lock was unreliable when the hub was at the far end of the building! Z-Wave relies on Always-On (110VAC powered) nodes to build a “mesh” network. The mesh is the key to Z-Wave reliability. Every Always-On node acts as a repeater in the mesh and is able to forward a message from one node to another in the mesh. But only the Always-On nodes can forward a message. Battery powered devices like door locks and battery powered thermostats cannot forward messages. Only the Always-On nodes can.

Solution: If some devices are not reliable, add more Always-On devices. Add a Z-Wave repeater or any device like a lamp dimmer. Even if you don’t use the lamp dimmer it will act as a repeater and improve the network. I have a few lamp switches I use for my Christmas lights which I leave plugged in year round because they help the Z-Wave network since these nodes are at the periphery of my home.

Distance between nodes is not always the criteria for adding more nodes in a network. The Z-Wave radio signals may bounce off metal objects like mirrors or appliances and cause two nodes that are only a few feet apart be completely unable to talk to each other due to reflections of the radio signals. Adding more nodes in the mesh provide alternate routes to nodes that otherwise might be in a dead zone due to these reflections cancelling out the radio signals.

3. Place the hub in a central location

Putting the hub in a corner of the basement might be convenient, but its a terrible idea for Z-Wave. The hub is the most important node in the network and should have the best location possible. While Z-Wave is a mesh network and can route or hop thru other nodes in the mesh, each hop is a significant delay and chokes up the network with more traffic. Ideally the hub should reach 90% of the nodes in your Smart Home without relying on routing. If the hub has Wifi then putting it in a central location is easy, you just need a wall outlet to plug it in. I have my hub hung off the back of a TV cabinet in roughly the middle of the first floor of my home.

4. Heal the Network

Once a Z-Wave network is built, it has to be “healed” so every node can use all the other nodes in the network to route messages. This healing process can take many minutes to even hours depending on the size of the network. When you first build a Z-Wave network, the first node added only knows that the hub is in the network. When you add a second node, the hub knows that both the nodes are in the network but the first node you added has no idea that node 2 is there – unless you heal the network. So any time you add a node, you need to heal at least a few nodes in the network if not the entire network. Be cautious with the healing process – it uses 100% of the Z-Wave bandwidth during the process and every node will wake up every FliR node (door locks) at least once which will drain the batteries of the FLiR node. Generally only heal when nodes have been added or removed or if there seems to be a problem in the network.

Z-Wave is able to self-heal automatically. Z-Wave nodes will try various routes to get their message thru if at first it doesn’t succeed.  The node will remember the Last Working Route and try that one first for the next message. But if the nodes have no idea there are other nodes in the network they have no way of knowing what routes to try so at least one full heal of the network is required.

HomeSeer

homeseerhealHomeSeer has several platforms so the precise method might be slightly different than shown here. From the web interface home page select the menu Plug-Ins->Z-Wave->Controller Management then select the Action “Fully Optimize a Network”. The network wide heal will take some time depending on the size of the network.

SmartThings

SmartThings Expert Z-Wave Eric Ryherd DrZwaveSmartThings  user interface is thru their app which makes finding the network heal a bit of a challenge. Start from the dashboard and click on the three lines in the upper left corner. Your Hub should be the first choice in the menu that slides out, click on your hub. A new menu comes up, click on the last choice “Z-Wave Utilities”. The last choice on the next menu that slides in is “Repair Z-Wave Network” so click on it and then click on “Start Z-Wave Network Repair”. The repair will take from minutes to over an hour depending on the size of your network.

Vera

verahealVera has several versions of their UI but each of them has a similar menu structure so these instructions should work on any version. The Vera version shown here is UI7. Use a PC to log into GetVera.com and select your hub. From the Dashboard, select Settings->Z-Wave Settings and then click on the advanced tab. At the bottom of the advanced tab is the GO button to run the “Update Node Neighbors”. Depending on the size of the Z-Wave network this process will take several minutes to over an hour.

5. If a device doesn’t pair, first exclude it, then include it

You’ve taken the brand new Z-Wave IoT widget out of the box and you’ve tried to pair it (the Z-Wave term is “inclusion”) but it just won’t include! Arrrghhh! The first thing to try is to exclude the node first and then try including it. Any hub can “reset” or exclude a Z-Wave device even if that device was previously connected to another network. Some manufacturers occasionally fail to exclude the device during testing so the device may already be connected to their test network. Z-Wave Expert IoT WirelessOr you may have inadvertently included the device but the inclusion process failed somehow and the hub is confused. Excluding the node should reset it to the factory fresh state. Newer Z-Wave Plus devices (which have this logo on them) are required to have a way to reset them to factory defaults using just the device itself. Every device is different so you’ll have to refer to the device manual to perform a factory reset but if all else fails this should make the device ready to pair. Naturally having the hub physically close to the device being paired will also help though most devices can be paired from a distance.

Secure devices like door locks are particularly challenging to pair. First the secure device has to join the Z-Wave network, then the AES-128 encryption keys have to be exchanged and if that process fails (which it does on occasion), then you have to exclude and try the inclusion process all over again. Secure devices definitely want to be within a few feet of the hub during inclusion to ensure reliable and speedy Z-Wave communication.

6. Battery life and how to maximize it

When a battery powered Z-Wave device wakes up and turns on its radio, it uses 10,000 times more battery power than when it’s asleep. So the entire trick to making batteries last is to minimize the amount of time the device is awake. Some devices naturally have other battery draining activities mostly involving motors to throw a deadbolt or raise a window shade. Obviously any motor will use a lot more battery power than the Z-Wave radio but the radio will play a significant role in battery life.

When a battery powered device is added to a Z-Wave network the hub should do two things:

  1. Assign the Association Group 1 NodeID to the hub
    1. Association Group 1 is the “LifeLine” in Z-Wave and devices use this lifeline to send all sensor data and alerts to this node
    2. All hubs are required to assign Group 1 but double check this assignment
  2. Set the Wake Up Interval to no more than once per hour and ideally only a few times per day
    1. Every hub assigns the WakeUpInterval differently and largely handles it behind the scenes so this may be difficult to verify or change
    2. If the device is waking up every few minutes and sends a sensor reading then its battery life isn’t going to be more than a few weeks
    3. The battery level of the device is usually reported at the WakeUpInterval  rate

Many sensors have other Association Groups or Configuration Parameters that will let you specify the frequency of sensor readings. Realize that the more often the sensors report in, the shorter the battery life.

7.  Dead nodes in your controller

One of the big problems in Z-Wave network maintenance is eliminating “dead” nodes. When a device fails or for whatever reason is no longer in use, then it needs to be removed from the controller. If it remains in the controller then the controller will try to route thru this dead node on occasion resulting in delays in delivering messages. Eventually the self-healing aspects of Z-Wave will make this less likely but various devices will on occasion attempt to route thru it. Since the node is dead, that wastes valuable Z-Wave bandwidth and potentially battery power of sleeping devices. Occasionally running a Heal on the network will remove the node from the routing tables but it will remain in the controllers routing tables. It is best to completely remove this dead node. Each hub has a different method for removing dead nodes and usually requires going into an advanced Z-Wave menu.

Following these guidelines will help your Z-Wave experience be more robust. If you have more questions please feel feel to reach out via email to drzwave at expresscontrols.com.