This page describes the pre-release validation standards for firmware and application software used by Intel Research Seattle developers. Polly Powledge maintains this page.

"First, do no harm"


Here is a situation that's common to lots of software development projects:

You're working away on version 3.0 of your project, Foobar. You've been handed a list of requested features, and spend all your time whipping through the feature list. SInce the code's new, you focus all your energy on the new functionality, and go light on regression testing of the functionality from previous versions of Foobar. You forget to test some of the less visible aspects of Foobar -- like performance -- altogether.

You make your release, then 4.0, then 5.0, before anyone notices that some of the earlier functionality is now broken, and performance is a good 25% off from what it used to be. Because there have been multiple releases of Foobar before the problems were detected, it's hard to figure out which set of changes caused the problem.

Going through an established validation suite with every release helps you to spot these problems right away. Because your changeset is smaller, it helps you figure out what you did that broke things much quicker. Basically, it's a checklist that should be gone through before every update cycle.

Validation Procedure - WISP Firmware


1) Start by recording the WISP hardware version, the WISP number, the reader firmware version, the protocol, and the modulation method in the device's implementation notes page. Describe the firmware changes you made to the version. Note any known issues (e.g., "X axis broken") and any hardware changes made ("Alanson twinned the VS output to pin 3.4").
2) Using harvested power, record the distances at which you get 20, 10 and 5 Hz responses to query-acks on the implementation notes page. Record the maximum distance at which the device will "cold-start" on the implementation notes page. Be sure to note what reader application and any special settings you were using for your test.
3) Make sure all the other apps work, too (e.g. query-ack-sensor, read command with hardcoded values, and read command with sensor sampling). Record the maximum distances for each of these apps on the implementation notes page.
4) Run a stability test overnight. The device should be in range of a reader and still running the next morning.
5) Run a power torture test overnight. Aim the reader antenna so you get a Vout level that goes above and below the voltage regulator's rating repeatedly.
6) Make sure the device gets depowered such that it goes out of RAM retention mode and then successfully starts up again when powered up again. Repeat 20 times.
7) Make sure the device still works in the presence of other WISPs, and other RFID tags.

Here are a couple of other best practices to follow when creating a new firmware release:
1) Please, no generic main.c filenames! We want to be able to look at a file's name and know exactly what hardware it was designed for. If you're working with newly modified hardware, make sure you change the firmware source file's name to reflect this.
2) Where possible, try to maintain a common code base. Source code fragmentation is really confusing.
3) Use SVN. Use svn log comments liberally.

Validation Procedure - Reader Applications


1) Make sure the reader app works with WISPs. Record the WISP hardware and firmware versions you tested with, as well as what reader and any special settings you used. For the WISP firmware, record any unusual #define settings you used (e.g., application type). Make sure that the cold start and response rate requirements don't deviate from the documented numbers for the WISP firmware. Put all this information on the Wiki.
2) Document what compilers and third party libraries (including versions) are required to run your app. Include a README telling people how to get up and running with your app. Document known issues.
3) Give your app a version number, and bump it for every release.
4) Use CVS. Use CVS log comments liberally.
5) Run a stability test overnight. The app should still be operational the next morning.
6) Take the WISP out of range and bring it back into range. The application should see the WISP when it comes back into range.
7) The reader app should work in an environment with multiple WISPs and other RFID tags.
8) FIXME: Reader software developers: add your additional validation steps here.