Sniff is a "Scratch-like" programming language that's designed to help Scratchers move gently from Scratch to more conventional languages. They can start writing programs, without having to learn a new language because Sniff is based on Scratch. They learn a little more about variables, compiling, syntax errors (!), and they can have fun controlling real hardware while they're doing it.

Friday, 2 December 2016

Motorised Pot

I've been looking at a lot of smart home stuff recently and having fun with things like the Philips Hue and cheer lights. Ironically one of the best features of Hue is its light switch! It shouldn't come as much as a surprise that the best way to turn lights on and off is with a switch! Taking your phone out of your pocket is way too clunky. Even using Siri on Apple Watch you need to press the button and say "turn on the lights", when you could just press a button in the first place!

That's not to say Hue isn't really great - it is really great (and addictive and expensive!), but the best bits are having lights turn off automatically in rooms you're not using, being able to dim them exactly as you want, and turning on and off lamps from the switch by the door. Seriously that lamp in the far corner will see a lot more use when you don't have to turn on the main lights, walk across the room, turn it on, walk back then turn off the main light, then walk back and sit down! (then repeat when its time to leave).

Most of the cool tricks involve being able to control the lights in more flexible ways in addition to just a basic switch (and the ability to place that switch anywhere).

We've already implemented Hue support in Sniff so you can use and Arduino or PC as part of that control mechanism. Specifically I wanted to be able to use a Pot connected to an Arduino to dim lights. While an up/button works OK, the good old fashioned dial is something that's sorely missing from our modern lives. It would be really easy to have a pot connected to an Arduino, control the brightness of a Hue light, but with physical controls there's a big catch - turning the dial can easily dim the lights, but what happens when I dim the lights on my phone? Now the dial is out of sync with the lights...

What we need is a motorised pot - which is exactly what it sounds like: a Pot with a motor attached. These can be quite expensive, but I found a cheap source on eBay and picked one up for about £3.

Conceptually they couldn't be easier - the pot at the front can be driven by the motor at the back. The only tricky part was figuring out the pins, as for some reason there are 8 pins on the pot instead of the usual 3. With a bit of googling and poking around with a multi-meter, I found the connections I've penciled on in the picture:
  • Pin 2: V+
  • Pin 6: 0V
  • Pin 7: Signal

Signal just connects to any analog in, and we can read the position of the pot.

make dial analog input A0

when start
.forever
..say [dial]
..wait 1 secs



Hooking up the motor is simple but requires a motor driver. I could have used a motor shield, but I had some stand alone boards around. We don't need much power, so I just powered the motor board from the 5V of the Arduino. Connect the motor wires to the outputs of the board, and then connect two wires from Arduino digital pins to the control pins of the motor board.

Coding this is simple but quite fun:

make dial analog input A0
make motorCW digital output D2
make motorCCW digital output D3

make targetDial number

Here I've set up the pins, and created a variable targetDial. The only thing to watch is that the motor pins are the correct way around, and CW does turn the pot clockwise and CCW counter-clockwise (and similarly that turning CW increases the value of dial). Now we need to just change dial until it matches targetDial:

when adjustDial
.repeat until abs of (dial-targetDial)<0.01
..set motorCW to targetDial>dial
..set motorCCW to targetDial<dial
.set motorCW to no
.set motorCCW to no

We've made a loop, and we keep going round the loop until the difference between the desired and actual values are less than 0.01. We use abs because we don't care if the difference is positive or negative.

The contents of the loop might look a bit odd, but its really pretty simple. When ever we use a greater than (or less than) its actually testing something to see if its true or false, so the answer to the question "is targetDial>dial" is either a yes or a no. So if targetDial is bigger than the actual position (dial) then we set motorCW to yes, and the motor turns clockwise. If it isn't then motorCW becomes no, but motorCCW probably becomes less and we turn the pot CCW.

This will keep adjusting the dial until it reaches the target position. The neat thing is that if it overshoots, then it will adjust back the other way automatically, until its in exactly the right position and the loop exits. Then we turn the motor off, and we're done.

when start
.forever
..say ""
..say join "Current Val:" [dial]
..set targetDial to ( pick random 0 to 10 ) *0.1
..say join "Target Val:" [targetDial]
..broadcast adjustDial and wait
..say [dial]
..wait 5 secs

To test the adjustDial script I wrote the start script above. It picks a random value between 0 and 1, and adjusts the dial to that position. It then waits 5 seconds (during which you can turn the dial manually) before starting over again.

The next step would be to read values from the Hue system, and move the pot appropriately, and when the pot is turned, push values form the pot back to Hue. There are some updates in the next release to make that all very easy, so I'll cover that another time, but in the mean time consider what else needs a nice rotary control...

Wednesday, 23 November 2016

Sniff Documentation-fest

Lets be honest... no one likes writing documentation, and the Sniff docs haven't been the best. It's just more fun writing code than writing it up. We'd claim that all the information you need it there, either here on the blog, or in the examples folder but as the blog has got bigger (great - more resources and ideas!) its increasingly harder to find older posts. I'll admit I've googled my own posts to try and find instructions for a project I developed months ago.

While the examples folder is still the goto-place for code to get you started, the last few weeks have been spent trying to put together something more tangible, so you've a solid base of reference material you can work from. 

In fact we've always had a manual... you did read the manual right? Most of it was written way back when Sniff was very new. It's in the Sniff/docs folder when you download Sniff. We've now given it a bit of an overhaul, adding a few extra chapters to the tutorial section, updating the reference section, and adding an index of blocks, so you can look up "blocks" in alphabetical order. While the updated version will still be in the docs folder for download the pdf of the latest version is in the new online docs folder.

In addition we've updated the Device docs. "Devices" are external code to talk to specific hardware, so there are quite a lot of these (currently 145, though some of those are interdependent), and keeping track of how to use each device can be quite hard. Again there were some docs for a couple of the Devices: Filesystem and Sprite, as these are two of the more complex Devices with many commands, but now we've added documents for some of the simpler devices on the basis that these are the ones you'll encounter first, and need the most help with. They should be suitable for printing out, and keeping a few copies around when you're working with a particular device.

For each documented device, there's a page on the hardware (of appropriate), a page defining the Variables used, and then a list of all the commands that the device recognise. For each method, the variables that it uses as parameters are shown in a table, with a brief statement on their meaning. Finally there's an example.

We're still refining the format and the documentation only exists for a small subset of the available devices, but hopefully by establishing a standard format for documenting devices, we can do better in future (and at least document new devices better).

Currently there are documents for DHT11/DHT22, DS18b20, text LCD's, the Motor Controller shield, NeoPixels, IR Receiver for use with Arduino, and for Hosted systems there's Filesystem, Sprites, Flotilla and Window. There's also a generic DisplayDevices which applies to all bitmapped screens.

You can get The Docs online now, and we'll be including them as PDF's in future distributions of Sniff. The Sniff distribution will also include the LaTeX source for the main manual, and Pages files for the Devices so you can edit them (If you're PC based, then you can still edit pages on iCloud).

Sunday, 6 November 2016

Simple Reation Timer

Yesterday I ran the first of a series of workshops, making fun stuff with Sniff on Arduino. We started with an Arduino and an LED, then added a button and a potentiometer (dial). That neatly covers inputs and outputs, both digital and analog, so we can declare all that hardware with something like:

make led1 digital output D13
make led2 analog output D9
make button digital input D4
make dial analog input A0

Then we can use the button to control the "digital led"

when start
.forever
..set led1 to not button

use the potentiometer  to control the brightness of the analog led:

when start
.forever
..set led2 to dial

or we can use the dial to set the speed of a flash:

when start
.forever
..set led1 to on
..wait dial secs
..set led1 to off
..wait dial secs

we can even make the button control a flashing LED

when start
.forever
..if not button
...set led1 to on
...wait dial secs
...set led1 to off
...wait dial secs

The idea was then that we could build something fun or useful using the components. The code for a simple ration timer is:

when start
.forever
..set led to off
..wait pick random 1 to 5 secs
..set led1 to on
..reset timer
..wait until not button
..say [timer]

You can extend this a little to prevent cheating by checking that the button isn't pressed before the LED comes on:

when start
.forever
..set led to off
..wait pick random 1 to 5 secs
..if not button
...say "Cheater!"
..else
...set led1 to on
...reset timer
...wait until not button
...say [timer]

Hopefully everyone had a fun time with this, and I've got some ideas for a Xmas/Winter project next month...

Friday, 4 November 2016

Flotilla and MQTT

I've not played with the Flotilla for a few months, but when I was playing with MQTT last month, I kept thinking that I should integrate the flotilla Weather sensor into the MQTT network that we built.

Previously I used and Arduino with a temperature sensor, pushed the data to MQTT, processed the data in Node-Red and finally pushed a message back through MQTT to an LCD screen. You'll need MQTT and Node-Red servers running already, along with the Node-Red flow I built in the last post.

Rewriting it for the Flotilla is mainly a case of changing a few declarations:

make mqtt device
make networkPeer string
make networkConnected boolean
make clientid string

make message string
make topic string
make retain boolean


make flotilla device
make sensor flotillaWeather device
make temperature number
make pressure number


We don't need an ethernet device,  as the MQTT device doesn't need to drive the hardware directly. And I've just replaced the DHT11 with a flotilla dock and weather sensor.

when start
.set clientid to "flotillaSensor"
.set networkPeer to "raspberrypi.local." 
.
.repeat until networkConnected
..tell mqtt to "connect"
.
.say "connected"
.
.broadcast measureStuff
.
.set topic to "text"
.tell mqtt to "subscribe"
.set topic to "office/text"
.tell mqtt to "subscribe"
.forever
..tell mqtt to "loop"
..if not message = ""
...say message

There are a few minor tweaks to change the name of the client, and specific that the MQTT server is on raspberrypi.local before we subscribe to the text topics  Currently there's no text based screen for the flotilla (an i2c 16x2 LCD would be nice...) so when we get the message we just print it on the terminal with "say".

when measureStuff
.forever
..tell sensor to "update"
..
..set retain to yes
..set topic to "office/temperature"
..set message to [temperature]
..tell mqtt to "publish"
..
..set retain to yes
..set topic to "office/pressure"
..set message to [pressure]
..tell mqtt to "publish"
..
..wait 10 secs

The script to do the measuring is essentially the same too, though now we're recording temperature and pressure, rather than temp and humidity.

And that's it! Neat huh? Lots of scope for fun... Have one set of sensors running, and let everyone tap into them, or place sensors all over the place and collect data from them. Have one group of kids responsible for posting data, and other for displaying it - on the same or separate computers.

Saturday, 15 October 2016

MQTT Part 4: Sniff meets Node-Red

So far we've used MQTT to publish sensor data from an Arduino, and subscribe to receive messages to display on a screen. Having written code to do each of those things, one of Sniff's strengths is that its pretty trivial to do both at the same time, just by running the script to do each at the same time. Here's the complete code to support both a screen and a sensor at the some time:

make i2c device 2
make display lcdi2c device 2
make displayFlush boolean

make spi device
make ethernet device D10
make networkMAC string "b6:ee:63:ed:95:cb"
make networkIP string "192.168.0.200"
make networkConnected boolean
make networkPort number
make networkPeer string

make mqtt device
make clientid string
make message string
make topic string
make retain boolean

when start
.set clientid to networkMAC
.set networkPeer to "192.168.0.108" #No DNS on Arduino!
.
.repeat until networkConnected
..tell mqtt to "connect"
.
.broadcast measureStuff
.
.set topic to "text"
.tell mqtt to "subscribe"
.set topic to "office/text"
.tell mqtt to "subscribe"
.forever
..tell mqtt to "loop"
..if not message = ""
...tell display to "show"


make thermometer dht11 device D2
make temperature number
make humidity number
when measureStuff
.forever
..tell thermometer to "read"
..
..set retain to yes
..set topic to "office/temperature"
..set message to [temperature]
..tell mqtt to "publish"
..
..set retain to yes
..set topic to "office/humidity"
..set message to [humidity]
..tell mqtt to "publish"
..
..wait 10 secs

You might notice that we're subscribed to both text and office/text. The intention here is that we might have several of these devices in different rooms - so we can publish a message to all of them at once, or to each individually.

With the physical/embedded side out of the way, we now need to think about what we want to do with that data we've collected. That means we probably have something running on a desktop machine. While we could easily write that in Sniff, we could also use Node Red.

Node Red is a javascript based system that is specifically designed for collecting messages, processing them and passing them on.  It works particularly well with MQTT, which is sort of to be expected given that they're developed by the same people.

Grab a copy of Node Red, and run through the first tutorial to get the basic idea...

time passes... OK - you're back! Great!

Now that you've got the general idea of Node Red make an MQTT input node, and tell it to subscribe to office/temperature on your server/broker. Connect the output to a debug node, and hit deploy.




When the Sniff code is running on the Arduino, you should see the temperature messages appearing in the debug window.


Next we can connect the MQTT input node to an MQTT output node which publishes the received temperature back to office/text, so that the measured temperature appears on the screen.




Now that we can fetch the data from the sensor, and push it back, we can add some processing the he middle. For example let's make the message a little friendlier. Just add a Function node between the two MQTT nodes.



We read the msg.payload, modify and and write it back before passing it on to the send node. In fact we can make any changes we like. For example we can indulge the British habit of reporting cold temperatures in C (it was -2 last night!), and warm temperatures in F (awesome holiday - 80 degrees all week!). There's little confusion as we simply never use the values between 20 and 50!



The next step is to handle temperature and humidity at the same time. Getting the data is easy, as we can just subscribe to office/+, but there's serious gotcha - this also matches office/text. This means when we publish the results, we'll trigger an input, which will trigger an output.... that would be bad. To get round that add a switch node, which only outputs its result if the msg.topic!="office/text".





We can just check the message topic to find out what kind of information we've received. Slightly less obvious is that when we receive an update to temperature, we'll need to combine it with the previously received humidity. Fortunalty this is pretty easy, as each node in the flow has a "context". Context is a variable that we can store values in, and they'll be there the next time the function is executed. When we receive a value we store it in the context, and then generate a new payload message from the stored values.

One final thing to deal with is the that our simple Sniff program only handles one line of text at a time, while out Javascript function bundles the Humidity and into a single packet. While there are lots of ways we could handle this, the easiest way is to have a split node, which separates the two lines into two separate messages. This produces our final flow:

And one the Arduino we see:


While its true, we could have coded all of this in Sniff on the Arduino, without any networking involved, and it would have been much simpler its worth remembering that these kinds of sensor/display devices might be deployed all round your house, school or factory. When a few hundred are boxed up, and bolted to the wall, and you need a ladder to even reach some of them, going out and reprogramming them because you want to make a minor change isn't really practical. It makes a lot of sense to keep those boxes as dumb as possible, while the "business logic" - the stuff you need to change, and develop is all in one place where you can get to it.

In addition by pushing the data out to Node Red we get an easy connection to all sorts of other data streams, like email, twitter, and websites.



Friday, 14 October 2016

MQTT Part 3: Fancy publishing

So far with MQTT we've published and subscribed to basic messages from both desktop and embedded Sniff.

make spi device
make ethernet device D10
make networkMAC string "b6:ee:63:ed:95:cb"
make networkIP string "192.168.0.200"
make networkConnected boolean
make networkPort number
make networkPeer string

make mqtt device
make clientid string
make message string
make topic string
make retain boolean

when start
.set clientid to networkMAC
.set networkPeer to "192.168.0.108" #No DNS on Arduino!
.
.repeat until networkConnected
..tell mqtt to "connect"
.
.broadcast measureStuff
.
.forever
..tell mqtt to "loop"

That's the basic framework we need to create a connection to the broker, and keep everything ticking over. However this time around we're going to publish the temperature and humidity from a dht11 temperature and humidity sensor out to the world.


make retain boolean

make thermometer dht11 device D2
make temperature number
make humidity number

when measureStuff
.forever
..tell thermometer to "read"
..
..set retain to yes
..set topic to "office/temperature"
..set message to [temperature]
..tell mqtt to "publish" 
..
..set retain to yes
..set topic to "office/humidity"
..set message to [humidity]
..tell mqtt to "publish" 
..
..wait 60 secs

We've already included a broadcast in the start script to kick this script off, so all we need to do is measure the temperature by telling the thermometer to "read", and publish that data. Then we wait a minute and do it again.

Topic Hierarchy

However there are a couple of new tricks here that are worth looking at. Firstly look at the names of my topics - they've got a / in them, which allows us to group topics together. In this case I'm reporting  information about my office, so I'm grouping then under that name.

When someone subscribes they can subscribe to specific topics, but they can also subscribe to groups of topics, so "office/#" means every topic that begins "office/". That would include office/temperature and office/humidity but it would also include "office/lights/1/brightness". You can only use # as the last character of a subscription as it basically means "everything in this category".

On the other hand, maybe I'm interested in the temperature of lots if different rooms. In which case I can subscribe to "+/temperature", or if I want the brightness of all the lights in the house "+/lights/+/brightness". Each + represents exactly one level in the hierarchy, but you can use it anywhere in the path.

Retained Messages

The other thing to consider is that we're only publishing a measurement once per minute. That's fairly sensible, as air temperature doesn't change that quickly, so no need to flood the network. However what if a client connects to the broker and subscribes to  "+/temperature" to work out the average temperature of the house. It would have to wait up to a minute before it got messages from all the sensors, which would be frustrating. We could go further and have sensors push out values only when they change, which would leave our averaging program waiting for as long as it took before it gets all the data it needs (and it shouldn't need to know how many sensors there are either, so it would never know its got them all).

To get round this problem we've created a variable called "retained", and set that to yes before each call to publish. When a message is published in retained mode the broker sends it out to subscribers as normal, but keeps a copy. If any new subscribers come along, then they immediately get the last retained message on that topic, so in this case subscribing to "+/temperature" would immediately generate a set of messages with the last known temperature of every sensor. 

Last Will and Testament

Now that messages can be retained, we've created a new problem: My office publishes a message in the middle of the night, and tells the broker that its 16 degrees. It then gets disconnected from the network because a ferret unplugs the network cable (yes that really did happen to me!). The next morning its warmer, but when the heating subscribes to check the temperature, the last good value sent says its cold, so the heating turns on full blast until I reset the sensor.

Fortunately MQTT has a mechanism to check that everything is still running smoothly. Even when its not publishing, the client and broker occasionally send messages to each other just to make sure that they're still alive. If the broker spots that a client isn't working any more it can clean up a little bit.

However as the client is already disconnected the only way it can control what happens during this cleanup is by leaving a will - instructions for a message that should be sent upon its death. To make this happen our connection code might looks like:

.set retain to yes
.set topic to "office/temperature"
.set message to ""
.
.repeat until networkConnected
..tell mqtt to "connect"

When we connect this message (an empty string) gets sent to the broker, but the broker doesn't forward it to clients. Rather it hangs onto it, and if at some point in the future it looses connection to the client unexpectedly then it publishes the will-message on the dead clients behalf. Here I've set the message to "" to erase any previous temperature recorded, so it won't be used hours later, when its invalid.

If we had a client which published messages to control some kind of actuator, then the will-message could be used to reset the actuator to some kind of safe/recoverable state.

Note that will messages work with retain so if the will is retained, then its value will persist forever, while a non-retained-will, just gets send out at the time of failure (and an older retained value may persist and get sent to new subscribers!).

If you want to deliberately shut down cleanly then you can tell mqtt to "disconnect" which doesn't trigger the will, but in that scenario you can send whatever messages you like to leave the system in valid state.

There are still a few tricks that MQTT can do, but that's most of what you should need to start generating useful MQTT data sources. Next time I'll look at hooking that up to something server side.

Thursday, 13 October 2016

MQTT Part 2: Embedded!

MQTT is pretty handy for letting Sniff programs communicate just running on computers, but its real purpose is to pass messages between embedded devices like Arduino's which are sitting on the network, doing a single simple task - either measuring something (a temperature sensor?), or controlling something (a heating valve?).

The current Sniff implementation of MQTT runs great on Arduino, and it should run on other systems like MBED, though we've not had a chance to try that out yet. It requires a Wiznet5100 based ethernet interface, like the Arduino Ethernet Shield. Support to other network devices "real soon now".

So if you grab an Arduino, an Ethernet shield, and already have an MQTT broker set up from the previous tutorial, we'll begin.

Setup

First we need to set up the ethernet device:

make spi device
make ethernet device D10
make networkMAC string "b6:ee:63:ed:95:cb"
make networkIP string "192.168.0.200"
make networkConnected boolean
make networkPort number
make networkPeer string

Note that we also made an SPI device to handle the low level hardware, and we're using D10 as the select pin for the WizNet (as per Ethernet Shield). These chips come without a Mac address, so you'll need to generate one. It doesn't really matter what it is, but it has to be unique, so every board has its own. You also need an IP address, which you should get from your network administrator.

Subscribe

With that done, everything else is pretty much as it was previously.

make mqtt device
make clientid string
make message string
make topic string

when start
.set clientid to networkMAC
.set networkPeer to "192.168.0.108" #No DNS on Arduino!
.repeat until networkConnected
..tell mqtt to "connect"
.
.say "connected"
.
.set topic to "text"
.tell mqtt to "subscribe"
.forever
..tell mqtt to "loop"
..if not message = ""
...say message

I've done a couple of things slightly different, just because we're not running on a computer. First off I've used the MAC address as the clientID - you don't have to, but its a good way to ensure you've got a unique value. Next, I've used the raw IP address of the server rather than its hostname, as we don't have an easy way to look it up like we do on a "real" computer. In the desktop version I tried to connect, and gave up if I failed, but on an embedded device, we probably just want to keep trying until it works, which is what I've done here.

Control

Of course the real fun comes when we hook up some other hardware:

make i2c device
make display lcdi2c device 2 #2 line display
make displayFlush boolean

when start
etc
.
.forever
..tell mqtt to "loop"
..if not message = ""
...tell display to "show"

Now we're pushing the receive message onto an LCD screen connected via i2c. That's all it takes to do something really useful and exciting - now you can publish messages from any client, and they appear on the screen. You could have 1 screen, and dozens of sensors pushing alert messages to it, or dozens of screens with a central server sending messages to a dozen subscribers... or a thousand! This is essentially the architecture that bus and railway digital info boards would use.

You can publish from Arduino just as you would from the desktop too - once setup, MQTT works identically on both platforms, so setting up a sensor and publishing the results to the world is about as easy as it gets! However there are a few more tricks still to learn, so there's at least one more MQTT post to go...