Dynamic DNS update support for No-IP

My personal choice of a Dynamic DNS service is No-IP.

No-IP was one of DynDNS’s biggest competitors in the free dynamic DNS market, and now that DynDNS is gone they’re in a good position to snatch the crown for themselves. In fact, the two services are only one year apart in age (DynDNS launched 1998, No-IP launched 1999).
For a free account, you’ll get three subdomains on a single domain choice, but these subdomains will never expire as long as you confirm activity every 30 days. You also get port forwarding and URL forwarding, which can be useful depending on your use case. [ source ]

The story started when I received a new router from my ISP that didn't support No-IP. At that particular moment I decided to create a simple implementation of No-IP API into Jubito.

Now Jubito has a built-in function (%publicip%) that return the public IP address and a judo API that provides an update method for our hostname.

Let's proceed on how to configure your No-IP hostname.

In the settings menu, you will find a Dynamic DNS section. Click Setup, fill your hostname, username, password and save.

Next step is to create a new launcher that call the API for update. Expand Instructions menu and click on Add New Launcher. Give a name and type the action as follows:

judo noip update

We are now ready to create a schedule that updates the service in a given interval. Add new schedule and follow the screenshot, where interval sets the time of check (300000ms is 5 minutes) and action is the call we did in the previous step (launcher).

That was it. You are router independent!

Dropbox alternative for ARM Linux and Jubito integration

Once upon a time, when a netbook (Dell mini 9) with Ubuntu was my Jubito server, I used dropbox to sync my important files, such as camera video recordings of my security system. Then I migrated to a Banana Pi and currently to Banana Pro. Both derivatives of Raspberry Pi. Unfortunately dropbox has no support or plans for ARM Linux variants. A quick solution was to use ownCloud. All went well till ownCloud client stopped syncing for a number of reasons (incompatible updates/upgrades,etc).

I've made an effort to find out a solution that suits me and finally rclone came up.

Rclone is a command line program that sync files and directories to and from, and it comes with support of various storage systems (google drive, dropbox, Microsoft One Drive and more). My desirable choice is dropbox, so,  I started getting my hands wet.

I read of course that I would just have to perform the sync operation manually when I wanted things up to date but it didn't bother me in first place. I'll explain the reason afterwards.

Let's proceed to step by step process.

Download rclone to your home directory, unzip it and move forward to next paragraph, or go through command line.

cd
wget -c http://downloads.rclone.org/rclone-v1.35-linux-arm.zip
unzip rclone-v1.35-linux-arm.zip
Rename the directory for sake of simplicity
sudo mv /home/<user>/rclone-v1.35-linux-arm /home/<user>/rclone
cd rclone
Change file access permissions
sudo chmod 755 /home/<user>/rclone/rclone

Follow installation and configuration instructions. The guide use a generic name for the remote target, called remote. As I chose dropbox installation, I will call it dropbox instead.

After installation finishes try to synchronize your source with the destination. In this case the destination directory will be called jubito. Open a terminal and type:

./rclone sync /home/<user>/<source> dropbox:jubito

Time for Jubito integration. I will present three cases. Making a scheduled job, sync on-demand and trigger on event. Before we proceed we have to create invocation to rclone. We need a launcher that point to the action above.

Go to Settings->Instructions->Add New Launcher. On popup give as name "sync" and action like this:

./'/home/%user%/rclone/rclone' '/home/%user%/<source>
dropbox:jubito'

where <source> your current installation (in my case /home/john/bin), and press save. Dot slash (./) is a native implementation similar to Linux and Unix to execute a compiled program and %user% is a built-in function that gets the logged in username (alternatively, hard-coded your user). The result, if any, can be consumed by another Instruction Set. Learn more on (incomplete) Instruction Set anatomy.
This will create a pointer to the command inside AppConfig.xml which could be used at any case (scheduled, on-demand, etc).

Calling the method from CLI:

Physical location

Dropbox location

1) Scheduled

Go to Settings->Scheduler->Add New. Enter a name, select daily routine, time (e.g. 00:00) and finally sync action. This will create a schedule that runs daily at midnight and triggers the dropbox sync.

2) On-demand

From Settings->Instructions->Add New Instruction Set. Name it "dosync" or whatever you like, select from action list the sync command we created before and press add. It will generate a *sync pointer to the sync launcher. You can put some additional text if you wish. Expand Add to Dashboard and put some info (category, header) to populate a button to the dashboard list.

Now you'll be able from menu list to press and run the sync method...

 
3) Attach to other Instruction Set

First let's make an example based on DVR system by using an IP camera example. Scenario describe operations when security breach is detected (record a video and send email alerts). To extend it with dropbox synchronization, we're going to use the sync as the third action in the sequence. So, once again add the sync launcher.

<InstructionSet id="ip-cam">{ evalBool("%whereami%" == "absent"); rec sendmemail sync; ; }</InstructionSet>

If you want to use the UI and already have the ip-cam Instruction Set, please delete it and re-create it since there is no edit mode. Else, edit AppConfig.xml and change it with the line above.

However, you can create a new ip-cam with the same condition that calls only sync method like that:

<InstructionSet id="ip-cam">{ evalBool("%whereami%" == "absent"); sync; ; }</InstructionSet>

Instructions with the same name, run in order.

Upgraded Android client with location-based check-in/check-out process

Check-in/check-out refers to built-in functions (%checkin%, %checkout%) and used to change the user status from present to absent (this also could be translated as locked/unlocked or arm/disarm flag to some) and vice versa. It also raises oncheckin/oncheckout events as described in AppConfig.xml. You can bypass defaults by using custom events (e.g. mycheckin/mycheckout).

Some things in technology are meant to be deprecated, but yet, it's our responsibility to follow and make upgrades, new versions and provide new features.

I’m always looking for the best approach to perform check-in/check-out operations when I arrive or leave home. Jubito has the check-in/check-out functionality that allows us to perform a sequence of tasks depending on the event. For instance, on check-in we can turn on an assistive light, disable the camera and motion detection, checking email, etc, and inverse actions on check-out.

In the early days, I used bluetooth scan as a means to check-in when my cellphone was in range, and check-out, when it was unreachable. Drawbacks such as limitation of bluetooth range, auto-disable to save battery when phone was idle etc, didn't make it the most preferable choice.

Then I introduced the Jubito client Android app that had the capability to track my device's location, checking if I was within radius of 500m of the point I’ve set as home and do the appropriate actions as described earlier, more elegant and with more precision.

Far from perfect also. For instance, the app had to run in foreground, but besides that, Google changed the API and service eventually stop working.

The ping methodology came along, which share the same concept as the bluetooth, with some drawbacks too. Of course, it's very valuable in other kind of scenarios, but still, not good for my check-in/check-out expectations.

Lately, I found some time to study the new API and decide to modernize and upgrade the app. Along with its old functionality, now can run  as a service in background, even if the phone is locked. By far the most convenient way, especially when we don't want to be distracted by manual input on phone and be spared the extra effort.
It is a very nice feel, returning home, and this would be ready welcomes you by your preferences! Trust me on that!

Usage is straightforward. First you need to enable location from android settings. If not, and you have auto check-in/check-out checked, you will be prompted to. To take effect you need to kill the app from task manager and re-open it.

Next, from Jubito app settings, select your location by pressing the button "Get Location". On apply, selected address will be set as your home point.

Location services allow app to obtain periodic updates of the device's geographical location, and fire an application-specified event (check-in/check-out) when the device enters the proximity of a given geographical location. In our case, within 500m radius of the home location.

Before I close, let's recap what features app brings besides serving Jubito UI.

• Speech Recognition & Synthesizer. It allows to execute Instruction Sets with speech dictation and speak the result if any.
• Location & Geocoder. Reverse geocoding is the process of transforming a (latitude, longitude) coordinate into a (partial) address. That allows the app to get location data (lat/long) and reverse it to an address in order to be user friendly. By pressing the "Get Location" button you can set your home location automatically.

Closing this post with the setup of my preferences.
oncheckin, oncheckout events:

<event id="oncheckin">%unmute%; *socket-d-on; salute; judo schedule enable mailcheck</event>
<event id="oncheckout">init; judo sleep 30000; goodbye;  lights_off; judo schedule disable mailcheck; %mute%</event>

Let's take a deeper look on implementation.

All steps can be made from the UI, but in this particular article I'd like to go behind the scenes for robust understanding.

Check-in Steps

%unmute% & %mute% are also built-in functions that disable/enable the speech synthesis. *socket-d-on is a pointer to the InstructionSet that turn on an assistive light. In my case the living room right. You can also evaluate the light conditions before proceeding.

<InstructionSet id="*socket-d-on">judo serial send 000000000101010100010001</InstructionSet>
<InstructionSet id="socket-d-on" descr="Turns on living room light" shortdescr="Living room light on" header="Socket D on" categ="Automation">*socket-d-on</InstructionSet>

Salute is what it says and implements like this...

<InstructionSet id="salute">Good %salute% %user%.</InstructionSet>

%salute% function returns the part of day e.g. morning, afternoon. %user% returns the logged user. So, the above will return "Good afternoon john" and vocalized by speech-synthesis.

judo schedule enable mailcheck, enable the gmail check that is scheduled to check every 5 minutes.

Check-out Steps
init command, just speak Initiate sequence.

<InstructionSet id="init">Initiate sequence</InstructionSet>

judo sleep 30000, is a 30 seconds sleep timer, to allow me leave the place without triggering anything.

goodbye command, again a simple goodbye verbal message.

<InstructionSet id="goodbye">Good bye %user%.</InstructionSet>

lights_off command, turn off any assistive light.

<InstructionSet id="*socket-a-off">judo serial send 000000000100010101011111</InstructionSet>
<InstructionSet id="*socket-b-off">judo serial send 000000000101000101010100</InstructionSet>
<InstructionSet id="*socket-c-off">judo serial send 000000000101010001010100</InstructionSet>
<InstructionSet id="*socket-d-off">judo serial send 000000000101010100010100</InstructionSet>
<InstructionSet id="lights_off" descr="Turns off the bedroom and living room lights" shortdescr="Bedroom and living room lights off" header="Lights off" categ="Automation">*socket-a-off *socket-b-off *socket-c-off *socket-d-off</InstructionSet>

An old showcase video.

Finally, the last two are self-described. judo schedule disable mailcheck; %mute%, disable mail check and speech-synthesis.

Binaries (000000000100010101011111) in above example, control appliances and explained in this how-to.

A basic Wiki for judo API and built-in functions.

In action...

I plan to make more enhancements in future versions. For example, change the radius, add zones on specific ranges and do different tasks (at 10m open garage door), etc.

Consuming RESTful data, create widgets and more

A common way to offer server-generated data to browsers, so that it can be used in client-side JavaScript is by formatting the data as JSON, and making it accessible through its own URL. A web server that respects itself, even a lightweight like Jubito, should conformed with these standards.

Until now, the core of Jubito was loading data from the server using a HTTP GET request over jQuery.get(). The return type was a simple html. In fact, still used and applied when mode is set to html (i.e. http://localhost:8080/www/?cmd=<commands>&mode=html). By default is text, or by setting mode to text. In case of html, the only element presented is the '<br />'. The main "disadvantage" is that there is no object to handle and each command need its own call, or a manipulation of the result in case of multiple commands (e.g. splitting the breaks '<br />').

In this post, I'm going to present native APIs for consuming data from the framework and the web. That way we'll be able to customize Jubito by creating widgets and UIs.

Before we begin, let's do some REST style examples from the browser in order to understand the behavior. URI has a typical structure and addressed like this:

http://<host>:<port>/www/?cmd=<sequence of commands as parameters delimited by ampersand>&<mode=json> 
e.g.: http://localhost:8080/www/?cmd=judo%20serial%20state%&%judo%20server%20state&%whereami%&%whoami%&mode=json

Explaining the functionality of each command:
judo serial state is an API call that returns the state/status of the serial port.
judo server state is an API call that returns the state/status of the web server.
%whereami% is a built-in function that returns the user status.
%whoami% is a built-in function that returns the user login.

Parameters in API respond:
1. Pair name - A simplified name of the requested command
2. Key - Represents the requested command
3. Value - Result of the requested command

For more information about built-in functions and API please visit Wiki

Next step is to create a jQuery.ajax() request and see how to consume the response data. It provides a simple process of fetching and parsing JSON data through Ajax. An ideal and straightforward way to achieve our goal for tailor made widgets and UIs.

Sample HTML:

<!DOCTYPE html>
<html>
 <head>
  <meta charset="utf-8" />
  <title></title>
  <script type="text/javascript" src="https://ajax.googleapis.com/ajax/libs/jquery/2.2.2/jquery.min.js"></script> 
 </head>
 <body>
  <script type="text/javascript">
   $(document).ready(function() {
    getData();
   });
   
   setTimeout (function () {
    $("#data").empty();
    getData();
   }, 10000);
   
   function getData () {
    $.ajax({
     type: 'GET',
     contentType: 'application/json; charset=utf-8',
     dataType: "json",
     // requested built-in functions: %whereami% %whoami% %todayconditions% %todaylow% %todayhigh% %about%
     // request on judo API: judo server state
     // wiki https://github.com/jambelnet/janet-framework/wiki
     // uri form: <host>:<port>/www/?cmd=<sequence of commands as parameters>&<mode=json>
     url: "http://localhost:8080/www/?cmd=%whereami%&%whoami%&%todayconditions%&%todaylow%&%todayhigh%&judo%20server%20state&%about%&mode=json",
     }).done(function (data) {
      var template = 'Status: ' + data.whereami.Value + '<br/>User: ' + data.whoami.Value + '<br />';
      template += 'Conditions: ' + data.todayconditions.Value + '<br />';
      template += 'Low: ' + data.todaylow.Value + 'C&deg;<br />';
      template += 'High: ' + data.todayhigh.Value + 'C&deg;<br />';
      template += 'Server status: ' + data.judo_server_state.Value + '<br />';
      template += 'About: ' + data.about.Value + '<br />';
      
      $("#data").append(template);
     }).fail(function() {
      alert( "Oops!" );
     });
    }
  </script>
  
  <pre id="data"></pre>
 </body>
</html>

http://hilite.me/

The sample above will serve a simple page with data displayed and refreshed every 10 seconds:

Another powerful feature which you may find interesting, is the HTTP GET request supported by the API. It provides raw data from any endpoint.
judo http get <endpoint>

An example of its use can be found here on how to send and read data from RaZberry Z-Wave controller.

So, to recap, there is a lot of new things to expect, but till then, you should probably start honing your skills and finally create your own UI or a widget to the existing.

I tried to describe this post as much as possible less technical, in order to be understood by beginners.

Inexpensive Z-Wave gateway using a Banana Pro and a RaZberry daughter card

Lately Jubito grows its family by contributions of individuals and vendors and I thank them in public each time I make a reference. More details in social networks e.g.: facebook.

Every piece of hardware I received has an interesting aspect that opens a new challenge to the Jubito ecosystem. For example, Z-Wave integration. An interesting technology, but still, a little bit expensive.

While I'm studying the Z-Way JSON API that allows to execute commands on server side using HTTP GET/POST requests, I'd like to share some early developments for those who want to learn the basics and for the adventurous. In this post I will deal with the RaZberry module by Z-Wave.me on a Banana Pro.

Since I currently owned only one device (Fibaro wall plug) I will focus on SwitchBinary Command Class (0x25/37). The Switch Binary Command Class is used to control all actuators with simple binary (on/off) switching functions, primarily electrical switches.

Those who have experience with the module will probably have a complete understanding of what I'm talking about.

But let's take a look at some methods. The command to execute is taken from the URL. All functions are executed in form:
http://YOURIP:8083/<URL>

• http://localhost:8083/ZWaveAPI/Run/controller.AddNodeToNetwork(1) - Start inclusion [1].

• http://localhost:8083/ZWaveAPI/Run/controller.AddNodeToNetwork(0) - Stop inclusion.

• http://localhost:8083/ZWaveAPI/Run/controller.RemoveNodeFromNetwork(1) - Start exclusion [1].

• http://localhost:8083/ZWaveAPI/Run/controller.RemoveNodeFromNetwork(0) - Stop exclusion.

• http://localhost:8083/ZWaveAPI/Run/devices[n].SendNoOperation - Refresh device list.

• http://localhost:8083/ZWaveAPI/Run/devices[n].instances[0].SwitchBinary.Set(value) - Binary values accept 0-99 for switching off and 255 for switch on.

• http://localhost:8083/ZWaveAPI/Run/devices[n].instances[0].SwitchBinary.data.level.value - Get the state of the device (true->on, false->off).

• http://localhost:8083/ZWaveAPI/Run/devices[n].instances[0].Meter.data[s].val.value - Measures the amount of electrical energy in kWh that was consumed.

• http://localhost:8083/ZWaveAPI/Run/devices[n].data.givenName.value='<A new name>' - Rename a device [2].

[1] You can include devices by pressing the 'Inclusion' button. This turns the controller into an inclusion mode that allows including a device. A led on the controller indicates this status. The inclusion of a device is typically confirmed with a triple press of a button of this particular device. However, please refer to the manual of this particular device for details how to include them into a Z-Wave network. The inclusion mode will time out after about 20 seconds or is aborted by pressing the 'Inclusion' button again.

The same applies for the 'Exclusion'.

[2] Rename a device:

Use <lock></lock> tags when you want to bypass parser and pass a URI with blank spaces as one argument.

To create an HTTP GET method as an Instruction Set for alternative use, you can move to Instructions->Add New Launcher and put your getter there. This will allow you to invoke a Z-Wave function into another Instruction Set. You can search in this blog for common actions, but I will come back later with analytical article (*).

You can review the whole object in real time to reveal all the available functions but I won't deepen on the data model -yet [ http://localhost:8083/ZWaveAPI/Data ]:

To have a human readable tree view, copy and paste the JSON object to an editor, such www.jsoneditoronline.org.

List of available devices [ http://localhost:8083/ZWaveAPI/Run/devices ]:

Let's stand here and see some examples on Jubito's side.
Jubito has a native API for HTTP GET method i.e.: judo http get <Request-URI>.

Try any of the above operations from the terminal.
Examples:

• judo http get http://localhost:8083/ZWaveAPI/Run/devices[n].data.givenName.value - Gets device name.

• judo http get http://localhost:8083/ZWaveAPI/Run/devices[n].instances[0].commandClass[37].Set(255) - This will switch on the plug. It is equivalent to http://localhost:8083/ZWaveAPI/Run/devices[n].instances[0].SwitchBinary.Set(255) or .Set(0) to switch off.

That was a quick introduction to the Z-Way API. Before you get confused, Jubito comes with a simple interface that implements the basics like, inclusion, exclusion and list of available devices with their actions (on/off), including some additional information.

While is still beta, I won't add a link on the menu button yet, but you can navigate to z-index.html:

http://<host>:<port>/www/z-index.html

Please make sure you have the latest version [0.2.8.299]:

(*) Let's consider this post as the first part of a Z-Wave session. In future articles I will present ways, on how to use a specific function, on a custom Instruction Set, and how these can interact and expand functionality and logic by other mechanisms, including evaluations and schedules.

Extracting data from JSON and XML

Web API connector is released and supports both JSON and XML response parsing. This feature opens up the possibility of connecting to a huge range of potential APIs and therefore reducing the cost for additional equipment. For example, if you want to create some tasks based on weather conditions, you can achieve that without a weather station, but by gathering information from the cloud.

In addition to this post let's do some JSON examples from Yahoo Weather API. I will also use the Code Beautify online code editor for a good human-readable representation of the response.

Step 1: Copy and Paste the endpoint URI from yahoo developer website and paste it to the editor.

Step 2: Go to Control Panel->Instructions->Add New Web Service. Address the node you want to retrieve and fill the form items (Name, Endpoint, Node Path). Notice that if the response have a tree structure, you should point the position index of your target in Node Path as seen below. By default will get the first element [index: 0].

By default some editors starts the tree viewer with root/array and then with the absolute tree structure. So, at the "Node Path" field of Jubito, ignore the above of starting path.

Step 3: Go to terminal tab and debug the new Instruction Set. If you get the value then everything went well.

Step 4: Now, let's make an interface and a scenario. Again from Control Panel->Instructions->Add New Instruction Set. Unfold the Add Action(s) and Add to Dashboard and follow the screenshot.

Save and return. Now you're able to make a call from the Dashboard screen.

Step 5: Some other things you can do, is to make a schedule to evaluate the weather and send an email notification when a specified weather acknowledgement applies. Let's take a look to the Evaluator. Once again Control Panel->Instructions->Add New Evaluator. Select the yahoo-test Instruction Set and add some additional text if you wish. As I always remind, the asterisk in front of the Instruction Set is the reflector of a value.

'Then' condition corresponds to another Instruction Set that is responsible to send mail. You can search this blog with 'send mail' keyword to see how to make one.

Step 6: Finally there all set to create the Schedule. Go to Control Panel->Scheduler->Add New. Type a name, repeat, interval at your choice (3600000ms is one hour) and select the Evaluator's Instruction Set.

This schedule will run every hour and evaluate the condition.

All of the above steps are described as a proof of concept. The use-cases may vary depending your needs.

The same logic applies also to XML. You can be advised by this post. If you find any difficulties or discover a bug, please do not hesitate to drop me a message.

Changes applied to jaNET Framework 0.2.6.555+. To check version go to terminal tab and type %about%.

The power of reflection

In this post we've explained a scenario that consisted by a number (3) of alternative responses. Someone might consider, what if I want to use the response or some other action, like evaluation, to another activity? Should I make duplicates or a number of responses for each one of them? The answer is no. This can be solved by creating the response or some action as a launcher. A launcher creates two instances. One that holds the action and the other the reflection. The reflection is indicated by asterisk symbol.

For example, if I create a new launcher 'demo_launcher' that holds 'Hello %user%' as action, then the system will create:

1. *demo_launcher: Hello %user%
2. demo_launcher: *demo_launcher

When 'demo_launcher' is called the response will look like this...

Hello john (where john the system login. Check Built-in functions and API for more info)

Now, from a different Instruction Set that returns for example the user status, *demo_launcher can be invoked like this..

get_user_stat: *demo_launcher. Your status is set to %whereami%.

which will return...
Hello john, your status is set to present.

I think it's a very straightforward methodology and will become a standard in the future versions.

Screenshots are from the development version which is not yet available. However the functionality remains.

Of course these actions can be applied to any system object, such the evaluator.

Chain reaction behind the scenes.

Launchers are extremely versatile. You can combine anything that supported from the framework in a single action.
So, the verdict is that if you want a complex Instruction Set it is wise first to create launcher components and then mixed in a single one.

You can review a complex AppConfig.xml configuration file.

Web of Things and what's new on the upcoming version

In our days, there is a trend to talk about the Internet of Things, Machine to Machine, Web of Things, etc. However all of the above are considered an integral part of the IoT. I won't discuss any of them in detail, since there are too many articles over the internet.

A vast amount of information, is distributed via web services. Web services are an efficient way of communication, that support interoperable machine-to-machine interaction, over a network. As a developer, I need my platform to be able to consume all these services and produce innovative use-cases.

In this post I would like to share some of the latest developments, that would be available in the next version of Jubito. I'm talking about tailor made methods for native XML and JSON deserialization from web service/web API responses.

Some first tests using Yahoo Weather API.

XML:

 

 

JSON:

The current release, based on 0.2.5 jaNET Framework comes with full partial support [ Still under development ]. To check the version, go to Terminal tab and type %about%.