Turbostick Huawei E8372 for remote access

Turbostick Huawei E8372 for remote accessEdit

Advantages:

• does not require a custom SG kernel or hardware drivers

Disadvantages

• no straightforward way for the SG to power the cellular modem up and down, so the device is
  always consuming power.  Approximate power consumption ~ 1.5 Watts added to usual
  SG consumption.

Turbostick Set-up

 Before using this turbostick with an SG:

1. perform intial set up following the instructions here.
    
2. connect at least once to the internet using the turbostick on a laptop,
   following instructions here. This ensures you won't face mysterious charges on your bill
   due to the sensorgnome being the first device to activate the stick.  You should
   also make sure to set a strong admin password.
    
3. turn off the turbostick WiFi Hotspot access point.  Use the instructions here, 
   except that instead of setting a WiFi password, disable the WLAN module.  Leaving
   this on would consume more power and would allow anyone with a nearby smart phone to
   use the turbostick's internet connection, and steal your data plan minutes.
    
4. turn *on* the data connection and leave it on when you remove the stick from the laptop.
   This ensures the turbostick will try connect to the cellular network when you attach it to an
   SG. 
    
5. In the field, when deploying: 
   • plug the turbostick into a laptop to verify your provider's cellular network has sufficient
     coverage at your site.
      
   • make sure the turbostick's data connection is on (use a web-browser to connect to http://192.168.8.1
     while the turbostick is plugged into the laptop; you'll need to know the admin password for the turbostick).
     If you don't do this, your turbostick won't work in the SG. 
     The SG cannot turn on the turbostick's data connection.

Sensorgnome set-up

Currently, using this device requires the (pending) 2016-04-13 release of the SG software.
You should also modify the deployment.txt file to add the field remoteStatusUpdatePeriodMinutes
like so:

{
    "info": "default deployment",
    "who": "who is responsible for this deployment",
    "contact": "contact information for responsible party",
    "shortLabel": "CHAP3",
    "remoteStatusUpdatePeriodMinutes": 5,
    "acquire": {
    
    ...

(don't forget the trailing comma).  You can of course set this to a different value, but updates
will not be sent more than once every 10 seconds.  Reboot the SG to have this setting take effect.

Dataplan Usage

With the default configuration, a message is sent every 5 minutes, and with no tags around, messages are ~580 bytes.
Each unique tag detected between messages will probably add around 15 bytes (a given tag is only included once
in a message, and only if was detected since the previous message was sent out).  If there are 100 tags regularly
detected, then this would be another 1500 bytes per message.
There is also another 2K or so of traffic every 20 minutes to check for remote commands.

So data usage with a 5-minute update period and 20-minute remote command checking looks like:
 

Of course, if remote commands are used to establish a secure tunnel to provide software updates
or troubleshooting, usage will go up.  Also, there might be other connection overhead not included
in the above totals.

What it does

self-registration: A new SG first registers itself with our server using TCP port 59022 at sensorgnome.org 
This provides it with a unique set of cryptographic keys.  The SG attempts to register itself once per minute until
it succeeds.  If your SG was built by compudata.ca, it has already registered itself.

Update messages:  The 2016-04-13 software release is the first to provide datagram-based communication for remote status
updates.  Every 5 minutes (or whatever period you selected above), the connection to the turbostick is
enabled, and a data message is sent to UDP port 59022 on the server at sensorgnome.org.  This message
currently has these items:

• status of all attached funcubedongles (how many raw sample frames they have
  processed, when they were last restarted, etc.)
   
• full IDs and antenna # for all known tags detected since the previous update
  message.  Known tags are those in the on-board tag registration database,
  /boot/uboot/SG_tag_database.csv, which can be modified remotely
  (see below)
   
• receiver GPS fix [FIXME: not present in message] and local clock

Messages are cryptographically signed, to reduce the risk of spoofed or corrupt messages.
Routing of these messages from our server to end users is a work in progress.
 

Push commands:   Every 20 minutes, the SG will send an HTTP request to this URL:

https://archived.sensorgnome.org/recv/1234BBBK5678

where the last 12 characters are the SG serial number.  Our server will respond with the most
recent push command, if any.  The SG will look at the serial number of the command, and if it is
higher than that of any previous commands, the new command will be processed.

So far, the available push commands are:

• ssh-tunnel XXXX: open a secure shell connection to sensorgnome.org:59022, to allow remote login
  from our server for troubleshooting, software updates, etc.   The tunnel is opened for XXXX seconds,
  then closed.
   
• add-tags: add the following tag registration records to the SG's on-board database, and restart the tag finder
   
• set-tags: replace the SG's on-board database with the following lines, and restart the tag finder.

Push commands are created manually by sensorgnome.org admins, in response to user requests.

Role of the onboard tag database

• an optional feature that does not affect which tags will be detected from raw SG data files when these are
  eventually sent manually to sensorgnome.org for processing
   
• displays live detections of known tags on the web interface, so you can see them when attached to the
  SG in the field via USB or ethernet cable
   
• sends summaries of tags seen in each periodic remote update message.

Update Message format (for the record, not for end-user consumption)

Messages arrive from SGs with this format:

+--SIGLEN--+ SIGNATURE-----+--------PAYLOAD-------------------------------------------+
|          |               |                                                          |
|          |               +-SERNO------+-MACADDR--+-MSGLEN-+-COMPRESSED_MESSAGE------+
| 1 byte   | SIGLEN bytes  |12 alphanum | 6 bytes  | 2 bytes| MSGLEN bytes            |

where SIGNATURE is the SHA512 signature of the compressed message using the SG's private
key, SERNO is e.g. 3214BBBK7260, MACADDR is the raw 6-byte MAC address of the eth0 adapter,
and the COMPRESSED_MESSAGE is a gzip-compressed JSON string encoding receiver status and
tag detection summaries.  MSGLEN is the length of the *compressed* message.  SERNO and MACADDR
are used to look up the public key used to verify the signature.

Here is an example, with the serial number and MAC address shown as text, and message payload:

2814BBBK4765,c65f33699ed9:
{"ts":1460563468.776,"gps":null,"devList":[],"param":[],"VAHstatus":{
"4":{"type":"AlsaMinder","device":"hw:0","rate":48000,"hwRate":48000,"numChan":2,
    "startTimestamp":1460563169.3085,"stopTimestamp":1460563168.3514,"running":true,"hasError":-77,"totalFrames":14366426},
"5":{"type":"AlsaMinder","device":"hw:1","rate":48000,"hwRate":48000,"numChan":2,
    "startTimestamp":1460563169.3435,"stopTimestamp":1460563168.4039,"running":true,"hasError":-77,"totalFrames":14366438},
"7":{"type":"AlsaMinder","device":"hw:2","rate":48000,"hwRate":48000,"numChan":2,
    "startTimestamp":1460563169.3786,"stopTimestamp":1460563168.4449,"running":true,"hasError":-77,"totalFrames":14366442},
"Socket#4":{"type":"TCPConnection","fileDescriptor":4,"timeConnected":1460563168.3136},
"Socket#5":{"type":"TCPConnection","fileDescriptor":5,"timeConnected":1460563168.3163},
"Socket:/tmp/VAH.sock":{"type":"TCPListener","socket":"/tmp/VAH.sock"},
"p4":{"type":"PluginRunner","devLabel":"4","libraryName":"lotek-plugins.so","pluginID":"findpulsefdbatch","pluginOutput":"pulses","totalFrames":14366426,"totalFeatures":1139},
"p5":{"type":"PluginRunner","devLabel":"5","libraryName":"lotek-plugins.so","pluginID":"findpulsefdbatch","pluginOutput":"pulses","totalFrames":14366438,"totalFeatures":991},
"p7":{"type":"PluginRunner","devLabel":"7","libraryName":"lotek-plugins.so","pluginID":"findpulsefdbatch","pluginOutput":"pulses","totalFrames":14366442,"totalFeatures":1084}},

"tagsSeen":{"Taylr#500@166.38:8":"5"},

"machInfo":{"bootCount":"000006","version":"Tue, 02 Feb 2016 13:00:09 GMT"}
}

This format is subject to change, and we will likely remove some of the rarely-changing items.
The yellow lines show status of each of the funcubes as reported by vamp-alsa-host.
The blue lines list tags seen detected since the last update message, and the antenna of their
most recent detections.