# XBee Update script

This script is intended as a helper for use updating networks with old versions
of XBee3 and S2C firmware present. When run, it will discover modules on the network
that need updating, then go through them one by one and update them using the
gateway's RCI interface.

The following modules will be updated:

* Any S2C radios not at version 40A7 will be up/downgraded to 40A7.
* XBee 3 radios with versions 1005-100A will be upgraded to 1A10.
* XBee 3 radios with version 1003 will be upgraded to 1B04, then to 1A10.

## Configuring the script

The version to update radios to, as well as some other configurable behaviors,
are determined by a few variables set at the start of the script. To use this
script to update to a different version, modify the variables in the section
marked `# Configurable values` near the start of the script. The filenames
referenced in the script need to be installed in the gateway's firmware
directory.

## Running the script

Load the script onto the gateway, then using ssh run the following at the
gateway's console:

```
python xbee_update_tool.py run
```

The script will build a list of modules to update, then go through them and
update them all. To iterate through the process multiple times, run

```
python xbee_update_tool.py run --loop 3
```

This will go through the whole process 3 times: ie discover nodes, update them,
discover nodes again (which will find both new nodes and ones that failed to
update on the last time through), update those nodes, etc.

Giving the loop argument without a number will cause the script to go through
the discover/update loop indefinitely.

Our testing has determined that older firmwares have a higher chance of success
if other transmissions are kept to a minimum during the update. Becuase of this,
the discovery and updating steps should be kept separate (this is why when using
the `run` command, discovery isn't done while updating or between updates).

The `run` command leaves enough time between discovery and updating on its own,
but it is also possible to run the discovery and updating steps separately.

### Discovery only

Running `python xbee_update_tool.py discover` will run only the discovery part
of the update, and will output a list of addresses of radios that need updating,
one per line. The addresses will be output in the order they should be updated,
as determined by the discovery code. This list can be saved to a file:

```
python xbee_update_tool.py discover > radios_to_update.txt
```

and modified or used directly for the update step.

### Updating only

`python xbee_update_tool.py update` takes a list of addresses, one per line,
from stdin and updates them one by one. To use a list from a file, run:

```
cat radios_to_update.txt | python xbee_update_tool.py update
```

### Excluding radios from updating

The `run` command can be given a list of radios to exclude if certain radios
are needed to be skipped during the update process. 
`python xbee_update_tool.py run` takes a list of addresses, one per line,
from stdin and filters them out of the update list. To use a list from a file, run:

```
cat radios_to_exclude.txt | python xbee_update_tool.py run --exclude-radios True
```

### Enabling/disabling source routing

It can be advantageous to be able to turn source routing on and off for testing and
diagnostic purposes. The `--ar-value` argument takes an integer value between 0 and 255
and sets AR on the gateway to that value. Setting AR to 255 disables source routing
while any value from 0 to 254 will enable source routing. This command will default to 
255 if not specified.

```
python xbee_update_tool.py run --ar-value 255
```

### Rescuing orphaned S2C nodes

A text file called "orphans.txt" will need to be uploaded to the same directory
as the script. It should contain a single line of comma separated addresses of
orphaned radios to search for in this format: 
`00:13:a2:00:40:dd:bf:83!,00:13:a2:00:40:dd:ac:91!,00:13:a2:00:40:dd:bf:76!`

This step will attempt to use three different methods to recover S2Cs
that are currently in the bootloader but still on the network. The first method
will only be used on S2Cs that are using the 0x40A7 firmware; this method will automatically
search for and recover nodes. The second method will attempt to find and recover the radios 
listed in "orphans.txt" that are in the bootloader but still on the original network channel.
The third method will attempt to find and recover listed radios that are in the bootloader 
but using the recovery channel.

To intiate the recovery of orphaned nodes, run

```
python xbee_update_tool.py rescue-orphans
```

An example of running the rescue orphans step, using logging level 7, 
clearing any previous logs, and enabling saving logs during OTA updates:

```
python xbee_update_tool.py rescue-orphans --logging-level 7 --clear-logs True --save-logs True
```

## Saving logs during OTA updates

The optional argument `--save-logs {bool}` will create a logging file at `/WEB/python/copied_logs/{address of remote node}/` which contains the contents of xbee.log over the course of the OTA update process. The script will remove that file at the end of the OTA process if the process was successful to save on memory.
The script will also only create 10 files worth of failed OTA update logs to ensure it doesn't overrun memory. The script will also retain the first successful S2C and XBee3 update logs to enable a comparison
between failed updates and successful updates.
Make sure to reclaim space between runs by using the `--clear-logs True` argument. Log files will be renamed with the current timestamp as well as the EUI of the radio undergoing the OTA update.
i.e. `2022-01-10 09_41_42.760092-00_13_a2_00_40_e5_68_cc!.log`.

An example of running the full run step, using logging level 7,
clearing any previous logs, and enabling saving logs during OTA updates:
```
python xbee_update_tool.py run --logging-level 7 --clear-logs True --save-logs True
```

## Changing the logging level

The optional argument `--logging-level {#}` will change the logging level of the gateway where # is the level.
By default, the script will set the logging level to 7 which is the debug level.

## Saving logs

The gateway doesn't have a lot of hard drive space for logs, so if log files are
going to be saved they can be saved on another computer. This can be done
using ssh. From a terminal on a computer on the same network as the gateway, run:

```
ssh -t -t python@{gateway_ip_addr} python xbee_update_tool.py run --loop 3 2>&1 | tee xbee_update.log
```

That will run `python xbee_update_tool.py run --loop 3` on the gateway, and
output the logs both to the screen and to `xbee_update.log` on the computer.

The `-t -t` parameters to `ssh` are necessary because without them, the python
script will keep running even if it is cancelled from the computer (if the
operator hits ctrl-C, for example).

Note that a similar strategy can be used to save the gateway's internal logs, to
get around the fast rollover when logging at level 7:

```
ssh python@{gateway_ip_addr} tail /WEB/logging/xbee.log -F | tee gateway_xbee.log
```

## Recovering S2C's stuck on the Recovery Channel

The `recover` command can be specified to attempt to recover S2C updater nodes 
that have switched to the recovery channel but failed to switch back on to the
original network channel. Make sure that the gateway firwmare has a firmware version
capable of setting CH such as 0x40A1.

```
python xbee_update_tool.py recover
```