Updating/Upgrading SeisComP¶

You will …

Update/Upgrade a SeisComP system
Migrate a SeisComP3 system to a newer SeisComP version

Pre-requisites for this tutorial:

Documentation of installation
Tutorial on installation and SeisComP previously installed

Afterwards/Results/Outcomes:

Upgraded SeisComP

Time range estimate:

60 minutes

Background¶

Updating/upgrading SeisComP to a higher release version is typically simple and the steps described in Normal Update/Upgrade can be applied. More actions are required when

Upgrading the major version of SeisComP as described in Normal Update/Upgrade.
Upgrading from SeisComP3 to SeisComP in version 4.0.0. or higher.
Upgrading from SeisComP3 Jakarta-2018.327 or older to Jakarta-2020.330 or SeisComP in version 4 or higher.

Upgrade vs. update¶

Here we understand that upgrades increase the major release version number while updates only increase the minor version or the patch number, see section SeisComP Versions.

Upgrading multiple machines¶

Applications can only connect to a messaging system that runs with a database in an equal or lower data base schema version. However, upgrading SeisComP to a higher major release version typically increases the database scheme version. Therefore, in distributed SeisComP systems where one machine hosts the messaging system and the database and all other machines are connected to this messaging, the SeisComP installation on the machine operating the messaging is always upgraded last.

Example: A distributed system includes a processing system with the messaging system and database and a GUI work station connected to the processing system:

Upgrade the GUI work station
Upgrade the processing system, take actions to upgrade the database version.

Note

Always stop all SeisComP modules before upgrading:

seiscomp stop

Package download¶

Get and install the SeisComP package in the any available version from gempa GmbH [31] or from the download website of Helmholtz-Centre Potsdam - GFZ German Research Centre for Geosciences and gempa GmbH [76] as described in section Installation of Packages.

Documentation of changes¶

The important novelties, optimizations and changes that are available after upgrading are documented in the Changelog which is part of this documentation and which can also be accessed through the Docs panel of scconfig. The installed locally installed file is $SEISCOMP_ROOT/share/doc/seiscomp/CHANGELOG The Changelog can also be read online but care should be take that the version number matches your.´

Note

New major release with the features are regularly advertised and described in detail on the News website of gempa GmbH and on the SeisComP forum [26].

Upgrade database schema version¶

When upgrading SeisComP to a higher major version number as set out in section Normal Update/Upgrade, upgrading the database schema is typically required, too. The database version will be tested and the required actions will be reported when executing:

seiscomp update-config**Special case:**

or when pressing the Update Configuration button in scconfig. An upgrade from version SeisComP3 jakarta-2017.334 to SeisComP in version 5.1.0 will give, e.g.:

seiscomp update-config
* starting kernel modules
starting scmaster
* configure kernel
* configure scmaster
INFO: checking DB schema version of queue: production
  * check database write access ... OK
  * database schema version is 0.10
  * last migration version is 0.12
  * migration to the current version is required. apply the following
    scripts in exactly the given order:
    * mysql -u sysop -p -D seiscomp -h localhost < /home/sysop/seiscomp/share/db/migrations/mysql/0_10_to_0_11.sql
    * mysql -u sysop -p -D seiscomp -h l

**Special case:**ocalhost < /home/sysop/seiscomp/share/db/migrations/mysql/0_11_to_0_12.sql: error: updating configuration for scmaster failed

The shown migration scripts can be used directly as given and in the given order but you need to first stop all modules writing to or deleting objects from the database such as scmaster, scdb, scardac or scdbstrip running on the same or any connected machine, e.g.

seiscomp stop scmaster

Special case: Thereafter, apply the migration according to your database:

MySQL/MariaDB:

mysql -u sysop -p -D seiscomp -h localhost < /home/sysop/seiscomp/share/db/migrations/mysql/0_10_to_0_11.sql
mysql -u sysop -p -D seiscomp -h localhost < /home/sysop/seiscomp/share/db/migrations/mysql/0_11_to_0_12.sql

PostgreSQL:

psql -U sysop -d seiscomp -h localhost -W -f /home/sysop/seiscomp/share/db/migrations/postgresql/0_10_to_0_11.sql
psql -U sysop -d seiscomp -h localhost -W -f /home/sysop/seiscomp/share/db/migrations/postgresql/0_11_to_0_12.sql

Using the migration scripts provides a more user friendly way than copying the lines of MySQL code from the changelog which was practise in past SeisComP3 versions. In future versions we might add the option to automatically run the migrations.

Warning

Upgrading the database make take some time. Do no interrupt the process! During this time, the SeisComP messaging system is unavailable causing a downtime of the system.

After applying the migration scripts the database should be at the correct version. Test again with:

seiscomp update-config

After successfully upgrading the database continue your previous upgrade procedure.

Normal Update/Upgrade¶

Normal updates/upgrades of the major version of SeisComP takes only a few steps:

If you are tracking your SeisComP installation using git (recommended), ensure all previous changes are committed and the installation is clean.
Stop all SeisComP modules:
```
seiscomp stop
```
Download and install the SeisComP package as described in section Installation of Packages.
Understand the Changelog of the installed SeisComP as described in section Documentation of changes.
Infrequently configuration parameters require changes which are documented in the Changelog. You may also scan your configuration for deprecated or obsolete configuration parameters or values using gempa-checkSCconfig.py provided by gempa GmbH [31]. Apply changes accordingly.
The software dependencies may have changed after upgrading. Install them as described in section Software Dependencies.
Test the database schema version and update bindings
```
seiscomp update-config
```
Upgrade the database schema version if mismatches are reported. The mismatch reports will also give the upgrade instructions. Repeat seiscomp update-config after upgrading the database schema version.

Background: When upgrading the major SeisComP version you typically need to upgrade the database scheme version.
After a successful update/upgrade, start all modules again and observe the status:
```
seiscomp start
seiscomp status started
```
If you are tracking your SeisComP installation using git (recommended), commit all changes.

Upgrading from SeisComP3¶

SeisComP in version >=4 has major differences to SeisComP3 which require adjustments. The main differences are in the directories of the SeisComP installation and the messaging system. The changes and the required actions are explained below. They must be considered in addition to the steps set out in section Normal Update/Upgrade.

Files and directories¶

With SeisComP3 all the default installation typically required all modules and configurations in the directories

seiscomp3/ , typically $HOME/seiscomp3 or /opt/seiscomp3/
$HOME/.seiscomp3/

As of SeisComP in version 4 the directories are:

seiscomp/ , typically $HOME/seiscomp/ or /opt/seiscomp/
$HOME/.seiscomp/

All configuration files must be migrated to the new structures. This includes:

Configurations and inventory in seiscomp3/:
- seiscomp3/etc/*.cfg
- seiscomp3/etc/inventory/
- seiscomp3/etc/keys/
Configurations in $HOME/.seiscomp3/
Logs in $HOME/.seiscomp3/log (optional)
All user-defined files and directories in seiscomp3/share/
All user-defined seedlink and other templates in seiscomp3/share/templates/
The waveform archive and other archives typically in seiscomp3/var/lib/
User-defined files and directories in other places.
Warning

Some configuration default and description files have changed. Spread, arclink and arclinkproxy are not part of SeisComP anymore. Therefore, do not migrate:
- any default configuration, description and init files. Better enable the desired daemon modules again:
  seiscomp/bin/seiscomp enable [module]
- any file related to spread or the arclink and arclinkproxy servers.

Configurations containing absolute paths, e.g. /home/sysop/seiscomp3/share/scautoloc/grid_custom.conf, must be adjusted. Better use internal SeisComP variables such as @DATADIR@ instead of seiscomp3/share or seiscomp/share.

Software dependencies¶

The software dependencies may have changed. Install the missing ones.

System variables¶

The system environment variables must be updated, e.g. in $HOME/.bashrc. Remove or uncomment the lines $HOME/.bashrc referring to the depreciated SeisComP3 version. Then execute

seiscomp/bin/seiscomp print env >> $HOME/.bashrc
source $HOME/.bashrc

Pipelines¶

When using pipelines or alias modules, create and enable the alias module names again, e.g.

seiscomp alias create [alias] [module]
seiscomp enable [alias]

Migrate the module and bindings configurations of the alias modules including all related additional files which are referred to in the configurations.

Messaging system¶

One of the main changes SeisComP3 to SeisComP in version 4.0 is the messaging system. Spread does not exist anymore and only scmaster is started initially for the messaging system. scmaster allows to operate several queues in parallel with different databases. This flexibility comes with additional parameters which require configuration. Migrate the legacy database parameters and configure the new one:

Remove or comment the obsolete dbplugin plugin manually from scmaster.cfg and global.cfg
```
# plugins = dbplugin
```
Set up the messaging queues in the configuration of scmaster in scmaster.cfg.
- Add and configure a new queue or stay with the default ones.
  - production considers a database by default.
  - playback considers no database by default. Here, parameters can be exchanged through the messaging without storing in the database.
  In the following examples, the production queue shall be assumed.
  
  Note
  
  The production queue is used by default by all modules connected to the messaging system. When removing this queue and a database shall be used, another queue must exist and the queue name must be configured for all modules in the global connection.server parameter. See below for an example.
  - Add the required plugins per queue. Currently only dbstore is supported. Example for the production queue:
    queues.production.plugins = dbstore
  - Add non-default message groups, e.g. L1PICK and L1LOCATION to the list of groups in one of the ways:
    - Recommended: Add groups per queues to defaults in queues.$name.groups, e.g. for the production group. This convenient configuration per queue considers the default groups in defaultGroups and simply adds new groups in the configuration of queues
      queues.production.groups = ${defaultGroups}, L1PICK, L1LOCATION
    - Special case: Set groups per queue in queues.$name.groups, ignoring groups in defaultGroups
      queues.production.groups = L1PICK, L1LOCATION, AMPLITUDE, PICK, LOCATION, MAGNITUDE, FOCMECH, EVENT, QC, PUBLICATION, GUI, INVENTORY, ROUTING, CONFIG, LOGGING, IMPORT_GROUP, SERVICE_REQUEST, SERVICE_PROVIDE
    - Special case: Set groups in defaultGroups
      defaultGroups = L1PICK, L1LOCATION, AMPLITUDE, PICK, LOCATION, MAGNITUDE, FOCMECH, EVENT, QC, PUBLICATION, GUI, INVENTORY, ROUTING, CONFIG, LOGGING, IMPORT_GROUP, SERVICE_REQUEST, SERVICE_PROVIDE
    Warning
    
    When setting groups in the queues all groups configured in defaultGroups will be ignored unless ${defaultGroups} is used. Add all groups from defaultGroups to the queues to keep the default groups.
  - Add the interface name, currently only dbstore is supported. Example for a queue names production
    queues.production.processors.messages = dbstore
  - Add the database parameters which can be used from the legacy configuration
    queues.production.processors.messages.dbstore.driver = mysql queues.production.processors.messages.dbstore.read = sysop:sysop@localhost/seiscomp3 queues.production.processors.messages.dbstore.write = sysop:sysop@localhost/seiscomp3
    Note
    
    The name of the database can be freely chosen. The example assumes that the database named seiscomp3 exists already and that it shall be continued to be used with the new SeisComP in version 4.x.x.
- Add one or more of the queues to the queues parameter to register them by their names
```
queues = production, playback
```
Configure the connection parameters of all modules connecting to the messaging system in the global configuration, e.g. in global.cfg. As in SeisComP3 the connection server is localhost. The queue name is added to the host by “/”. The default queue is production, e.g.
```
connection.server = localhost/production
```
Note

If production shall be used, then no additional configuration is required.

Database¶

After adjusting the structure, variables and configuration parameters, check if the database requires an upgrade as well.

Seedlink¶

When upgrading from SeisComp3 in version Jakrata-2018.327 or older and using seedlink, consider the section Upgrading From SeisComP3 <= Jakarta-2018.327.

Automatic module check¶

If applied, adjust the settings for automatic module status check, e.g. crontab entries. For crontab use:

crontab -e

System daemon¶

If SeisComP is controlled by the system daemon, e.g. to start enabled SeisComP modules automatically during computer startup, then the startup script must be adjusted.

Upgrading From SeisComP3 <= Jakarta-2018.327¶

SeedLink buffer¶

In SeisComP3 prior to Jakarta-2020.330 two stations with the same station but different network code were mixed in one buffer directory. As of Jakarta-2020.330 and SeisComP in version 4 the buffer directories are now unique! Before upgrading seedlink, you should therefore rename the buffer directories accordingly.

Warning

You may discover data gaps if you do not rename the buffer directories.

Example:

Check the current situation:

sysop@host:~/seiscomp3/var/lib/seedlink/buffer$ ls
  PB02

Rename the directories properly:

Stop seedlink:
```
sysop@host:seiscomp stop seedlink
```
Upgrade to SeisComP3-jakarta-2020.330 or SeisComP in version 4 or higher.

Rename all seedlink buffer directories to NET.STA, e.g.

sysop@host:~/seiscomp3/var/lib/seedlink/buffer$ mv PB02 CX.PB02
sysop@host:~/seiscomp3/var/lib/seedlink/buffer$ ls
  CX.PB02

Update configuration:
```
sysop@host:seiscomp update-config
```
Start SeedLink
```
sysop@host:seiscomp start seedlink
```

Script for renaming the seedlink buffer directories:

#!/bin/bash

if [ -z ${SEISCOMP_ROOT+x} ]; then
        echo "Environment variable SEISCOMP_ROOT is not set."
        echo "Either use 'seiscomp exec [script]' or set SEISCOMP_ROOT to the installation "
     exit 1
     echo "path of your SeisComP installation."
fi

grep -A 2 ^station $SEISCOMP_ROOT/var/lib/seedlink/seedlink.ini | while read a b c; do
    if [ "$a" = station -a "$b" != .dummy ]; then
             id=$b
             sta=""
             net=""
             while read a b c; do
                     case $a in
                             --) break;;tutorials_upgrade_seedlink
                             name) eval sta=$c;;
                             network) eval net=$c;;
                     esac
             done
             if [ -z "$id" -o -z "$sta" -o -z "$net" ]; then
                     echo "Error parsing seedlink.ini"
                     break
             fi

             if [ "$id" != "$net.$sta" ]; then
                     mv -v "$SEISCOMP_ROOT/var/lib/seedlink/buffer/$id" "$SEISCOMP_ROOT/var/lib/seedlink/buffer/$net.$sta"
             else
                     echo "$id: No renaming required"
             fi
     fi
done

SeedLink stream processor¶

Since SeisComP3 in version Jakarta-2020.030 and SeisComP in version 4.0.0, SeedLink stream processors (proc parameter) can be attached to both, stations and plugin instances. In order to distinguish between the two cases, either proc (attach to station) or sources.*.proc (attach to plugin instance) parameter (or both) can be used in SeedLink bindings.

chain plugin¶

In case of the chain plugin for seedlink, there is normally just one instance, so stream processors attached to this instance apply to all stations. This is normally not what we want. Therefore the chain plugin does not support the sources.*.proc option.

Before SeisComP3 in version Jakarta-2020.030 and SeisComP in version 4.0.0, stream processors were always attached to stations, even when sources.*.proc was used. This means when upgrading:

sources.chain.proc must be renamed to proc
streams_*.tpl templates must be moved one level up, from $SEISCOMP_ROOT/seiscomp/share/templates/seedlink/chain/ to $SEISCOMP_ROOT/seiscomp/share/templates/seedlink/.

Note

Using a stream processor with chain_plugin makes only sense when raw data is generated (sources.chain.channels.unpack).

Background¶

A stream processor is an object defined in XML, which is used to create MiniSEED from raw data and optionally downsample the data. What is the difference between attaching a stream processor to station and plugin instance?

Let’s take a look at the following stream processor definition in $SEISCOMP_ROOT/share/templates/seedlink/streams_stream100.tpl:

<proc name="stream100">
  <tree>
    <input name="Z" channel="Z" location="" rate="100"/>
    <input name="N" channel="N" location="" rate="100"/>
    <input name="E" channel="E" location="" rate="100"/>
    <node filter="FS2D5" stream="BH">
      <node filter="F96C">
        <node filter="ULP" stream="LH">
          <node filter="VLP" stream="VH"/>
        </node>
      </node>
    </node>
  </tree>
</proc>

This creates 20Hz BH*, 1Hz LH* and 0.1Hz VH* streams from 100Hz Z, N, E raw data. If one plugin instance is used for the station, it does not make a difference whether this is attached to station or plugin instance. But suppose the station is using two plugin instances—one for broad-band and the other for strong-motion data—, both sending Z, N and E channels. Now if the stream processor is attached to station, data from both plugin instances would mixed up. We must attach a different stream processor to each plugin instance—one producing BH*, LH* and VH* and the other one producing BN* and so on.

Updating/Upgrading SeisComP¶

Background¶

Upgrade vs. update¶

Upgrading multiple machines¶

Package download¶

Documentation of changes¶

Upgrade database schema version¶

Normal Update/Upgrade¶

Upgrading from SeisComP3¶

Files and directories¶

Software dependencies¶

System variables¶

Pipelines¶

Messaging system¶

Database¶

Seedlink¶

Automatic module check¶

System daemon¶

Upgrading From SeisComP3 <= Jakarta-2018.327¶

SeedLink buffer¶

SeedLink stream processor¶

chain plugin¶

Background¶

Table of Contents

Previous topic

Next topic

This Page