Upgrading SeisComP

You will …

  • Upgrade a SeisComP system

  • Migrate a SeisComP3 system to a newer SeisComP version

Pre-requisites for this tutorial:

  • Tutorial on installation and SeisComP previously installed

Afterwards/Results/Outcomes:

  • Upgraded SeisComP

Time range estimate:

  • 60 minutes


Background

SeisComP versions

SeisComP has developed over time. The versions can be distinguished by the name of the release:

  • SeisComP since version 4.0.0 uses release version numbers

  • SeisComP3 uses release versions, names, numbers and patch numbers.

    Full example: SeisComP3-jakarta-2020.330.02

    • 3: release version

    • jakarta: release name

    • 2020.330: release number

    • 02: patch number

    Names are adjusted depending on changes in source code:

    • Release version: major changes in module groups, functionality, concepts, data model. Example: SeisComp3 is SeisComP in version 3.0 in comparison to version 2.5 the GUIs were introduced.

    • Release name: major changes in functionality, concepts, data model. Example: with SeisComP3-Seattle the new user friendly configuration GUI scconfig was introduced.

    • Release number: changes in data model version and/or major changes in applications and optimizations. The numbers include the year and the day of the year of the software release. Example: Jakarta-2018.327

    • Patch number: optimizations of applications without changes in the data model version.

Upgrade SeisComP on multiple machines

Applications can only connect to a messaging system that runs with a database in an equal or lower data model version. Therefore, the SeisComP system which operates the messaging system is always updated last.

Example: A distributed system includes a processing system with the messaging system and database and a GUI work station connected to the processing system:

  1. Upgrade the GUI work station

  2. Upgrade the processing system, take actions to upgrade the database version.

Note

Always stop all SeisComP modules before upgrading:

seiscomp stop

Package download

Get the latest or older SeisComP release packages from gempa GmbH or from the download website of Helmholtz-Centre Potsdam - GFZ German Research Centre for Geosciences and gempa GmbH [61].

Documentation of changes

The important novelties, optimizations and changes that are available after upgrading are documented in the change log. It is recommend to read the change log before taking further actions. The details can be found in the file

$SEISCOMP_ROOT/share/doc/seiscomp/CHANGELOG

The change log can be directly accessed from the SeisComP documentation or from the Docs panel in scconfig.

Note

New features are regularly advertised and described in detail on the News website of gempa GmbH and on the SeisComP forum [23].

Upgrade to a higher release number

Installing a new SeisComP release or version is typically simple. More actions are required when

The normal upgrade takes only a few steps:

  1. Download the SeisComP package

  2. Stop all SeisComP modules:

    seiscomp stop
    
  3. Install the new packages

    Note

    Users of external, e.g. gempa modules must ensure that the gempa modules match the SeisComP release version if they depend on SeisComP libraries.

  4. When installing a new SeisComP release, upgrading the database may be required. The database version will be tested and the required actions will be shown when executing:

    seiscomp update-config
    

    or when pressing the Update Configuration button in scconfig. An upgrade from version SeisComP3 jakarta-2017.334 to SeisComP in version 4.1.0 will give, e.g.:

    seiscomp update-config
    * starting kernel modules
    starting scmaster
    * configure kernel
    * configure scmaster
    INFO: checking DB schema version of queue: production
      * check database write access ... OK
      * database schema version is 0.10
      * last migration version is 0.11
      * migration to the current version is required. apply the following
        scripts in exactly the given order:
        * /home/sysop/seiscomp/share/db/migrations/mysql/0_10_to_0_11.sql
    error: updating configuration for scmaster failed
    

    The shown migration scripts can be used directly with the database command for upgrading:

    • MySQL / MariaDB:

      mysql -u sysop -p -D seiscomp -h localhost < /home/sysop/seiscomp/share/db/migrations/mysql/0_10_to_0_11.sql
      
    • PostgreSQL:

      psql -U sysop -d seiscomp -h localhost -W -f /home/sysop/seiscomp/share/db/migrations/postgresql/0_10_to_0_11.sql
      

    Using the migration scripts provides a more user friendly way than copying the lines of MySQL code from the changelog. In future versions we might add the option to automatically run the migrations.

    Warning

    Upgrading the database make take some time. Do no interrupt the process! During this time, the SeisComP messaging system is unavailable causing a downtime of the system.

    After applying the migration scripts the database should be at the correct version. Test again with:

    seiscomp update-config
    
  5. After a successful upgrade, start all modules again and observe the status:

    seiscomp start
    seiscomp status
    

Migrate from SeisComP3 to version >=4

SeisComP in version 4 has some major differences to SeisComP3 which require adjustments. The main differences are in the directories of the SeisComP installation and the messaging system. The changes and the required actions are explained below. They must be considered in addition to the steps set out in section Upgrade to a higher release number.

Files and directories

With SeisComP3 all the default installation typically required all modules and configurations in the directories

  • seiscomp3/ , typically $HOME/seiscomp3 or /opt/seiscomp3/

  • $HOME/.seiscomp3/

As of SeisComP in version 4 the directories are:

  • seiscomp/ , typically $HOME/seiscomp/ or /opt/seiscomp/

  • $HOME/.seiscomp/

All configuration files must be migrated to the new structures. This includes:

  • Configurations and inventory in seiscomp3/:

    • seiscomp3/etc/*.cfg

    • seiscomp3/etc/inventory/

    • seiscomp3/etc/keys/

  • Configurations in $HOME/.seiscomp3/

  • Logs in $HOME/.seiscomp3/log (optional)

  • All user-defined files and directories in seiscomp3/share/

  • All user-defined seedlink and other templates in seiscomp3/share/templates/

  • The waveform archive and other archives typically in seiscomp3/var/lib/

  • User-defined files and directories in other places.

    Warning

    Some configuration default and description files have changed. Spread, arclink and arclinkproxy are not part of SeisComP anymore. Therefore, do not migrate:

    • any default configuration, description and init files. Better enable the desired daemon modules again:

      seiscomp/bin/seiscomp enable [module]
      
    • any file related to spread or the arclink and arclinkproxy servers.

Configurations containing absolute paths, e.g. /home/sysop/seiscomp3/share/scautoloc/grid_custom.conf, must be adjusted. Better use internal SeisComP variables such as @DATADIR@ instead of seiscomp3/share or seiscomp/share.

Software dependencies

The software dependencies may have changed. Install the missing ones.

System variables

The system environment variables must be updated, e.g. in $HOME/.bashrc. Remove or uncomment the lines $HOME/.bashrc referring to the depreciated SeisComP3 version. Then execute

seiscomp/bin/seiscomp print env >> $HOME/.bashrc
source $HOME/.bashrc

Pipelines

When using pipelines or alias modules, create and enable the alias module names again, e.g.

seiscomp alias create [alias] [module]
seiscomp enable [alias]

Migrate the module and bindings configurations of the alias modules including all related additional files which are referred to in the configurations.

Messaging system

One of the main changes SeisComP3 to SeisComP in version 4.0 is the messaging system. Spread does not exist anymore and only scmaster is started initially for the messaging system. scmaster allows to operate several queues in parallel with different databases. This flexibility comes with additional parameters which require configuration. Migrate the legacy database parameters and configure the new one:

  1. Remove or comment the obsolete dbplugin plugin manually from scmaster.cfg and global.cfg

    # plugins = dbplugin
    
  2. Set up the messaging queues in the configuration of scmaster in scmaster.cfg.

    • Add and configure a new queue or stay with the default ones.

      • production considers a database by default.

      • playback considers no database by default. Here, parameters can be exchanged through the messaging without storing in the database.

      In the following examples, the production queue shall be assumed.

      Note

      The production queue is used by default by all modules connected to the messaging system. When removing this queue and a database shall be used, another queue must exist and the queue name must be configured for all modules in the global connection.server parameter. See below for an example.

      • Add the required plugins per queue. Currently only dbstore is supported. Example for the production queue:

        queues.production.plugins = dbstore
        
      • Add non-default message groups, e.g. L1PICK and L1LOCATION to the list of groups in one of the ways:

        • Set defaultGroups

          defaultGroups = L1PICK, L1LOCATION, AMPLITUDE,PICK,LOCATION,MAGNITUDE,FOCMECH,EVENT,QC,PUBLICATION,GUI,INVENTORY,ROUTING,CONFIG,LOGGING,IMPORT_GROUP,SERVICE_REQUEST,SERVICE_PROVIDE
          
        • Set groups per queue in queues.$name.groups, ignoring groups in defaultGroups

          queues.production.groups = L1PICK, L1LOCATION, AMPLITUDE,PICK,LOCATION,MAGNITUDE,FOCMECH,EVENT,QC,PUBLICATION,GUI,INVENTORY,ROUTING,CONFIG,LOGGING,IMPORT_GROUP,SERVICE_REQUEST,SERVICE_PROVIDE
          
        • Add groups per queues to defaults in queues.$name.groups, e.g. for the production group. This convenient configuration per queue considers the default groups in defaultGroups and simply adds new groups in the configuration of queues

          queues.production.groups = ${defaultGroups}, L1PICK, L1LOCATION
          

        Warning

        When setting groups in the queues all groups configured in defaultGroups will be ignored unless ${defaultGroups} is used. Add all groups from defaultGroups to the queues to keep the default groups.

      • Add the interface name, currently only dbstore is supported. Example for a queue names production

        queues.production.processors.messages = dbstore
        
      • Add the database parameters which can be used from the legacy configuration

        queues.production.processors.messages.dbstore.driver = mysql
        queues.production.processors.messages.dbstore.read = sysop:sysop@localhost/seiscomp3
        queues.production.processors.messages.dbstore.write = sysop:sysop@localhost/seiscomp3
        

        Note

        The name of the database can be freely chosen. The example assumes that the database named seiscomp3 exists already and that it shall be continued to be used with the new SeisComP in version 4.x.x.

    • Add one or more of the queues to the queues parameter to register them by their names

      queues = production, playback
      
  3. Configure the connection parameters of all modules connecting to the messaging system in the global configuration, e.g. in global.cfg. As in SeisComP3 the connection server is localhost. The queue name is added to the host by “/”. The default queue is production, e.g.

    connection.server = localhost/production
    

    Note

    If production shall be used, then no additional configuration is required.

Database

After adjusting the structure, variables and configuration parameters, check if the database requires an upgrade as well.

Automatic module check

If applied, adjust the settings for automatic module status check, e.g. crontab entries. For crontab use:

crontab -e

System daemon

If SeisComP is controlled by the system daemon, e.g. to start enabled SeisComP modules automatically during computer startup, then the startup script must be adjusted.

Upgrade from SeisComP3 Jakarta-2018.327 or before