.. highlight:: rst

.. _scdispatch:

##########
scdispatch
##########

**Read objects (event, origin, etc) from a SCML file and sends the objects
to the messaging system.**


Description
===========

scdispatch reads an :term:`SCML` file and creates notifier objects for them that
are sent to the :ref:`messaging <concepts_messaging>` and the corresponding
messaging groups (see :option:`--routingtable`). In contrast to :ref:`scdb`
which writes SCML files directly into the :ref:`database <concepts_database>`
scdispatch uses the messaging bus. If :ref:`scmaster` is configured with the
database plugin, messages will end up in the database as well.


Modes
-----

scdispatch can work in two modes applying different
:ref:`operations <scdispatch-operations>`:

* *Without database check:* One of the :ref:`operations <scdispatch-operations>`
  *add*, *update* or *remove* is selected along with the option :option:`-O`. In
  that case all objects in the :term:`SCML` are encapsulated in a notifier with
  that specific operation and sent to the messaging. No check is performed if
  the object is already in the database or not.
* *With database check:* The option :option:`-O` is not given or the
  option is used along with one of the :ref:`operations <scdispatch-operations>`
  *merge* or *merge-without-remove*. scdispatch first tries to load the corresponding
  objects from the database and calculates differences. It will then create the
  corresponding notifiers with operations *add*, *update* or *remove* and sends
  them to the messaging. That mode is quite close to a sync operation with the
  exception that top level objects (such as origin or event) that are not part
  of the input SCML are left untouched in the database. It can be used to
  synchronize event information from one system with another.


.. _scdispatch-operations:

Operations
----------

Different operations can be chosen along with the option :option:`-O`.
If :option:`-O` is not given, *merge* is assumed by default.

* *Add*: All objects are sent trying to be added to the database. If they
  already exist in the database, they will be rejected and not spread through
  the messaging. Modules connected to the messaging will not receive rejected
  objects.
* *Remove*: All sent objects with all their attributes and child objects are
  removed from the database. Modules connected to the messaging will not receive
  any sent object.
* *Update*: All objects are sent trying to be updated to the database along with
  all of their child objects and attributes. Sent objects not existing in the
  database will be ignored and not received by any module connected to the
  messaging. Child objects and attributes existing in the database but not
  included in the sent object will be removed as well.
* *Merge* (default): Applies *Add* and *Update* and requires a database
  connection.
* *Merge-without-remove*: Applies *Add* and *Update* and requires a database
  connection. However, no objects are removed from the database.

.. note::

   All |scname| objects along are listed and described along with their child
   objects and attributes in the :ref:`API documentation <api-datamodel-python>`.


Examples
--------

#. Send different objects from a :term:`SCML` file for merging (adding or
   updating). The option :option:`-O` can be ommitted because the default
   behavior is to merge:

   .. code-block:: sh

      scdispatch -i test.xml -O merge
      scdispatch -i test.xml

#. Send all objects by ignoring events. When :ref:`scevent` receives origins it
   will create new events or associate the origins to existing ones. The ignored
   events may be already existing with different IDs. Hence, event duplication
   is avoided by ignoring them.

   .. code-block:: sh

      scdispatch -i test.xml -e

#. Send new objects to be added:

   .. code-block:: sh

      scdispatch -i test.xml -O add

#. Send an update of objects:

   .. code-block:: sh

      scdispatch -i test.xml -O update

#. Send objects to be removed:

   .. code-block:: sh

      scdispatch -i test.xml -O remove

#. Compare new objects with the database content and send the difference (optionally without removing objects):

   .. code-block:: sh

      scdispatch -i test.xml -O merge
      scdispatch -i test.xml -O merge-without-remove

#. Offline mode: all operations can be performed without the messaging system using xml files:

   .. code-block:: sh

      scdispatch -i test.xml -O operation --create-notifier > notifier.xml

   then:

   .. code-block:: sh

      scdb -i notifier.xml

#. Subsets of SCML Objects

   It can be useful to import a subset of QuakeML objects, e.g. Origins from other
   agencies and then allow :ref:`scevent` to associate them to existing
   events (and possibly prefer them based on the rules in scevent) or create new
   events for the origins. If the event objects from a SCML file are not required
   to be sent to the messaging then either they should be removed (e.g. using XSLT)
   and all the remaining objects in the file added:

   .. code-block:: sh

      scdispatch -i test.xml -O add

   or the **event objects** can be left out of the routing table, e.g.

   .. code-block:: sh

      scdispatch -i test.xml -O add \
                 --routingtable Pick:PICK, \
                                Amplitude:AMPLITUDE, \
                                Origin:LOCATION,StationMagnitude:MAGNITUDE, \
                                Magnitude:MAGNITUDE

   .. hint::

      The option :option:`--no-event` is a wrapper for removing Event:EVENT from
      the routing table. With this option no event objects will be sent which may
      be useful if just the origins with magnitudes, amplitudes, arrivals, picks, etc.
      shall be integrated, e.g. after XML-based playbacks.


#. Testing

   For testing it is useful to watch the results of dispatch with :ref:`scolv` or
   :ref:`scxmldump`. It is also useful to clean the database and logs to remove
   objects from persistent storage to allow repeated reloading of a file.

   .. note::

      The following will clear all events from the database and any other
      other object persistence. Modify the mysql command to suit your db setup.

      .. code-block:: sh

         mysql -u root --password='my$q1' -e "DROP DATABASE IF EXISTS seiscomp; \
           CREATE DATABASE seiscomp CHARACTER SET utf8 COLLATE utf8_bin; \
           GRANT ALL ON seiscomp.* TO 'sysop'@'localhost' IDENTIFIED BY 'sysop'; \
           USE seiscomp;source seiscomp/trunk/share/db/mysql.sql;"

         seiscomp start


.. _scdispatch_configuration:

Module Configuration
====================

| :file:`etc/defaults/global.cfg`
| :file:`etc/defaults/scdispatch.cfg`
| :file:`etc/global.cfg`
| :file:`etc/scdispatch.cfg`
| :file:`~/.seiscomp/global.cfg`
| :file:`~/.seiscomp/scdispatch.cfg`

scdispatch inherits :ref:`global options<global-configuration>`.




Command-Line Options
====================

.. program:: scdispatch

:program:`scdispatch [options]`


Generic
-------

.. option:: -h, --help

   Show help message.

.. option:: -V, --version

   Show version information.

.. option:: --config-file arg

   Use alternative configuration file. When this option is
   used the loading of all stages is disabled. Only the
   given configuration file is parsed and used. To use
   another name for the configuration create a symbolic
   link of the application or copy it. Example:
   scautopick \-> scautopick2.

.. option:: --plugins arg

   Load given plugins.

.. option:: -D, --daemon

   Run as daemon. This means the application will fork itself
   and doesn't need to be started with \&.

.. option:: --auto-shutdown arg

   Enable\/disable self\-shutdown because a master module shutdown.
   This only works when messaging is enabled and the master
   module sends a shutdown message \(enabled with \-\-start\-stop\-msg
   for the master module\).

.. option:: --shutdown-master-module arg

   Set the name of the master\-module used for auto\-shutdown.
   This is the application name of the module actually
   started. If symlinks are used, then it is the name of
   the symlinked application.

.. option:: --shutdown-master-username arg

   Set the name of the master\-username of the messaging
   used for auto\-shutdown. If \"shutdown\-master\-module\" is
   given as well, this parameter is ignored.


Verbosity
---------

.. option:: --verbosity arg

   Verbosity level [0..4]. 0:quiet, 1:error, 2:warning, 3:info,
   4:debug.

.. option:: -v, --v

   Increase verbosity level \(may be repeated, eg. \-vv\).

.. option:: -q, --quiet

   Quiet mode: no logging output.

.. option:: --component arg

   Limit the logging to a certain component. This option can
   be given more than once.

.. option:: -s, --syslog

   Use syslog logging backend. The output usually goes to
   \/var\/lib\/messages.

.. option:: -l, --lockfile arg

   Path to lock file.

.. option:: --console arg

   Send log output to stdout.

.. option:: --debug

   Execute in debug mode.
   Equivalent to \-\-verbosity\=4 \-\-console\=1 .

.. option:: --log-file arg

   Use alternative log file.


Messaging
---------

.. option:: -u, --user arg

   Overrides configuration parameter :confval:`connection.username`.

.. option:: -H, --host arg

   Overrides configuration parameter :confval:`connection.server`.

.. option:: -t, --timeout arg

   Overrides configuration parameter :confval:`connection.timeout`.

.. option:: -g, --primary-group arg

   Overrides configuration parameter :confval:`connection.primaryGroup`.

.. option:: -S, --subscribe-group arg

   A group to subscribe to.
   This option can be given more than once.

.. option:: --content-type arg

   Overrides configuration parameter :confval:`connection.contentType`.

.. option:: --start-stop-msg arg

   Set sending of a start and a stop message.


Database
--------

.. option:: --db-driver-list

   List all supported database drivers.

.. option:: -d, --database arg

   The database connection string, format:
   service:\/\/user:pwd\@host\/database.
   \"service\" is the name of the database driver which
   can be queried with \"\-\-db\-driver\-list\".

.. option:: --config-module arg

   The config module to use.

.. option:: --inventory-db arg

   Load the inventory from the given database or file, format:
   [service:\/\/]location .

.. option:: --db-disable

   Do not use the database at all


Dispatch
--------

.. option:: -e, --no-events

   Do not send any event object which are ignored when reading
   event parameters.

.. option:: -i, --input arg

   File from which the content is dispatched to the messaging.

.. option:: -O, --operation arg

   Merge and merge\-without\-remove require a database
   connection. Both will read the object corresponding to the
   sent object from the database and calculate the differences.
   Merge\-without\-remove behaves like merge with the exception
   that remove operations will be filtered out and no objects
   in the database will be removed.
   
   If add, update or remove is specified, then all objects in
   XML are sent with the given operation regardless of their
   existence in the database.

.. option:: --print-objects

   Print names of routable objects.

.. option:: --print-routingtable

   Print routing table.

.. option:: --routingtable arg

   Specify routing table as comma separated list of object:group
   pairs, e.g. \"Origin:LOCATION,Event:EVENT\". When an
   object should be routed to a group and no table entry for
   that particular class type is available, all parent objects
   are checked for valid routing entries and the first found is
   used. E.g. if only \"Origin:LOCATION\" is specified
   but the input file contains also Arrivals which are child
   objects of Origin then the routing entry of Origin is used
   because of the parent\-child relationship between Origin and
   Arrival.

.. option:: --test

   Test mode. Does not send any object.

.. option:: --create-notifier

   Do not send any object. All notifiers will be written to
   standard output in XML format.

