422 lines
12 KiB
ReStructuredText
422 lines
12 KiB
ReStructuredText
.. highlight:: rst
|
|
|
|
.. _scdispatch:
|
|
|
|
##########
|
|
scdispatch
|
|
##########
|
|
|
|
**Read objects (event, origin, etc) from a SCML file and sends the objects
|
|
to the messaging system.**
|
|
|
|
|
|
Description
|
|
===========
|
|
|
|
scdispatch reads an :term:`SCML` file and creates notifier objects for them that
|
|
are sent to the :ref:`messaging <concepts_messaging>` and the corresponding
|
|
messaging groups (see :option:`--routingtable`). In contrast to :ref:`scdb`
|
|
which writes SCML files directly into the :ref:`database <concepts_database>`
|
|
scdispatch uses the messaging bus. If :ref:`scmaster` is configured with the
|
|
database plugin, messages will end up in the database as well.
|
|
|
|
|
|
Modes
|
|
-----
|
|
|
|
scdispatch can work in two modes applying different
|
|
:ref:`operations <scdispatch-operations>`:
|
|
|
|
* *Without database check:* One of the :ref:`operations <scdispatch-operations>`
|
|
*add*, *update* or *remove* is selected along with the option :option:`-O`. In
|
|
that case all objects in the :term:`SCML` are encapsulated in a notifier with
|
|
that specific operation and sent to the messaging. No check is performed if
|
|
the object is already in the database or not.
|
|
* *With database check:* The option :option:`-O` is not given or the
|
|
option is used along with one of the :ref:`operations <scdispatch-operations>`
|
|
*merge* or *merge-without-remove*. scdispatch first tries to load the corresponding
|
|
objects from the database and calculates differences. It will then create the
|
|
corresponding notifiers with operations *add*, *update* or *remove* and sends
|
|
them to the messaging. That mode is quite close to a sync operation with the
|
|
exception that top level objects (such as origin or event) that are not part
|
|
of the input SCML are left untouched in the database. It can be used to
|
|
synchronize event information from one system with another.
|
|
|
|
|
|
.. _scdispatch-operations:
|
|
|
|
Operations
|
|
----------
|
|
|
|
Different operations can be chosen along with the option :option:`-O`.
|
|
If :option:`-O` is not given, *merge* is assumed by default.
|
|
|
|
* *Add*: All objects are sent trying to be added to the database. If they
|
|
already exist in the database, they will be rejected and not spread through
|
|
the messaging. Modules connected to the messaging will not receive rejected
|
|
objects.
|
|
* *Remove*: All sent objects with all their attributes and child objects are
|
|
removed from the database. Modules connected to the messaging will not receive
|
|
any sent object.
|
|
* *Update*: All objects are sent trying to be updated to the database along with
|
|
all of their child objects and attributes. Sent objects not existing in the
|
|
database will be ignored and not received by any module connected to the
|
|
messaging. Child objects and attributes existing in the database but not
|
|
included in the sent object will be removed as well.
|
|
* *Merge* (default): Applies *Add* and *Update* and requires a database
|
|
connection.
|
|
* *Merge-without-remove*: Applies *Add* and *Update* and requires a database
|
|
connection. However, no objects are removed from the database.
|
|
|
|
.. note::
|
|
|
|
All |scname| objects along are listed and described along with their child
|
|
objects and attributes in the :ref:`API documentation <api-datamodel-python>`.
|
|
|
|
|
|
Examples
|
|
--------
|
|
|
|
#. Send different objects from a :term:`SCML` file for merging (adding or
|
|
updating). The option :option:`-O` can be ommitted because the default
|
|
behavior is to merge:
|
|
|
|
.. code-block:: sh
|
|
|
|
scdispatch -i test.xml -O merge
|
|
scdispatch -i test.xml
|
|
|
|
#. Send all objects by ignoring events. When :ref:`scevent` receives origins it
|
|
will create new events or associate the origins to existing ones. The ignored
|
|
events may be already existing with different IDs. Hence, event duplication
|
|
is avoided by ignoring them.
|
|
|
|
.. code-block:: sh
|
|
|
|
scdispatch -i test.xml -e
|
|
|
|
#. Send new objects to be added:
|
|
|
|
.. code-block:: sh
|
|
|
|
scdispatch -i test.xml -O add
|
|
|
|
#. Send an update of objects:
|
|
|
|
.. code-block:: sh
|
|
|
|
scdispatch -i test.xml -O update
|
|
|
|
#. Send objects to be removed:
|
|
|
|
.. code-block:: sh
|
|
|
|
scdispatch -i test.xml -O remove
|
|
|
|
#. Compare new objects with the database content and send the difference (optionally without removing objects):
|
|
|
|
.. code-block:: sh
|
|
|
|
scdispatch -i test.xml -O merge
|
|
scdispatch -i test.xml -O merge-without-remove
|
|
|
|
#. Offline mode: all operations can be performed without the messaging system using xml files:
|
|
|
|
.. code-block:: sh
|
|
|
|
scdispatch -i test.xml -O operation --create-notifier > notifier.xml
|
|
|
|
then:
|
|
|
|
.. code-block:: sh
|
|
|
|
scdb -i notifier.xml
|
|
|
|
#. Subsets of SCML Objects
|
|
|
|
It can be useful to import a subset of QuakeML objects, e.g. Origins from other
|
|
agencies and then allow :ref:`scevent` to associate them to existing
|
|
events (and possibly prefer them based on the rules in scevent) or create new
|
|
events for the origins. If the event objects from a SCML file are not required
|
|
to be sent to the messaging then either they should be removed (e.g. using XSLT)
|
|
and all the remaining objects in the file added:
|
|
|
|
.. code-block:: sh
|
|
|
|
scdispatch -i test.xml -O add
|
|
|
|
or the **event objects** can be left out of the routing table, e.g.
|
|
|
|
.. code-block:: sh
|
|
|
|
scdispatch -i test.xml -O add \
|
|
--routingtable Pick:PICK, \
|
|
Amplitude:AMPLITUDE, \
|
|
Origin:LOCATION,StationMagnitude:MAGNITUDE, \
|
|
Magnitude:MAGNITUDE
|
|
|
|
.. hint::
|
|
|
|
The option :option:`--no-event` is a wrapper for removing Event:EVENT from
|
|
the routing table. With this option no event objects will be sent which may
|
|
be useful if just the origins with magnitudes, amplitudes, arrivals, picks, etc.
|
|
shall be integrated, e.g. after XML-based playbacks.
|
|
|
|
|
|
#. Testing
|
|
|
|
For testing it is useful to watch the results of dispatch with :ref:`scolv` or
|
|
:ref:`scxmldump`. It is also useful to clean the database and logs to remove
|
|
objects from persistent storage to allow repeated reloading of a file.
|
|
|
|
.. note::
|
|
|
|
The following will clear all events from the database and any other
|
|
other object persistence. Modify the mysql command to suit your db setup.
|
|
|
|
.. code-block:: sh
|
|
|
|
mysql -u root --password='my$q1' -e "DROP DATABASE IF EXISTS seiscomp; \
|
|
CREATE DATABASE seiscomp CHARACTER SET utf8 COLLATE utf8_bin; \
|
|
GRANT ALL ON seiscomp.* TO 'sysop'@'localhost' IDENTIFIED BY 'sysop'; \
|
|
USE seiscomp;source seiscomp/trunk/share/db/mysql.sql;"
|
|
|
|
seiscomp start
|
|
|
|
|
|
.. _scdispatch_configuration:
|
|
|
|
Module Configuration
|
|
====================
|
|
|
|
| :file:`etc/defaults/global.cfg`
|
|
| :file:`etc/defaults/scdispatch.cfg`
|
|
| :file:`etc/global.cfg`
|
|
| :file:`etc/scdispatch.cfg`
|
|
| :file:`~/.seiscomp/global.cfg`
|
|
| :file:`~/.seiscomp/scdispatch.cfg`
|
|
|
|
scdispatch inherits :ref:`global options<global-configuration>`.
|
|
|
|
|
|
|
|
|
|
Command-Line Options
|
|
====================
|
|
|
|
.. program:: scdispatch
|
|
|
|
:program:`scdispatch [options]`
|
|
|
|
|
|
Generic
|
|
-------
|
|
|
|
.. option:: -h, --help
|
|
|
|
Show help message.
|
|
|
|
.. option:: -V, --version
|
|
|
|
Show version information.
|
|
|
|
.. option:: --config-file arg
|
|
|
|
Use alternative configuration file. When this option is
|
|
used the loading of all stages is disabled. Only the
|
|
given configuration file is parsed and used. To use
|
|
another name for the configuration create a symbolic
|
|
link of the application or copy it. Example:
|
|
scautopick \-> scautopick2.
|
|
|
|
.. option:: --plugins arg
|
|
|
|
Load given plugins.
|
|
|
|
.. option:: -D, --daemon
|
|
|
|
Run as daemon. This means the application will fork itself
|
|
and doesn't need to be started with \&.
|
|
|
|
.. option:: --auto-shutdown arg
|
|
|
|
Enable\/disable self\-shutdown because a master module shutdown.
|
|
This only works when messaging is enabled and the master
|
|
module sends a shutdown message \(enabled with \-\-start\-stop\-msg
|
|
for the master module\).
|
|
|
|
.. option:: --shutdown-master-module arg
|
|
|
|
Set the name of the master\-module used for auto\-shutdown.
|
|
This is the application name of the module actually
|
|
started. If symlinks are used, then it is the name of
|
|
the symlinked application.
|
|
|
|
.. option:: --shutdown-master-username arg
|
|
|
|
Set the name of the master\-username of the messaging
|
|
used for auto\-shutdown. If \"shutdown\-master\-module\" is
|
|
given as well, this parameter is ignored.
|
|
|
|
|
|
Verbosity
|
|
---------
|
|
|
|
.. option:: --verbosity arg
|
|
|
|
Verbosity level [0..4]. 0:quiet, 1:error, 2:warning, 3:info,
|
|
4:debug.
|
|
|
|
.. option:: -v, --v
|
|
|
|
Increase verbosity level \(may be repeated, eg. \-vv\).
|
|
|
|
.. option:: -q, --quiet
|
|
|
|
Quiet mode: no logging output.
|
|
|
|
.. option:: --component arg
|
|
|
|
Limit the logging to a certain component. This option can
|
|
be given more than once.
|
|
|
|
.. option:: -s, --syslog
|
|
|
|
Use syslog logging backend. The output usually goes to
|
|
\/var\/lib\/messages.
|
|
|
|
.. option:: -l, --lockfile arg
|
|
|
|
Path to lock file.
|
|
|
|
.. option:: --console arg
|
|
|
|
Send log output to stdout.
|
|
|
|
.. option:: --debug
|
|
|
|
Execute in debug mode.
|
|
Equivalent to \-\-verbosity\=4 \-\-console\=1 .
|
|
|
|
.. option:: --log-file arg
|
|
|
|
Use alternative log file.
|
|
|
|
|
|
Messaging
|
|
---------
|
|
|
|
.. option:: -u, --user arg
|
|
|
|
Overrides configuration parameter :confval:`connection.username`.
|
|
|
|
.. option:: -H, --host arg
|
|
|
|
Overrides configuration parameter :confval:`connection.server`.
|
|
|
|
.. option:: -t, --timeout arg
|
|
|
|
Overrides configuration parameter :confval:`connection.timeout`.
|
|
|
|
.. option:: -g, --primary-group arg
|
|
|
|
Overrides configuration parameter :confval:`connection.primaryGroup`.
|
|
|
|
.. option:: -S, --subscribe-group arg
|
|
|
|
A group to subscribe to.
|
|
This option can be given more than once.
|
|
|
|
.. option:: --content-type arg
|
|
|
|
Overrides configuration parameter :confval:`connection.contentType`.
|
|
|
|
.. option:: --start-stop-msg arg
|
|
|
|
Set sending of a start and a stop message.
|
|
|
|
|
|
Database
|
|
--------
|
|
|
|
.. option:: --db-driver-list
|
|
|
|
List all supported database drivers.
|
|
|
|
.. option:: -d, --database arg
|
|
|
|
The database connection string, format:
|
|
service:\/\/user:pwd\@host\/database.
|
|
\"service\" is the name of the database driver which
|
|
can be queried with \"\-\-db\-driver\-list\".
|
|
|
|
.. option:: --config-module arg
|
|
|
|
The config module to use.
|
|
|
|
.. option:: --inventory-db arg
|
|
|
|
Load the inventory from the given database or file, format:
|
|
[service:\/\/]location .
|
|
|
|
.. option:: --db-disable
|
|
|
|
Do not use the database at all
|
|
|
|
|
|
Dispatch
|
|
--------
|
|
|
|
.. option:: -e, --no-events
|
|
|
|
Do not send any event object which are ignored when reading
|
|
event parameters.
|
|
|
|
.. option:: -i, --input arg
|
|
|
|
File from which the content is dispatched to the messaging.
|
|
|
|
.. option:: -O, --operation arg
|
|
|
|
Merge and merge\-without\-remove require a database
|
|
connection. Both will read the object corresponding to the
|
|
sent object from the database and calculate the differences.
|
|
Merge\-without\-remove behaves like merge with the exception
|
|
that remove operations will be filtered out and no objects
|
|
in the database will be removed.
|
|
|
|
If add, update or remove is specified, then all objects in
|
|
XML are sent with the given operation regardless of their
|
|
existence in the database.
|
|
|
|
.. option:: --print-objects
|
|
|
|
Print names of routable objects.
|
|
|
|
.. option:: --print-routingtable
|
|
|
|
Print routing table.
|
|
|
|
.. option:: --routingtable arg
|
|
|
|
Specify routing table as comma separated list of object:group
|
|
pairs, e.g. \"Origin:LOCATION,Event:EVENT\". When an
|
|
object should be routed to a group and no table entry for
|
|
that particular class type is available, all parent objects
|
|
are checked for valid routing entries and the first found is
|
|
used. E.g. if only \"Origin:LOCATION\" is specified
|
|
but the input file contains also Arrivals which are child
|
|
objects of Origin then the routing entry of Origin is used
|
|
because of the parent\-child relationship between Origin and
|
|
Arrival.
|
|
|
|
.. option:: --test
|
|
|
|
Test mode. Does not send any object.
|
|
|
|
.. option:: --create-notifier
|
|
|
|
Do not send any object. All notifiers will be written to
|
|
standard output in XML format.
|
|
|