589 lines
11 KiB
ReStructuredText
589 lines
11 KiB
ReStructuredText
.. highlight:: rst
|
|
|
|
.. _caps2caps:
|
|
|
|
#########
|
|
caps2caps
|
|
#########
|
|
|
|
**caps2caps synchronizes CAPS servers in real-time**
|
|
|
|
|
|
Description
|
|
===========
|
|
|
|
*caps2caps* can connect two |appname| server instances to synchronize the data
|
|
in real time. When one server 1 fails and the
|
|
other one, server 2, continues to operate, the server 1 can back fill the data
|
|
as soon as it becomes alive again.
|
|
|
|
*caps2caps* can run on either side to pull the data from the other server or to
|
|
push the data to this server:
|
|
|
|
* For **pulling data** from a remote to a local server configure the input and the
|
|
output parameters with the remote and the local server, respectively.
|
|
* For **pushing data** from a local to a remote server configure the input and the
|
|
output parameters with the local and the remote server, respectively.
|
|
|
|
.. _fig-caps2caps:
|
|
|
|
.. figure:: media/caps2caps.png
|
|
:width: 18cm
|
|
:align: center
|
|
|
|
caps2caps instances connecting two |appname| servers pulling data from the
|
|
remote into the local server.
|
|
|
|
|
|
Examples
|
|
========
|
|
|
|
* Run caps2caps as daemon module.
|
|
|
|
#. Configure input and output hosts (:confval:`input.address`,
|
|
:confval:`output.address`) in caps2caps module configuration,
|
|
:file:`caps2caps.cfg`.
|
|
#. Enable and start caps2caps
|
|
|
|
.. code-block:: bash
|
|
|
|
seiscomp enable caps2caps
|
|
seiscomp start caps2caps
|
|
|
|
* Run caps2caps on demand in a terminal with specific, explicitly specifying
|
|
input and output hosts without encryption
|
|
|
|
.. code-block:: bash
|
|
|
|
caps2caps -I caps://inputServer:18002 -O caps://outputServer:18003
|
|
|
|
The same as above but with encrypted data transfer controlled by user name and
|
|
password
|
|
|
|
.. code-block:: bash
|
|
|
|
caps2caps -I capss://user:password@inputServer:18002 -O capss://user:password@inputServer:output:18003
|
|
|
|
* Pull or push data depending on module configuration but ignore the journal
|
|
file. This allows resending the data
|
|
|
|
.. code-block:: bash
|
|
|
|
caps2caps -j ""
|
|
|
|
|
|
|
|
Module Configuration
|
|
====================
|
|
|
|
| :file:`etc/defaults/global.cfg`
|
|
| :file:`etc/defaults/caps2caps.cfg`
|
|
| :file:`etc/global.cfg`
|
|
| :file:`etc/caps2caps.cfg`
|
|
| :file:`~/.seiscomp/global.cfg`
|
|
| :file:`~/.seiscomp/caps2caps.cfg`
|
|
|
|
caps2caps inherits :ref:`global options<global-configuration>`.
|
|
|
|
.. note::
|
|
|
|
Modules/plugins may require a license file. The default path to license
|
|
files is :file:`@DATADIR@/licenses/` which can be overridden by global
|
|
configuration of the parameter :confval:`gempa.licensePath`. Example: ::
|
|
|
|
gempa.licensePath = @CONFIGDIR@/licenses
|
|
|
|
|
|
|
|
.. confval:: streams
|
|
|
|
Type: *string*
|
|
|
|
Comma separated list of streams. Stream format: NET.STA.LOC.CHA.
|
|
Streams may contain wildcards
|
|
|
|
|
|
.. confval:: begin
|
|
|
|
Type: *string*
|
|
|
|
Start time of data time window, default is 'GMT'. Date time format:
|
|
[YYYY\-MM\-DD HH:MM:SS].
|
|
|
|
|
|
.. confval:: end
|
|
|
|
Type: *string*
|
|
|
|
End time of data time window. Date time format:
|
|
[YYYY\-MM\-DD HH:MM:SS].
|
|
|
|
|
|
.. confval:: maxDays
|
|
|
|
Default: ``-1``
|
|
|
|
Unit: *day*
|
|
|
|
Type: *int*
|
|
|
|
Maximum number of days to acquire regardless if the time window
|
|
is configured or read from journal. A value of 0 or less disables
|
|
the check.
|
|
|
|
|
|
.. confval:: days
|
|
|
|
Default: ``-1``
|
|
|
|
Unit: *day*
|
|
|
|
Type: *int*
|
|
|
|
Use to set the start time of data time window n days before the current time.
|
|
|
|
|
|
.. confval:: daysBefore
|
|
|
|
Default: ``-1``
|
|
|
|
Unit: *day*
|
|
|
|
Type: *int*
|
|
|
|
Use to set the end time of data time window n days before the current time.
|
|
|
|
|
|
.. confval:: timeWindowUpdateInterval
|
|
|
|
Default: ``-1``
|
|
|
|
Unit: *s*
|
|
|
|
Type: *int*
|
|
|
|
Sets the interval in seconds at which the relative request
|
|
time window defined by option days and\/or daysBefore is
|
|
updated.
|
|
Use a value less or equal zero to disable the update.
|
|
This feature is supported in archive mode only.
|
|
A typical use case is when data has to be transmitted
|
|
continuously with a time delay.
|
|
|
|
|
|
.. confval:: maxRealTimeGap
|
|
|
|
Default: ``-1``
|
|
|
|
Unit: *s*
|
|
|
|
Type: *int*
|
|
|
|
Sets the maximum real\-time data gap in seconds. This means,
|
|
if the start time of the requested time window of a channel
|
|
is before this value with respect to the current system time
|
|
then the request is split into a real\-time request starting
|
|
at system time \- marginRealTimeGap and a backfill request
|
|
from requested start time to time \- marginRealTimeGap.
|
|
That prioritizes real\-time data and backfills old data in
|
|
parallel.
|
|
|
|
|
|
.. confval:: marginRealTimeGap
|
|
|
|
Default: ``60``
|
|
|
|
Unit: *s*
|
|
|
|
Type: *int*
|
|
|
|
The time margin used to request real\-time data in combination
|
|
with maxRealTimeGap with respect to system time.
|
|
|
|
|
|
.. confval:: realtime
|
|
|
|
Default: ``true``
|
|
|
|
Type: *boolean*
|
|
|
|
Enable real\-time mode. Archived data is not fetched.
|
|
|
|
|
|
.. confval:: outOfOrder
|
|
|
|
Default: ``false``
|
|
|
|
Type: *boolean*
|
|
|
|
Enable out of order mode. Allows transfering data
|
|
which is not in timely order.
|
|
|
|
|
|
.. _input:
|
|
|
|
|
|
.. note::
|
|
|
|
**input.\***
|
|
*Configuration of data input host.*
|
|
|
|
|
|
.. confval:: input.address
|
|
|
|
Type: *string*
|
|
|
|
URL. Format: [[caps\|capss]:\/\/][user:pass\@]host[:port] .
|
|
|
|
|
|
.. _output:
|
|
|
|
|
|
.. note::
|
|
|
|
**output.\***
|
|
*Configuration of data output host.*
|
|
|
|
|
|
.. confval:: output.address
|
|
|
|
Default: ``localhost:18003``
|
|
|
|
Type: *string*
|
|
|
|
URL. Format: [[caps\|capss]:\/\/][user:pass\@]host[:port] .
|
|
|
|
|
|
.. confval:: output.bufferSize
|
|
|
|
Default: ``1048576``
|
|
|
|
Unit: *byte*
|
|
|
|
Type: *uint*
|
|
|
|
Size of the packet buffer.
|
|
|
|
|
|
.. confval:: output.backfillingBufferSize
|
|
|
|
Default: ``0``
|
|
|
|
Unit: *s*
|
|
|
|
Type: *uint*
|
|
|
|
Length of backfilling buffer which is a tool to mitigate
|
|
out\-of\-order data. Whenever a gap is detected, records
|
|
will be held in a buffer and not sent out. Records are
|
|
flushed from front to back if the buffer size is
|
|
exceeded. A value of 0 disables this feature.
|
|
|
|
|
|
.. confval:: output.mseed
|
|
|
|
Default: ``false``
|
|
|
|
Type: *boolean*
|
|
|
|
Enable Steim2 encoding for received RAW packets.
|
|
|
|
|
|
.. confval:: output.timeout
|
|
|
|
Default: ``60``
|
|
|
|
Unit: *s*
|
|
|
|
Type: *int*
|
|
|
|
Timeout when sending a packet. If the timeout expires,
|
|
the connection will be closed and re\-established.
|
|
|
|
|
|
.. _journal:
|
|
|
|
|
|
.. confval:: journal.file
|
|
|
|
Default: ``@ROOTDIR@/var/run/caps2caps/journal``
|
|
|
|
Type: *string*
|
|
|
|
File to store stream states.
|
|
|
|
|
|
.. confval:: journal.flush
|
|
|
|
Default: ``10``
|
|
|
|
Unit: *s*
|
|
|
|
Type: *uint*
|
|
|
|
Flush stream states to disk in the given interval.
|
|
|
|
|
|
.. confval:: journal.waitForAck
|
|
|
|
Default: ``60``
|
|
|
|
Unit: *s*
|
|
|
|
Type: *uint*
|
|
|
|
Wait when a sync has been forced, up to given seconds.
|
|
|
|
|
|
.. confval:: journal.waitForLastAck
|
|
|
|
Default: ``5``
|
|
|
|
Unit: *s*
|
|
|
|
Type: *uint*
|
|
|
|
Wait on shutdown to receive acknownledgement messages, up to the
|
|
given seconds.
|
|
|
|
|
|
|
|
|
|
Command-Line Options
|
|
====================
|
|
|
|
|
|
.. _Generic:
|
|
|
|
|
|
Generic
|
|
-------
|
|
|
|
.. option:: -h, --help
|
|
|
|
Show help message.
|
|
|
|
.. option:: -V, --version
|
|
|
|
Show version information.
|
|
|
|
.. option:: --config-file arg
|
|
|
|
Use alternative configuration file. When this option is
|
|
used the loading of all stages is disabled. Only the
|
|
given configuration file is parsed and used. To use
|
|
another name for the configuration create a symbolic
|
|
link of the application or copy it. Example:
|
|
scautopick \-> scautopick2.
|
|
|
|
.. option:: --plugins arg
|
|
|
|
Load given plugins.
|
|
|
|
.. option:: -D, --daemon
|
|
|
|
Run as daemon. This means the application will fork itself
|
|
and doesn't need to be started with \&.
|
|
|
|
|
|
.. _Verbosity:
|
|
|
|
|
|
Verbosity
|
|
---------
|
|
|
|
.. option:: --verbosity arg
|
|
|
|
Verbosity level [0..4]. 0:quiet, 1:error, 2:warning, 3:info,
|
|
4:debug.
|
|
|
|
.. option:: -v, --v
|
|
|
|
Increase verbosity level \(may be repeated, eg. \-vv\).
|
|
|
|
.. option:: -q, --quiet
|
|
|
|
Quiet mode: no logging output.
|
|
|
|
.. option:: --print-component arg
|
|
|
|
For each log entry print the component right after the
|
|
log level. By default the component output is enabled
|
|
for file output but disabled for console output.
|
|
|
|
.. option:: --component arg
|
|
|
|
Limit the logging to a certain component. This option can
|
|
be given more than once.
|
|
|
|
.. option:: -s, --syslog
|
|
|
|
Use syslog logging backend. The output usually goes to
|
|
\/var\/lib\/messages.
|
|
|
|
.. option:: -l, --lockfile arg
|
|
|
|
Path to lock file.
|
|
|
|
.. option:: --console arg
|
|
|
|
Send log output to stdout.
|
|
|
|
.. option:: --debug
|
|
|
|
Execute in debug mode.
|
|
Equivalent to \-\-verbosity\=4 \-\-console\=1 .
|
|
|
|
.. option:: --trace
|
|
|
|
Execute in trace mode.
|
|
Equivalent to \-\-verbosity\=4 \-\-console\=1 \-\-print\-component\=1
|
|
\-\-print\-context\=1 .
|
|
|
|
.. option:: --log-file arg
|
|
|
|
Use alternative log file.
|
|
|
|
|
|
.. _Input:
|
|
|
|
|
|
Input
|
|
-----
|
|
|
|
.. option:: -I, --input arg
|
|
|
|
Overrides configuration parameter :confval:`input.address`.
|
|
|
|
URL of data input host. Format:
|
|
[[caps\|capss]:\/\/][user:password\@]host[:port] .
|
|
|
|
.. option:: --max-real-time-gap
|
|
|
|
Maximum length of data gap after reconnecting. If exceeded,
|
|
a real\-time stream and backfilling stream will be created in
|
|
parallel. Setting this value will give highest priority to
|
|
real\-time streams, e.g., for rapid response systems.
|
|
|
|
|
|
.. _Streams:
|
|
|
|
|
|
Streams
|
|
-------
|
|
|
|
.. option:: -i, --inventory arg
|
|
|
|
Inventory XML defining the streams to add.
|
|
|
|
.. option:: -A, --add-stream arg
|
|
|
|
List of streamIDs [NET.STA.LOC.CHA] to add. Wildcards are
|
|
supported. Use comma\-separation without blanks for multiple
|
|
IDs.
|
|
|
|
.. option:: --begin arg
|
|
|
|
Start time of data request. Applied only on streams not
|
|
found in the journal. Format: 'YYYY\-MM\-DD hh:mm:ss.sss'.
|
|
|
|
.. option:: --end arg
|
|
|
|
End time of data request. Format: 'YYYY\-MM\-DD hh:mm:ss.sss'.
|
|
|
|
.. option:: --max-days arg
|
|
|
|
Unit: *day*
|
|
|
|
Maximum number of days to acquire regardless if the time
|
|
window is configured or read from journal. A value of 0 or
|
|
less disables the check.
|
|
|
|
.. option:: --days arg
|
|
|
|
Unit: *day*
|
|
|
|
Begin of data request time window given as days before current time.
|
|
Applied only on streams not found in the journal.
|
|
|
|
.. option:: --days-before arg
|
|
|
|
Unit: *day*
|
|
|
|
End of data request time window given as number of days
|
|
before current time.
|
|
|
|
|
|
.. _Mode:
|
|
|
|
|
|
Mode
|
|
----
|
|
|
|
.. option:: --archive
|
|
|
|
Disable real\-time mode. Only archived data is fetched and
|
|
missing records are ignored.
|
|
|
|
.. option:: --out-of-order
|
|
|
|
Use to enable out\-of\-order mode. Allows transfering data
|
|
which is not in timely order.
|
|
|
|
|
|
.. _Output:
|
|
|
|
|
|
Output
|
|
------
|
|
|
|
.. option:: -O, --output arg
|
|
|
|
Overrides configuration parameter :confval:`output.address`.
|
|
|
|
URL of data output host. Format:
|
|
[[caps\|capss]:\/\/][user:password\@]host[:port] .
|
|
|
|
.. option:: -b, --buffer-size arg
|
|
|
|
Size \(bytes\) of the journal buffer. If exceeded, a sync of
|
|
the journal is forced.
|
|
|
|
.. option:: --mseed
|
|
|
|
Enables Steim2 encoding for received RAW packets.
|
|
|
|
|
|
.. _Journal:
|
|
|
|
|
|
Journal
|
|
-------
|
|
|
|
.. option:: -j, --journal arg
|
|
|
|
Journal file to store stream states. Use an empty string to
|
|
ignore the journal file which will transfer the data
|
|
independent of previous transfers.
|
|
|
|
.. option:: --flush arg
|
|
|
|
Unit: *s*
|
|
|
|
Flush stream states to disk every given seconds.
|
|
|
|
.. option:: --waitForAck arg
|
|
|
|
Unit: *s*
|
|
|
|
Wait when a sync has been forced, up to the given seconds.
|
|
|
|
.. option:: -w, --waitForLastAck arg
|
|
|
|
Unit: *s*
|
|
|
|
Wait on shutdown to receive acknowledgment messages, up
|
|
to the given seconds.
|
|
|
|
|