[seiscomp, scanloc] Install, add .gitignore
This commit is contained in:
364
share/doc/seiscomp/html/_sources/apps/scart.rst.txt
Normal file
364
share/doc/seiscomp/html/_sources/apps/scart.rst.txt
Normal file
@ -0,0 +1,364 @@
|
||||
.. highlight:: rst
|
||||
|
||||
.. _scart:
|
||||
|
||||
#####
|
||||
scart
|
||||
#####
|
||||
|
||||
**Import/export MiniSEED data to/from SDS archives.**
|
||||
|
||||
|
||||
Description
|
||||
===========
|
||||
|
||||
The archive tool scart reads and writes :term:`SDS` archives and files
|
||||
in miniSEED format and checks miniSEED archives or prints stream information.
|
||||
|
||||
* **Dump mode:** Create miniSEED files (multiplexed), e.g. for playbacks, from
|
||||
:term:`SDS` structured data (e.g. created by slarchive).
|
||||
* **Dump mode:** Play back records directly out of an SDS structure.
|
||||
* **Import mode:** Import multiplexed miniSEED files into a local SDS waveform
|
||||
archive.
|
||||
* **Import mode:** Import data using the :ref:`global_recordstream` interface
|
||||
into a local SDS waveform archive.
|
||||
* **Import mode:** Read data from any :ref:`global_recordstream` interface
|
||||
and dump it to file.
|
||||
* **Check mode:** Check an archive of miniSEED files for out-of-order records in
|
||||
files.
|
||||
|
||||
.. warning::
|
||||
|
||||
* When creating :term:`SDS` archives, scart simply appends the new records to
|
||||
existing ones. Multiple imports of the same data result in duplication.
|
||||
* Out-of-order imports of waveforms into a SDS archive result in out-of-order
|
||||
records which may not be processed. Clean your archive using :ref:`scmssort`.
|
||||
* **Before importing miniSEED data** into an SDS archive they must be sorted
|
||||
by time and duplicate records must be removed. Otherwise, the SDS archive
|
||||
may not be correctly readable by other modules. Therefore, combine scart
|
||||
with :ref:`scmssort` for multiplexing and removal of duplicates.
|
||||
|
||||
.. hint::
|
||||
|
||||
In dump and import mode output streams may be filtered by
|
||||
|
||||
* Time windows (:option:`-t`),
|
||||
* Network-station-location-channel (NSLC) lists (:option:`--nslc`) created,
|
||||
e.g., with
|
||||
|
||||
* :ref:`scinv` from inventories,
|
||||
* :ref:`scdumpcfg` from bindings configuration,
|
||||
* scart itself from other miniSEED files or achives.
|
||||
|
||||
In dump mode output streams may also be filtered by
|
||||
|
||||
* Time window - stream lists (:option:`--list`, dump mode) generated by
|
||||
:ref:`scevtstreams` for particular events.
|
||||
|
||||
Time strings may be given in
|
||||
* ISO time format, e.g., 2023-03-28T15:48:00
|
||||
* or the 'old' SeisComP time format with empty character between date and time, e.g.,
|
||||
'2023-03-28 15:48:00'.
|
||||
|
||||
When omitting seconds, minutes or hours values of zero are implied.
|
||||
|
||||
|
||||
.. _scart-config:
|
||||
|
||||
Configuration
|
||||
=============
|
||||
|
||||
scart can make use of :ref:`global_recordstream` implementations which are
|
||||
provided by additional plugins. For loading additional plugins, e.g. the *xyz*
|
||||
plugin create and configure :file:`scart.cfg`:
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
plugins = xyz
|
||||
|
||||
|
||||
Examples
|
||||
========
|
||||
|
||||
.. hint::
|
||||
|
||||
The usage of wildcards in place of network, station, location or channel code
|
||||
is allowed in many options (:option:`-n`, :option:`-c`, :option:`-l`,
|
||||
:option:`--list`, :option:`--nslc`) and follows these rules:
|
||||
|
||||
* Import mode: the wildcards are passed to the :ref:`global_recordstream` interface,
|
||||
that interprets them. Normally both "*" and "?" are supported by RecordStreams.
|
||||
* Dump mode: the wildcards are interpreted by scart command that supports "*" for
|
||||
network, station, location codes and "*", "?", "(", ")", "|" for channel code.
|
||||
|
||||
#. Extract data from the default :term:`SDS` archive in :file:`$SEISCOMP_ROOT/var/lib/archive`
|
||||
or from a local :term:`SDS` archive [SDS archive] into a miniSEED file :file:`file.mseed`
|
||||
and sort by end time of the records:
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
scart -dsvE -t '[start-time]~[end-time]' [SDS archive] > [file.mseed]
|
||||
scart -dsvE -t '[start-time]~[end-time]' > file.mseed
|
||||
scart -dsvE -t '[start-time]~[end-time]' -n '[NET1],[NET2]' > file.mseed
|
||||
scart -dsvE -t '[start-time]~[end-time]' -n '[NET]' -c '(E,H)H(1,2,3)' > file.mseed
|
||||
scart -dsvE -t '[start-time]~[end-time]' -n '[N1.S1.L1.C1],[N2.S2.L2.C2]' > file.mseed
|
||||
scart -dsvE -t '[start-time]~[end-time]' --nslc list.file > file.mseed
|
||||
scart -dsvE -t --list list.file > file.mseed
|
||||
|
||||
|
||||
It is possible to achieve the same result of the dump mode using a
|
||||
combination of the input mode and the :ref:`scmssort` command, which allows
|
||||
to read the input data from any supported :ref:`global_recordstream`,
|
||||
not only an SDS archive:
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
scart -I [record-stream] --list list.file --stdout | scmssort -u -E -v > file.mseed
|
||||
|
||||
.. note::
|
||||
|
||||
Sorting data is computational expensive but required for waveform playbacks.
|
||||
|
||||
#. Push miniSEED data from file :file:`file.mseed` or standard input
|
||||
(stdin) into a local :term:`SDS` archive or a file. Additionally, you may
|
||||
check if the records of archived files are correctly ordered, filter by time
|
||||
and/or with NSLC list and print the output streams:
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
scmssort -u -E [file.mseed] > sorted.mseed
|
||||
scart -I sorted.mseed --with-filecheck [SDS archive]
|
||||
|
||||
scart -I [file.mseed] -t '[start-time]~[end-time]' --print-streams --nslc list.file -o [out.mseed]
|
||||
|
||||
cat sorted.mseed | scart -I - [SDS archive]
|
||||
cat sorted.mseed | scart [SDS archive]
|
||||
|
||||
#. Collect data using the :ref:`global_recordstream` interface (e.g. FDSNWS server)
|
||||
and write to a miniSEED file or import it into a local :term:`SDS` archive. The
|
||||
data streams and the time spans can be defined in several ways. The data streams
|
||||
(:option:`--list`) can be automatically generated, e.g., by :ref:`scevtstreams`.
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
scart -I fdsnws://[server]:80 --list list.file [SDS archive]
|
||||
scart -I fdsnws://[server]:80 --list list.file -o file.mseed
|
||||
scart -I fdsnws://[server]:80 -t '[start-time]~[end-time]' --nslc list.file [SDS archive]
|
||||
scart -I fdsnws://[server]:80 -t '[start-time]~[end-time]' -n '[NET1],[NET2]' [SDS archive]
|
||||
scart -I fdsnws://[server]:80 -t '[start-time]~[end-time]' -n '[NET]' -c 'EH?' [SDS archive]
|
||||
scart -I fdsnws://[server]:80 -t '[start-time]~[end-time]' -n '[N1.S1.L1.C1],[N2.S2.L2.C2]' [SDS archive]
|
||||
|
||||
#. Check all files of an SDS archive or other directory structure for
|
||||
miniSEED files with out-of-order records:
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
scart --check [archive]
|
||||
|
||||
#. Print stream information from miniSEED files in archives or from
|
||||
:term:`RecordStream` without actually writing miniSEED data. In dump and
|
||||
import mode use the :option:`--test` if miniSEED data shall be read but not
|
||||
written.
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
scart --print-streams -I [miniSEED file] --test
|
||||
scart --print-streams -d -t [time span] --test [SDS archive]
|
||||
scart --print-streams --check [archive]
|
||||
|
||||
The output looks like this:
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
# streamID start end records samples samplingRate
|
||||
GE.RGN..BHZ 2022-12-08T15:34:41.895Z 2022-12-08T15:52:19.145Z 58 21145 20.0
|
||||
|
||||
where the header and the text body are printed to stderr.
|
||||
|
||||
|
||||
|
||||
Command-Line Options
|
||||
====================
|
||||
|
||||
.. program:: scart
|
||||
|
||||
:program:`scart [options] {archive-dir}`
|
||||
|
||||
The last option has to be the archive directory when dump mode is enabled.
|
||||
When no archive directory is explicitly given,
|
||||
\$SEISCOMP_ROOT\/var\/lib\/archive or the current directory
|
||||
is used depending on whether \$SEISCOMP_ROOT has been set or not.
|
||||
The default operation mode is import. That means that a multiplexed
|
||||
MiniSEED file or another record source such as ArcLink is used to import
|
||||
records into a SDS structure.
|
||||
|
||||
|
||||
Verbosity
|
||||
---------
|
||||
|
||||
.. option:: -v, --verbose
|
||||
|
||||
Verbose mode.
|
||||
|
||||
.. option:: -h, --help
|
||||
|
||||
Display a help message.
|
||||
|
||||
|
||||
Mode
|
||||
----
|
||||
|
||||
.. option:: --check
|
||||
|
||||
Check mode. Check all files
|
||||
in the given directory for erroneous miniSEED records.
|
||||
All sub\-directories are included. If no directory is given,
|
||||
the default SDS archive is scanned. Checks are only complete
|
||||
for files containing exactly one stream. More complete checks
|
||||
are made with scmssort.
|
||||
|
||||
.. option:: -d, --dump
|
||||
|
||||
Set export \(dump\) mode. Records are retrieved from an archive and
|
||||
written to standard output.
|
||||
|
||||
.. option:: -I
|
||||
|
||||
Import mode \(default\): Specify the recordstream URL to read
|
||||
the data from for archiving. When using any other
|
||||
recordstream than file, a stream list file is needed.
|
||||
Specifying \- implies file:\/\/\- \(stdin\). If no mode is
|
||||
explicitly specified, \-I file:\/\/\- is assumed.
|
||||
|
||||
|
||||
Processing
|
||||
----------
|
||||
|
||||
.. option:: -c channels
|
||||
|
||||
Channel filter to be applied to the data streams.
|
||||
Default for Dump: \"\(B\|E\|H\|M\|S\)\(D\|H\|L\|N\)\(E\|F\|N\|Z\|1\|2\|3\)\"
|
||||
Default for Import: \"\*\"
|
||||
|
||||
.. option:: -E
|
||||
|
||||
Dump mode: sort records according to their end time.
|
||||
Default: start time.
|
||||
|
||||
.. option:: --files count
|
||||
|
||||
Dump mode: Specify the number of file handles to cache.
|
||||
Default: 100.
|
||||
|
||||
.. option:: -i
|
||||
|
||||
Ignore records without data samples.
|
||||
|
||||
.. option:: -l, --list file
|
||||
|
||||
Import, dump mode: Use a stream list file with time windows instead
|
||||
of defined networks and channels \(\-n, \-c and \-t are ignored\).
|
||||
The list can be generated from events by scevtstreams. One
|
||||
line per stream. Line format: starttime;endtime;streamID
|
||||
The time format is the same as described in option '\-t'.
|
||||
|
||||
Example:
|
||||
|
||||
2019\-07\-17 02:00:00;2019\-07\-17 02:10:00;GR.CLL..BH?
|
||||
|
||||
.. option:: -m, --modify
|
||||
|
||||
Dump mode: Modify the record time for real time playback.
|
||||
The first record time is NOW. The relative time of
|
||||
successive records to the first one are kept.
|
||||
|
||||
.. option:: -n networks
|
||||
|
||||
Import, dump mode: Data stream selection as a comma separated list
|
||||
\"stream1,stream2,streamX\" where each stream can be NET or NET.STA
|
||||
or NET.STA.LOC or NET.STA.LOC.CHA.
|
||||
If CHA is omitted, it defaults to the value of \-c option.
|
||||
Default: \"\*\"
|
||||
|
||||
.. option:: --nslc file
|
||||
|
||||
Import, dump mode: Stream list file to be used instead of
|
||||
defined networks and channels \(\-n and \-c are ignored\)
|
||||
for filtering the data by the given streams. Dump mode:
|
||||
Use in combination with \-t\! One line per stream, line
|
||||
format: NET.STA.LOC.CHA
|
||||
|
||||
Example:
|
||||
|
||||
GR.CLL..BH?
|
||||
|
||||
.. option:: --rename rule
|
||||
|
||||
Import, dump mode: Rename stream data according to the provided
|
||||
rule\(s\). A rule is \"[match\-stream:]rename\-stream\" and match\-stream
|
||||
is optional. match\-stream and rename\-stream are in the
|
||||
\"NET.STA.LOC.CHA\" format. match\-stream supports special
|
||||
charactes \"?\" \"\*\" \"\|\" \"\(\" \"\)\". rename\-stream supports the
|
||||
special character \"\-\" that can be used in place of NET, STA,
|
||||
LOC, CHA codes with the meaning of not renaming those.
|
||||
\"\-\" can also be used as the last character in CHA code.
|
||||
Multiple rules can be provided as a comma separated list
|
||||
or by providing multiple \-\-rename options.
|
||||
|
||||
.. option:: -s, --sort
|
||||
|
||||
Dump mode: Sort records by [start\-]time. To sort records by their
|
||||
end time use \-E.
|
||||
|
||||
.. option:: --speed value
|
||||
|
||||
Dump mode: Specify the speed to dump the records. A value of 0 means
|
||||
no delay otherwise speed is a multiplier of the real time difference
|
||||
between the records. When feeding the records directly into the replay
|
||||
pipe a value of 1 \(real time\) is recommended.
|
||||
|
||||
.. option:: -t, --time-window timeWindow
|
||||
|
||||
Import, dump mode: Specify the time window \(as one properly
|
||||
quoted string\) to dump records for. Times are UTC and
|
||||
separated by a tilde \"\~\". To dump one hour of
|
||||
waveform data between 2008\/01\/01 00:00:00 and 2008\/01\/01
|
||||
01:00:00 use
|
||||
<\-t 2008\-01\-01T00:00:00\~2008\-01\-01T01:00:00>.
|
||||
|
||||
|
||||
Output
|
||||
------
|
||||
|
||||
.. option:: -o, --output
|
||||
|
||||
Dump, Import mode: Write data to given file instead of creating
|
||||
a SDS archive. Deactivates \-\-stdout. Deactivated by \-\-test.
|
||||
|
||||
.. option:: --print-streams
|
||||
|
||||
Print stream information only and exit. Works in import, dump
|
||||
and check mode. Output: NET.STA.LOC.CHA StartTime EndTime.
|
||||
|
||||
.. option:: --stdout
|
||||
|
||||
Import mode: Write to stdout instead of creating a SDS archive.
|
||||
Deactivated by \-\-test and \-\-output.
|
||||
|
||||
.. option:: --test
|
||||
|
||||
Test input only, deactivate all miniSEED output. This switch is
|
||||
useful for debugging and printing stream information with
|
||||
\-\-print\-streams.
|
||||
|
||||
.. option:: --with-filecheck
|
||||
|
||||
Import mode: Check all accessed files. Unsorted or unreadable
|
||||
files are reported to stderr. Checks are only complete
|
||||
for files containing exactly one stream. More complete
|
||||
checks are made with scmssort.
|
||||
|
||||
.. option:: --with-filename
|
||||
|
||||
Import mode: Print all accessed files to stderr after import.
|
||||
|
||||
Reference in New Issue
Block a user