You cannot select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
593 lines
17 KiB
Groff
593 lines
17 KiB
Groff
.\" Man page generated from reStructuredText.
|
|
.
|
|
.TH "SLARCHIVE" "1" "Jan 10, 2023" "5.3.0" "SeisComP"
|
|
.SH NAME
|
|
slarchive \- SeisComP Documentation
|
|
.
|
|
.nr rst2man-indent-level 0
|
|
.
|
|
.de1 rstReportMargin
|
|
\\$1 \\n[an-margin]
|
|
level \\n[rst2man-indent-level]
|
|
level margin: \\n[rst2man-indent\\n[rst2man-indent-level]]
|
|
-
|
|
\\n[rst2man-indent0]
|
|
\\n[rst2man-indent1]
|
|
\\n[rst2man-indent2]
|
|
..
|
|
.de1 INDENT
|
|
.\" .rstReportMargin pre:
|
|
. RS \\$1
|
|
. nr rst2man-indent\\n[rst2man-indent-level] \\n[an-margin]
|
|
. nr rst2man-indent-level +1
|
|
.\" .rstReportMargin post:
|
|
..
|
|
.de UNINDENT
|
|
. RE
|
|
.\" indent \\n[an-margin]
|
|
.\" old: \\n[rst2man-indent\\n[rst2man-indent-level]]
|
|
.nr rst2man-indent-level -1
|
|
.\" new: \\n[rst2man-indent\\n[rst2man-indent-level]]
|
|
.in \\n[rst2man-indent\\n[rst2man-indent-level]]u
|
|
..
|
|
.sp
|
|
\fBSeedLink client for data stream archiving\fP
|
|
.SH DESCRIPTION
|
|
.sp
|
|
slarchive connects to a SeedLink server, requests data streams and writes received
|
|
packets into directory/file structures (archives). The precise layout
|
|
of the directories and files is defined in a format string.
|
|
.sp
|
|
The implemented layouts are:
|
|
.INDENT 0.0
|
|
.IP \(bu 2
|
|
\fI\%SDS\fP: The SeisComP Data Structure, default in \fISeisComP\fP
|
|
.IP \(bu 2
|
|
BUD: Buffer of Uniform Data structure
|
|
.IP \(bu 2
|
|
DLOG: The old SeisComP/datalog structure for backwards compatibility
|
|
.UNINDENT
|
|
.sp
|
|
The duration for which the data are kept in archive is controlled by the bindings
|
|
parameter \fI\%keep\fP\&. slarchive itself does not clean the archive. For removing
|
|
old data execute \fB$SEISCOMP_ROOT/var/lib/slarchive/purge_datafiles\fP\&. A
|
|
regular clean\-up is suggested by
|
|
.INDENT 0.0
|
|
.INDENT 3.5
|
|
.sp
|
|
.nf
|
|
.ft C
|
|
seiscomp print crontab
|
|
.ft P
|
|
.fi
|
|
.UNINDENT
|
|
.UNINDENT
|
|
.sp
|
|
The resulting line, e.g.
|
|
.INDENT 0.0
|
|
.INDENT 3.5
|
|
.sp
|
|
.nf
|
|
.ft C
|
|
20 3 * * * /home/sysop/seiscomp/var/lib/slarchive/purge_datafiles >/dev/null 2>&1
|
|
.ft P
|
|
.fi
|
|
.UNINDENT
|
|
.UNINDENT
|
|
.sp
|
|
can be adjusted and added to crontab.
|
|
.SH BACKGROUND EXECUTION
|
|
.sp
|
|
When starting slarchive in \fISeisComP\fP as a daemon module in the background SDS is
|
|
considered and the packets are written without modification:
|
|
.INDENT 0.0
|
|
.INDENT 3.5
|
|
.sp
|
|
.nf
|
|
.ft C
|
|
$ seiscomp start slarchive
|
|
.ft P
|
|
.fi
|
|
.UNINDENT
|
|
.UNINDENT
|
|
.SH COMMAND-LINE EXECUTION
|
|
.sp
|
|
Writing to \fBother layouts\fP or to \fBmultiple archives\fP and other options are
|
|
supported when executing slarchive on the command line.
|
|
E.g. to write to more than one archive simply specify multiple format definitions
|
|
(or presets).
|
|
.sp
|
|
For more command\-line option read the help:
|
|
.INDENT 0.0
|
|
.INDENT 3.5
|
|
.sp
|
|
.nf
|
|
.ft C
|
|
$ slarchive \-h
|
|
.ft P
|
|
.fi
|
|
.UNINDENT
|
|
.UNINDENT
|
|
.SH MULTIPLE INSTANCES
|
|
.sp
|
|
slarchive allows generating aliases, e.g. for running in multiple instances with
|
|
different module and bindings configurations. For creating/removing aliases use the
|
|
seiscomp script, e.g.
|
|
.INDENT 0.0
|
|
.INDENT 3.5
|
|
.sp
|
|
.nf
|
|
.ft C
|
|
$ seiscomp alias create slarchive2 slarchive
|
|
.ft P
|
|
.fi
|
|
.UNINDENT
|
|
.UNINDENT
|
|
.SH SDS DEFINITION
|
|
.sp
|
|
SDS is the basic directory and file layout in \fISeisComP\fP for waveform archives. The
|
|
archive base directory is defined by \fI\%archive\fP\&. The SDS layout is defined
|
|
as:
|
|
.INDENT 0.0
|
|
.INDENT 3.5
|
|
.sp
|
|
.nf
|
|
.ft C
|
|
<SDSdir>
|
|
+ year
|
|
+ network code
|
|
+ station code
|
|
+ channel code
|
|
+ one file per day and location, e.g. NET.STA.LOC.CHAN.D.YEAR.DOY
|
|
.ft P
|
|
.fi
|
|
.UNINDENT
|
|
.UNINDENT
|
|
.sp
|
|
File example: \fB<SDSdir>/Year/NET/STA/CHAN.TYPE/NET.STA.LOC.CHAN.TYPE.YEAR.DAY\fP\&.
|
|
.TS
|
|
center;
|
|
|l|l|.
|
|
_
|
|
T{
|
|
Field
|
|
T} T{
|
|
Description
|
|
T}
|
|
_
|
|
T{
|
|
SDSdir
|
|
T} T{
|
|
Arbitrary base directory
|
|
T}
|
|
_
|
|
T{
|
|
YEAR
|
|
T} T{
|
|
4 digit YEAR
|
|
T}
|
|
_
|
|
T{
|
|
NET
|
|
T} T{
|
|
Network code/identifier, 1\-8 characters,
|
|
no spaces
|
|
T}
|
|
_
|
|
T{
|
|
STA
|
|
T} T{
|
|
Station code/identifier, 1\-8 characters,
|
|
no spaces
|
|
T}
|
|
_
|
|
T{
|
|
CHAN
|
|
T} T{
|
|
Channel code/identifier, 1\-8 characters,
|
|
no spaces
|
|
T}
|
|
_
|
|
T{
|
|
TYPE
|
|
T} T{
|
|
1 character, indicating the data type,
|
|
provided types are:
|
|
.nf
|
|
\fBD\fP Waveform data
|
|
\fBE\fP Detection data
|
|
\fBL\fP Log data
|
|
\fBT\fP Timing data
|
|
\fBC\fP Calibration data
|
|
\fBR\fP Response data
|
|
\fBO\fP Opaque data
|
|
.fi
|
|
T}
|
|
_
|
|
T{
|
|
LOC
|
|
T} T{
|
|
Location identifier, 1\-8 characters,
|
|
no spaces
|
|
T}
|
|
_
|
|
T{
|
|
DAY
|
|
T} T{
|
|
3 digit day of year, padded with zeros
|
|
T}
|
|
_
|
|
.TE
|
|
.SH MODULE CONFIGURATION
|
|
.sp
|
|
\fBNOTE:\fP
|
|
.INDENT 0.0
|
|
.INDENT 3.5
|
|
slarchive is a standalone module and does not inherit global options\&.
|
|
.UNINDENT
|
|
.UNINDENT
|
|
.nf
|
|
\fBetc/defaults/slarchive.cfg\fP
|
|
\fBetc/slarchive.cfg\fP
|
|
\fB~/.seiscomp/slarchive.cfg\fP
|
|
.fi
|
|
.sp
|
|
.INDENT 0.0
|
|
.TP
|
|
.B address
|
|
Default: \fB127.0.0.1\fP
|
|
.sp
|
|
Type: \fIstring\fP
|
|
.sp
|
|
Host of the Seedlink server to connect to. If the acquisition
|
|
is running on one system nothing needs to be changed.
|
|
.UNINDENT
|
|
.INDENT 0.0
|
|
.TP
|
|
.B port
|
|
Default: \fB18000\fP
|
|
.sp
|
|
Type: \fIint\fP
|
|
.sp
|
|
The port of the Seedlink server to connect to. If the acquisition
|
|
is running on one system this port must match the configured
|
|
local Seedlink port.
|
|
.UNINDENT
|
|
.INDENT 0.0
|
|
.TP
|
|
.B archive
|
|
Default: \fBvar/lib/archive\fP
|
|
.sp
|
|
Type: \fIstring\fP
|
|
.sp
|
|
Path to waveform archive where all data is stored. Relative paths
|
|
(as the default) are treated relative to the installation
|
|
directory ($SEISCOMP_ROOT).
|
|
.UNINDENT
|
|
.INDENT 0.0
|
|
.TP
|
|
.B buffer
|
|
Default: \fB1000\fP
|
|
.sp
|
|
Type: \fIint\fP
|
|
.sp
|
|
Number of records (512 byte units) to buffer before flushing to
|
|
disk.
|
|
.UNINDENT
|
|
.INDENT 0.0
|
|
.TP
|
|
.B delay
|
|
Default: \fB30\fP
|
|
.sp
|
|
Type: \fIint\fP
|
|
.sp
|
|
Unit: \fIs\fP
|
|
.sp
|
|
The network reconnect delay (in seconds) for the connection
|
|
to the SeedLink server. If the connection breaks for any
|
|
reason this will govern how soon a reconnection should be
|
|
attempted. The default value is 30 seconds.
|
|
.UNINDENT
|
|
.INDENT 0.0
|
|
.TP
|
|
.B networkTimeout
|
|
Default: \fB900\fP
|
|
.sp
|
|
Type: \fIint\fP
|
|
.sp
|
|
Unit: \fIs\fP
|
|
.sp
|
|
The network timeout (in seconds) for the connection to the
|
|
SeedLink server. If no data [or keep alive packets?] are received
|
|
in this time range the connection is closed and re\-established
|
|
(after the reconnect delay has expired). The default value is
|
|
600 seconds. A value of 0 disables the timeout.
|
|
.UNINDENT
|
|
.INDENT 0.0
|
|
.TP
|
|
.B idleTimeout
|
|
Default: \fB300\fP
|
|
.sp
|
|
Type: \fIint\fP
|
|
.sp
|
|
Unit: \fIs\fP
|
|
.sp
|
|
Timeout for closing idle data stream files in seconds. The idle
|
|
time of the data streams is only checked when some packets has
|
|
arrived. If no packets arrived no idle stream files will be
|
|
closed. There is no reason to change this parameter except for
|
|
the unusual cases where the process is running against an open
|
|
file number limit. Default is 300 seconds.
|
|
.UNINDENT
|
|
.INDENT 0.0
|
|
.TP
|
|
.B keepalive
|
|
Default: \fB0\fP
|
|
.sp
|
|
Type: \fIint\fP
|
|
.sp
|
|
Unit: \fIs\fP
|
|
.sp
|
|
Interval (in seconds) at which keepalive (heartbeat) packets
|
|
are sent to the server. Keepalive packets are only sent if
|
|
nothing is received within the interval. This requires a
|
|
Seedlink version >= 3.
|
|
.UNINDENT
|
|
.INDENT 0.0
|
|
.TP
|
|
.B validation.certs
|
|
Default: \fBvar/lib/certs\fP
|
|
.sp
|
|
Type: \fIstring\fP
|
|
.sp
|
|
Path to cerificate store where all certificates and CRLs are stored. Relative
|
|
paths(as the default) are treated relative to the installation
|
|
directory ($SEISCOMP_ROOT).
|
|
If the signature check is enabled slarchive loads all files at start. The store
|
|
uses the OpenSSl store format. From the offical OpenSSL documentation:
|
|
"The directory should contain one certificate or CRL per file in PEM format,
|
|
with a file name of the form hash.N for a certificate, or hash.rN for a CRL.
|
|
The .N or .rN suffix is a sequence number that starts at zero, and is incremented
|
|
consecutively for each certificate or CRL with the same hash value. Gaps in the
|
|
sequence numbers are not supported, it is assumed that there are no more objects
|
|
with the same hash beyond the first missing number in the sequence.The .N or .rN suffix
|
|
is a sequence number that starts at zero, and is incremented consecutively for
|
|
each certificate or CRL with the same hash value. Gaps in the sequence numbers
|
|
are not supported, it is assumed that there are no more objects with the same
|
|
hash beyond the first missing number in the sequence."
|
|
The hash value can be obtained as follows:
|
|
.sp
|
|
openssl x509 \-hash \-noout \-in <file>
|
|
.UNINDENT
|
|
.INDENT 0.0
|
|
.TP
|
|
.B validation.mode
|
|
Default: \fBignore\fP
|
|
.sp
|
|
Type: \fIstring\fP
|
|
.sp
|
|
Signatures are expected to be carried in blockette 2000
|
|
as opaque data. Modes:
|
|
.sp
|
|
ignore : Signatures will be ignored and no further actions
|
|
will be taken.
|
|
warning: Signatures will be checked and all received records
|
|
which do not carry a valid signature or no signature
|
|
at all will be logged with at warning level.
|
|
skip : All received records without a valid signature
|
|
will be ignored and will not be processed.
|
|
.UNINDENT
|
|
.SH BINDINGS PARAMETERS
|
|
.INDENT 0.0
|
|
.TP
|
|
.B selectors
|
|
Type: \fIlist:string\fP
|
|
.sp
|
|
List of stream selectors. If left empty all available
|
|
streams will be requested. See slarchive manpage for
|
|
more information.
|
|
.UNINDENT
|
|
.INDENT 0.0
|
|
.TP
|
|
.B keep
|
|
Default: \fB30\fP
|
|
.sp
|
|
Type: \fIint\fP
|
|
.sp
|
|
Unit: \fIday\fP
|
|
.sp
|
|
Number of days the data is kept in the archive. This
|
|
requires purge_datafile to be run as cronjob.
|
|
.UNINDENT
|
|
.SH COMMAND-LINE OPTIONS
|
|
.sp
|
|
\fBslarchive [OPTION]... [host][:][port]\fP
|
|
.sp
|
|
Address ([host][:][port]) is a required argument. It specifies the address
|
|
of the SeedLink server in host:port format. Either the host, port or both
|
|
can be omitted. If host is omitted then localhost is assumed,
|
|
i.e. \(aq:18000\(aq implies \(aqlocalhost:18000\(aq. If the port is omitted
|
|
then 18000 is assumed, i.e. \(aqlocalhost\(aq implies \(aqlocalhost:18000\(aq.
|
|
If only \(aq:\(aq is specified \(aqlocalhost:18000\(aq is assumed.
|
|
.INDENT 0.0
|
|
.TP
|
|
.B \-V
|
|
Print program version and exit.
|
|
.UNINDENT
|
|
.INDENT 0.0
|
|
.TP
|
|
.B \-h
|
|
Print program usage and exit.
|
|
.UNINDENT
|
|
.INDENT 0.0
|
|
.TP
|
|
.B \-v
|
|
Be more verbose. This flag can be used multiple times ("\-v \-v" or "\-vv")
|
|
for more verbosity. One flag: report basic handshaking (link configuration) details and
|
|
briefly report each packet received. Two flags: report the details of the handshaking,
|
|
each packet received and detailed connection diagnostics.
|
|
.UNINDENT
|
|
.INDENT 0.0
|
|
.TP
|
|
.B \-p
|
|
Print details of received Mini\-SEED data records. This flag can be used multiple times
|
|
("\-p \-p" or "\-pp") for more detail. One flag: a single summary line
|
|
for each data packet received. Two flags: details of the Mini\-SEED data records received,
|
|
including information from fixed header and 100/1000/1001 blockettes.
|
|
.UNINDENT
|
|
.INDENT 0.0
|
|
.TP
|
|
.B \-nd delay
|
|
The network reconnect delay (in seconds) for the connection to the SeedLink server.
|
|
If the connection breaks for any reason this will govern how soon a reconnection should
|
|
be attempted. The default value is 30 seconds.
|
|
.UNINDENT
|
|
.INDENT 0.0
|
|
.TP
|
|
.B \-nt timeout
|
|
The network timeout (in seconds) for the connection to the SeedLink server. If no data
|
|
[or keep alive packets?] are received in this time range the connection is closed and
|
|
re\-established (after the reconnect delay has expired). The default value is 600 seconds.
|
|
A value of 0 disables the timeout.
|
|
.UNINDENT
|
|
.INDENT 0.0
|
|
.TP
|
|
.B \-k keepalive
|
|
Interval (in seconds) at which keepalive (heartbeat) packets are sent to the server.
|
|
Keepalive packets are only sent if nothing is received within the interval. Requires SeedLink
|
|
version >= 3.
|
|
.UNINDENT
|
|
.INDENT 0.0
|
|
.TP
|
|
.B \-x statefile[:interval]
|
|
During client shutdown the last received sequence numbers and time stamps (start times)
|
|
for each data stream will be saved in this file. If this file exists upon startup the information
|
|
will be used to resume the data streams from the point at which they were stopped. In this way the
|
|
client can be stopped and started without data loss, assuming the data are still available on the
|
|
server. If an interval is specified the state will be saved every interval in that packets are
|
|
received. Otherwise the state will be saved only on normal program termination.
|
|
.UNINDENT
|
|
.INDENT 0.0
|
|
.TP
|
|
.B \-i timeout
|
|
Timeout for closing idle data stream files in seconds. The idle time of the data streams is
|
|
only checked when some packets has arrived. If no packets arrived no idle stream files will be
|
|
closed. There is no reason to change this parameter except for the unusual cases where the
|
|
process is running against an open file number limit. Default is 300 seconds.
|
|
.UNINDENT
|
|
.INDENT 0.0
|
|
.TP
|
|
.B \-d
|
|
Configure the connection in "dial\-up" mode. The remote server will close the connection when
|
|
it has sent all of the data in its buffers for the selected data streams. This is opposed to
|
|
the normal behavior of waiting indefinitely for data.
|
|
.UNINDENT
|
|
.INDENT 0.0
|
|
.TP
|
|
.B \-b
|
|
Configure the connection in "batch" mode.
|
|
.UNINDENT
|
|
.INDENT 0.0
|
|
.TP
|
|
.B \-Fi[:overlap]
|
|
Future check initially. Check the last Mini\-SEED data record in an existing archive file
|
|
and do not write new data to that file if it is older than a certain overlap. The default
|
|
overlap limit is 2 seconds; the overlap can be specified by appending a colon and the desired
|
|
overlap limit in seconds to the option. If the overlap is exceeded an error message will be
|
|
logged once for each time the file is opened. This option makes sense only for archive formats
|
|
where each unique data stream is written to a unique file (e.g. SDS format). If a data stream
|
|
is closed due to timeout (see option \-i) the initial future check will be preformed when the
|
|
file is re\-opened.
|
|
.UNINDENT
|
|
.INDENT 0.0
|
|
.TP
|
|
.B \-Fc[:overlap]
|
|
Future check continuously. Available only for archive Mini\-SEED data records. Check if the
|
|
first sample of the record is older than the last sample of the previous record for a given
|
|
archive file, within a certain overlap. The default overlap limit is 2 seconds; the overlap
|
|
can be specified by appending a colon and the desired overlap limit in seconds to the option.
|
|
If the overlap is exceeded an error message will be logged once until either a non\-overlapping
|
|
packet is received or a new archive file is used. This option only makes sense for archive
|
|
formats where each unique data stream is written to a unique file (e.g. SDS format).
|
|
.UNINDENT
|
|
.INDENT 0.0
|
|
.TP
|
|
.B \-A format
|
|
If specified, all received packets (Mini\-SEED records) will be appended to a directory/file
|
|
structure defined by format. All directories implied in the format string will be created if
|
|
necessary. The option may be used multiple times to write received packets to multiple archives.
|
|
See the section "archiving data".
|
|
.UNINDENT
|
|
.INDENT 0.0
|
|
.TP
|
|
.B \-SDS path
|
|
If specified, all received packets (Mini\-SEED records) will be saved into a Simple Data
|
|
Structure (SDS) dir/file structure starting at the specified directory. This directory and
|
|
all subdirectories will be created if necessary. This option is a preset of the \(aq\-A\(aq option.
|
|
The SDS dir/file structure is:
|
|
.sp
|
|
<SDSdir>/<YEAR>/<NET>/<STA>/<CHAN.TYPE>/NET.STA.LOC.CHAN.TYPE.YEAR.DAY
|
|
.sp
|
|
Details are mentioned later on.
|
|
.UNINDENT
|
|
.INDENT 0.0
|
|
.TP
|
|
.B \-BUD path
|
|
If specified, all received waveform data packets (Mini\-SEED data records) will be saved into
|
|
a Buffer of Uniform Data (BUD) dir/file structure starting at the specified directory.
|
|
This directory and all subdirectories will be created if necessary. This option is a preset
|
|
of the \(aq\-A\(aq option. The BUD dir/file structure is:
|
|
.sp
|
|
<BUDdir>/<NET>/<STA>/STA.NET.LOC.CHAN.YEAR.DAY
|
|
.UNINDENT
|
|
.INDENT 0.0
|
|
.TP
|
|
.B \-DLOG DLOGdir
|
|
If specified, all received packets (Mini\-SEED data records) will be saved into an old style
|
|
SeisComP/datalog dir/file structure starting at the specified directory. This directory and
|
|
all subdirectories will be created if necessary. This option is a preset of the \(aq\-A\(aq option.
|
|
The DLOG dir/file structure is:
|
|
.sp
|
|
<DLOGdir>/<STA>/[LOC.]<CHAN>.<TYPE>/STA.NET.CHAN.TYPE.YEAR.DAY.HHMM
|
|
.UNINDENT
|
|
.INDENT 0.0
|
|
.TP
|
|
.B \-l streamfile
|
|
The given file contains a list of streams. This option implies multi\-station mode.
|
|
The format of the stream list file is given below in the section "stream list file".
|
|
.UNINDENT
|
|
.INDENT 0.0
|
|
.TP
|
|
.B \-s selectors
|
|
Defining default selectors. If no multi\-station data streams are configured these selectors
|
|
will be used for uni\-station mode. Otherwise these selectors will be used when no selectors
|
|
are specified for a given stream with the \(aq\-S\(aq or \(aq\-l\(aq options.
|
|
.UNINDENT
|
|
.INDENT 0.0
|
|
.TP
|
|
.B \-S stream[:selectors]
|
|
The connection will be configured in multi\-station mode with optional SeedLink selectors
|
|
for each station, see examples below. Stream should be provided in NET_STA format. If no
|
|
selectors are provided for a given stream, the default selectors will be used, if defined.
|
|
.sp
|
|
Requires SeedLink >= 2.5.
|
|
.UNINDENT
|
|
.INDENT 0.0
|
|
.TP
|
|
.B \-tw start:[end]
|
|
Specifying a time window for the data streams that is applied by the server. The format
|
|
for both times is year,month,day,hour,min,sec; for example: "2002,08,05,14,00:2002,08,05,14,15,00".
|
|
The end time is optional but the colon must be present. If no end time is specified the
|
|
server will send data indefinitely. This option will override any saved state information.
|
|
.sp
|
|
Warning: time windowing might be disabled on the remote server.
|
|
.sp
|
|
Requires SeedLink >= 3.
|
|
.UNINDENT
|
|
.SH AUTHOR
|
|
gempa GmbH, GFZ Potsdam
|
|
.SH COPYRIGHT
|
|
gempa GmbH, GFZ Potsdam
|
|
.\" Generated by docutils manpage writer.
|
|
.
|