|
|
|
|
<!DOCTYPE html>
|
|
|
|
|
|
|
|
|
|
<html >
|
|
|
|
|
<head>
|
|
|
|
|
<meta charset="utf-8" />
|
|
|
|
|
<title>Set up local waveform archiving — SeisComP Release documentation</title>
|
|
|
|
|
<link rel="stylesheet" href="../../_static/pygments.css" type="text/css" />
|
|
|
|
|
<link rel="stylesheet" href="../../_static/seiscomp.css" type="text/css" />
|
|
|
|
|
<link rel="stylesheet" type="text/css" href="../../_static/graphviz.css" />
|
|
|
|
|
<script type="text/javascript" src="../../_static/seiscomp.js"></script>
|
|
|
|
|
<script type="text/javascript" id="documentation_options" data-url_root="../../" src="../../_static/documentation_options.js"></script>
|
|
|
|
|
<script src="../../_static/jquery.js"></script>
|
|
|
|
|
<script src="../../_static/underscore.js"></script>
|
|
|
|
|
<script src="../../_static/doctools.js"></script>
|
|
|
|
|
<script src="../../_static/language_data.js"></script>
|
|
|
|
|
<link rel="shortcut icon" href="../../_static/favicon.ico"/>
|
|
|
|
|
<link rel="index" title="Index" href="../../genindex.html" />
|
|
|
|
|
<link rel="search" title="Search" href="../../search.html" />
|
|
|
|
|
<link rel="next" title="Configure real-time processing" href="processing.html" />
|
|
|
|
|
<link rel="prev" title="Get real-time data from a remote Seedlink server (single station)" href="waveforms.html" />
|
|
|
|
|
</head>
|
|
|
|
|
<body>
|
|
|
|
|
<div class="header">
|
|
|
|
|
<div class="container">
|
|
|
|
|
<div class="brand">
|
|
|
|
|
<img class="logo" src="../../_static/brands/seiscomp/text/white.svg"/>
|
|
|
|
|
<!-- span class="title">SeisComP Release</span -->
|
|
|
|
|
<span class="version">5.3.0</span>
|
|
|
|
|
</div>
|
|
|
|
|
</div>
|
|
|
|
|
</div>
|
|
|
|
|
<div class="nav">
|
|
|
|
|
<div class="container">
|
|
|
|
|
<div class="content"><a class="pull-right" id="sidebar-toggle">TOC</a>
|
|
|
|
|
|
|
|
|
|
<div class="related" role="navigation" aria-label="related navigation">
|
|
|
|
|
<ul>
|
|
|
|
|
<li class="right">
|
|
|
|
|
<a href="../../genindex.html" title="General Index"
|
|
|
|
|
accesskey="I">
|
|
|
|
|
index
|
|
|
|
|
</a>
|
|
|
|
|
</li>
|
|
|
|
|
<li class="right">
|
|
|
|
|
<a href="processing.html" title="Configure real-time processing"
|
|
|
|
|
accesskey="N">
|
|
|
|
|
next
|
|
|
|
|
</a>
|
|
|
|
|
</li>
|
|
|
|
|
<li class="right">
|
|
|
|
|
<a href="waveforms.html" title="Get real-time data from a remote Seedlink server (single station)"
|
|
|
|
|
accesskey="P">
|
|
|
|
|
previous
|
|
|
|
|
</a>
|
|
|
|
|
</li>
|
|
|
|
|
<li class="nav-item nav-item-0">
|
|
|
|
|
<a href="../../index.html">Home</a>
|
|
|
|
|
</li>
|
|
|
|
|
<li class="nav-item nav-item-1">
|
|
|
|
|
<a href="../tutorials.html" accesskey="U">Tutorials</a>
|
|
|
|
|
</li>
|
|
|
|
|
</ul>
|
|
|
|
|
</div>
|
|
|
|
|
</div>
|
|
|
|
|
</div>
|
|
|
|
|
</div>
|
|
|
|
|
<div class="container">
|
|
|
|
|
<div class="fitted content" id="anchors-container">
|
|
|
|
|
<div class="body" role="main">
|
|
|
|
|
|
|
|
|
|
<div class="section" id="set-up-local-waveform-archiving">
|
|
|
|
|
<span id="tutorials-archiving"></span><h1>Set up local waveform archiving<a class="headerlink" href="#set-up-local-waveform-archiving" title="Permalink to this headline">¶</a></h1>
|
|
|
|
|
<p>You will …</p>
|
|
|
|
|
<ul class="simple">
|
|
|
|
|
<li><p>Set up <a class="reference internal" href="../../apps/slarchive.html#slarchive"><span class="std std-ref">slarchive</span></a> with its necessary bindings</p></li>
|
|
|
|
|
<li><p>Set up <cite>purge_datafiles</cite> in crontab</p></li>
|
|
|
|
|
</ul>
|
|
|
|
|
<p>Pre-requisites for this tutorial:</p>
|
|
|
|
|
<ul class="simple">
|
|
|
|
|
<li><p>Have SeisComP installed</p></li>
|
|
|
|
|
<li><p>Tutorial on <a class="reference internal" href="addstation.html#tutorials-addstation"><span class="std std-ref">adding a new station</span></a></p></li>
|
|
|
|
|
<li><p>Tutorial on <a class="reference internal" href="geofon_waveforms.html#tutorials-geofon-waveforms"><span class="std std-ref">real-time data acquisition</span></a>
|
|
|
|
|
so that you have local data for GE stations.
|
|
|
|
|
Alternatively you may already obtain real-time waveform data from
|
|
|
|
|
somewhere else (see <a class="reference internal" href="waveforms.html#tutorials-waveforms"><span class="std std-ref">Get real-time data from a remote Seedlink server (single station)</span></a>).</p></li>
|
|
|
|
|
</ul>
|
|
|
|
|
<p>Afterwards/Results/Outcomes:</p>
|
|
|
|
|
<ul class="simple">
|
|
|
|
|
<li><p>Save real-time data in a local archive for later processing.</p></li>
|
|
|
|
|
<li><p>See <a class="reference internal" href="../glossary.html#term-miniseed"><span class="xref std std-term">miniSEED</span></a> day files for GE stations in your local <a class="reference internal" href="../concepts/waveformarchives.html#concepts-waveformarchives"><span class="std std-ref">waveform archive</span></a>.</p></li>
|
|
|
|
|
</ul>
|
|
|
|
|
<p>Time range estimate:</p>
|
|
|
|
|
<ul class="simple">
|
|
|
|
|
<li><p>5 minutes</p></li>
|
|
|
|
|
</ul>
|
|
|
|
|
<p>Related tutorial(s):</p>
|
|
|
|
|
<ul class="simple">
|
|
|
|
|
<li><p>Tutorial on <a class="reference internal" href="servefdsnws.html#tutorials-servefdsnws"><span class="std std-ref">Enable local FDSNWS server</span></a></p></li>
|
|
|
|
|
<li><p>Tutorial on <a class="reference internal" href="waveformplayback.html#tutorials-waveformplayback"><span class="std std-ref">Play back archived waveforms</span></a></p></li>
|
|
|
|
|
</ul>
|
|
|
|
|
<hr class="docutils" />
|
|
|
|
|
<p><strong>Motivation</strong>:
|
|
|
|
|
Without activating archiving, your local Seedlink server
|
|
|
|
|
will only keep waveforms for a short time.
|
|
|
|
|
This makes it hard to review old events, for example.</p>
|
|
|
|
|
<p>In this example, we’ll arrange for keeping waveforms for one week.
|
|
|
|
|
Before starting, you’ll need bindings for your stations;
|
|
|
|
|
see the tutorials <a class="reference internal" href="geofon_waveforms.html#tutorials-geofon-waveforms"><span class="std std-ref">Add real-time stations from GEOFON</span></a>
|
|
|
|
|
or <a class="reference internal" href="waveforms.html#tutorials-waveforms"><span class="std std-ref">Get real-time data from a remote Seedlink server (single station)</span></a>.</p>
|
|
|
|
|
<p>The <strong class="program">slarchive</strong> collects data and archives it
|
|
|
|
|
locally using a <a class="reference internal" href="../glossary.html#term-sds"><span class="xref std std-term">SDS</span></a> file system structure of
|
|
|
|
|
nested subdirectories and systematically named files.</p>
|
|
|
|
|
<div class="section" id="in-scconfig">
|
|
|
|
|
<h2>In scconfig<a class="headerlink" href="#in-scconfig" title="Permalink to this headline">¶</a></h2>
|
|
|
|
|
<ol class="arabic">
|
|
|
|
|
<li><p>Under the Modules tab, go to Acquisition, and select <strong class="program">slarchive</strong>.
|
|
|
|
|
Here you can see the default parameters used.
|
|
|
|
|
By default, <strong class="program">slarchive</strong> connects to your local Seedlink server,
|
|
|
|
|
and archives to your local disk.</p></li>
|
|
|
|
|
<li><p>Under the System tab, select the line for <strong class="program">slarchive</strong>, and click
|
|
|
|
|
“Enable module(s)” button at the top.</p></li>
|
|
|
|
|
<li><p>Under Bindings:
|
|
|
|
|
On RHS right-click “slarchive” to add an slarchive profile.
|
|
|
|
|
Name it ‘week’, to keep waveforms for 7 days, and click ‘Ok’.
|
|
|
|
|
The new profile appears in the (bottom right corner of <strong class="program">scconfig</strong>.
|
|
|
|
|
Double click on the profile to open its settings.
|
|
|
|
|
Unlock the box labeled “keep”, and change the default from 30 to 7.</p>
|
|
|
|
|
<p>Once you have a binding profile, drag it over all the stations it
|
|
|
|
|
should apply to, under “Networks” on the left-hand side of the
|
|
|
|
|
bindings tool.</p>
|
|
|
|
|
</li>
|
|
|
|
|
</ol>
|
|
|
|
|
<div class="admonition warning">
|
|
|
|
|
<p class="admonition-title">Warning</p>
|
|
|
|
|
<p>The name ‘week’ is just a label.
|
|
|
|
|
Its functionality comes from changing the value of the <cite>keep</cite> parameter.
|
|
|
|
|
Changing the <em>name</em> of a binding profile does not change its function.</p>
|
|
|
|
|
</div>
|
|
|
|
|
<div class="admonition note">
|
|
|
|
|
<p class="admonition-title">Note</p>
|
|
|
|
|
<p>You can also choose which channels should be archived,
|
|
|
|
|
using the “<a class="reference internal" href="../../apps/slarchive.html#confval-selectors"><code class="xref std std-confval docutils literal notranslate"><span class="pre">selectors</span></code></a>” box.
|
|
|
|
|
For instance, you may collect data at several sample rates,
|
|
|
|
|
and only wish to archive the highest rate.
|
|
|
|
|
If you collect LH, BH, HH streams at 0.1, 20, and 100 samples
|
|
|
|
|
per second, respectively, you might retain only the HH streams,
|
|
|
|
|
by setting “<a class="reference internal" href="../../apps/slarchive.html#confval-selectors"><code class="xref std std-confval docutils literal notranslate"><span class="pre">selectors</span></code></a>” to “HH”.</p>
|
|
|
|
|
</div>
|
|
|
|
|
<ol class="arabic simple">
|
|
|
|
|
<li><p>Then return to System, and click ‘Update configuration’.
|
|
|
|
|
Make sure the <strong class="program">slarchive</strong> module, or no module, is selected.</p></li>
|
|
|
|
|
<li><p>Restart <strong class="program">slarchive</strong>.</p></li>
|
|
|
|
|
<li><p>Adjust the <a class="reference internal" href="../concepts/recordstream.html#concepts-recordstream"><span class="std std-ref">RecordStream</span></a> for making use of the archived waveforms
|
|
|
|
|
from within a <a class="reference internal" href="../glossary.html#term-gui"><span class="xref std std-term">GUI</span></a> or automatic data processing modules.</p></li>
|
|
|
|
|
</ol>
|
|
|
|
|
</div>
|
|
|
|
|
<div class="section" id="command-line">
|
|
|
|
|
<h2>Command line<a class="headerlink" href="#command-line" title="Permalink to this headline">¶</a></h2>
|
|
|
|
|
<p>You will need to edit each of your top-level key files to refer to
|
|
|
|
|
a new binding profile.
|
|
|
|
|
e.g.:</p>
|
|
|
|
|
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>$ cd ~/seiscomp/etc/key
|
|
|
|
|
$ vi station_GR_CLL
|
|
|
|
|
</pre></div>
|
|
|
|
|
</div>
|
|
|
|
|
<p>Add the line <cite>slarchive:week</cite> to whatever lines are already there.
|
|
|
|
|
Afterwards it will look something like this:</p>
|
|
|
|
|
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="c1"># Binding references</span>
|
|
|
|
|
<span class="k">global</span><span class="p">:</span><span class="n">BH</span>
|
|
|
|
|
<span class="n">scautopick</span><span class="p">:</span><span class="n">default</span>
|
|
|
|
|
<span class="n">seedlink</span><span class="p">:</span><span class="n">geofon</span>
|
|
|
|
|
<span class="n">slarchive</span><span class="p">:</span><span class="n">week</span>
|
|
|
|
|
</pre></div>
|
|
|
|
|
</div>
|
|
|
|
|
<p>Repeat this for the top-level key file of each station
|
|
|
|
|
you wish this binding to apply to.
|
|
|
|
|
Now create the binding profile in the key directory.
|
|
|
|
|
This is a file with a name corresponding to the binding profile name; here: ‘week’</p>
|
|
|
|
|
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>$ cd ~/seiscomp/etc/key
|
|
|
|
|
$ mkdir slarchive
|
|
|
|
|
$ vi slarchive/profile_week
|
|
|
|
|
# Number of days the data is kept in the archive. This requires purge_datafile
|
|
|
|
|
# to be run as cronjob.
|
|
|
|
|
keep = 7
|
|
|
|
|
|
|
|
|
|
$ seiscomp enable slarchive
|
|
|
|
|
$ seiscomp update-config slarchive
|
|
|
|
|
$ seiscomp restart slarchive
|
|
|
|
|
slarchive is not running
|
|
|
|
|
starting slarchive
|
|
|
|
|
</pre></div>
|
|
|
|
|
</div>
|
|
|
|
|
<div class="admonition note">
|
|
|
|
|
<p class="admonition-title">Note</p>
|
|
|
|
|
<p>Left unattended, your disk will eventually fill up with archived data.
|
|
|
|
|
To prevent this you will need a script like <cite>purge_database</cite>,
|
|
|
|
|
which is provided with SeisComP.
|
|
|
|
|
This can be run once per day using the <cite>cron</cite> feature of your system.
|
|
|
|
|
The command:</p>
|
|
|
|
|
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>$ seiscomp print crontab
|
|
|
|
|
</pre></div>
|
|
|
|
|
</div>
|
|
|
|
|
<p>will print a number of lines to the terminal.
|
|
|
|
|
Type <cite>crontab -e</cite> and insert these lines into the crontab file for your
|
|
|
|
|
user (typically <cite>sysop</cite>).
|
|
|
|
|
Exit your crontab editor.
|
|
|
|
|
Displaying your crontab should now show a line for <cite>purge_database</cite>.:</p>
|
|
|
|
|
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>$ crontab -l
|
|
|
|
|
20 3 * * * /home/sysop/seiscomp/var/lib/slarchive/purge_datafiles >/dev/null 2>&1
|
|
|
|
|
[There may be other lines too.]
|
|
|
|
|
</pre></div>
|
|
|
|
|
</div>
|
|
|
|
|
<p>This shows you that the <cite>purge_datafiles</cite> script
|
|
|
|
|
will run every day at 3:20 a.m.</p>
|
|
|
|
|
</div>
|
|
|
|
|
<div class="admonition note">
|
|
|
|
|
<p class="admonition-title">Note</p>
|
|
|
|
|
<p>If you examine the <cite>purge_datafiles</cite> script, you will see that all it does
|
|
|
|
|
is look for files with a last modified time older than a certain number
|
|
|
|
|
of days ago.
|
|
|
|
|
The number of days to keep can be set station-by-station using the
|
|
|
|
|
ARCH_KEEP feature.
|
|
|
|
|
A convenient way to do this for many stations is with
|
|
|
|
|
multiple binding profiles, one for each length of time desired.</p>
|
|
|
|
|
</div>
|
|
|
|
|
</div>
|
|
|
|
|
<div class="section" id="checking-archiving-is-functioning">
|
|
|
|
|
<h2>Checking archiving is functioning<a class="headerlink" href="#checking-archiving-is-functioning" title="Permalink to this headline">¶</a></h2>
|
|
|
|
|
<ul>
|
|
|
|
|
<li><p>If <strong class="program">seedlink</strong> is configured correctly, a new station’s streams
|
|
|
|
|
appears in output from <strong class="program">slinktool</strong>:</p>
|
|
|
|
|
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>$ slinktool -Q : | grep CLL
|
|
|
|
|
GR CLL HHZ D 2020/04/01 01:11:57.6649 - 2020/04/01 07:28:49.0299
|
|
|
|
|
GR CLL HHE D 2020/04/01 01:11:57.6649 - 2020/04/01 07:28:45.0299
|
|
|
|
|
GR CLL HHN D 2020/04/01 01:11:57.6649 - 2020/04/01 07:28:39.2299
|
|
|
|
|
</pre></div>
|
|
|
|
|
</div>
|
|
|
|
|
<p>This shows three streams being acquired from station ‘CLL’.
|
|
|
|
|
The second time shown is the time of the most recent data for each stream.</p>
|
|
|
|
|
</li>
|
|
|
|
|
<li><p>If <strong class="program">slarchive</strong> is configured correctly, waveform data for the
|
|
|
|
|
station appears in <strong class="program">slarchive</strong>’s SDS archive directory:</p>
|
|
|
|
|
<blockquote>
|
|
|
|
|
<div><div class="highlight-sh notranslate"><div class="highlight"><pre><span></span>$ ls -l seiscomp/var/lib/archive/2020/GR/CLL
|
|
|
|
|
total <span class="m">12</span>
|
|
|
|
|
drwxr-xr-x <span class="m">2</span> user user <span class="m">4096</span> Apr <span class="m">1</span> <span class="m">06</span>:30 HHE.D
|
|
|
|
|
drwxr-xr-x <span class="m">2</span> user user <span class="m">4096</span> Apr <span class="m">1</span> <span class="m">06</span>:30 HHN.D
|
|
|
|
|
drwxr-xr-x <span class="m">2</span> user user <span class="m">4096</span> Apr <span class="m">1</span> <span class="m">06</span>:30 HHZ.D
|
|
|
|
|
|
|
|
|
|
$ ls -l seiscomp/var/lib/archive/2020/GR/CLL/HHZ.D/
|
|
|
|
|
total <span class="m">12728</span>
|
|
|
|
|
-rw-r--r-- <span class="m">1</span> user user <span class="m">5492224</span> Mar <span class="m">31</span> <span class="m">00</span>:04 GR.CLL..BHZ.D.2020.090
|
|
|
|
|
-rw-r--r-- <span class="m">1</span> user user <span class="m">7531008</span> Apr <span class="m">1</span> <span class="m">00</span>:03 GR.CLL..BHZ.D.2020.091
|
|
|
|
|
</pre></div>
|
|
|
|
|
</div>
|
|
|
|
|
</div></blockquote>
|
|
|
|
|
</li>
|
|
|
|
|
</ul>
|
|
|
|
|
</div>
|
|
|
|
|
</div>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
<div id="anchors-bottom"></div>
|
|
|
|
|
</div>
|
|
|
|
|
|
|
|
|
|
<div class="sidebar" role="navigation" aria-label="main navigation">
|
|
|
|
|
<div id="anchors-top"></div>
|
|
|
|
|
<div id="anchors" class="content">
|
|
|
|
|
<div id="searchbox" style="display: none" role="search">
|
|
|
|
|
<h3 id="searchlabel">Quick search</h3>
|
|
|
|
|
<div class="searchformwrapper">
|
|
|
|
|
<form class="search" action="../../search.html" method="get">
|
|
|
|
|
<input type="text" name="q" aria-labelledby="searchlabel" />
|
|
|
|
|
<input type="submit" value="Go" />
|
|
|
|
|
</form>
|
|
|
|
|
</div>
|
|
|
|
|
</div>
|
|
|
|
|
<script>$('#searchbox').show(0);</script>
|
|
|
|
|
<h3><a href="../../index.html">Table of Contents</a></h3>
|
|
|
|
|
<ul>
|
|
|
|
|
<li><a class="reference internal" href="#">Set up local waveform archiving</a><ul>
|
|
|
|
|
<li><a class="reference internal" href="#in-scconfig">In scconfig</a></li>
|
|
|
|
|
<li><a class="reference internal" href="#command-line">Command line</a></li>
|
|
|
|
|
<li><a class="reference internal" href="#checking-archiving-is-functioning">Checking archiving is functioning</a></li>
|
|
|
|
|
</ul>
|
|
|
|
|
</li>
|
|
|
|
|
</ul>
|
|
|
|
|
|
|
|
|
|
<h4>Previous topic</h4>
|
|
|
|
|
<p class="topless"><a href="waveforms.html"
|
|
|
|
|
title="previous chapter">Get real-time data from a remote Seedlink server (single station)</a></p>
|
|
|
|
|
<h4>Next topic</h4>
|
|
|
|
|
<p class="topless"><a href="processing.html"
|
|
|
|
|
title="next chapter">Configure real-time processing</a></p>
|
|
|
|
|
<div role="note" aria-label="source link">
|
|
|
|
|
<h3>This Page</h3>
|
|
|
|
|
<ul class="this-page-menu">
|
|
|
|
|
<li><a href="../../_sources/base/tutorials/archiving.rst.txt"
|
|
|
|
|
rel="nofollow">Show Source</a></li>
|
|
|
|
|
</ul>
|
|
|
|
|
</div>
|
|
|
|
|
</div>
|
|
|
|
|
</div>
|
|
|
|
|
</div>
|
|
|
|
|
</div>
|
|
|
|
|
<div class="footer">
|
|
|
|
|
<div class="container">
|
|
|
|
|
<div class="horizontal layout content">
|
|
|
|
|
<a class="fade-in" href="https://www.gempa.de" target="_blank">
|
|
|
|
|
<img class="brand" src="../../_static/brands/gempa.svg"/>
|
|
|
|
|
</a>
|
|
|
|
|
<div class="stretched align-center fitted content">
|
|
|
|
|
<div>
|
|
|
|
|
Version <b>5.3.0</b> Release
|
|
|
|
|
</div>
|
|
|
|
|
<div class="copyright">
|
|
|
|
|
Copyright © gempa GmbH, GFZ Potsdam.
|
|
|
|
|
</div>
|
|
|
|
|
</div>
|
|
|
|
|
<a class="fade-in" href="https://www.gfz-potsdam.de" target="_blank">
|
|
|
|
|
<img class="brand" src="../../_static/brands/gfz.svg"/>
|
|
|
|
|
</a>
|
|
|
|
|
</div>
|
|
|
|
|
</div>
|
|
|
|
|
</div>
|
|
|
|
|
</body>
|
|
|
|
|
</html>
|