You cannot select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

327 lines
16 KiB
HTML

<!DOCTYPE html>
<html >
<head>
<meta charset="utf-8" />
<title>Set up local waveform archiving &#8212; SeisComP Release documentation</title>
<link rel="stylesheet" href="../../_static/pygments.css" type="text/css" />
<link rel="stylesheet" href="../../_static/seiscomp.css" type="text/css" />
<link rel="stylesheet" type="text/css" href="../../_static/graphviz.css" />
<script type="text/javascript" src="../../_static/seiscomp.js"></script>
<script type="text/javascript" id="documentation_options" data-url_root="../../" src="../../_static/documentation_options.js"></script>
<script src="../../_static/jquery.js"></script>
<script src="../../_static/underscore.js"></script>
<script src="../../_static/doctools.js"></script>
<script src="../../_static/language_data.js"></script>
<link rel="shortcut icon" href="../../_static/favicon.ico"/>
<link rel="index" title="Index" href="../../genindex.html" />
<link rel="search" title="Search" href="../../search.html" />
<link rel="next" title="Configure real-time processing" href="processing.html" />
<link rel="prev" title="Get real-time data from a remote Seedlink server (single station)" href="waveforms.html" />
</head>
<body>
<div class="header">
<div class="container">
<div class="brand">
<img class="logo" src="../../_static/brands/seiscomp/text/white.svg"/>
<!-- span class="title">SeisComP Release</span -->
<span class="version">5.3.0</span>
</div>
</div>
</div>
<div class="nav">
<div class="container">
<div class="content"><a class="pull-right" id="sidebar-toggle">TOC</a>
<div class="related" role="navigation" aria-label="related navigation">
<ul>
<li class="right">
<a href="../../genindex.html" title="General Index"
accesskey="I">
index
</a>
</li>
<li class="right">
<a href="processing.html" title="Configure real-time processing"
accesskey="N">
next
</a>
</li>
<li class="right">
<a href="waveforms.html" title="Get real-time data from a remote Seedlink server (single station)"
accesskey="P">
previous
</a>
</li>
<li class="nav-item nav-item-0">
<a href="../../index.html">Home</a>
</li>
<li class="nav-item nav-item-1">
<a href="../tutorials.html" accesskey="U">Tutorials</a>
</li>
</ul>
</div>
</div>
</div>
</div>
<div class="container">
<div class="fitted content" id="anchors-container">
<div class="body" role="main">
<div class="section" id="set-up-local-waveform-archiving">
<span id="tutorials-archiving"></span><h1>Set up local waveform archiving<a class="headerlink" href="#set-up-local-waveform-archiving" title="Permalink to this headline"></a></h1>
<p>You will …</p>
<ul class="simple">
<li><p>Set up <a class="reference internal" href="../../apps/slarchive.html#slarchive"><span class="std std-ref">slarchive</span></a> with its necessary bindings</p></li>
<li><p>Set up <cite>purge_datafiles</cite> in crontab</p></li>
</ul>
<p>Pre-requisites for this tutorial:</p>
<ul class="simple">
<li><p>Have SeisComP installed</p></li>
<li><p>Tutorial on <a class="reference internal" href="addstation.html#tutorials-addstation"><span class="std std-ref">adding a new station</span></a></p></li>
<li><p>Tutorial on <a class="reference internal" href="geofon_waveforms.html#tutorials-geofon-waveforms"><span class="std std-ref">real-time data acquisition</span></a>
so that you have local data for GE stations.
Alternatively you may already obtain real-time waveform data from
somewhere else (see <a class="reference internal" href="waveforms.html#tutorials-waveforms"><span class="std std-ref">Get real-time data from a remote Seedlink server (single station)</span></a>).</p></li>
</ul>
<p>Afterwards/Results/Outcomes:</p>
<ul class="simple">
<li><p>Save real-time data in a local archive for later processing.</p></li>
<li><p>See <a class="reference internal" href="../glossary.html#term-miniseed"><span class="xref std std-term">miniSEED</span></a> day files for GE stations in your local <a class="reference internal" href="../concepts/waveformarchives.html#concepts-waveformarchives"><span class="std std-ref">waveform archive</span></a>.</p></li>
</ul>
<p>Time range estimate:</p>
<ul class="simple">
<li><p>5 minutes</p></li>
</ul>
<p>Related tutorial(s):</p>
<ul class="simple">
<li><p>Tutorial on <a class="reference internal" href="servefdsnws.html#tutorials-servefdsnws"><span class="std std-ref">Enable local FDSNWS server</span></a></p></li>
<li><p>Tutorial on <a class="reference internal" href="waveformplayback.html#tutorials-waveformplayback"><span class="std std-ref">Play back archived waveforms</span></a></p></li>
</ul>
<hr class="docutils" />
<p><strong>Motivation</strong>:
Without activating archiving, your local Seedlink server
will only keep waveforms for a short time.
This makes it hard to review old events, for example.</p>
<p>In this example, well arrange for keeping waveforms for one week.
Before starting, youll need bindings for your stations;
see the tutorials <a class="reference internal" href="geofon_waveforms.html#tutorials-geofon-waveforms"><span class="std std-ref">Add real-time stations from GEOFON</span></a>
or <a class="reference internal" href="waveforms.html#tutorials-waveforms"><span class="std std-ref">Get real-time data from a remote Seedlink server (single station)</span></a>.</p>
<p>The <strong class="program">slarchive</strong> collects data and archives it
locally using a <a class="reference internal" href="../glossary.html#term-sds"><span class="xref std std-term">SDS</span></a> file system structure of
nested subdirectories and systematically named files.</p>
<div class="section" id="in-scconfig">
<h2>In scconfig<a class="headerlink" href="#in-scconfig" title="Permalink to this headline"></a></h2>
<ol class="arabic">
<li><p>Under the Modules tab, go to Acquisition, and select <strong class="program">slarchive</strong>.
Here you can see the default parameters used.
By default, <strong class="program">slarchive</strong> connects to your local Seedlink server,
and archives to your local disk.</p></li>
<li><p>Under the System tab, select the line for <strong class="program">slarchive</strong>, and click
“Enable module(s)” button at the top.</p></li>
<li><p>Under Bindings:
On RHS right-click “slarchive” to add an slarchive profile.
Name it week, to keep waveforms for 7 days, and click Ok.
The new profile appears in the (bottom right corner of <strong class="program">scconfig</strong>.
Double click on the profile to open its settings.
Unlock the box labeled “keep”, and change the default from 30 to 7.</p>
<p>Once you have a binding profile, drag it over all the stations it
should apply to, under “Networks” on the left-hand side of the
bindings tool.</p>
</li>
</ol>
<div class="admonition warning">
<p class="admonition-title">Warning</p>
<p>The name week is just a label.
Its functionality comes from changing the value of the <cite>keep</cite> parameter.
Changing the <em>name</em> of a binding profile does not change its function.</p>
</div>
<div class="admonition note">
<p class="admonition-title">Note</p>
<p>You can also choose which channels should be archived,
using the “<a class="reference internal" href="../../apps/slarchive.html#confval-selectors"><code class="xref std std-confval docutils literal notranslate"><span class="pre">selectors</span></code></a>” box.
For instance, you may collect data at several sample rates,
and only wish to archive the highest rate.
If you collect LH, BH, HH streams at 0.1, 20, and 100 samples
per second, respectively, you might retain only the HH streams,
by setting “<a class="reference internal" href="../../apps/slarchive.html#confval-selectors"><code class="xref std std-confval docutils literal notranslate"><span class="pre">selectors</span></code></a>” to “HH”.</p>
</div>
<ol class="arabic simple">
<li><p>Then return to System, and click Update configuration.
Make sure the <strong class="program">slarchive</strong> module, or no module, is selected.</p></li>
<li><p>Restart <strong class="program">slarchive</strong>.</p></li>
<li><p>Adjust the <a class="reference internal" href="../concepts/recordstream.html#concepts-recordstream"><span class="std std-ref">RecordStream</span></a> for making use of the archived waveforms
from within a <a class="reference internal" href="../glossary.html#term-gui"><span class="xref std std-term">GUI</span></a> or automatic data processing modules.</p></li>
</ol>
</div>
<div class="section" id="command-line">
<h2>Command line<a class="headerlink" href="#command-line" title="Permalink to this headline"></a></h2>
<p>You will need to edit each of your top-level key files to refer to
a new binding profile.
e.g.:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>$ cd ~/seiscomp/etc/key
$ vi station_GR_CLL
</pre></div>
</div>
<p>Add the line <cite>slarchive:week</cite> to whatever lines are already there.
Afterwards it will look something like this:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="c1"># Binding references</span>
<span class="k">global</span><span class="p">:</span><span class="n">BH</span>
<span class="n">scautopick</span><span class="p">:</span><span class="n">default</span>
<span class="n">seedlink</span><span class="p">:</span><span class="n">geofon</span>
<span class="n">slarchive</span><span class="p">:</span><span class="n">week</span>
</pre></div>
</div>
<p>Repeat this for the top-level key file of each station
you wish this binding to apply to.
Now create the binding profile in the key directory.
This is a file with a name corresponding to the binding profile name; here: week</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>$ cd ~/seiscomp/etc/key
$ mkdir slarchive
$ vi slarchive/profile_week
# Number of days the data is kept in the archive. This requires purge_datafile
# to be run as cronjob.
keep = 7
$ seiscomp enable slarchive
$ seiscomp update-config slarchive
$ seiscomp restart slarchive
slarchive is not running
starting slarchive
</pre></div>
</div>
<div class="admonition note">
<p class="admonition-title">Note</p>
<p>Left unattended, your disk will eventually fill up with archived data.
To prevent this you will need a script like <cite>purge_database</cite>,
which is provided with SeisComP.
This can be run once per day using the <cite>cron</cite> feature of your system.
The command:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>$ seiscomp print crontab
</pre></div>
</div>
<p>will print a number of lines to the terminal.
Type <cite>crontab -e</cite> and insert these lines into the crontab file for your
user (typically <cite>sysop</cite>).
Exit your crontab editor.
Displaying your crontab should now show a line for <cite>purge_database</cite>.:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>$ crontab -l
20 3 * * * /home/sysop/seiscomp/var/lib/slarchive/purge_datafiles &gt;/dev/null 2&gt;&amp;1
[There may be other lines too.]
</pre></div>
</div>
<p>This shows you that the <cite>purge_datafiles</cite> script
will run every day at 3:20 a.m.</p>
</div>
<div class="admonition note">
<p class="admonition-title">Note</p>
<p>If you examine the <cite>purge_datafiles</cite> script, you will see that all it does
is look for files with a last modified time older than a certain number
of days ago.
The number of days to keep can be set station-by-station using the
ARCH_KEEP feature.
A convenient way to do this for many stations is with
multiple binding profiles, one for each length of time desired.</p>
</div>
</div>
<div class="section" id="checking-archiving-is-functioning">
<h2>Checking archiving is functioning<a class="headerlink" href="#checking-archiving-is-functioning" title="Permalink to this headline"></a></h2>
<ul>
<li><p>If <strong class="program">seedlink</strong> is configured correctly, a new stations streams
appears in output from <strong class="program">slinktool</strong>:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>$ slinktool -Q : | grep CLL
GR CLL HHZ D 2020/04/01 01:11:57.6649 - 2020/04/01 07:28:49.0299
GR CLL HHE D 2020/04/01 01:11:57.6649 - 2020/04/01 07:28:45.0299
GR CLL HHN D 2020/04/01 01:11:57.6649 - 2020/04/01 07:28:39.2299
</pre></div>
</div>
<p>This shows three streams being acquired from station CLL.
The second time shown is the time of the most recent data for each stream.</p>
</li>
<li><p>If <strong class="program">slarchive</strong> is configured correctly, waveform data for the
station appears in <strong class="program">slarchive</strong>s SDS archive directory:</p>
<blockquote>
<div><div class="highlight-sh notranslate"><div class="highlight"><pre><span></span>$ ls -l seiscomp/var/lib/archive/2020/GR/CLL
total <span class="m">12</span>
drwxr-xr-x <span class="m">2</span> user user <span class="m">4096</span> Apr <span class="m">1</span> <span class="m">06</span>:30 HHE.D
drwxr-xr-x <span class="m">2</span> user user <span class="m">4096</span> Apr <span class="m">1</span> <span class="m">06</span>:30 HHN.D
drwxr-xr-x <span class="m">2</span> user user <span class="m">4096</span> Apr <span class="m">1</span> <span class="m">06</span>:30 HHZ.D
$ ls -l seiscomp/var/lib/archive/2020/GR/CLL/HHZ.D/
total <span class="m">12728</span>
-rw-r--r-- <span class="m">1</span> user user <span class="m">5492224</span> Mar <span class="m">31</span> <span class="m">00</span>:04 GR.CLL..BHZ.D.2020.090
-rw-r--r-- <span class="m">1</span> user user <span class="m">7531008</span> Apr <span class="m">1</span> <span class="m">00</span>:03 GR.CLL..BHZ.D.2020.091
</pre></div>
</div>
</div></blockquote>
</li>
</ul>
</div>
</div>
<div id="anchors-bottom"></div>
</div>
<div class="sidebar" role="navigation" aria-label="main navigation">
<div id="anchors-top"></div>
<div id="anchors" class="content">
<div id="searchbox" style="display: none" role="search">
<h3 id="searchlabel">Quick search</h3>
<div class="searchformwrapper">
<form class="search" action="../../search.html" method="get">
<input type="text" name="q" aria-labelledby="searchlabel" />
<input type="submit" value="Go" />
</form>
</div>
</div>
<script>$('#searchbox').show(0);</script>
<h3><a href="../../index.html">Table of Contents</a></h3>
<ul>
<li><a class="reference internal" href="#">Set up local waveform archiving</a><ul>
<li><a class="reference internal" href="#in-scconfig">In scconfig</a></li>
<li><a class="reference internal" href="#command-line">Command line</a></li>
<li><a class="reference internal" href="#checking-archiving-is-functioning">Checking archiving is functioning</a></li>
</ul>
</li>
</ul>
<h4>Previous topic</h4>
<p class="topless"><a href="waveforms.html"
title="previous chapter">Get real-time data from a remote Seedlink server (single station)</a></p>
<h4>Next topic</h4>
<p class="topless"><a href="processing.html"
title="next chapter">Configure real-time processing</a></p>
<div role="note" aria-label="source link">
<h3>This Page</h3>
<ul class="this-page-menu">
<li><a href="../../_sources/base/tutorials/archiving.rst.txt"
rel="nofollow">Show Source</a></li>
</ul>
</div>
</div>
</div>
</div>
</div>
<div class="footer">
<div class="container">
<div class="horizontal layout content">
<a class="fade-in" href="https://www.gempa.de" target="_blank">
<img class="brand" src="../../_static/brands/gempa.svg"/>
</a>
<div class="stretched align-center fitted content">
<div>
Version <b>5.3.0</b> Release
</div>
<div class="copyright">
Copyright &copy; gempa GmbH, GFZ Potsdam.
</div>
</div>
<a class="fade-in" href="https://www.gfz-potsdam.de" target="_blank">
<img class="brand" src="../../_static/brands/gfz.svg"/>
</a>
</div>
</div>
</div>
</body>
</html>