202 lines
8.8 KiB
HTML
202 lines
8.8 KiB
HTML
<!doctype html>
|
|
|
|
<html lang="en">
|
|
<head>
|
|
<meta charset="utf-8" /><meta name="viewport" content="width=device-width, initial-scale=1" />
|
|
|
|
<title>Upgrading — CAPS: Common Acquisition Protocol Server documentation</title>
|
|
<link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
|
|
<link rel="stylesheet" href="../_static/gempa.css" type="text/css" />
|
|
<link rel="stylesheet" type="text/css" href="../_static/pygments.css?v=fa44fd50" />
|
|
<link rel="stylesheet" type="text/css" href="../_static/gempa.css?v=c960eebf" />
|
|
<link rel="stylesheet" type="text/css" href="../_static/graphviz.css?v=eafc0fe6" />
|
|
<script type="text/javascript" src="../_static/gempa.js"></script>
|
|
<script type="text/javascript" id="documentation_options" data-url_root="../" src="../_static/documentation_options.js"></script>
|
|
<script data-url_root="../" id="documentation_options" src="../_static/documentation_options.js?v=b3ba4146"></script>
|
|
<script src="../_static/doctools.js?v=888ff710"></script>
|
|
<script src="../_static/sphinx_highlight.js?v=4825356b"></script>
|
|
<link rel="index" title="Index" href="../genindex.html" />
|
|
<link rel="search" title="Search" href="../search.html" />
|
|
<link rel="next" title="Glossary" href="glossary.html" />
|
|
<link rel="prev" title="Introduction" href="introduction.html" />
|
|
</head>
|
|
<body>
|
|
<div class="header">
|
|
<div class="container">
|
|
<img class="background" src="../_static/icon.png"/>
|
|
<div class="content">
|
|
<span class="title">CAPS: Common Acquisition Protocol Server </span>
|
|
<span class="version">2025.254#f0c77b29e</span></h1>
|
|
</div>
|
|
</div>
|
|
</div>
|
|
<div class="nav">
|
|
<div class="container">
|
|
<div class="content"><a class="pull-right" id="sidebar-toggle">TOC</a>
|
|
|
|
<div class="related" role="navigation" aria-label="related navigation">
|
|
<ul>
|
|
<li class="right">
|
|
<a href="../genindex.html" title="General Index"
|
|
accesskey="I">
|
|
index
|
|
</a>
|
|
</li>
|
|
<li class="right">
|
|
<a href="glossary.html" title="Glossary"
|
|
accesskey="N">
|
|
next
|
|
</a>
|
|
</li>
|
|
<li class="right">
|
|
<a href="introduction.html" title="Introduction"
|
|
accesskey="P">
|
|
previous
|
|
</a>
|
|
</li>
|
|
<li class="nav-item nav-item-0">
|
|
<a href="../index.html">Home</a>
|
|
</li>
|
|
</ul>
|
|
</div>
|
|
</div>
|
|
</div>
|
|
</div>
|
|
<div class="container">
|
|
<div class="fitted content" id="anchors-container">
|
|
<div class="body" role="main">
|
|
|
|
<section id="upgrading">
|
|
<span id="sec-caps-upgrading"></span><h1>Upgrading<a class="headerlink" href="#upgrading" title="Permalink to this heading">¶</a></h1>
|
|
<section id="new-file-format">
|
|
<h2>New file format<a class="headerlink" href="#new-file-format" title="Permalink to this heading">¶</a></h2>
|
|
<p>Starting from version 2021.048 CAPS introduces a new file storage format.
|
|
Actually the files are still compatible and chunk based but two new chunk types
|
|
were added. The upgrade itself should run smoothly without interruption but due
|
|
to the new file format all files must be converted before they can be read.
|
|
CAPS will do that on-the-fly whenever a file is opened for reading or writing.</p>
|
|
<p>That can cause performance drops until all files have been converted. But it
|
|
should not cause any outages.</p>
|
|
</section>
|
|
<section id="rationale">
|
|
<h2>Rationale<a class="headerlink" href="#rationale" title="Permalink to this heading">¶</a></h2>
|
|
<p>The time to store an out-of-order record in CAPS increased the more records
|
|
were stored already. This was caused by a linear search of the insert position.
|
|
The more records were stored the more records had to be checked and the more
|
|
file content had to be paged in system memory which is a slow operation.
|
|
In addition a second index file had to be maintained which requires an additional
|
|
open file descriptor per data file. As we also looked for way to reduce
|
|
disc fragmentation and to allow file size pre-allocation on any operating system
|
|
we decided to redesign the way how individual records are stored within a data
|
|
file. What we wanted was:</p>
|
|
<ul class="simple">
|
|
<li><p>Fast insert operations</p></li>
|
|
<li><p>Fast data retrieval</p></li>
|
|
<li><p>Portable file size pre-allocations</p></li>
|
|
<li><p>Efficient OS memory paging</p></li>
|
|
</ul>
|
|
<p>CAPS now implements a B+tree index per data file. No additional index file is
|
|
required. The index is maintained as additional chunks in the data file itself.
|
|
Furthermore CAPS maintains a meta chunk at the end of the file with information
|
|
about the logical and pyhsical file size, the index chunks and so on. If that
|
|
chunk is not available or is not valid then the data file will be re-scanned
|
|
and converted. This is what actually happens after an upgrade.</p>
|
|
<p>As a consequence, time window requests will be much faster with respect to
|
|
CPU time. Also file accesses are less frequent and reading file content overhead
|
|
while extracting arbitrary time windows is less than before.</p>
|
|
<p>As the time range stored in the data file is now part of the meta data a full
|
|
re-scan is not necessary when restarting CAPS without its archive log. When
|
|
dealing with many channels it will speed up re-scanning an archive a lot.</p>
|
|
</section>
|
|
<section id="manual-archive-conversion">
|
|
<h2>Manual archive conversion<a class="headerlink" href="#manual-archive-conversion" title="Permalink to this heading">¶</a></h2>
|
|
<p>If a controlled conversion of the archive files is desired then the following
|
|
procedure can be applied:</p>
|
|
<ol class="arabic">
|
|
<li><p>Stop caps</p>
|
|
<div class="highlight-sh notranslate"><div class="highlight"><pre><span></span>$<span class="w"> </span>seiscomp<span class="w"> </span>stop<span class="w"> </span>caps
|
|
</pre></div>
|
|
</div>
|
|
</li>
|
|
<li><p>Enter the configured archve directory</p>
|
|
<div class="highlight-sh notranslate"><div class="highlight"><pre><span></span>$<span class="w"> </span><span class="nb">cd</span><span class="w"> </span>seiscomp/var/lib/caps/archive
|
|
</pre></div>
|
|
</div>
|
|
</li>
|
|
<li><p>Check all files and trigger a conversion</p>
|
|
<div class="highlight-sh notranslate"><div class="highlight"><pre><span></span>$<span class="w"> </span>find<span class="w"> </span>-name<span class="w"> </span>*.data<span class="w"> </span>-exec<span class="w"> </span>rifftest<span class="w"> </span><span class="o">{}</span><span class="w"> </span>check<span class="w"> </span><span class="se">\;</span>
|
|
</pre></div>
|
|
</div>
|
|
</li>
|
|
<li><p>Start caps</p>
|
|
<div class="highlight-sh notranslate"><div class="highlight"><pre><span></span>$<span class="w"> </span>seiscomp<span class="w"> </span>start<span class="w"> </span>caps
|
|
</pre></div>
|
|
</div>
|
|
</li>
|
|
</ol>
|
|
<p>Depending on the size of the archive step 3 can take some time.</p>
|
|
</section>
|
|
</section>
|
|
|
|
|
|
<div id="anchors-bottom"></div>
|
|
</div>
|
|
|
|
<div class="sidebar" role="navigation" aria-label="main navigation">
|
|
<div id="anchors-top"></div>
|
|
<div id="anchors" class="content">
|
|
<div id="searchbox" style="display: none" role="search">
|
|
<h3 id="searchlabel">Quick search</h3>
|
|
<div class="searchformwrapper">
|
|
<form class="search" action="../search.html" method="get">
|
|
<input type="text" name="q" aria-labelledby="searchlabel" autocomplete="off" autocorrect="off" autocapitalize="off" spellcheck="false"/>
|
|
<input type="submit" value="Go" />
|
|
</form>
|
|
</div>
|
|
</div>
|
|
<script>document.getElementById('searchbox').style.display = "block"</script>
|
|
<div>
|
|
<h3><a href="../index.html">Table of Contents</a></h3>
|
|
<ul>
|
|
<li><a class="reference internal" href="#">Upgrading</a><ul>
|
|
<li><a class="reference internal" href="#new-file-format">New file format</a></li>
|
|
<li><a class="reference internal" href="#rationale">Rationale</a></li>
|
|
<li><a class="reference internal" href="#manual-archive-conversion">Manual archive conversion</a></li>
|
|
</ul>
|
|
</li>
|
|
</ul>
|
|
|
|
</div>
|
|
<div>
|
|
<h4>Previous topic</h4>
|
|
<p class="topless"><a href="introduction.html"
|
|
title="previous chapter">Introduction</a></p>
|
|
</div>
|
|
<div>
|
|
<h4>Next topic</h4>
|
|
<p class="topless"><a href="glossary.html"
|
|
title="next chapter">Glossary</a></p>
|
|
</div>
|
|
<div role="note" aria-label="source link">
|
|
<h3>This Page</h3>
|
|
<ul class="this-page-menu">
|
|
<li><a href="../_sources/base/upgrading.rst.txt"
|
|
rel="nofollow">Show Source</a></li>
|
|
</ul>
|
|
</div>
|
|
</div>
|
|
</div>
|
|
</div>
|
|
</div>
|
|
<div class="footer">
|
|
<div class="container">
|
|
<div class="content">
|
|
Release <b></b> version <b>2025.254#f0c77b29e</b>
|
|
<div class="copyright">
|
|
© Copyright <a href="https://www.gempa.de">2013, gempa GmbH.</a>
|
|
</div>
|
|
</div>
|
|
</div>
|
|
</div>
|
|
</body>
|
|
</html> |