597 lines
28 KiB
HTML
597 lines
28 KiB
HTML
<!doctype html>
|
||
|
||
<html lang="en">
|
||
<head>
|
||
<meta charset="utf-8" /><meta name="viewport" content="width=device-width, initial-scale=1" />
|
||
|
||
<title>Data Management — CAPS: Common Acquisition Protocol Server documentation</title>
|
||
<link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
|
||
<link rel="stylesheet" href="../_static/gempa.css" type="text/css" />
|
||
<link rel="stylesheet" type="text/css" href="../_static/pygments.css?v=fa44fd50" />
|
||
<link rel="stylesheet" type="text/css" href="../_static/gempa.css?v=c960eebf" />
|
||
<link rel="stylesheet" type="text/css" href="../_static/graphviz.css?v=eafc0fe6" />
|
||
<script type="text/javascript" src="../_static/gempa.js"></script>
|
||
<script type="text/javascript" id="documentation_options" data-url_root="../" src="../_static/documentation_options.js"></script>
|
||
<script data-url_root="../" id="documentation_options" src="../_static/documentation_options.js?v=b3ba4146"></script>
|
||
<script src="../_static/doctools.js?v=888ff710"></script>
|
||
<script src="../_static/sphinx_highlight.js?v=4825356b"></script>
|
||
<link rel="index" title="Index" href="../genindex.html" />
|
||
<link rel="search" title="Search" href="../search.html" />
|
||
<link rel="next" title="Execution and Automatic Startup" href="configuration.html" />
|
||
<link rel="prev" title="CAPS Server Application" href="server.html" />
|
||
</head>
|
||
<body>
|
||
<div class="header">
|
||
<div class="container">
|
||
<img class="background" src="../_static/icon.png"/>
|
||
<div class="content">
|
||
<span class="title">CAPS: Common Acquisition Protocol Server </span>
|
||
<span class="version">2025.254#f0c77b29e</span></h1>
|
||
</div>
|
||
</div>
|
||
</div>
|
||
<div class="nav">
|
||
<div class="container">
|
||
<div class="content"><a class="pull-right" id="sidebar-toggle">TOC</a>
|
||
|
||
<div class="related" role="navigation" aria-label="related navigation">
|
||
<ul>
|
||
<li class="right">
|
||
<a href="../genindex.html" title="General Index"
|
||
accesskey="I">
|
||
index
|
||
</a>
|
||
</li>
|
||
<li class="right">
|
||
<a href="configuration.html" title="Execution and Automatic Startup"
|
||
accesskey="N">
|
||
next
|
||
</a>
|
||
</li>
|
||
<li class="right">
|
||
<a href="server.html" title="CAPS Server Application"
|
||
accesskey="P">
|
||
previous
|
||
</a>
|
||
</li>
|
||
<li class="nav-item nav-item-0">
|
||
<a href="../index.html">Home</a>
|
||
</li>
|
||
<li class="nav-item nav-item-1">
|
||
<a href="server.html" accesskey="U">CAPS Server Application</a>
|
||
</li>
|
||
</ul>
|
||
</div>
|
||
</div>
|
||
</div>
|
||
</div>
|
||
<div class="container">
|
||
<div class="fitted content" id="anchors-container">
|
||
<div class="body" role="main">
|
||
|
||
<section id="data-management">
|
||
<span id="sec-archive"></span><h1>Data Management<a class="headerlink" href="#data-management" title="Permalink to this heading">¶</a></h1>
|
||
<p><a class="reference internal" href="glossary.html#term-CAPS"><span class="xref std std-term">CAPS</span></a> uses the <a class="reference internal" href="glossary.html#term-SDS"><span class="xref std std-term">SDS</span></a> directory
|
||
structure for its archives shown in figure <a href="#fig-archive">5</a>. SDS organizes
|
||
the data in directories by year, network, station and channel.
|
||
This tree structure eases archiving of data. One complete year may be
|
||
moved to an external storage, e.g. a tape library.</p>
|
||
<figure class="align-default" id="id2">
|
||
<span id="fig-archive"></span><a class="reference internal image-reference" href="../_images/sds.png"><img alt="../_images/sds.png" src="../_images/sds.png" style="width: 12cm;" /></a>
|
||
<figcaption>
|
||
<p><span class="caption-text">SDS archive structure of a CAPS archive</span><a class="headerlink" href="#id2" title="Permalink to this image">¶</a></p>
|
||
</figcaption>
|
||
</figure>
|
||
<p>The data are stored in the channel directories. One file is created per sensor
|
||
location for each day of the year. File names take the form
|
||
<code class="file docutils literal notranslate"><span class="pre">$net.$sta.$loc.$cha.$year.$yday.data</span></code> with</p>
|
||
<ul class="simple">
|
||
<li><p><strong>net</strong>: network code, e.g. ‘II’</p></li>
|
||
<li><p><strong>sta</strong>: station code, e.g. ‘BFO’</p></li>
|
||
<li><p><strong>loc</strong>: sensor location code, e.g. ‘00’. Empty codes are supported</p></li>
|
||
<li><p><strong>cha</strong>: channel code, e.g. ‘BHZ’</p></li>
|
||
<li><p><strong>year</strong>: calender year, e.g. ‘2021’</p></li>
|
||
<li><p><strong>yday</strong>: day of the year starting with ‘000’ on 1 January</p></li>
|
||
</ul>
|
||
<div class="admonition note">
|
||
<p class="admonition-title">Note</p>
|
||
<p>In contrast to CAPS archives, in SDS archives created with
|
||
<a class="reference external" href="https://docs.gempa.de/seiscomp/current/apps/slarchive.html">slarchive</a>
|
||
the first day of the year, 1 January, is referred to by index ‘001’.</p>
|
||
</div>
|
||
<section id="file-format">
|
||
<span id="sec-caps-archive-file-format"></span><h2>File Format<a class="headerlink" href="#file-format" title="Permalink to this heading">¶</a></h2>
|
||
<p><a class="reference internal" href="glossary.html#term-CAPS"><span class="xref std std-term">CAPS</span></a> uses the <a class="reference external" href="http://de.wikipedia.org/wiki/Resource_Interchange_File_Format">RIFF</a> file format
|
||
for data storage. A RIFF file consists of <code class="docutils literal notranslate"><span class="pre">chunks</span></code>. Each chunk starts with a 8
|
||
byte chunk header followed by data. The first 4 bytes denote the chunk type, the
|
||
next 4 bytes the length of the following data block. Currently the following
|
||
chunk types are supported:</p>
|
||
<ul class="simple">
|
||
<li><p><strong>SID</strong> - stream ID header</p></li>
|
||
<li><p><strong>HEAD</strong> - data information header</p></li>
|
||
<li><p><strong>DATA</strong> - data block</p></li>
|
||
<li><p><strong>BPT</strong> - b-tree index page</p></li>
|
||
<li><p><strong>META</strong> - meta chunk of the entire file containing states and a checksum</p></li>
|
||
</ul>
|
||
<p>Figure <a href="#fig-file-one-day">6</a> shows the possible structure of an archive
|
||
file consisting of the different chunk types.</p>
|
||
<figure class="align-default" id="id3">
|
||
<span id="fig-file-one-day"></span><a class="reference internal image-reference" href="../_images/file_one_day.png"><img alt="../_images/file_one_day.png" src="../_images/file_one_day.png" style="width: 18cm;" /></a>
|
||
<figcaption>
|
||
<p><span class="caption-text">Possible structure of an archive file</span><a class="headerlink" href="#id3" title="Permalink to this image">¶</a></p>
|
||
</figcaption>
|
||
</figure>
|
||
<section id="sid-chunk">
|
||
<h3>SID Chunk<a class="headerlink" href="#sid-chunk" title="Permalink to this heading">¶</a></h3>
|
||
<p>A data file may start with a SID chunk which defines the stream id of the
|
||
data that follows in DATA chunks. In the absence of a SID chunk, the stream ID
|
||
is retrieved from the file name.</p>
|
||
<table class="docutils align-default">
|
||
<thead>
|
||
<tr class="row-odd"><th class="head"><p>content</p></th>
|
||
<th class="head"><p>type</p></th>
|
||
<th class="head"><p>bytes</p></th>
|
||
</tr>
|
||
</thead>
|
||
<tbody>
|
||
<tr class="row-even"><td><p>id=”SID”</p></td>
|
||
<td><p>char[4]</p></td>
|
||
<td><p>4</p></td>
|
||
</tr>
|
||
<tr class="row-odd"><td><p>chunkSize</p></td>
|
||
<td><p>int32</p></td>
|
||
<td><p>4</p></td>
|
||
</tr>
|
||
<tr class="row-even"><td><p>networkCode + ‘\0’</p></td>
|
||
<td><p>char*</p></td>
|
||
<td><p>len(networkCode) + 1</p></td>
|
||
</tr>
|
||
<tr class="row-odd"><td><p>stationCode + ‘\0’</p></td>
|
||
<td><p>char*</p></td>
|
||
<td><p>len(stationCode) + 1</p></td>
|
||
</tr>
|
||
<tr class="row-even"><td><p>locationCode + ‘\0’</p></td>
|
||
<td><p>char*</p></td>
|
||
<td><p>len(locationCode) + 1</p></td>
|
||
</tr>
|
||
<tr class="row-odd"><td><p>channelCode + ‘\0’</p></td>
|
||
<td><p>char*</p></td>
|
||
<td><p>len(channelCode) + 1</p></td>
|
||
</tr>
|
||
</tbody>
|
||
</table>
|
||
</section>
|
||
<section id="head-chunk">
|
||
<h3>HEAD Chunk<a class="headerlink" href="#head-chunk" title="Permalink to this heading">¶</a></h3>
|
||
<p>The HEAD chunk contains information about subsequent DATA chunks. It has a fixed
|
||
size of 15 bytes and is inserted under the following conditions:</p>
|
||
<ul class="simple">
|
||
<li><p>before the first data chunk (beginning of file)</p></li>
|
||
<li><p>packet type changed</p></li>
|
||
<li><p>unit of measurement changed</p></li>
|
||
</ul>
|
||
<table class="docutils align-default">
|
||
<thead>
|
||
<tr class="row-odd"><th class="head"><p>content</p></th>
|
||
<th class="head"><p>type</p></th>
|
||
<th class="head"><p>bytes</p></th>
|
||
</tr>
|
||
</thead>
|
||
<tbody>
|
||
<tr class="row-even"><td><p>id=”HEAD”</p></td>
|
||
<td><p>char[4]</p></td>
|
||
<td><p>4</p></td>
|
||
</tr>
|
||
<tr class="row-odd"><td><p>chunkSize (=7)</p></td>
|
||
<td><p>int32</p></td>
|
||
<td><p>4</p></td>
|
||
</tr>
|
||
<tr class="row-even"><td><p>version</p></td>
|
||
<td><p>int16</p></td>
|
||
<td><p>2</p></td>
|
||
</tr>
|
||
<tr class="row-odd"><td><p>packetType</p></td>
|
||
<td><p>char</p></td>
|
||
<td><p>1</p></td>
|
||
</tr>
|
||
<tr class="row-even"><td><p>unitOfMeasurement</p></td>
|
||
<td><p>char[4]</p></td>
|
||
<td><p>4</p></td>
|
||
</tr>
|
||
</tbody>
|
||
</table>
|
||
<p>The <code class="docutils literal notranslate"><span class="pre">packetType</span></code> entry refers to one of the supported types described in
|
||
section <a class="reference internal" href="#sec-packet-types"><span class="std std-ref">Packet Types</span></a>.</p>
|
||
</section>
|
||
<section id="data-chunk">
|
||
<h3>DATA Chunk<a class="headerlink" href="#data-chunk" title="Permalink to this heading">¶</a></h3>
|
||
<p>The DATA chunk contains the actually payload, which may be further structured
|
||
into header and data parts.</p>
|
||
<table class="docutils align-default">
|
||
<thead>
|
||
<tr class="row-odd"><th class="head"><p>content</p></th>
|
||
<th class="head"><p>type</p></th>
|
||
<th class="head"><p>bytes</p></th>
|
||
</tr>
|
||
</thead>
|
||
<tbody>
|
||
<tr class="row-even"><td><p>id=”DATA”</p></td>
|
||
<td><p>char[4]</p></td>
|
||
<td><p>4</p></td>
|
||
</tr>
|
||
<tr class="row-odd"><td><p>chunkSize</p></td>
|
||
<td><p>int32</p></td>
|
||
<td><p>4</p></td>
|
||
</tr>
|
||
<tr class="row-even"><td><p>data</p></td>
|
||
<td><p>char*</p></td>
|
||
<td><p>chunkSize</p></td>
|
||
</tr>
|
||
</tbody>
|
||
</table>
|
||
<p>Section <a class="reference internal" href="#sec-packet-types"><span class="std std-ref">Packet Types</span></a> describes the currently supported packet types.
|
||
Each packet type defines its own data structure. Nevertheless <a class="reference internal" href="glossary.html#term-CAPS"><span class="xref std std-term">CAPS</span></a>
|
||
requires each type to supply a <code class="docutils literal notranslate"><span class="pre">startTime</span></code> and <code class="docutils literal notranslate"><span class="pre">endTime</span></code> information for
|
||
each record in order to create seamless data streams. The <code class="docutils literal notranslate"><span class="pre">endTime</span></code> may be
|
||
stored explicitly or may be derived from <code class="docutils literal notranslate"><span class="pre">startTime</span></code>, <code class="docutils literal notranslate"><span class="pre">chunkSize</span></code>,
|
||
<code class="docutils literal notranslate"><span class="pre">dataType</span></code> and <code class="docutils literal notranslate"><span class="pre">samplingFrequency</span></code>.</p>
|
||
<p>In contrast to a data streams, <a class="reference internal" href="glossary.html#term-CAPS"><span class="xref std std-term">CAPS</span></a> also supports storing of individual
|
||
measurements. These measurements are indicated by setting the sampling frequency
|
||
to 1/0.</p>
|
||
</section>
|
||
<section id="bpt-chunk">
|
||
<h3>BPT Chunk<a class="headerlink" href="#bpt-chunk" title="Permalink to this heading">¶</a></h3>
|
||
<p>BPT chunks hold information about the file index. All data records are indexed
|
||
using a B+ tree. The index key is the tuple of start time and end time of each
|
||
data chunk to allow very fast time window lookup and to minimize disc accesses.
|
||
The value is a structure and holds the following information:</p>
|
||
<ul class="simple">
|
||
<li><p>File position of the format header</p></li>
|
||
<li><p>File position of the record data</p></li>
|
||
<li><p>Timestamp of record reception</p></li>
|
||
</ul>
|
||
<p>This chunk holds a single index tree page with a fixed size of 4kb
|
||
(4096 byte). More information about B+ trees can be found at
|
||
<a class="reference external" href="https://en.wikipedia.org/wiki/B%2B_tree">https://en.wikipedia.org/wiki/B%2B_tree</a>.</p>
|
||
</section>
|
||
<section id="meta-chunk">
|
||
<h3>META Chunk<a class="headerlink" href="#meta-chunk" title="Permalink to this heading">¶</a></h3>
|
||
<p>Each data file contains a META chunk which holds information about the state of
|
||
the file. The META chunk is always at the end of the file at a fixed position.
|
||
Because CAPS supports pre-allocation of file sizes without native file system
|
||
support to minimize disc fragmentation it contains information such as:</p>
|
||
<ul class="simple">
|
||
<li><p>effectively used bytes in the file (virtual file size)</p></li>
|
||
<li><p>position of the index root node</p></li>
|
||
<li><p>the number of records in the file</p></li>
|
||
<li><p>the covered time span</p></li>
|
||
</ul>
|
||
<p>and some other internal information.</p>
|
||
</section>
|
||
</section>
|
||
<section id="optimization">
|
||
<span id="sec-optimization"></span><h2>Optimization<a class="headerlink" href="#optimization" title="Permalink to this heading">¶</a></h2>
|
||
<p>After a plugin packet is received and before it is written to disk,
|
||
<a class="reference internal" href="glossary.html#term-CAPS"><span class="xref std std-term">CAPS</span></a> tries to optimize the file data in order reduce the overall data
|
||
size and to increase the access time. This includes:</p>
|
||
<ul class="simple">
|
||
<li><p><strong>merging</strong> data chunks for continuous data blocks</p></li>
|
||
<li><p><strong>splitting</strong> data chunks on the date limit</p></li>
|
||
<li><p><strong>trimming</strong> overlapped data</p></li>
|
||
</ul>
|
||
<section id="merging-of-data-chunks">
|
||
<h3>Merging of Data Chunks<a class="headerlink" href="#merging-of-data-chunks" title="Permalink to this heading">¶</a></h3>
|
||
<p><a class="reference internal" href="glossary.html#term-CAPS"><span class="xref std std-term">CAPS</span></a> tries to create large continues blocks of data by reducing the
|
||
number of data chunks. The advantage of large chunks is that less disk space is
|
||
occupied by data chunk headers. Also seeking to a particular time stamp is
|
||
faster because less data chunk headers need to be read.</p>
|
||
<p>Data chunks can be merged if the following conditions apply:</p>
|
||
<ul class="simple">
|
||
<li><p>merging is supported by packet type</p></li>
|
||
<li><p>previous data header is compatible according to packet specification, e.g.
|
||
<code class="docutils literal notranslate"><span class="pre">samplingFrequency</span></code> and <code class="docutils literal notranslate"><span class="pre">dataType</span></code> matches</p></li>
|
||
<li><p><code class="docutils literal notranslate"><span class="pre">endTime</span></code> of last record equals <code class="docutils literal notranslate"><span class="pre">startTime</span></code> of new record (no gap)</p></li>
|
||
</ul>
|
||
<p>Figure <a href="#fig-file-merge">7</a> shows the arrival of a new plugin packet. In
|
||
alternative A) the merge failed and a new data chunk is created. In alternative B)
|
||
the merger succeeds. In the latter case the new data is appended to the existing
|
||
data block and the original chunk header is updated to reflect the new chunk
|
||
size.</p>
|
||
<figure class="align-default" id="id4">
|
||
<span id="fig-file-merge"></span><a class="reference internal image-reference" href="../_images/file_merge.png"><img alt="../_images/file_merge.png" src="../_images/file_merge.png" style="width: 18cm;" /></a>
|
||
<figcaption>
|
||
<p><span class="caption-text">Merging of data chunks for seamless streams</span><a class="headerlink" href="#id4" title="Permalink to this image">¶</a></p>
|
||
</figcaption>
|
||
</figure>
|
||
</section>
|
||
<section id="splitting-of-data-chunks">
|
||
<h3>Splitting of Data Chunks<a class="headerlink" href="#splitting-of-data-chunks" title="Permalink to this heading">¶</a></h3>
|
||
<p>Figure <a href="#fig-file-split">8</a> shows the arrival of a plugin packet containing
|
||
data of 2 different days. If possible, the data is split on the date limit. The
|
||
first part is appended to the existing data file. For the second part a new day
|
||
file is created, containing a new header and data chunk. This approach ensures
|
||
that a sample is stored in the correct data file and thus increases the access
|
||
time.</p>
|
||
<p>Splitting of data chunks is only supported for packet types providing the
|
||
<code class="docutils literal notranslate"><span class="pre">trim</span></code> operation.</p>
|
||
<figure class="align-default" id="id5">
|
||
<span id="fig-file-split"></span><a class="reference internal image-reference" href="../_images/file_split.png"><img alt="../_images/file_split.png" src="../_images/file_split.png" style="width: 18cm;" /></a>
|
||
<figcaption>
|
||
<p><span class="caption-text">Splitting of data chunks on the date limit</span><a class="headerlink" href="#id5" title="Permalink to this image">¶</a></p>
|
||
</figcaption>
|
||
</figure>
|
||
</section>
|
||
<section id="trimming-of-overlaps">
|
||
<h3>Trimming of Overlaps<a class="headerlink" href="#trimming-of-overlaps" title="Permalink to this heading">¶</a></h3>
|
||
<p>The received plugin packets may contain overlapping time spans. If supported by
|
||
the packet type <a class="reference internal" href="glossary.html#term-CAPS"><span class="xref std std-term">CAPS</span></a> will trim the data to create seamless data streams.</p>
|
||
</section>
|
||
</section>
|
||
<section id="packet-types">
|
||
<span id="sec-packet-types"></span><h2>Packet Types<a class="headerlink" href="#packet-types" title="Permalink to this heading">¶</a></h2>
|
||
<p><a class="reference internal" href="glossary.html#term-CAPS"><span class="xref std std-term">CAPS</span></a> currently supports the following packet types:</p>
|
||
<ul class="simple">
|
||
<li><p><strong>RAW</strong> - generic time series data</p></li>
|
||
<li><p><strong>ANY</strong> - any possible content</p></li>
|
||
<li><p><strong>MiniSeed</strong> - native <a class="reference internal" href="glossary.html#term-miniSeed"><span class="xref std std-term">MiniSeed</span></a></p></li>
|
||
</ul>
|
||
<section id="raw">
|
||
<span id="sec-pt-raw"></span><h3>RAW<a class="headerlink" href="#raw" title="Permalink to this heading">¶</a></h3>
|
||
<p>The RAW format is a lightweight format for uncompressed time series data with a
|
||
minimal header. The chunk header is followed by a 16 byte data header:</p>
|
||
<table class="docutils align-default">
|
||
<thead>
|
||
<tr class="row-odd"><th class="head"><p>content</p></th>
|
||
<th class="head"><p>type</p></th>
|
||
<th class="head"><p>bytes</p></th>
|
||
</tr>
|
||
</thead>
|
||
<tbody>
|
||
<tr class="row-even"><td><p>dataType</p></td>
|
||
<td><p>char</p></td>
|
||
<td><p>1</p></td>
|
||
</tr>
|
||
<tr class="row-odd"><td><p><em>startTime</em></p></td>
|
||
<td><p>TimeStamp</p></td>
|
||
<td><p>[11]</p></td>
|
||
</tr>
|
||
<tr class="row-even"><td><p> year</p></td>
|
||
<td><p>int16</p></td>
|
||
<td><p>2</p></td>
|
||
</tr>
|
||
<tr class="row-odd"><td><p> yDay</p></td>
|
||
<td><p>uint16</p></td>
|
||
<td><p>2</p></td>
|
||
</tr>
|
||
<tr class="row-even"><td><p> hour</p></td>
|
||
<td><p>uint8</p></td>
|
||
<td><p>1</p></td>
|
||
</tr>
|
||
<tr class="row-odd"><td><p> minute</p></td>
|
||
<td><p>uint8</p></td>
|
||
<td><p>1</p></td>
|
||
</tr>
|
||
<tr class="row-even"><td><p> second</p></td>
|
||
<td><p>uint8</p></td>
|
||
<td><p>1</p></td>
|
||
</tr>
|
||
<tr class="row-odd"><td><p> usec</p></td>
|
||
<td><p>int32</p></td>
|
||
<td><p>4</p></td>
|
||
</tr>
|
||
<tr class="row-even"><td><p>samplingFrequencyNumerator</p></td>
|
||
<td><p>uint16</p></td>
|
||
<td><p>2</p></td>
|
||
</tr>
|
||
<tr class="row-odd"><td><p>samplingFrequencyDenominator</p></td>
|
||
<td><p>uint16</p></td>
|
||
<td><p>2</p></td>
|
||
</tr>
|
||
</tbody>
|
||
</table>
|
||
<p>The number of samples is calculated by the remaining <code class="docutils literal notranslate"><span class="pre">chunkSize</span></code> divided by
|
||
the size of the <code class="docutils literal notranslate"><span class="pre">dataType</span></code>. The following data types value are supported:</p>
|
||
<table class="docutils align-default">
|
||
<thead>
|
||
<tr class="row-odd"><th class="head"><p>id</p></th>
|
||
<th class="head"><p>type</p></th>
|
||
<th class="head"><p>bytes</p></th>
|
||
</tr>
|
||
</thead>
|
||
<tbody>
|
||
<tr class="row-even"><td><p>1</p></td>
|
||
<td><p>double</p></td>
|
||
<td><p>8</p></td>
|
||
</tr>
|
||
<tr class="row-odd"><td><p>2</p></td>
|
||
<td><p>float</p></td>
|
||
<td><p>4</p></td>
|
||
</tr>
|
||
<tr class="row-even"><td><p>100</p></td>
|
||
<td><p>int64</p></td>
|
||
<td><p>8</p></td>
|
||
</tr>
|
||
<tr class="row-odd"><td><p>101</p></td>
|
||
<td><p>int32</p></td>
|
||
<td><p>4</p></td>
|
||
</tr>
|
||
<tr class="row-even"><td><p>102</p></td>
|
||
<td><p>int16</p></td>
|
||
<td><p>2</p></td>
|
||
</tr>
|
||
<tr class="row-odd"><td><p>103</p></td>
|
||
<td><p>int8</p></td>
|
||
<td><p>1</p></td>
|
||
</tr>
|
||
</tbody>
|
||
</table>
|
||
<p>The RAW format supports the <code class="docutils literal notranslate"><span class="pre">trim</span></code> and <code class="docutils literal notranslate"><span class="pre">merge</span></code> operation.</p>
|
||
</section>
|
||
<section id="any">
|
||
<span id="sec-pt-any"></span><h3>ANY<a class="headerlink" href="#any" title="Permalink to this heading">¶</a></h3>
|
||
<p>The ANY format was developed to store any possible content in <a class="reference internal" href="glossary.html#term-CAPS"><span class="xref std std-term">CAPS</span></a>. The chunk
|
||
header is followed by a 31 byte data header:</p>
|
||
<table class="docutils align-default">
|
||
<thead>
|
||
<tr class="row-odd"><th class="head"><p>content</p></th>
|
||
<th class="head"><p>type</p></th>
|
||
<th class="head"><p>bytes</p></th>
|
||
</tr>
|
||
</thead>
|
||
<tbody>
|
||
<tr class="row-even"><td><p>type</p></td>
|
||
<td><p>char[4]</p></td>
|
||
<td><p>4</p></td>
|
||
</tr>
|
||
<tr class="row-odd"><td><p>dataType (=103, unused)</p></td>
|
||
<td><p>char</p></td>
|
||
<td><p>1</p></td>
|
||
</tr>
|
||
<tr class="row-even"><td><p><em>startTime</em></p></td>
|
||
<td><p>TimeStamp</p></td>
|
||
<td><p>[11]</p></td>
|
||
</tr>
|
||
<tr class="row-odd"><td><p> year</p></td>
|
||
<td><p>int16</p></td>
|
||
<td><p>2</p></td>
|
||
</tr>
|
||
<tr class="row-even"><td><p> yDay</p></td>
|
||
<td><p>uint16</p></td>
|
||
<td><p>2</p></td>
|
||
</tr>
|
||
<tr class="row-odd"><td><p> hour</p></td>
|
||
<td><p>uint8</p></td>
|
||
<td><p>1</p></td>
|
||
</tr>
|
||
<tr class="row-even"><td><p> minute</p></td>
|
||
<td><p>uint8</p></td>
|
||
<td><p>1</p></td>
|
||
</tr>
|
||
<tr class="row-odd"><td><p> second</p></td>
|
||
<td><p>uint8</p></td>
|
||
<td><p>1</p></td>
|
||
</tr>
|
||
<tr class="row-even"><td><p> usec</p></td>
|
||
<td><p>int32</p></td>
|
||
<td><p>4</p></td>
|
||
</tr>
|
||
<tr class="row-odd"><td><p>samplingFrequencyNumerator</p></td>
|
||
<td><p>uint16</p></td>
|
||
<td><p>2</p></td>
|
||
</tr>
|
||
<tr class="row-even"><td><p>samplingFrequencyDenominator</p></td>
|
||
<td><p>uint16</p></td>
|
||
<td><p>2</p></td>
|
||
</tr>
|
||
<tr class="row-odd"><td><p>endTime</p></td>
|
||
<td><p>TimeStamp</p></td>
|
||
<td><p>11</p></td>
|
||
</tr>
|
||
</tbody>
|
||
</table>
|
||
<p>The ANY data header extends the RAW data header by a 4 character <code class="docutils literal notranslate"><span class="pre">type</span></code>
|
||
field. This field is indented to give a hint on the stored data. E.g. an image
|
||
from a Web cam could be announced by the string <code class="docutils literal notranslate"><span class="pre">JPEG</span></code>.</p>
|
||
<p>Since the ANY format removes the restriction to a particular data type, the
|
||
<code class="docutils literal notranslate"><span class="pre">endTime</span></code> can no longer be derived from the <code class="docutils literal notranslate"><span class="pre">startTime</span></code> and
|
||
<code class="docutils literal notranslate"><span class="pre">samplingFrequency</span></code>. Consequently the <code class="docutils literal notranslate"><span class="pre">endTime</span></code> is explicitly specified in
|
||
the header.</p>
|
||
<p>Because the content of the ANY format is unspecified it neither supports the
|
||
<code class="docutils literal notranslate"><span class="pre">trim</span></code> nor the <code class="docutils literal notranslate"><span class="pre">merge</span></code> operation.</p>
|
||
</section>
|
||
<section id="miniseed">
|
||
<span id="sec-pt-miniseed"></span><h3>MiniSeed<a class="headerlink" href="#miniseed" title="Permalink to this heading">¶</a></h3>
|
||
<p><a class="reference external" href="http://www.iris.edu/data/miniseed.htm">MiniSeed</a> is the standard for the
|
||
exchange of seismic time series. It uses a fixed record length and applies data
|
||
compression.</p>
|
||
<p><a class="reference internal" href="glossary.html#term-CAPS"><span class="xref std std-term">CAPS</span></a> adds no additional header to the <a class="reference internal" href="glossary.html#term-miniSeed"><span class="xref std std-term">MiniSeed</span></a> data. The
|
||
<a class="reference internal" href="glossary.html#term-miniSeed"><span class="xref std std-term">MiniSeed</span></a> record is directly stored after the 8-byte data chunk header.
|
||
All meta information needed by <a class="reference internal" href="glossary.html#term-CAPS"><span class="xref std std-term">CAPS</span></a> is extracted from the
|
||
<a class="reference internal" href="glossary.html#term-miniSeed"><span class="xref std std-term">MiniSeed</span></a> header. The advantage of this native <a class="reference internal" href="glossary.html#term-miniSeed"><span class="xref std std-term">MiniSeed</span></a> support
|
||
is that existing plugin and client code may be reused. Also the transfer and
|
||
storage volume is minimized.</p>
|
||
<p>Because of the fixed record size requirement neither the <code class="docutils literal notranslate"><span class="pre">trim</span></code> nor the
|
||
<code class="docutils literal notranslate"><span class="pre">merge</span></code> operation is supported.</p>
|
||
</section>
|
||
</section>
|
||
</section>
|
||
|
||
|
||
<div id="anchors-bottom"></div>
|
||
</div>
|
||
|
||
<div class="sidebar" role="navigation" aria-label="main navigation">
|
||
<div id="anchors-top"></div>
|
||
<div id="anchors" class="content">
|
||
<div id="searchbox" style="display: none" role="search">
|
||
<h3 id="searchlabel">Quick search</h3>
|
||
<div class="searchformwrapper">
|
||
<form class="search" action="../search.html" method="get">
|
||
<input type="text" name="q" aria-labelledby="searchlabel" autocomplete="off" autocorrect="off" autocapitalize="off" spellcheck="false"/>
|
||
<input type="submit" value="Go" />
|
||
</form>
|
||
</div>
|
||
</div>
|
||
<script>document.getElementById('searchbox').style.display = "block"</script>
|
||
<div>
|
||
<h3><a href="../index.html">Table of Contents</a></h3>
|
||
<ul>
|
||
<li><a class="reference internal" href="#">Data Management</a><ul>
|
||
<li><a class="reference internal" href="#file-format">File Format</a><ul>
|
||
<li><a class="reference internal" href="#sid-chunk">SID Chunk</a></li>
|
||
<li><a class="reference internal" href="#head-chunk">HEAD Chunk</a></li>
|
||
<li><a class="reference internal" href="#data-chunk">DATA Chunk</a></li>
|
||
<li><a class="reference internal" href="#bpt-chunk">BPT Chunk</a></li>
|
||
<li><a class="reference internal" href="#meta-chunk">META Chunk</a></li>
|
||
</ul>
|
||
</li>
|
||
<li><a class="reference internal" href="#optimization">Optimization</a><ul>
|
||
<li><a class="reference internal" href="#merging-of-data-chunks">Merging of Data Chunks</a></li>
|
||
<li><a class="reference internal" href="#splitting-of-data-chunks">Splitting of Data Chunks</a></li>
|
||
<li><a class="reference internal" href="#trimming-of-overlaps">Trimming of Overlaps</a></li>
|
||
</ul>
|
||
</li>
|
||
<li><a class="reference internal" href="#packet-types">Packet Types</a><ul>
|
||
<li><a class="reference internal" href="#raw">RAW</a></li>
|
||
<li><a class="reference internal" href="#any">ANY</a></li>
|
||
<li><a class="reference internal" href="#miniseed">MiniSeed</a></li>
|
||
</ul>
|
||
</li>
|
||
</ul>
|
||
</li>
|
||
</ul>
|
||
|
||
</div>
|
||
<div>
|
||
<h4>Previous topic</h4>
|
||
<p class="topless"><a href="server.html"
|
||
title="previous chapter">CAPS Server Application</a></p>
|
||
</div>
|
||
<div>
|
||
<h4>Next topic</h4>
|
||
<p class="topless"><a href="configuration.html"
|
||
title="next chapter">Execution and Automatic Startup</a></p>
|
||
</div>
|
||
<div role="note" aria-label="source link">
|
||
<h3>This Page</h3>
|
||
<ul class="this-page-menu">
|
||
<li><a href="../_sources/base/archive.rst.txt"
|
||
rel="nofollow">Show Source</a></li>
|
||
</ul>
|
||
</div>
|
||
</div>
|
||
</div>
|
||
</div>
|
||
</div>
|
||
<div class="footer">
|
||
<div class="container">
|
||
<div class="content">
|
||
Release <b></b> version <b>2025.254#f0c77b29e</b>
|
||
<div class="copyright">
|
||
© Copyright <a href="https://www.gempa.de">2013, gempa GmbH.</a>
|
||
</div>
|
||
</div>
|
||
</div>
|
||
</div>
|
||
</body>
|
||
</html> |