344 lines
21 KiB
HTML
344 lines
21 KiB
HTML
<!DOCTYPE html>
|
||
|
||
<html lang="en">
|
||
<head>
|
||
<meta charset="utf-8" /><meta name="viewport" content="width=device-width, initial-scale=1" />
|
||
|
||
<title>Messaging system — SeisComP Release documentation</title>
|
||
<link rel="stylesheet" href="../../_static/pygments.css" type="text/css" />
|
||
<link rel="stylesheet" href="../../_static/seiscomp.css" type="text/css" />
|
||
<link rel="stylesheet" type="text/css" href="../../_static/pygments.css?v=72bcf2f2" />
|
||
<link rel="stylesheet" type="text/css" href="../../_static/seiscomp.css?v=c6da7ce6" />
|
||
<link rel="stylesheet" type="text/css" href="../../_static/graphviz.css?v=eafc0fe6" />
|
||
<script type="text/javascript" src="../../_static/seiscomp.js"></script>
|
||
<script type="text/javascript" id="documentation_options" data-url_root="../../" src="../../_static/documentation_options.js"></script>
|
||
<script data-url_root="../../" id="documentation_options" src="../../_static/documentation_options.js?v=823bb831"></script>
|
||
<script src="../../_static/doctools.js?v=888ff710"></script>
|
||
<script src="../../_static/sphinx_highlight.js?v=4825356b"></script>
|
||
<link rel="index" title="Index" href="../../genindex.html" />
|
||
<link rel="search" title="Search" href="../../search.html" />
|
||
<link rel="next" title="SeisComP modules" href="modules.html" />
|
||
<link rel="prev" title="Concepts" href="../concepts.html" />
|
||
</head>
|
||
<body>
|
||
<div class="header">
|
||
<div class="container">
|
||
<div class="brand">
|
||
<img class="logo" src="../../_static/brands/seiscomp/text/white.svg"/>
|
||
<!-- span class="title">SeisComP Release</span -->
|
||
<span class="version">6.9.0</span>
|
||
</div>
|
||
</div>
|
||
</div>
|
||
<div class="nav">
|
||
<div class="container">
|
||
<div class="content"><a class="pull-right" id="sidebar-toggle">TOC</a>
|
||
|
||
<div class="related" role="navigation" aria-label="related navigation">
|
||
<ul>
|
||
<li class="right">
|
||
<a href="../../genindex.html" title="General Index"
|
||
accesskey="I">
|
||
index
|
||
</a>
|
||
</li>
|
||
<li class="right">
|
||
<a href="modules.html" title="SeisComP modules"
|
||
accesskey="N">
|
||
next
|
||
</a>
|
||
</li>
|
||
<li class="right">
|
||
<a href="../concepts.html" title="Concepts"
|
||
accesskey="P">
|
||
previous
|
||
</a>
|
||
</li>
|
||
<li class="nav-item nav-item-0">
|
||
<a href="../../index.html">Home</a>
|
||
</li>
|
||
<li class="nav-item nav-item-1">
|
||
<a href="../concepts.html" accesskey="U">Concepts</a>
|
||
</li>
|
||
</ul>
|
||
</div>
|
||
</div>
|
||
</div>
|
||
</div>
|
||
<div class="container">
|
||
<div class="fitted content" id="anchors-container">
|
||
<div class="body" role="main">
|
||
|
||
<section id="messaging-system">
|
||
<span id="concepts-messaging"></span><h1>Messaging system<a class="headerlink" href="#messaging-system" title="Permalink to this heading">¶</a></h1>
|
||
<section id="scope">
|
||
<h2>Scope<a class="headerlink" href="#scope" title="Permalink to this heading">¶</a></h2>
|
||
<p>This chapter describes the messaging system used for exchanging parameter messages between
|
||
different <a class="reference internal" href="modules.html#concepts-modules"><span class="std std-ref">SeisComP modules</span></a> during runtime.</p>
|
||
</section>
|
||
<section id="overview">
|
||
<h2>Overview<a class="headerlink" href="#overview" title="Permalink to this heading">¶</a></h2>
|
||
<p>A typical real-time <cite>SeisComP</cite> system consists of data processing and other
|
||
modules (clients) and a messaging bus. The messaging bus connects all the
|
||
clients letting them exchange information by messages through this bus.
|
||
The messaging system is a fundamental concept of SeisComP following the
|
||
publish-subscribe pattern <span id="id1">[<a class="reference internal" href="../references.html#id186" title="Publish–subscribe pattern. Wikipedia. URL: https://en.wikipedia.org/wiki/Publish%E2%80%93subscribe_pattern.">21</a>]</span>. It is provided by the <cite>SeisComP</cite>
|
||
module <a class="reference internal" href="../../apps/scmaster.html#scmaster"><span class="std std-ref">scmaster</span></a>.</p>
|
||
<p>Clients can be producers (sending messages) and/or consumers (receiving
|
||
messages). The basic concept is really simple: A producer sends a message
|
||
(some event parameter, arbitrary content) and a consumer read that message and tries to
|
||
process it. That’s it. That is a very generic approach which requires clients
|
||
to understand the message content of each other.</p>
|
||
<p>In <cite>SeisComP</cite> this common messaging language consists of well defined message
|
||
types and contents. The vast majority of messages sent around are so called
|
||
<strong>notifiers</strong>.</p>
|
||
<p>Notifiers inform about changes on the data model. They carry just three
|
||
attributes:</p>
|
||
<ul class="simple">
|
||
<li><p>The unique identifier of the parent object</p></li>
|
||
<li><p>The operation to apply (add, update or remove)</p></li>
|
||
<li><p>The subject</p></li>
|
||
</ul>
|
||
<p>Speaking XML, a notifier looks like this:</p>
|
||
<div class="highlight-xml notranslate"><div class="highlight"><pre><span></span><span class="cp"><?xml version="1.0" encoding="UTF-8"?></span>
|
||
<span class="nt"><seiscomp</span><span class="w"> </span><span class="na">xmlns=</span><span class="s">"http://geofon.gfz-potsdam.de/ns/seiscomp-schema/0.11"</span><span class="w"> </span><span class="na">version=</span><span class="s">"0.11"</span><span class="nt">></span>
|
||
<span class="w"> </span><span class="nt"><Notifier</span><span class="w"> </span><span class="na">parentID=</span><span class="s">"EventParameters"</span><span class="w"> </span><span class="na">operation=</span><span class="s">"add"</span><span class="nt">></span>
|
||
<span class="w"> </span><span class="nt"><Pick</span><span class="w"> </span><span class="na">publicID=</span><span class="s">"Pick/...."</span><span class="nt">></span>
|
||
<span class="w"> </span><span class="cm"><!-- All pick attributes will be placed here --></span>
|
||
<span class="w"> </span><span class="nt"></Pick></span>
|
||
<span class="w"> </span><span class="nt"></Notifier></span>
|
||
<span class="nt"></seiscomp></span>
|
||
</pre></div>
|
||
</div>
|
||
<p>Each client that receives such a notifier will take further actions if the
|
||
information meets its interest. A simple example is <a class="reference internal" href="../../apps/scautoloc.html#scautoloc"><span class="std std-ref">scautoloc</span></a>. It
|
||
receives Pick objects in the form as written above. It stores the pick to its
|
||
internal data structures and when there are enough picks to form an Origin,
|
||
it does its processing and probably sends an Origin as a result (again as
|
||
notifier).</p>
|
||
<figure class="align-center" id="id2">
|
||
<img alt="sketch of a distributed system" src="../../_images/system.png" />
|
||
<figcaption>
|
||
<p><span class="caption-text">Schematic view on a distributed SeisComP system.</span><a class="headerlink" href="#id2" title="Permalink to this image">¶</a></p>
|
||
</figcaption>
|
||
</figure>
|
||
</section>
|
||
<section id="queues">
|
||
<span id="messaging-queue"></span><h2>Queues<a class="headerlink" href="#queues" title="Permalink to this heading">¶</a></h2>
|
||
<p>The messaging server (<a class="reference internal" href="../../apps/scmaster.html#scmaster"><span class="std std-ref">scmaster</span></a>) manages a set of queues. Each queue is
|
||
independent and clients can only connect to <em>one</em> queue at a time within one
|
||
connection. Messages sent to a queue are sorted by their arrival time and
|
||
forwarded to clients in exactly that order. That is called a FIFO (first in
|
||
first out) queue.</p>
|
||
<p>For each queue a set of processors can be added where each message is
|
||
piped through. One example is the dbstore processor which applies notifiers to
|
||
a configured relational database management system (RDBMS).</p>
|
||
<p>One could configure scmaster with two queues, one for production which
|
||
populates a database and one for playbacks which does not populate a database.</p>
|
||
<p>A client can be directed to either the one or the other queue:</p>
|
||
<div class="highlight-sh notranslate"><div class="highlight"><pre><span></span>connection.server<span class="w"> </span><span class="o">=</span><span class="w"> </span>localhost/production
|
||
</pre></div>
|
||
</div>
|
||
<p>or</p>
|
||
<div class="highlight-sh notranslate"><div class="highlight"><pre><span></span>scautopick<span class="w"> </span>-H<span class="w"> </span>localhost/production
|
||
</pre></div>
|
||
</div>
|
||
</section>
|
||
<section id="groups">
|
||
<span id="messaging-groups"></span><h2>Groups<a class="headerlink" href="#groups" title="Permalink to this heading">¶</a></h2>
|
||
<p>Groups are like topic filters for a message. A message is always being sent to
|
||
a specific group or a specific client (peer-to-peer). A client can subscribe
|
||
to one or many groups and it will only receive messages tagged with a group it
|
||
has subscribed to.</p>
|
||
<p>For example <a class="reference internal" href="../../apps/scautopick.html#scautopick"><span class="std std-ref">scautopick</span></a> processes waveforms sending detections or
|
||
picks as pick notifiers to the PICK group. All clients subscribed to that group
|
||
would also receive this information. If the queue is configured with the
|
||
dbstore processor then it is guaranteed that the pick has been stored already
|
||
in the database at the time of message arrival at the clients.</p>
|
||
<p>A consumer of the PICK group messages is <a class="reference internal" href="../../apps/scautoloc.html#scautoloc"><span class="std std-ref">scautoloc</span></a>. Whenever it receives
|
||
a new pick it will try to associate it or to locate a seismic event. After it
|
||
succeeded it will send an origin to the LOCATION group and so on.</p>
|
||
</section>
|
||
<section id="scheme">
|
||
<span id="messaging-scheme"></span><h2>Scheme<a class="headerlink" href="#scheme" title="Permalink to this heading">¶</a></h2>
|
||
<p>The messaging allows unencrypted or encrypted connections which are specified by
|
||
the scheme parameter of the connection:</p>
|
||
<ul class="simple">
|
||
<li><p><cite>scmp</cite> (default): unencrypted,</p></li>
|
||
<li><p><cite>scmps</cite>: encrypted.</p></li>
|
||
</ul>
|
||
<p>Scheme, host, port and <a class="reference internal" href="#messaging-scheme"><span class="std std-ref">queue</span></a> together form the
|
||
connection URL of the messaging system which is configurable in
|
||
<a class="reference internal" href="../../apps/global.html#confval-connection.server"><code class="xref std std-confval docutils literal notranslate"><span class="pre">connection.server</span></code></a> or set by the command-line option <code class="docutils literal notranslate"><span class="pre">-H</span></code>.</p>
|
||
<p>Examples:</p>
|
||
<ul>
|
||
<li><p>Connect to the production queue of the messaging on localhost with default port.
|
||
Not using encryption <cite>scmp</cite> can be assumed implicitly in global configuration</p>
|
||
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">connection</span><span class="o">.</span><span class="n">server</span> <span class="o">=</span> <span class="n">localhost</span><span class="o">/</span><span class="n">production</span>
|
||
</pre></div>
|
||
</div>
|
||
<p>On the command line use, e.g.</p>
|
||
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>$ scolv -H localhost
|
||
</pre></div>
|
||
</div>
|
||
</li>
|
||
<li><p>Connect to the production queue of the messaging on localhost with default port.
|
||
Using non-default secure encryption and port the scheme and the port must be
|
||
provided explicitly in global configuration</p>
|
||
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">connection</span><span class="o">.</span><span class="n">server</span> <span class="o">=</span> <span class="n">scmps</span><span class="p">:</span><span class="o">//</span><span class="n">localhost18181</span><span class="o">/</span><span class="n">production</span>
|
||
</pre></div>
|
||
</div>
|
||
<p>On the command line use, e.g.</p>
|
||
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>$ scolv -H scmps://localhost:18181/production
|
||
</pre></div>
|
||
</div>
|
||
</li>
|
||
</ul>
|
||
</section>
|
||
<section id="database">
|
||
<span id="messaging-db"></span><h2>Database<a class="headerlink" href="#database" title="Permalink to this heading">¶</a></h2>
|
||
<p>The <a class="reference internal" href="database.html#concepts-database"><span class="std std-ref">database</span></a> is populated by <a class="reference internal" href="../../apps/scmaster.html#scmaster"><span class="std std-ref">scmaster</span></a> with all
|
||
kind of information which might depend on
|
||
each other. It is crucial for operations that write operations are well
|
||
synchronized. Part of the messaging contract is that clients get read-only
|
||
database access from <a class="reference internal" href="../../apps/scmaster.html#scmaster"><span class="std std-ref">scmaster</span></a> and that they will never attempt to write
|
||
into the database. In a fully-fledged <cite>SeisComP</cite> system the only instance that is
|
||
allowed to populate the database is the dbstore plugin of <a class="reference internal" href="../../apps/scmaster.html#scmaster"><span class="std std-ref">scmaster</span></a>.</p>
|
||
<p>When connecting from a client to the messaging, the database connection parameters
|
||
are reported by <a class="reference internal" href="../../apps/scmaster.html#scmaster"><span class="std std-ref">scmaster</span></a>. They can be overridden by the global configuration
|
||
or command-line option <code class="docutils literal notranslate"><span class="pre">-d</span></code>.</p>
|
||
<p>Example:</p>
|
||
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">scolv</span> <span class="o">-</span><span class="n">H</span> <span class="n">localhost</span> <span class="o">-</span><span class="n">d</span> <span class="n">mysql</span><span class="p">:</span><span class="o">//</span><span class="n">sysop</span><span class="p">:</span><span class="n">sysop</span><span class="nd">@localhost</span><span class="o">/</span><span class="n">seiscomp</span><span class="o">-</span><span class="n">test</span>
|
||
</pre></div>
|
||
</div>
|
||
</section>
|
||
<section id="module-distribution">
|
||
<span id="messaging-distribution"></span><h2>Module distribution<a class="headerlink" href="#module-distribution" title="Permalink to this heading">¶</a></h2>
|
||
<p>The messaging server accepts TCP/IP connections enabling a module actually
|
||
running anywhere in the network. It happens that this concept is not understood
|
||
and users run on each computer another <a class="reference internal" href="../../apps/scmaster.html#scmaster"><span class="std std-ref">scmaster</span></a> instance to offload
|
||
processing to more computers. Unfortunately each scmaster instance is
|
||
configured with the same database connection which will most likely cause data
|
||
inconsistencies because each group of modules which is connected to an scmaster
|
||
instance does not know about the other scmaster clients and therefore ignore
|
||
the sent updates.</p>
|
||
<p>The correct approach is to run <strong>one</strong> scmaster on computer A which populates
|
||
database X. Modules running on computer A, use</p>
|
||
<div class="highlight-sh notranslate"><div class="highlight"><pre><span></span>connection.server<span class="w"> </span><span class="o">=</span><span class="w"> </span>localhost/production
|
||
</pre></div>
|
||
</div>
|
||
<p>while modules running on computer B use</p>
|
||
<div class="highlight-sh notranslate"><div class="highlight"><pre><span></span>connection.server<span class="w"> </span><span class="o">=</span><span class="w"> </span>computerA/production
|
||
</pre></div>
|
||
</div>
|
||
<p>The database connection which is used by <a class="reference internal" href="../../apps/scmaster.html#scmaster"><span class="std std-ref">scmaster</span></a> will be reported to the
|
||
clients when they connect so no explicit database configuration is necessary.</p>
|
||
<p>The messaging connection can be explicitly provided on the command line using
|
||
the option <code class="docutils literal notranslate"><span class="pre">-H</span></code> considering the comments on the
|
||
<a class="reference internal" href="#messaging-scheme"><span class="std std-ref">scheme</span></a> and <a class="reference internal" href="#messaging-queue"><span class="std std-ref">queue</span></a>, e.g.</p>
|
||
<div class="highlight-sh notranslate"><div class="highlight"><pre><span></span>scolv<span class="w"> </span>-H<span class="w"> </span>computerA/production
|
||
</pre></div>
|
||
</div>
|
||
</section>
|
||
<section id="web-frontend">
|
||
<h2>Web Frontend<a class="headerlink" href="#web-frontend" title="Permalink to this heading">¶</a></h2>
|
||
<p>When running <a class="reference internal" href="../../apps/scmaster.html#scmaster"><span class="std std-ref">scmaster</span></a> a web frontend is available which can be adjusted to
|
||
provide system information. The default port to access the web frontend is 18180:</p>
|
||
<div class="highlight-sh notranslate"><div class="highlight"><pre><span></span>firefox<span class="w"> </span>localhost:18180
|
||
</pre></div>
|
||
</div>
|
||
<figure class="align-center" id="id3">
|
||
<a class="reference internal image-reference" href="../../_images/scmaster_web.png"><img alt="scmaster: web frontend" src="../../_images/scmaster_web.png" style="width: 10cm;" /></a>
|
||
<figcaption>
|
||
<p><span class="caption-text">scmaster: Web frontend</span><a class="headerlink" href="#id3" title="Permalink to this image">¶</a></p>
|
||
</figcaption>
|
||
</figure>
|
||
</section>
|
||
<section id="related-modules">
|
||
<h2>Related Modules<a class="headerlink" href="#related-modules" title="Permalink to this heading">¶</a></h2>
|
||
<ul class="simple">
|
||
<li><p><a class="reference internal" href="../../apps/scmaster.html#scmaster"><span class="std std-ref">scmaster</span></a></p></li>
|
||
<li><p><a class="reference internal" href="../../apps/scm.html#scm"><span class="std std-ref">scm</span></a></p></li>
|
||
<li><p><a class="reference internal" href="../../apps/scmm.html#scmm"><span class="std std-ref">scmm</span></a></p></li>
|
||
</ul>
|
||
</section>
|
||
</section>
|
||
|
||
|
||
<div id="anchors-bottom"></div>
|
||
</div>
|
||
|
||
<div class="sidebar" role="navigation" aria-label="main navigation">
|
||
<div id="anchors-top"></div>
|
||
<div id="anchors" class="content">
|
||
<div id="searchbox" style="display: none" role="search">
|
||
<h3 id="searchlabel">Quick search</h3>
|
||
<div class="searchformwrapper">
|
||
<form class="search" action="../../search.html" method="get">
|
||
<input type="text" name="q" aria-labelledby="searchlabel" autocomplete="off" autocorrect="off" autocapitalize="off" spellcheck="false"/>
|
||
<input type="submit" value="Go" />
|
||
</form>
|
||
</div>
|
||
</div>
|
||
<script>document.getElementById('searchbox').style.display = "block"</script>
|
||
<div>
|
||
<h3><a href="../../index.html">Table of Contents</a></h3>
|
||
<ul>
|
||
<li><a class="reference internal" href="#">Messaging system</a><ul>
|
||
<li><a class="reference internal" href="#scope">Scope</a></li>
|
||
<li><a class="reference internal" href="#overview">Overview</a></li>
|
||
<li><a class="reference internal" href="#queues">Queues</a></li>
|
||
<li><a class="reference internal" href="#groups">Groups</a></li>
|
||
<li><a class="reference internal" href="#scheme">Scheme</a></li>
|
||
<li><a class="reference internal" href="#database">Database</a></li>
|
||
<li><a class="reference internal" href="#module-distribution">Module distribution</a></li>
|
||
<li><a class="reference internal" href="#web-frontend">Web Frontend</a></li>
|
||
<li><a class="reference internal" href="#related-modules">Related Modules</a></li>
|
||
</ul>
|
||
</li>
|
||
</ul>
|
||
|
||
</div>
|
||
<div>
|
||
<h4>Previous topic</h4>
|
||
<p class="topless"><a href="../concepts.html"
|
||
title="previous chapter">Concepts</a></p>
|
||
</div>
|
||
<div>
|
||
<h4>Next topic</h4>
|
||
<p class="topless"><a href="modules.html"
|
||
title="next chapter">SeisComP modules</a></p>
|
||
</div>
|
||
<div role="note" aria-label="source link">
|
||
<h3>This Page</h3>
|
||
<ul class="this-page-menu">
|
||
<li><a href="../../_sources/base/concepts/messaging.rst.txt"
|
||
rel="nofollow">Show Source</a></li>
|
||
</ul>
|
||
</div>
|
||
</div>
|
||
</div>
|
||
</div>
|
||
</div>
|
||
<div class="footer">
|
||
<div class="container">
|
||
<div class="horizontal layout content">
|
||
<a class="fade-in" href="https://www.gempa.de" target="_blank">
|
||
<img class="brand" src="../../_static/brands/gempa.svg"/>
|
||
</a>
|
||
<div class="stretched align-center fitted content">
|
||
<div>
|
||
Version <b>6.9.0</b> Release
|
||
</div>
|
||
<div class="copyright">
|
||
Copyright © gempa GmbH, GFZ Potsdam.
|
||
</div>
|
||
</div>
|
||
<a class="fade-in" href="https://www.gfz-potsdam.de" target="_blank">
|
||
<img class="brand" src="../../_static/brands/gfz.svg"/>
|
||
</a>
|
||
</div>
|
||
</div>
|
||
</div>
|
||
</body>
|
||
</html> |