You cannot select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

333 lines
20 KiB
HTML

<!DOCTYPE html>
<html >
<head>
<meta charset="utf-8" />
<title>Messaging system &#8212; SeisComP Release documentation</title>
<link rel="stylesheet" href="../../_static/pygments.css" type="text/css" />
<link rel="stylesheet" href="../../_static/seiscomp.css" type="text/css" />
<link rel="stylesheet" type="text/css" href="../../_static/graphviz.css" />
<script type="text/javascript" src="../../_static/seiscomp.js"></script>
<script type="text/javascript" id="documentation_options" data-url_root="../../" src="../../_static/documentation_options.js"></script>
<script src="../../_static/jquery.js"></script>
<script src="../../_static/underscore.js"></script>
<script src="../../_static/doctools.js"></script>
<script src="../../_static/language_data.js"></script>
<link rel="shortcut icon" href="../../_static/favicon.ico"/>
<link rel="index" title="Index" href="../../genindex.html" />
<link rel="search" title="Search" href="../../search.html" />
<link rel="next" title="SeisComP modules" href="modules.html" />
<link rel="prev" title="Concepts" href="../concepts.html" />
</head>
<body>
<div class="header">
<div class="container">
<div class="brand">
<img class="logo" src="../../_static/brands/seiscomp/text/white.svg"/>
<!-- span class="title">SeisComP Release</span -->
<span class="version">5.3.0</span>
</div>
</div>
</div>
<div class="nav">
<div class="container">
<div class="content"><a class="pull-right" id="sidebar-toggle">TOC</a>
<div class="related" role="navigation" aria-label="related navigation">
<ul>
<li class="right">
<a href="../../genindex.html" title="General Index"
accesskey="I">
index
</a>
</li>
<li class="right">
<a href="modules.html" title="SeisComP modules"
accesskey="N">
next
</a>
</li>
<li class="right">
<a href="../concepts.html" title="Concepts"
accesskey="P">
previous
</a>
</li>
<li class="nav-item nav-item-0">
<a href="../../index.html">Home</a>
</li>
<li class="nav-item nav-item-1">
<a href="../concepts.html" accesskey="U">Concepts</a>
</li>
</ul>
</div>
</div>
</div>
</div>
<div class="container">
<div class="fitted content" id="anchors-container">
<div class="body" role="main">
<div class="section" id="messaging-system">
<span id="concepts-messaging"></span><h1>Messaging system<a class="headerlink" href="#messaging-system" title="Permalink to this headline"></a></h1>
<div class="section" id="scope">
<h2>Scope<a class="headerlink" href="#scope" title="Permalink to this headline"></a></h2>
<p>This chapter describes the messaging system used for exchanging parameter messages between
different <a class="reference internal" href="modules.html#concepts-modules"><span class="std std-ref">SeisComP modules</span></a> during runtime.</p>
</div>
<div class="section" id="overview">
<h2>Overview<a class="headerlink" href="#overview" title="Permalink to this headline"></a></h2>
<p>A typical real-time <cite>SeisComP</cite> system consists of data processing and other
modules (clients) and a messaging bus. The messaging bus connects all the
clients letting them exchange information by messages through this bus.
The messaging system is a fundamental concept of SeisComP following the
publish-subscribe pattern <span id="id1">[<a class="reference internal" href="../references.html#id173" title="Publishsubscribe pattern. Wikipedia. URL: https://en.wikipedia.org/wiki/Publish%E2%80%93subscribe_pattern.">20</a>]</span>. It is provided by the <cite>SeisComP</cite>
module <a class="reference internal" href="../../apps/scmaster.html#scmaster"><span class="std std-ref">scmaster</span></a>.</p>
<p>Clients can be producers (sending messages) and/or consumers (receiving
messages). The basic concept is really simple: A producer sends a message
(some event parameter, arbitrary content) and a consumer read that message and tries to
process it. Thats it. That is a very generic approach which requires clients
to understand the message content of each other.</p>
<p>In <cite>SeisComP</cite> this common messaging language consists of well defined message
types and contents. The vast majority of messages sent around are so called
<strong>notifiers</strong>.</p>
<p>Notifiers inform about changes on the data model. They carry just three
attributes:</p>
<ul class="simple">
<li><p>The unique identifier of the parent object</p></li>
<li><p>The operation to apply (add, update or remove)</p></li>
<li><p>The subject</p></li>
</ul>
<p>Speaking XML, a notifier looks like this:</p>
<div class="highlight-xml notranslate"><div class="highlight"><pre><span></span><span class="cp">&lt;?xml version=&quot;1.0&quot; encoding=&quot;UTF-8&quot;?&gt;</span>
<span class="nt">&lt;seiscomp</span> <span class="na">xmlns=</span><span class="s">&quot;http://geofon.gfz-potsdam.de/ns/seiscomp-schema/0.11&quot;</span> <span class="na">version=</span><span class="s">&quot;0.11&quot;</span><span class="nt">&gt;</span>
<span class="nt">&lt;Notifier</span> <span class="na">parentID=</span><span class="s">&quot;EventParameters&quot;</span> <span class="na">operation=</span><span class="s">&quot;add&quot;</span><span class="nt">&gt;</span>
<span class="nt">&lt;Pick</span> <span class="na">publicID=</span><span class="s">&quot;Pick/....&quot;</span><span class="nt">&gt;</span>
<span class="c">&lt;!-- All pick attributes will be placed here --&gt;</span>
<span class="nt">&lt;/Pick&gt;</span>
<span class="nt">&lt;/Notifier&gt;</span>
<span class="nt">&lt;/seiscomp&gt;</span>
</pre></div>
</div>
<p>Each client that receives such a notifier will take further actions if the
information meets its interest. A simple example is <a class="reference internal" href="../../apps/scautoloc.html#scautoloc"><span class="std std-ref">scautoloc</span></a>. It
receives Pick objects in the form as written above. It stores the pick to its
internal data structures and when there are enough picks to form an Origin,
it does its processing and probably sends an Origin as a result (again as
notifier).</p>
<div class="figure align-center" id="id2">
<img alt="sketch of a distributed system" src="../../_images/system.png" />
<p class="caption"><span class="caption-text">Schematic view on a distributed SeisComP system.</span><a class="headerlink" href="#id2" title="Permalink to this image"></a></p>
</div>
</div>
<div class="section" id="queues">
<span id="messaging-queue"></span><h2>Queues<a class="headerlink" href="#queues" title="Permalink to this headline"></a></h2>
<p>The messaging server (<a class="reference internal" href="../../apps/scmaster.html#scmaster"><span class="std std-ref">scmaster</span></a>) manages a set of queues. Each queue is
independent and clients can only connect to <em>one</em> queue at a time within one
connection. Messages sent to a queue are sorted by their arrival time and
forwarded to clients in exactly that order. That is called a FIFO (first in
first out) queue.</p>
<p>For each queue a set of processors can be added where each message is
piped through. One example is the dbstore processor which applies notifiers to
a configured relational database management system (RDBMS).</p>
<p>One could configure scmaster with two queues, one for production which
populates a database and one for playbacks which does not populate a database.</p>
<p>A client can be directed to either the one or the other queue:</p>
<div class="highlight-sh notranslate"><div class="highlight"><pre><span></span>connection.server <span class="o">=</span> localhost/production
</pre></div>
</div>
<p>or</p>
<div class="highlight-sh notranslate"><div class="highlight"><pre><span></span>scautopick -H localhost/production
</pre></div>
</div>
</div>
<div class="section" id="groups">
<span id="messaging-groups"></span><h2>Groups<a class="headerlink" href="#groups" title="Permalink to this headline"></a></h2>
<p>Groups are like topic filters for a message. A message is always being sent to
a specific group or a specific client (peer-to-peer). A client can subscribe
to one or many groups and it will only receive messages tagged with a group it
has subscribed to.</p>
<p>For example <a class="reference internal" href="../../apps/scautopick.html#scautopick"><span class="std std-ref">scautopick</span></a> processes waveforms sending detections or
picks as pick notifiers to the PICK group. All clients subscribed to that group
would also receive this information. If the queue is configured with the
dbstore processor then it is guaranteed that the pick has been stored already
in the database at the time of message arrival at the clients.</p>
<p>A consumer of the PICK group messages is <a class="reference internal" href="../../apps/scautoloc.html#scautoloc"><span class="std std-ref">scautoloc</span></a>. Whenever it receives
a new pick it will try to associate it or to locate a seismic event. After it
succeeded it will send an origin to the LOCATION group and so on.</p>
</div>
<div class="section" id="scheme">
<span id="messaging-scheme"></span><h2>Scheme<a class="headerlink" href="#scheme" title="Permalink to this headline"></a></h2>
<p>The messaging allows unencrypted or encrypted connections which are specified by
the scheme parameter of the connection:</p>
<ul class="simple">
<li><p><cite>scmp</cite> (default): unencrypted,</p></li>
<li><p><cite>scmps</cite>: encrypted.</p></li>
</ul>
<p>Scheme, host, port and <a class="reference internal" href="#messaging-scheme"><span class="std std-ref">queue</span></a> together form the
connection URL of the messaging system which is configurable in
<a class="reference internal" href="../../apps/global.html#confval-connection.server"><code class="xref std std-confval docutils literal notranslate"><span class="pre">connection.server</span></code></a> or set by the command-line option <code class="docutils literal notranslate"><span class="pre">-H</span></code>.</p>
<p>Examples:</p>
<ul>
<li><p>Connect to the production queue of the messaging on localhost with default port.
Not using encryption <cite>scmp</cite> can be assumed implicitly in global configuration</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">connection</span><span class="o">.</span><span class="n">server</span> <span class="o">=</span> <span class="n">localhost</span><span class="o">/</span><span class="n">production</span>
</pre></div>
</div>
<p>On the command line use, e.g.</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>$ scolv -H localhost
</pre></div>
</div>
</li>
<li><p>Connect to the production queue of the messaging on localhost with default port.
Using non-default secure encryption and port the scheme and the port must be
provided explicitly in global configuration</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">connection</span><span class="o">.</span><span class="n">server</span> <span class="o">=</span> <span class="n">scmps</span><span class="p">:</span><span class="o">//</span><span class="n">localhost18181</span><span class="o">/</span><span class="n">production</span>
</pre></div>
</div>
<p>On the command line use, e.g.</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>$ scolv -H scmps://localhost:18181/production
</pre></div>
</div>
</li>
</ul>
</div>
<div class="section" id="database">
<span id="messaging-db"></span><h2>Database<a class="headerlink" href="#database" title="Permalink to this headline"></a></h2>
<p>The <a class="reference internal" href="database.html#concepts-database"><span class="std std-ref">database</span></a> is populated by <a class="reference internal" href="../../apps/scmaster.html#scmaster"><span class="std std-ref">scmaster</span></a> with all
kind of information which might depend on
each other. It is crucial for operations that write operations are well
synchronized. Part of the messaging contract is that clients get read-only
database access from <a class="reference internal" href="../../apps/scmaster.html#scmaster"><span class="std std-ref">scmaster</span></a> and that they will never attempt to write
into the database. In a fully-fledged <cite>SeisComP</cite> system the only instance that is
allowed to populate the database is the dbstore plugin of <a class="reference internal" href="../../apps/scmaster.html#scmaster"><span class="std std-ref">scmaster</span></a>.</p>
<p>When connecting from a client to the messaging, the database connection parameters
are reported by <a class="reference internal" href="../../apps/scmaster.html#scmaster"><span class="std std-ref">scmaster</span></a>. They can be overridden by the global configuration
or command-line option <code class="docutils literal notranslate"><span class="pre">-d</span></code>.</p>
<p>Example:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">scolv</span> <span class="o">-</span><span class="n">H</span> <span class="n">localhost</span> <span class="o">-</span><span class="n">d</span> <span class="n">mysql</span><span class="p">:</span><span class="o">//</span><span class="n">sysop</span><span class="p">:</span><span class="n">sysop</span><span class="nd">@localhost</span><span class="o">/</span><span class="n">seiscomp</span><span class="o">-</span><span class="n">test</span>
</pre></div>
</div>
</div>
<div class="section" id="module-distribution">
<span id="messaging-distribution"></span><h2>Module distribution<a class="headerlink" href="#module-distribution" title="Permalink to this headline"></a></h2>
<p>The messaging server accepts TCP/IP connections enabling a module actually
running anywhere in the network. It happens that this concept is not understood
and users run on each computer another <a class="reference internal" href="../../apps/scmaster.html#scmaster"><span class="std std-ref">scmaster</span></a> instance to offload
processing to more computers. Unfortunately each scmaster instance is
configured with the same database connection which will most likely cause data
inconsistencies because each group of modules which is connected to an scmaster
instance does not know about the other scmaster clients and therefore ignore
the sent updates.</p>
<p>The correct approach is to run <strong>one</strong> scmaster on computer A which populates
database X. Modules running on computer A, use</p>
<div class="highlight-sh notranslate"><div class="highlight"><pre><span></span>connection.server <span class="o">=</span> localhost/production
</pre></div>
</div>
<p>while modules running on computer B use</p>
<div class="highlight-sh notranslate"><div class="highlight"><pre><span></span>connection.server <span class="o">=</span> computerA/production
</pre></div>
</div>
<p>The database connection which is used by <a class="reference internal" href="../../apps/scmaster.html#scmaster"><span class="std std-ref">scmaster</span></a> will be reported to the
clients when they connect so no explicit database configuration is necessary.</p>
<p>The messaging connection can be explicitly provided on the command line using
the option <code class="docutils literal notranslate"><span class="pre">-H</span></code> considering the comments on the
<a class="reference internal" href="#messaging-scheme"><span class="std std-ref">scheme</span></a> and <a class="reference internal" href="#messaging-queue"><span class="std std-ref">queue</span></a>, e.g.</p>
<div class="highlight-sh notranslate"><div class="highlight"><pre><span></span>scolv -H computerA/production
</pre></div>
</div>
</div>
<div class="section" id="web-frontend">
<h2>Web Frontend<a class="headerlink" href="#web-frontend" title="Permalink to this headline"></a></h2>
<p>When running <a class="reference internal" href="../../apps/scmaster.html#scmaster"><span class="std std-ref">scmaster</span></a> a web frontend is available which can be adjusted to
provide system information. The default port to access the web frontend is 18180:</p>
<div class="highlight-sh notranslate"><div class="highlight"><pre><span></span>firefox localhost:18180
</pre></div>
</div>
<div class="figure align-center" id="id3">
<a class="reference internal image-reference" href="../../_images/scmaster_web.png"><img alt="scmaster: web frontend" src="../../_images/scmaster_web.png" style="width: 10cm;" /></a>
<p class="caption"><span class="caption-text">scmaster: Web frontend</span><a class="headerlink" href="#id3" title="Permalink to this image"></a></p>
</div>
</div>
<div class="section" id="related-modules">
<h2>Related Modules<a class="headerlink" href="#related-modules" title="Permalink to this headline"></a></h2>
<ul class="simple">
<li><p><a class="reference internal" href="../../apps/scmaster.html#scmaster"><span class="std std-ref">scmaster</span></a></p></li>
<li><p><a class="reference internal" href="../../apps/scm.html#scm"><span class="std std-ref">scm</span></a></p></li>
<li><p><a class="reference internal" href="../../apps/scmm.html#scmm"><span class="std std-ref">scmm</span></a></p></li>
</ul>
</div>
</div>
<div id="anchors-bottom"></div>
</div>
<div class="sidebar" role="navigation" aria-label="main navigation">
<div id="anchors-top"></div>
<div id="anchors" class="content">
<div id="searchbox" style="display: none" role="search">
<h3 id="searchlabel">Quick search</h3>
<div class="searchformwrapper">
<form class="search" action="../../search.html" method="get">
<input type="text" name="q" aria-labelledby="searchlabel" />
<input type="submit" value="Go" />
</form>
</div>
</div>
<script>$('#searchbox').show(0);</script>
<h3><a href="../../index.html">Table of Contents</a></h3>
<ul>
<li><a class="reference internal" href="#">Messaging system</a><ul>
<li><a class="reference internal" href="#scope">Scope</a></li>
<li><a class="reference internal" href="#overview">Overview</a></li>
<li><a class="reference internal" href="#queues">Queues</a></li>
<li><a class="reference internal" href="#groups">Groups</a></li>
<li><a class="reference internal" href="#scheme">Scheme</a></li>
<li><a class="reference internal" href="#database">Database</a></li>
<li><a class="reference internal" href="#module-distribution">Module distribution</a></li>
<li><a class="reference internal" href="#web-frontend">Web Frontend</a></li>
<li><a class="reference internal" href="#related-modules">Related Modules</a></li>
</ul>
</li>
</ul>
<h4>Previous topic</h4>
<p class="topless"><a href="../concepts.html"
title="previous chapter">Concepts</a></p>
<h4>Next topic</h4>
<p class="topless"><a href="modules.html"
title="next chapter">SeisComP modules</a></p>
<div role="note" aria-label="source link">
<h3>This Page</h3>
<ul class="this-page-menu">
<li><a href="../../_sources/base/concepts/messaging.rst.txt"
rel="nofollow">Show Source</a></li>
</ul>
</div>
</div>
</div>
</div>
</div>
<div class="footer">
<div class="container">
<div class="horizontal layout content">
<a class="fade-in" href="https://www.gempa.de" target="_blank">
<img class="brand" src="../../_static/brands/gempa.svg"/>
</a>
<div class="stretched align-center fitted content">
<div>
Version <b>5.3.0</b> Release
</div>
<div class="copyright">
Copyright &copy; gempa GmbH, GFZ Potsdam.
</div>
</div>
<a class="fade-in" href="https://www.gfz-potsdam.de" target="_blank">
<img class="brand" src="../../_static/brands/gfz.svg"/>
</a>
</div>
</div>
</div>
</body>
</html>