[installation] Init with inital config for global
This commit is contained in:
401
share/doc/seiscomp/html/_sources/base/api-python-client.rst.txt
Normal file
401
share/doc/seiscomp/html/_sources/base/api-python-client.rst.txt
Normal file
@ -0,0 +1,401 @@
|
||||
.. _api-client-python:
|
||||
|
||||
seiscomp.client
|
||||
===============
|
||||
|
||||
.. py:module:: seiscomp.client
|
||||
|
||||
Modules are meant to be standalone programs doing a particular job. The
|
||||
seiscomp.client package focuses on three main aspects:
|
||||
|
||||
* Communicate with other modules
|
||||
* Access station- and event metadata through a database
|
||||
* Fetch waveform data
|
||||
|
||||
Therefore a client package has been developed combining these concepts in an
|
||||
easy way with only a couple of API calls. Since |scname| has been developed in
|
||||
C++ and uses the object oriented paradigm forcefully, modules build on the
|
||||
Application (C++: :class:`Seiscomp::Client::Application`, Python:
|
||||
:class:`seiscomp.client.Application`) class. It manages the messaging connection
|
||||
and waveform sources in a transparent way.
|
||||
|
||||
The class :class:`Seiscomp::Client::Application` is the base class for
|
||||
all |scname| applications. It manages messaging and database
|
||||
connections, provides access to command line options and configuration
|
||||
parameters and also handles and interprets notifier messages.
|
||||
|
||||
Blocking network operations like reading messages are moved into threads that
|
||||
are synchronized in a single blocking message queue. This queue allows pushing
|
||||
elements from different threads and unblocks when a new element is ready to be
|
||||
popped. If the queue is full (currently 10 elements are allowed) the pushing
|
||||
threads also block until an element can be pushed again.
|
||||
|
||||
This way applications do not have to poll and thus do not burn CPU cycles.
|
||||
|
||||
The application class is event driven. It runs the event loop which pops the
|
||||
message queue and dispatches events with their handlers. Handler methods are
|
||||
prefixed with *handle*, e.g. :func:`handleMessage`.
|
||||
|
||||
.. note::
|
||||
|
||||
When overriding handlers it is always good practise to call the base
|
||||
handlers before running custom code.
|
||||
|
||||
|
||||
Application class
|
||||
-----------------
|
||||
|
||||
The application class is part of the seiscomp.client package. It needs to
|
||||
be imported first.
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
import seiscomp.client
|
||||
|
||||
A common strategy to write a module with that class is to derive from it and
|
||||
run it in a Python main method.
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
import sys
|
||||
import seiscomp.client
|
||||
|
||||
# Class definition
|
||||
class MyApp(seiscomp.client.Application):
|
||||
def __init__(self, argc, argv):
|
||||
seiscomp.client.Application.__init__(self, argc, argv)
|
||||
|
||||
# Main method to call the app
|
||||
def main(argc, argv):
|
||||
app = MyApp(argc, argv)
|
||||
return app()
|
||||
|
||||
# Call the main method if run as script
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main(len(sys.argv), sys.argv))
|
||||
|
||||
|
||||
An application can be called with the parenthesis operator :func:`()` which
|
||||
returns the applications result code and serves as input to :func:`sys.exit`.
|
||||
Operator() is a wrapper for :func:`Application.exec`.
|
||||
|
||||
|
||||
The workflow of :func:`Application.exec` looks as follows:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
def exec(self):
|
||||
self.returnCode = 1
|
||||
|
||||
if self.init() and self.run():
|
||||
self.returnCode = 0
|
||||
|
||||
self.done()
|
||||
|
||||
return self.returnCode
|
||||
|
||||
:func:`init`, :func:`run` and :func:`done` are explained in more detail in
|
||||
the next sections.
|
||||
|
||||
|
||||
Constructor
|
||||
^^^^^^^^^^^
|
||||
|
||||
To create an application, derive from the seiscomp.client.Application class
|
||||
and configure it in the constructor.
|
||||
|
||||
.. code-block:: python
|
||||
:linenos:
|
||||
:emphasize-lines: 7,9,11
|
||||
|
||||
class MyApp(seiscomp.client.Application):
|
||||
# MyApp constructor
|
||||
def __init__(self, argc, argv):
|
||||
# IMPORTANT: call the base class constructor
|
||||
seiscomp.client.Application.__init__(self, argc, argv)
|
||||
# Default is TRUE
|
||||
self.setMessagingEnabled(False)
|
||||
# Default is TRUE, TRUE
|
||||
self.setDatabaseEnabled(False, False)
|
||||
# Default is TRUE
|
||||
self.setDaemonEnabled(False)
|
||||
|
||||
As marked in line 4, the call of the constructor of the base class is very
|
||||
important. It takes the command line parameters and sets up internal
|
||||
application variables. Without this call the application will either not run
|
||||
at all or show undefined/unexpected behaviour.
|
||||
|
||||
The constructor takes also the initial parameters of the application such as
|
||||
enabling a messaging connection and enabling database access.
|
||||
|
||||
Messaging, database and daemon mode is enabled by default. The daemon mode is
|
||||
important if the application should be started as service and therefore should
|
||||
support the option ``-D, --daemon``. Utilities and non daemon applications
|
||||
should disable that mode.
|
||||
|
||||
Example calls to this options are shown in the highlighted lines of the above
|
||||
code block.
|
||||
|
||||
If messaging is enabled, the messaging username is derived from the binary
|
||||
called (*not the class name*). If the script is called test.py then the username
|
||||
selected is **test**. The username can be overridden either in the configuration
|
||||
file (:ref:`global`) or using the API.
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
self.setMessagingUsername("test")
|
||||
|
||||
Setting the username to an empty string results in a random username selected
|
||||
by the messaging server.
|
||||
|
||||
All application methods are defined in the C++ header file
|
||||
:file:`src/trunk/libs/seiscomp/client/application.h`.
|
||||
|
||||
|
||||
Init
|
||||
^^^^
|
||||
|
||||
The workflow of the init function looks like this:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
init (virtual)
|
||||
initConfiguration (virtual)
|
||||
initCommandLine (virtual)
|
||||
createCommandLineDescription (virtual)
|
||||
parseCommandLine (virtual)
|
||||
printUsage (virtual)
|
||||
validateParameters (virtual)
|
||||
loadPlugins
|
||||
forkDaemon
|
||||
initMessaging
|
||||
initDatabase
|
||||
loadInventory or loadStations
|
||||
loadDBConfigModule
|
||||
loadCities
|
||||
|
||||
Methods marked with virtual can be overridden. :func:`init` itself calls
|
||||
a lot of handlers that can be customized. Typical handlers are
|
||||
:func:`initConfiguration`, :func:`createCommandLineDescription`
|
||||
and :func:`validateParameters`.
|
||||
|
||||
:func:`initConfiguration` is used to read parameters of the configuration files
|
||||
and to populate the internal state. If something fails or if configured values
|
||||
are out of bounds, False can be returned which causes :func:`init` to return
|
||||
False and to exit the application with a non-zero result code.
|
||||
|
||||
An example is show below:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
def initConfiguration(self):
|
||||
if not seiscomp.client.Application.initConfiguration(self):
|
||||
return False
|
||||
|
||||
try:
|
||||
self._directory = self.configGetString("directory")
|
||||
except:
|
||||
pass
|
||||
|
||||
return True
|
||||
|
||||
This method reads the directory parameter from the configuration file(s) and
|
||||
sets it internally. If the directory is not given in any of the modules
|
||||
configuration files, it logs an error and aborts the application by returning
|
||||
False.
|
||||
|
||||
:func:`createCommandLineDescription` is used to add custom command line options.
|
||||
This is a void function and does not return any value. It is also not necessary
|
||||
to call the base class method although it does not hurt.
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
def createCommandLineDescription(self):
|
||||
self.commandline().addGroup("Storage")
|
||||
self.commandline().addStringOption("Storage", "directory,o", "Specify the storage directory")
|
||||
|
||||
A new command line option group is added with :func:`addGroup` and then a new
|
||||
option is added to this group which is a string option.
|
||||
Four types can be added
|
||||
as options: string, int, double and bool: :func:`addStringOption`, :func:`addIntOption`,
|
||||
:func:`addDoubleOption` and :func:`addBoolOption`.
|
||||
|
||||
:func:`validateParameters` can be used to fetch the values of previously added
|
||||
command line options and to validate each parameter. If False is returned, the
|
||||
application is aborted with a non-zero result code.
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
def validateParameters(self):
|
||||
try:
|
||||
self._directory = self.commandline().optionString("directory")
|
||||
except:
|
||||
pass
|
||||
|
||||
# The directory validity is checked to avoid duplicate checks in
|
||||
# initConfiguration.
|
||||
if not self._directory:
|
||||
seiscomp.logging.error("directory not set")
|
||||
return False
|
||||
|
||||
if not exists(self._directory):
|
||||
seiscomp.logging.error(
|
||||
"directory {} does not exist".format(self._directory))
|
||||
return False
|
||||
|
||||
return True
|
||||
|
||||
|
||||
Custom initialization code after checking all parameters can be placed in the
|
||||
overridden method :func:`init`.
|
||||
|
||||
.. code-block: python
|
||||
|
||||
def init(self):
|
||||
if not seiscomp.client.Application.init(self):
|
||||
return False
|
||||
|
||||
# Custom initialization code runs here.
|
||||
setupCustomConnections()
|
||||
readMyDataFiles()
|
||||
|
||||
return True
|
||||
|
||||
But be aware that the process forked already if started as daemon. To run before
|
||||
the fork, it needs to be put into :func:`validateParameters`.
|
||||
|
||||
|
||||
Run
|
||||
^^^
|
||||
|
||||
The workflow of the run method looks like this:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
run (virtual)
|
||||
startMessageThread
|
||||
messageLoop
|
||||
readMessage
|
||||
dispatchMessage (virtual)
|
||||
handleMessage (virtual)
|
||||
addObject (virtual)
|
||||
updateObject (virtual)
|
||||
removeObject (virtual)
|
||||
handleReconnect (virtual)
|
||||
handleDisconnect (virtual)
|
||||
handleTimeout (virtual)
|
||||
handleAutoShutdown (virtual)
|
||||
|
||||
The run method starts the event loop and waits for new events in the queue.
|
||||
In case of messaging a thread is started that sits and waits for messages
|
||||
and feeds them to the queue and to the event loop in :func:`run`. Without
|
||||
messaging the run loop would do nothing but waiting for SIGTERM or
|
||||
a timer event enabled with :func:`enableTimer`. If the event loop is not needed
|
||||
because no timer and messages are needed, it should be overridden and the
|
||||
code should be placed there. This will disable the event loop.
|
||||
|
||||
:func:`run` is expected to return True on success and False otherwise. If False
|
||||
is returned the application exists with a non-zero return code. Custom return
|
||||
codes can always be set with :func:`Application.exit`.
|
||||
|
||||
If the scmaster sends a message to the client it is received in the applications
|
||||
message thread and pushed to the queue. The event loop pops the message from
|
||||
the queue and calls :func:`handleMessage`. The default implementation uses two
|
||||
settings when handling a messages that can be controlled with
|
||||
:func:`enableInterpretNotifier` and :func:`enableAutoApplyNotifier`.
|
||||
|
||||
:func:`enableInterpretNotifier` controls whether the Application queries the
|
||||
message type and extracts notifier objects. For each notifier it parses the
|
||||
operation and dispatches the parentID and the object either to
|
||||
:func:`addObject`, :func:`updateObject` or :func:`removeObject` handler. This
|
||||
behaviour is enabled by default. If disabled, a clients needs to parse the
|
||||
messages by itself and implement this method.
|
||||
|
||||
:func:`enableAutoApplyNotifier` controls whether incoming notifier objects are
|
||||
applied automatically to objects in local memory. If the client has already
|
||||
an object in memory and an update notifier for this object is received, the object
|
||||
in the notifier is copied to the local object. This behaviour is enabled by default.
|
||||
|
||||
|
||||
Done
|
||||
^^^^
|
||||
|
||||
The workflow of the done method looks like this:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
done (virtual)
|
||||
closeTimer
|
||||
closeMessaging
|
||||
closeDatabase
|
||||
|
||||
:func:`done` is usually not overridden. If custom code and clean up procedures
|
||||
need to be placed in :func:`done`, the base class **must** be called. :func:`done` is a
|
||||
void function.
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
def done(self):
|
||||
seiscomp.client.Application.done()
|
||||
|
||||
# Custom clean ups
|
||||
closeMyDataFiles()
|
||||
closeCustomConnections()
|
||||
|
||||
|
||||
|
||||
StreamApplication class
|
||||
-----------------------
|
||||
|
||||
The application class has another occurrence: :class:`seiscomp.client.StreamApplication`.
|
||||
|
||||
The class :class:`StreamApplication` extends the :class:`Application`
|
||||
in terms of record acquisition. It spawns another thread that reads the records
|
||||
from a configurable source and adds a new handler method
|
||||
:func:`StreamApplication.handleRecord` to handle these records.
|
||||
|
||||
|
||||
Its workflow looks like this:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
init (virtual)
|
||||
+initRecordStream
|
||||
run (virtual)
|
||||
+startAcquisitionThread
|
||||
+storeRecord
|
||||
Application.messageLoop
|
||||
dispatchMessage (virtual)
|
||||
+handleRecord (virtual)
|
||||
done (virtual)
|
||||
+closeRecordStream
|
||||
|
||||
|
||||
Received records can be handled with :func:`handleRecord`.
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
def handleRecord(self, rec):
|
||||
print rec.streamID()
|
||||
|
||||
The stream subscription should be done in :func:`init`. :func:`recordStream`
|
||||
returns the RecordStream instance which can be used to add stream requests.
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
def init(self):
|
||||
if not seiscomp.client.StreamApplication.init(self):
|
||||
return False
|
||||
|
||||
# Subscribe to some streams
|
||||
self.recordStream().addStream("GE", "MORC", "", "BHZ")
|
||||
return True
|
||||
|
||||
|
||||
The record stream service is configured either with configuration files
|
||||
(:confval:`recordstream`) or
|
||||
via command-line options ``-I`, ``--record-url``.
|
||||
|
||||
The application finishes if the record stream read EOF. Running a :class:`StreamApplication`
|
||||
with :ref:`Seedlink<seedlink>` would probably never terminate since it is a
|
||||
real time connection and handles reconnects automatically.
|
||||
Reference in New Issue
Block a user