You cannot select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
1225 lines
33 KiB
ReStructuredText
1225 lines
33 KiB
ReStructuredText
.. highlight:: rst
|
|
|
|
.. _fdsnws:
|
|
|
|
######
|
|
fdsnws
|
|
######
|
|
|
|
**Provide FDSN Web Services.**
|
|
|
|
|
|
Description
|
|
===========
|
|
|
|
fdsnws is a server that provides event and station information by FDSN Web Services
|
|
(:cite:t:`fdsn`)
|
|
from a SeisComP database and waveforms from a :ref:`global_recordstream` source.
|
|
Also it may be configured to serve data
|
|
availability information as described in the IRIS DMC FDSNWS availability Web
|
|
Service Documentation (:cite:t:`iris-dmc`).
|
|
|
|
.. caution::
|
|
If you expose the FDSN Web Service as a public service, make sure that
|
|
the database connection is read-only. fdsnws will never attempt to write
|
|
into the database.
|
|
|
|
|
|
Service Overview
|
|
================
|
|
|
|
The following services are available:
|
|
|
|
.. csv-table::
|
|
:header: "Service", "Provides", "Provided format"
|
|
|
|
":ref:`fdsnws-dataselect <sec-dataSelect>`", "time series data", "`miniSEED <http://www.iris.edu/data/miniseed.htm>`_"
|
|
":ref:`fdsnws-station <sec-station>`", "network, station, channel, response metadata", "`FDSN Station XML <http://www.fdsn.org/xml/station/>`_, `StationXML <http://www.data.scec.org/station/xml.html>`_, :term:`SCML`"
|
|
":ref:`fdsnws-event <sec-event>`", "earthquake origin and magnitude estimates", "`QuakeML <https://quake.ethz.ch/quakeml>`_, :term:`SCML`"
|
|
":ref:`ext-availability <sec-avail>`", "waveform data availability information", "text, geocsv, json, sync, request (`fdsnws-dataselect <https://service.iris.edu/fdsnws/dataselect/1>`_)"
|
|
|
|
|
|
The available services can be reached from the fdsnws start page. The services
|
|
also provide an interactive URL builder constructing the request URL based on
|
|
user inputs. The FDSN specifications can be found on :cite:t:`fdsn`.
|
|
|
|
**URL**
|
|
|
|
* http://localhost:8080/fdsnws
|
|
|
|
If ``fdsnws`` is started, it accepts connections by default on port 8080 which
|
|
can be changed in the configuration. Also please read :ref:`sec-port` for
|
|
running the services on a privileged port, e.g. port 80 as requested by the
|
|
FDSNWS specification.
|
|
|
|
.. note::
|
|
|
|
If you decide to run the service on a different URL than ``localhost:8080``
|
|
you have to change the URL string in the ``*.wadl`` documents located under
|
|
``$DATADIR/fdsnws``.
|
|
|
|
|
|
.. _sec-dataSelect:
|
|
|
|
DataSelect
|
|
----------
|
|
|
|
* Provides time series data in miniSEED format
|
|
* Request type: HTTP-GET, HTTP-POST
|
|
|
|
|
|
URL
|
|
~~~
|
|
|
|
* http://localhost:8080/fdsnws/dataselect/1/builder
|
|
* http://localhost:8080/fdsnws/dataselect/1/query
|
|
* http://localhost:8080/fdsnws/dataselect/1/queryauth
|
|
* http://localhost:8080/fdsnws/dataselect/1/version
|
|
* http://localhost:8080/fdsnws/dataselect/1/application.wadl
|
|
|
|
|
|
Example
|
|
~~~~~~~
|
|
|
|
* Request URL for querying waveform data from the GE station BKNI, all BH channels
|
|
on 11 April 2013 between 00:00:00 and 12:00:00:
|
|
|
|
``http://localhost:8080/fdsnws/dataselect/1/query?net=GE&sta=BKNI&cha=BH?&start=2013-04-11T00:00:00&end=2013-04-11T12:00:00``
|
|
|
|
To submit HTTP-POST requests the command line tool ``curl`` may be used:
|
|
|
|
.. code-block:: sh
|
|
|
|
sysop@host:~$ curl -X POST --data-binary @request.txt "http://localhost:8080/fdsnws/dataselect/1/query"
|
|
|
|
where *request.txt* contains the POST message body. For details read the
|
|
FDSN specifications.
|
|
|
|
|
|
Feature Notes
|
|
~~~~~~~~~~~~~
|
|
|
|
* ``quality`` parameter not implemented (information not available in SeisComP)
|
|
* ``minimumlength`` parameter is not implemented
|
|
* ``longestonly`` parameter is not implemented
|
|
* Access to restricted networks and stations is only granted through the
|
|
``queryauth`` method
|
|
* The data channels exposed by this service may be restricted by defining an
|
|
inventory filter, see section :ref:`sec-inv-filter`.
|
|
* **No** trimming of miniSEED records to requested time window -- This FDSNWS
|
|
implementation returns the records as available in its data source, e.g., SDS
|
|
archive. It is guaranteed that the requested time range is within the returned
|
|
data (if available in the archive) but not that it is exactly the requested
|
|
time range. FDSNWS does not trim and therefore re-encode miniSEED records. The
|
|
rationale for that is that miniSEED records are probably further distributed
|
|
and stored in other archives and we do not see the point in modifying them.
|
|
Furthermore we do not want to increase the load on the web server with that
|
|
extra processing step.
|
|
|
|
|
|
Service Configuration
|
|
~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
* Activate :confval:`serveDataSelect` in the module configuration
|
|
* Configure the :ref:`global_recordstream` in the module's global configuration.
|
|
If the data is stored in a local waveform archive the
|
|
:ref:`rs-sdsarchive` provides fast access to the data. For archives on remote hosts
|
|
use :ref:`rs-arclink` or :ref:`rs-fdsnws` instead.
|
|
|
|
.. warning::
|
|
|
|
Requesting future or delayed data may block the :ref:`sec-dataSelect` service.
|
|
Therefore, real-time :ref:`global_recordstream` requests such as :ref:`rs-slink`
|
|
should be avoided.
|
|
If :ref:`rs-slink` is inevitable make use of the ``timeout`` and
|
|
``retries`` parameters. E.g. set the :confval:`recordstream` to
|
|
``slink://localhost:18000?timeout=1&retries=0`` or in case of the :ref:`rs-combined`
|
|
service to
|
|
``combined://slink/localhost:18000?timeout=1&retries=0;sdsarchive//home/sysop/seiscomp/var/lib/archive``.
|
|
|
|
|
|
.. _sec-station:
|
|
|
|
Station
|
|
-------
|
|
|
|
* Provides network, station, channel, response metadata
|
|
* Request type: HTTP-GET, HTTP-POST
|
|
* Stations may be filtered e.g. by geographic region and time, also the
|
|
information depth level is selectable
|
|
* Optionally handles time-based conditional HTTP-GET requests as specified by
|
|
`RFC 7232 <https://tools.ietf.org/html/rfc7232>`_.
|
|
|
|
|
|
URL
|
|
~~~
|
|
|
|
* http://localhost:8080/fdsnws/station/1/builder
|
|
* http://localhost:8080/fdsnws/station/1/query
|
|
* http://localhost:8080/fdsnws/station/1/version
|
|
* http://localhost:8080/fdsnws/station/1/application.wadl
|
|
|
|
|
|
Example
|
|
~~~~~~~
|
|
|
|
* Request URL for querying the information for the GE network on response level:
|
|
|
|
http://localhost:8080/fdsnws/station/1/query?net=GE&cha=BH%3F&level=response&nodata=404
|
|
|
|
|
|
Feature Notes
|
|
~~~~~~~~~~~~~
|
|
|
|
* To enable FDSNXML or StationXML support load the plugin ``fdsnxml``. The
|
|
plugin is loaded by default configuration.
|
|
* ``updatedafter`` request parameter not implemented: The last modification time
|
|
in |scname| is tracked on the object level. If a child of an object is updated
|
|
the update time is not propagated to all parents. In order to check if a
|
|
station was updated all children must be evaluated recursively. This operation
|
|
would be much too expensive.
|
|
* ``formatted``: boolean, default: ``false``
|
|
* Additional values of request parameters:
|
|
|
|
* format:
|
|
|
|
* standard: ``[xml, text]``
|
|
* additional: ``[fdsnxml (=xml), stationxml, sc3ml]``
|
|
* default: ``xml``
|
|
|
|
The inventory exposed by this service may be restricted, see section
|
|
:ref:`sec-inv-filter`.
|
|
|
|
|
|
.. _sec-event:
|
|
|
|
Event
|
|
-----
|
|
|
|
* Provides earthquake origin and magnitude estimates
|
|
* Request type: HTTP-GET
|
|
* Events may be filtered e.g. by hypocenter, time and magnitude
|
|
|
|
|
|
URL
|
|
~~~
|
|
|
|
* http://localhost:8080/fdsnws/event/1/builder
|
|
* http://localhost:8080/fdsnws/event/1/query
|
|
* http://localhost:8080/fdsnws/event/1/catalogs
|
|
* http://localhost:8080/fdsnws/event/1/contributors
|
|
* http://localhost:8080/fdsnws/event/1/version
|
|
* http://localhost:8080/fdsnws/event/1/application.wadl
|
|
|
|
|
|
Example
|
|
~~~~~~~
|
|
|
|
* Request URL for fetching the event parameters within 10 degrees around 50°N/11°E
|
|
starting on 18 April 2013:
|
|
|
|
http://localhost:8080/fdsnws/event/1/query?start=2018-06-01&lat=50&lon=11&maxradius=10&nodata=404
|
|
|
|
|
|
Feature Notes
|
|
~~~~~~~~~~~~~
|
|
|
|
* |scname| does not distinguish between catalogs and contributors, but
|
|
supports agencyIDs. Hence, if specified, the value of the ``contributor``
|
|
parameter is mapped to the agencyID. The file
|
|
``@DATADIR@/share/fdsn/contributors.xml`` has to be filled manually with all
|
|
available agency ids
|
|
* Origin and magnitude filter parameters are always applied to preferred origin
|
|
resp. preferred magnitude
|
|
* ``updatedafter`` request parameter not implemented: The last modification time
|
|
in |scname| is tracked on the object level. If a child of an object is updated
|
|
the update time is not propagated to all parents. In order to check if a
|
|
station was updated all children must be evaluated recursively. This operation
|
|
would be much too expensive.
|
|
* Additional request parameters:
|
|
|
|
* ``includepicks``: boolean, default: ``false``, works only in combination
|
|
with ``includearrivals`` set to ``true``
|
|
* ``includecomments``: boolean, default: ``true``
|
|
* ``formatted``: boolean, default: ``false``
|
|
|
|
* Additional values of request parameters:
|
|
|
|
* format:
|
|
|
|
* standard: ``[xml, text]``
|
|
* additional: ``[qml (=xml), qml-rt, sc3ml, csv]``
|
|
* default: ``xml``
|
|
|
|
|
|
.. _sec-avail:
|
|
|
|
Data Availability
|
|
-----------------
|
|
|
|
The data availability web service returns detailed time span information of
|
|
what time series data is available at the DMC archive. The availability information
|
|
can be created by :ref:`scardac` in the |scname| database from where it is
|
|
fetched by fdsnws.
|
|
|
|
The availability service is no official standard yet. This implementation aims
|
|
to be compatible with the IRIS DMC availability FDSN Web Service
|
|
(:cite:t:`iris-dmc`) implementation.
|
|
|
|
* request type: HTTP-GET, HTTP-POST
|
|
* results may be filtered e.g. by channel code, time and quality
|
|
|
|
|
|
URL
|
|
~~~
|
|
|
|
* http://localhost:8080/ext/availability/1/extent - Produces list of available
|
|
time extents (earliest to latest) for selected channels (network, station,
|
|
location and quality) and time ranges.
|
|
* http://localhost:8080/ext/availability/1/builder-extent - URL builder helping
|
|
you to form your data extent requests
|
|
* http://localhost:8080/ext/availability/1/query - Produces list of contiguous
|
|
time spans for selected channels (network, station, location, channel and
|
|
quality) and time ranges.
|
|
* http://localhost:8080/ext/availability/1/builder - URL builder helping you to
|
|
form your data time span requests
|
|
* http://localhost:8080/ext/availability/1/version
|
|
|
|
|
|
Examples
|
|
~~~~~~~~
|
|
|
|
* Request URL for data extents of seismic network ``IU``:
|
|
|
|
http://localhost:8080/fdsnws/ext/availability/1/extent?net=IU
|
|
|
|
* Further limit the extents to those providing data for August 1st 2018:
|
|
|
|
http://localhost:8080/fdsnws/ext/availability/1/extent?net=IU&start=2018-08-01
|
|
|
|
* Request URL for continues time spans of station ``ANMO`` in July 2018:
|
|
|
|
http://localhost:8080/fdsnws/ext/availability/1/query?sta=ANMO&start=2018-07-01&end=2018-08-01
|
|
|
|
.. note::
|
|
|
|
Use :ref:`scardac` for creating the availability information.
|
|
|
|
|
|
Feature Notes
|
|
~~~~~~~~~~~~~
|
|
|
|
* The IRISWS availability implementation truncates the time spans of the returned
|
|
data extents and segments to the requested start and end times (if any). This
|
|
implementation truncates the start and end time only for the formats: ``sync``
|
|
and ``request``. The ``text``, ``geocsv`` and ``json`` format will return the
|
|
exact time windows extracted from the waveform archive.
|
|
|
|
The reasons for this derivation are:
|
|
|
|
* performance: With the ``/extent`` query the ``text``, ``geocsv`` and
|
|
``json`` offer the display of the number of included time spans
|
|
(``show=timespancount``). The data model offers no efficient way to
|
|
recalculate the number of time spans represented by an extent if the extents
|
|
time window is altered by the requested start and end times. The ``sync``
|
|
and ``request`` formats do not provided this counter and it is convenient to
|
|
use their outputs for subsequent data requests.
|
|
* by truncating the time windows information is lost. There would be no
|
|
efficient way for a client to retrieve the exact time windows falling into a
|
|
specific time span.
|
|
* network and station epochs returned by the :ref:`sec-station` service are also
|
|
not truncated to the requested start and end times.
|
|
* truncation can easily be done on client side. No additional network traffic is
|
|
generated.
|
|
|
|
|
|
.. _sec-inv-filter:
|
|
|
|
Filtering Inventories
|
|
=====================
|
|
|
|
The channels served by the :ref:`sec-station` and :ref:`sec-dataSelect` services
|
|
may be filtered by specified an INI file in the ``stationFilter`` and
|
|
``dataSelectFilter`` configuration parameter. You may use the same file for both
|
|
services or define a separate configuration set. **Note:** If distinct file
|
|
names are specified and both services are activated, the inventory is loaded
|
|
twice which will increase the memory consumption of this module.
|
|
|
|
.. code-block:: ini
|
|
|
|
[Chile]
|
|
code = CX.*.*.*
|
|
|
|
[!Exclude station APE]
|
|
code = GE.APE.*.*
|
|
|
|
[German (not restricted)]
|
|
code = GE.*.*.*
|
|
restricted = false
|
|
shared = true
|
|
archive = GFZ
|
|
|
|
The listing above shows a configuration example which includes all Chile
|
|
stations. Also all not restricted German stations, with exception of the station
|
|
GE.APE, are included.
|
|
|
|
The configuration is divided into several rules. The rule name is given in
|
|
square brackets. A name starting with an exclamation mark defines an exclude
|
|
rule, else the rule is an include. The rule name is not evaluated by the
|
|
application but is plotted when debugging the rule set, see configuration
|
|
parameter ``debugFilter``.
|
|
|
|
Each rule consists of a set of attributes. The first and mandatory attribute is
|
|
``code`` which defines a regular expression for the channel code (network,
|
|
station, location, channel). In addition the following optional attributes
|
|
exist:
|
|
|
|
.. csv-table::
|
|
:header: "Attribute", "Type", "Network", "Station", "Location", "Channel"
|
|
|
|
"**restricted**", "Boolean", "X", "X", "", "X"
|
|
"**shared**", "Boolean", "X", "X", "", "X"
|
|
"**netClass**", "String", "X", "", "", ""
|
|
"**archive**", "String", "X", "X", "", ""
|
|
|
|
A rule matches if all of its attributes match. The optional attributes are
|
|
evaluated bottom-up where ever they are applicable. E.g. if a rule defines
|
|
``restricted = false`` but the restricted flag is not present on channel level
|
|
then it is searched on station and then on network level. If no ``restricted``
|
|
attribute is found in the hierarchy, the rule will not match even if the value
|
|
was set to ``false``.
|
|
|
|
The individual rules are evaluated in order of their definition. The processing
|
|
stops once a matching rule is found and the channel is included or excluded
|
|
immediately. So the order of the rules is important.
|
|
|
|
One may decided to specify a pure whitelist, a pure blacklist, or to mix include
|
|
and exclude rules. If neither a matching include nor exclude rule is found, then
|
|
channel is only added if no other include rule exists in the entire rule set.
|
|
|
|
|
|
.. _sec-port:
|
|
|
|
Changing the Service Port
|
|
=========================
|
|
|
|
The FDSN Web service specification defines that the Service SHOULD be available
|
|
under port 80. Typically |scname| runs under a user without root permissions
|
|
and therefore is not allowed to bind to privileged ports (<1024).
|
|
To serve on port 80 you may for instance
|
|
|
|
* Run |scname| with root privileged (not recommended)
|
|
* Use a proxy Webserver, e.g. Apache with
|
|
`mod-proxy <http://httpd.apache.org/docs/2.2/mod/mod_proxy.html>`_ module
|
|
* Configure and use :ref:`sec-authbind`
|
|
* Setup :ref:`sec-firewall` redirect rules
|
|
|
|
|
|
.. _sec-authbind:
|
|
|
|
Authbind
|
|
========
|
|
|
|
``authbind`` allows a program which does not or should not run as root to bind
|
|
to low-numbered ports in a controlled way. Please refer to ``man authbind`` for
|
|
program descriptions. The following lines show how to install and setup authbind
|
|
for the user ``sysop`` under the Ubuntu OS.
|
|
|
|
.. code-block:: sh
|
|
|
|
sysop@host:~$ sudo apt-get install authbind
|
|
sysop@host:~$ sudo touch /etc/authbind/byport/80
|
|
sysop@host:~$ sudo chown sysop /etc/authbind/byport/80
|
|
sysop@host:~$ sudo chmod 500 /etc/authbind/byport/80
|
|
|
|
Once ``authbind`` is configured correctly the FDSN Web services may be started
|
|
as follows:
|
|
|
|
.. code-block:: sh
|
|
|
|
sysop@host:~$ authbind --deep seiscomp exec fdsnws
|
|
|
|
In order use ``authbind`` when starting ``fdsnws`` as |scname| service the last
|
|
line in the ``~/seiscomp/etc/init/fdsnws.py`` have to be commented in.
|
|
|
|
|
|
.. _sec-firewall:
|
|
|
|
Firewall
|
|
========
|
|
|
|
All major Linux distributions ship with their own firewall implementations which
|
|
are front-ends for the ``iptables`` kernel functions. The following line
|
|
temporary adds a firewall rule which redirects all incoming traffic on port 8080
|
|
to port 80.
|
|
|
|
.. code-block:: sh
|
|
|
|
sysop@host:~$ sudo iptables -t nat -A PREROUTING -p tcp --dport 80 -j REDIRECT --to 8080
|
|
|
|
Please refer to the documentation of your particular firewall solution on how to
|
|
set up this rule permanently.
|
|
|
|
|
|
Authentication Extension
|
|
========================
|
|
|
|
The FDSNWS standard requires HTTP digest authentication as the
|
|
authentication mechanism. The "htpasswd" configuration option is used to
|
|
define the location of the file storing usernames and passwords of users
|
|
who are allowed to get restricted data. Any user with valid credentials
|
|
would have access to all restricted data.
|
|
|
|
An extension to the FDSNWS protocol has been developed in order to use
|
|
email-pattern-based access control lists, which is an established
|
|
authorization mechanism in SeisComP3 (used by Arclink). It works as follows:
|
|
|
|
* The user contacts an authentication service (based on eduGAIN AAI,
|
|
e-mail, etc.) and receives a list of attributes (a token), signed by the
|
|
authentication service. The validity of the token is typically 30 days.
|
|
|
|
* The user presents the token to /auth method (HTTPS) of the dataselect
|
|
service. This method is the only extension to standard FDSNWS that is
|
|
required.
|
|
|
|
* If the digital signature is valid, a temporary account for /queryauth
|
|
is created. The /auth method returns username and password of this
|
|
account, separated by ':'. The account is typically valid for 24 hours.
|
|
|
|
* The username and password are to be used with /queryauth as usual.
|
|
|
|
* Authorization is based on user's e-mail address in the token and
|
|
arclink-access bindings.
|
|
|
|
|
|
Configuration
|
|
-------------
|
|
|
|
The authentication extension is enabled by setting the "auth.enable"
|
|
configuration option to "true" and pointing "auth.gnupgHome" to a directory
|
|
where GPG stores its files. Let's use the directory
|
|
~/seiscomp/var/lib/gpg, which is the default.
|
|
|
|
* First create the direcory and your own signing key:
|
|
|
|
.. code-block:: sh
|
|
|
|
sysop@host:~$ mkdir -m 700 ~/seiscomp/var/lib/gpg
|
|
sysop@host:~$ gpg --homedir ~/seiscomp/var/lib/gpg --gen-key
|
|
|
|
* Now import GPG keys of all authentication services you trust:
|
|
|
|
.. code-block:: sh
|
|
|
|
sysop@host:~$ gpg --homedir ~/seiscomp/var/lib/gpg --import <keys.asc
|
|
|
|
* Finally sign all imported keys with your own key (XXXXXXXX is the ID of
|
|
an imported key):
|
|
|
|
.. code-block:: sh
|
|
|
|
sysop@host:~$ gpg --homedir ~/seiscomp/var/lib/gpg --edit-key XXXXXXXX sign save
|
|
|
|
* ...and set auth.enable, either using the "scconfig" tool or:
|
|
|
|
.. code-block:: sh
|
|
|
|
sysop@host:~$ echo "auth.enable = true" >> ~/seiscomp/etc/fdsnws.cfg
|
|
|
|
|
|
Usage Example
|
|
-------------
|
|
|
|
A client like :ref:`fdsnws_fetch <sec-fdsnws-related>` is recommended, but also tools like wget and
|
|
curl can be used. As an example, let's request data from the restricted
|
|
station AAI (assuming that we are authorized to get data of this station).
|
|
|
|
* The first step is to obtain the token from an authentication service.
|
|
Assuming that the token is saved in "token.asc", credentials of the
|
|
temporary account can be requested using one of the following commands:
|
|
|
|
.. code-block:: sh
|
|
|
|
sysop@host:~$ wget --post-file token.asc https://geofon.gfz-potsdam.de/fdsnws/dataselect/1/auth -O cred.txt
|
|
sysop@host:~$ curl --data-binary @token.asc https://geofon.gfz-potsdam.de/fdsnws/dataselect/1/auth -o cred.txt
|
|
|
|
* The resulting file "cred.txt" contains username and password separated by
|
|
a colon, so one can conveniently use a shell expansion:
|
|
|
|
.. code-block:: sh
|
|
|
|
sysop@host:~$ wget "http://`cat cred.txt`@geofon.gfz-potsdam.de/fdsnws/dataselect/1/queryauth?starttime=2015-12-15T16:00:00Z&endtime=2015-12-15T16:10:00Z&network=IA&station=AAI" -O data.mseed
|
|
sysop@host:~$ curl --digest "http://`cat cred.txt`@geofon.gfz-potsdam.de/fdsnws/dataselect/1/queryauth?starttime=2015-12-15T16:00:00Z&endtime=2015-12-15T16:10:00Z&network=IA&station=AAI" -o data.mseed
|
|
|
|
* Using the :ref:`fdsnws_fetch <sec-fdsnws-related>` utility, the two steps above can be combined into
|
|
one:
|
|
|
|
.. code-block:: sh
|
|
|
|
sysop@host:~$ fdsnws_fetch -a token.asc -s 2015-12-15T16:00:00Z -e 2015-12-15T16:10:00Z -N IA -S AAI -o data.mseed
|
|
|
|
|
|
Logging
|
|
=======
|
|
|
|
In addition to normal |scname| logs, fdsnws can create a simple HTTP access log
|
|
and/or a detailed request log. The locations of log files are specified by
|
|
"accessLog" and "requestLog" in fdsnws.cfg.
|
|
|
|
Both logs are text-based and line-oriented. Each line of *access* log
|
|
contains the following fields, separated by '|' (some fields can be empty):
|
|
|
|
* service name;
|
|
* hostname of service;
|
|
* access time;
|
|
* hostname of user;
|
|
* IP address of user (proxy);
|
|
* length of data in bytes;
|
|
* processing time in milliseconds;
|
|
* error message;
|
|
* agent string;
|
|
* HTTP response code;
|
|
* username (if authenticated);
|
|
* network code of GET request;
|
|
* station code of GET request;
|
|
* location code of GET request;
|
|
* channel code of GET request;
|
|
|
|
Each line of *request* log contains a JSON object, which has the following
|
|
attributes:
|
|
|
|
service
|
|
service name
|
|
|
|
userID
|
|
anonymized (numeric) user ID for statistic purposes
|
|
|
|
clientID
|
|
agent string
|
|
|
|
userEmail
|
|
e-mail address of authenticated user if using restricted data
|
|
|
|
auth
|
|
True if user is authenticated (not anonymous)
|
|
|
|
userLocation
|
|
JSON object containing rough user location (eg., country) for statistic purposes
|
|
|
|
created
|
|
time of request creation
|
|
|
|
status
|
|
"OK", "NODATA", "ERROR" or "DENIED"
|
|
|
|
bytes
|
|
length of data in bytes
|
|
|
|
finished
|
|
time of request completion
|
|
|
|
trace
|
|
request content after wildcard expansion (array of JSON objects)
|
|
|
|
Each trace object has the following attributes:
|
|
|
|
net
|
|
network code
|
|
|
|
sta
|
|
station code
|
|
|
|
loc
|
|
location code
|
|
|
|
cha
|
|
channel code
|
|
|
|
start
|
|
start time
|
|
|
|
end
|
|
end time
|
|
|
|
restricted
|
|
True if the data requires authorization
|
|
|
|
status
|
|
"OK", "NODATA", "ERROR" or "DENIED"
|
|
|
|
bytes
|
|
length of trace in bytes
|
|
|
|
Both logs are rotated daily. In case of access log, one week of data is
|
|
kept. Request logs are compressed using bzip2 and not removed.
|
|
|
|
If trackdb.enable=true in fdsnws.cfg, then requests are additionally logged
|
|
into SeisComP database using the ArcLink request log schema. Be aware that the
|
|
number of requests in a production system can be rather large. For example,
|
|
the GEOFON datacentre is currently serving between 0.5..1 million FDSNWS
|
|
requests per day.
|
|
|
|
|
|
.. _sec-fdsnws-related:
|
|
|
|
Related Modules
|
|
===============
|
|
|
|
:term:`GEOFON` maintains scripts for FDSNWS :cite:p:`fdsnws_scripts`:
|
|
|
|
* The :program:`fdsnws_fetch` client is a convenient tool for requesting
|
|
waveforms from a FDSN web service hosted by :term:`EIDA` nodes.
|
|
* The :program:`fdsnws2sds` client is a tool for requesting waveforms
|
|
from a FDSN web service hosted by :term:`EIDA` nodes and to store them into an
|
|
:term:`SDS` archive.
|
|
|
|
|
|
Public FDSN Web Servers
|
|
=======================
|
|
|
|
IRIS maintains a list of data centers (:cite:t:`fdsn-datacenters`)
|
|
supporting FDSN Web Services (:cite:t:`fdsn`).
|
|
|
|
|
|
.. _fdsnws_configuration:
|
|
|
|
Module Configuration
|
|
====================
|
|
|
|
| :file:`etc/defaults/global.cfg`
|
|
| :file:`etc/defaults/fdsnws.cfg`
|
|
| :file:`etc/global.cfg`
|
|
| :file:`etc/fdsnws.cfg`
|
|
| :file:`~/.seiscomp/global.cfg`
|
|
| :file:`~/.seiscomp/fdsnws.cfg`
|
|
|
|
fdsnws inherits :ref:`global options<global-configuration>`.
|
|
|
|
|
|
|
|
.. confval:: listenAddress
|
|
|
|
Default: ``0.0.0.0``
|
|
|
|
Type: *IP*
|
|
|
|
Define the bind address of the server. \"0.0.0.0\" allows
|
|
any interface to connect to this server whereas \"127.0.0.0\"
|
|
only allows connections from localhost.
|
|
|
|
|
|
.. confval:: port
|
|
|
|
Default: ``8080``
|
|
|
|
Type: *int*
|
|
|
|
Server port to listen for incoming requests. Note: The FDSN Web
|
|
service specification defines the service port 80. Please refer
|
|
to the documentation on how to serve on privileged ports.
|
|
|
|
|
|
.. confval:: connections
|
|
|
|
Default: ``5``
|
|
|
|
Type: *int*
|
|
|
|
Number of maximum simultaneous requests.
|
|
|
|
|
|
.. confval:: queryObjects
|
|
|
|
Default: ``10000``
|
|
|
|
Type: *int*
|
|
|
|
Maximum number of objects per query, used in fdsnws\-station and
|
|
fdsnws\-event to limit main memory consumption.
|
|
|
|
|
|
.. confval:: realtimeGap
|
|
|
|
Type: *int*
|
|
|
|
Unit: *s*
|
|
|
|
Restrict end time of requests to current time \- realtimeGap
|
|
seconds. Negative values allowed. Used in fdsnws\-dataselect.
|
|
WARNING: If this value is unset and a realtime recordsource
|
|
\(e.g. slink\) is used, requests may block if end time in future
|
|
is requested.
|
|
|
|
|
|
.. confval:: samplesM
|
|
|
|
Type: *float*
|
|
|
|
Maximum number of samples \(in units of million\) per query, used
|
|
in fdsnws\-dataselect to prevent a single user to block one
|
|
connection with a large request.
|
|
|
|
|
|
.. confval:: recordBulkSize
|
|
|
|
Default: ``102400``
|
|
|
|
Type: *int*
|
|
|
|
Unit: *bytes*
|
|
|
|
Set the number of bytes to buffer for each chunk of waveform data
|
|
served to the client. The lower the buffer the higher the overhead
|
|
of Python Twisted. The higher the buffer the higher the memory
|
|
usage per request. 100kB seems to be a good trade\-off.
|
|
|
|
|
|
.. confval:: htpasswd
|
|
|
|
Default: ``@CONFIGDIR@/fdsnws.htpasswd``
|
|
|
|
Type: *string*
|
|
|
|
Path to password file used in fdsnws\-dataselect\/queryauth. The
|
|
format is 'username:password' separated by lines. Because of the
|
|
HTTP digest authentication method required by the FDSN
|
|
specification, the passwords have to be stored in plain text.
|
|
|
|
|
|
.. confval:: accessLog
|
|
|
|
Type: *string*
|
|
|
|
Path to access log file. If unset no access log is created.
|
|
|
|
|
|
.. confval:: requestLog
|
|
|
|
Type: *string*
|
|
|
|
Path to request log file. If unset no request log is created.
|
|
|
|
|
|
.. confval:: userSalt
|
|
|
|
Type: *string*
|
|
|
|
Secret salt for calculating userID.
|
|
|
|
|
|
.. confval:: corsOrigins
|
|
|
|
Default: ``*``
|
|
|
|
Type: *list:string:*
|
|
|
|
List of domain names Cross\-Origin Resource Sharing \(CORS\)
|
|
request may originate from. A value of '\*' allows any web page
|
|
to embed your service. An empty value will switch of CORS
|
|
requests entirely. An example of multiple domains might be:
|
|
'https:\/\/test.domain.de, https:\/\/production.domain.de'.
|
|
|
|
|
|
.. confval:: allowRestricted
|
|
|
|
Default: ``true``
|
|
|
|
Type: *boolean*
|
|
|
|
Enable\/disable access to restricted inventory data.
|
|
|
|
|
|
.. confval:: handleConditionalRequests
|
|
|
|
Default: ``false``
|
|
|
|
Type: *boolean*
|
|
|
|
Enable\/disable handling of time\-based conditional requests \(RFC
|
|
7232\) by the fdsnws\-station resource.
|
|
|
|
|
|
.. confval:: useArclinkAccess
|
|
|
|
Default: ``false``
|
|
|
|
Type: *boolean*
|
|
|
|
If enabled, then access to restricted waveform data is
|
|
controlled by arclink\-access bindings. By default authenticated
|
|
users have access to all data.
|
|
|
|
|
|
.. confval:: hideAuthor
|
|
|
|
Default: ``false``
|
|
|
|
Type: *boolean*
|
|
|
|
If enabled, author information is removed from any event
|
|
creationInfo element.
|
|
|
|
|
|
.. confval:: hideComments
|
|
|
|
Default: ``false``
|
|
|
|
Type: *boolean*
|
|
|
|
If enabled, event comment elements are no longer accessible.
|
|
|
|
|
|
.. confval:: evaluationMode
|
|
|
|
Type: *string*
|
|
|
|
If set, the event service will only return events having a
|
|
preferred origin with a matching evaluationMode property.
|
|
|
|
|
|
.. confval:: eventFormats
|
|
|
|
Type: *list:string*
|
|
|
|
List of enabled event formats. If unspecified, all supported
|
|
formats are enabled.
|
|
|
|
|
|
.. confval:: serveDataSelect
|
|
|
|
Default: ``true``
|
|
|
|
Type: *boolean*
|
|
|
|
Enable\/disable the DataSelect service.
|
|
|
|
|
|
.. confval:: serveEvent
|
|
|
|
Default: ``true``
|
|
|
|
Type: *boolean*
|
|
|
|
Enable\/disable the Event service.
|
|
|
|
|
|
.. confval:: serveStation
|
|
|
|
Default: ``true``
|
|
|
|
Type: *boolean*
|
|
|
|
Enable\/disable the Station service.
|
|
|
|
|
|
.. confval:: serveAvailability
|
|
|
|
Default: ``false``
|
|
|
|
Type: *boolean*
|
|
|
|
Enable\/disable the Availability service. Note: This is a non
|
|
standard FDSNWS extension served under fdsnws\/ext\/availability.
|
|
|
|
|
|
.. confval:: stationFilter
|
|
|
|
Type: *string*
|
|
|
|
Path to station inventory filter file.
|
|
|
|
|
|
.. confval:: dataSelectFilter
|
|
|
|
Type: *string*
|
|
|
|
Path to dataselect inventory filter file.
|
|
|
|
|
|
.. confval:: debugFilter
|
|
|
|
Default: ``false``
|
|
|
|
Type: *boolean*
|
|
|
|
If enabled, a debug line is written for each stream ID explaining
|
|
why a stream was added\/removed by a inventory filter.
|
|
|
|
|
|
.. confval:: fileNamePrefix
|
|
|
|
Default: ``fdsnws``
|
|
|
|
Type: *string*
|
|
|
|
Define the prefix for the default filenames if downloading and
|
|
saving data from within a browser.
|
|
For data loaded using dataselect, it is thus fdsnws.mseed by default.
|
|
|
|
|
|
.. confval:: eventType.whitelist
|
|
|
|
Type: *list:string*
|
|
|
|
List of enabled event types
|
|
|
|
|
|
.. confval:: eventType.blacklist
|
|
|
|
Type: *list:string*
|
|
|
|
List of disabled event types
|
|
|
|
|
|
.. note::
|
|
**dataAvailability.\***
|
|
*Provide access to waveform data availability information stored*
|
|
*in the SeisComP database. In case of a SDS archive, this*
|
|
*information may be collected by scardac (SeisComP archive*
|
|
*data availability collector).*
|
|
|
|
|
|
|
|
.. confval:: dataAvailability.enable
|
|
|
|
Default: ``false``
|
|
|
|
Type: *boolean*
|
|
|
|
Enable loading of data availabilty information from
|
|
SeisComP database. Availability information is used by
|
|
station and ext\/availability service.
|
|
|
|
|
|
.. confval:: dataAvailability.cacheDuration
|
|
|
|
Default: ``300``
|
|
|
|
Type: *int*
|
|
|
|
Unit: *s*
|
|
|
|
Number of seconds data availabilty information is considered
|
|
valid. If the duration time is exceeded, the information is
|
|
fetched again from the database.
|
|
|
|
|
|
.. confval:: dataAvailability.dccName
|
|
|
|
Default: ``DCC``
|
|
|
|
Type: *string*
|
|
|
|
Name of the archive use in sync format of dataavailability
|
|
extent service.
|
|
|
|
|
|
.. confval:: dataAvailability.repositoryName
|
|
|
|
Default: ``primary``
|
|
|
|
Type: *string*
|
|
|
|
Name of the archive use in some format of data availability
|
|
extent service.
|
|
|
|
|
|
.. confval:: trackdb.enable
|
|
|
|
Default: ``false``
|
|
|
|
Type: *boolean*
|
|
|
|
Save request log to database.
|
|
|
|
|
|
.. confval:: trackdb.defaultUser
|
|
|
|
Default: ``fdsnws``
|
|
|
|
Type: *string*
|
|
|
|
Default user.
|
|
|
|
|
|
.. confval:: auth.enable
|
|
|
|
Default: ``false``
|
|
|
|
Type: *boolean*
|
|
|
|
Enable auth extension.
|
|
|
|
|
|
.. confval:: auth.gnupgHome
|
|
|
|
Default: ``@ROOTDIR@/var/lib/gpg``
|
|
|
|
Type: *string*
|
|
|
|
GnuPG home directory.
|
|
|
|
|
|
.. confval:: auth.blacklist
|
|
|
|
Type: *list:string*
|
|
|
|
List of revoked token IDs.
|
|
|
|
|
|
|
|
Command-Line Options
|
|
====================
|
|
|
|
.. program:: fdsnws
|
|
|
|
:program:`fdsnws [options]`
|
|
|
|
|
|
Generic
|
|
-------
|
|
|
|
.. option:: -h, --help
|
|
|
|
Show help message.
|
|
|
|
.. option:: -V, --version
|
|
|
|
Show version information.
|
|
|
|
.. option:: --config-file arg
|
|
|
|
Use alternative configuration file. When this option is
|
|
used the loading of all stages is disabled. Only the
|
|
given configuration file is parsed and used. To use
|
|
another name for the configuration create a symbolic
|
|
link of the application or copy it. Example:
|
|
scautopick \-> scautopick2.
|
|
|
|
.. option:: --plugins arg
|
|
|
|
Load given plugins.
|
|
|
|
.. option:: -D, --daemon
|
|
|
|
Run as daemon. This means the application will fork itself
|
|
and doesn't need to be started with \&.
|
|
|
|
.. option:: --auto-shutdown arg
|
|
|
|
Enable\/disable self\-shutdown because a master module shutdown.
|
|
This only works when messaging is enabled and the master
|
|
module sends a shutdown message \(enabled with \-\-start\-stop\-msg
|
|
for the master module\).
|
|
|
|
.. option:: --shutdown-master-module arg
|
|
|
|
Set the name of the master\-module used for auto\-shutdown.
|
|
This is the application name of the module actually
|
|
started. If symlinks are used, then it is the name of
|
|
the symlinked application.
|
|
|
|
.. option:: --shutdown-master-username arg
|
|
|
|
Set the name of the master\-username of the messaging
|
|
used for auto\-shutdown. If \"shutdown\-master\-module\" is
|
|
given as well, this parameter is ignored.
|
|
|
|
|
|
Verbosity
|
|
---------
|
|
|
|
.. option:: --verbosity arg
|
|
|
|
Verbosity level [0..4]. 0:quiet, 1:error, 2:warning, 3:info,
|
|
4:debug.
|
|
|
|
.. option:: -v, --v
|
|
|
|
Increase verbosity level \(may be repeated, eg. \-vv\).
|
|
|
|
.. option:: -q, --quiet
|
|
|
|
Quiet mode: no logging output.
|
|
|
|
.. option:: --component arg
|
|
|
|
Limit the logging to a certain component. This option can
|
|
be given more than once.
|
|
|
|
.. option:: -s, --syslog
|
|
|
|
Use syslog logging backend. The output usually goes to
|
|
\/var\/lib\/messages.
|
|
|
|
.. option:: -l, --lockfile arg
|
|
|
|
Path to lock file.
|
|
|
|
.. option:: --console arg
|
|
|
|
Send log output to stdout.
|
|
|
|
.. option:: --debug
|
|
|
|
Execute in debug mode.
|
|
Equivalent to \-\-verbosity\=4 \-\-console\=1 .
|
|
|
|
.. option:: --log-file arg
|
|
|
|
Use alternative log file.
|
|
|
|
|
|
Database
|
|
--------
|
|
|
|
.. option:: --db-driver-list
|
|
|
|
List all supported database drivers.
|
|
|
|
.. option:: -d, --database arg
|
|
|
|
The database connection string, format:
|
|
service:\/\/user:pwd\@host\/database.
|
|
\"service\" is the name of the database driver which
|
|
can be queried with \"\-\-db\-driver\-list\".
|
|
|
|
.. option:: --config-module arg
|
|
|
|
The config module to use.
|
|
|
|
.. option:: --inventory-db arg
|
|
|
|
Load the inventory from the given database or file, format:
|
|
[service:\/\/]location .
|
|
|
|
.. option:: --db-disable
|
|
|
|
Do not use the database at all
|
|
|
|
|
|
Records
|
|
-------
|
|
|
|
.. option:: --record-driver-list
|
|
|
|
List all supported record stream drivers.
|
|
|
|
.. option:: -I, --record-url arg
|
|
|
|
The recordstream source URL, format:
|
|
[service:\/\/]location[#type].
|
|
\"service\" is the name of the recordstream driver
|
|
which can be queried with \"\-\-record\-driver\-list\".
|
|
If \"service\" is not given, \"file:\/\/\" is
|
|
used.
|
|
|
|
.. option:: --record-file arg
|
|
|
|
Specify a file as record source.
|
|
|
|
.. option:: --record-type arg
|
|
|
|
Specify a type for the records being read.
|
|
|