NOAA/seiscomp-training
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

[installation] Init with inital config for global

Browse Source
This commit is contained in:
Dirk Rössler
2025-10-30 15:08:17 +01:00
commit 7640b452ed
3678 changed files with 2200095 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

551
share/doc/caps/html/_sources/apps/ngl2caps.rst.txt Normal file
View File

@ -0,0 +1,551 @@
.. highlight:: rst
.. _ngl2caps:
########
ngl2caps
########
**NGL CAPS plugin. Reads GNSS data in kenv format and sends it to CAPS.**
Description
===========
Read GNSS data in kenv format, convert to RAW and send to a CAPS server.
Waveform Data
=============
Format describtions
-------------------
* **kenv Format**:
Final 5 min rapid solutions from stations [stat]:
http://geodesy.unr.edu/gps_timeseries/rapids_5min/kenv/[stat]/
* Sample rate: 1/300 sps
* Example: http://geodesy.unr.edu/gps_timeseries/rapids_5min/kenv/0ALM/
* Format (http://geodesy.unr.edu/gps_timeseries/README_kenv.txt):
.. code-block:: properties
----------------------------------------
.kenv format (east,north,up time series)
----------------------------------------
Note: first line of .kenv contains header fields to help human interpretation.
Column Header Example Description
------ ----------- --------- ------------------------------------
1 site JPLM 4-character station ID
2 sec-J2000 3705278100 GPS seconds since 2000-01-01 12:00:00
3 __MJD 55833 Modified Julian Day for GPS day
4 year 2011 Year
5 mm 9 Month
6 dd 29 Day of month
7 doy 272 Day of year
8 s-day 10800 Seconds of the GPS day
9 ___e-ref(m) -0.202261 East from reference longitude in llh
10 ___n-ref(m) 0.079096 North from reference latitude in llh
11 ___v-ref(m) -0.025883 Up from reference height in llh
12 _e-mean(m) -0.015904 East from daily mean position
13 _n-mean(m) -0.000944 North from daily mean position
14 _v-mean(m) 0.000232 Up from daily mean position
15 sig_e(m) 0.005700 Sigma east
16 sig_n(m) 0.006875 Sigma north
17 sig_v(m) 0.021739 Sigma up
* RINEX data web server: http://geodesy.unr.edu/magnet/rinex/
Data sources
------------
kenv data are provided by Nevada Geodetic Laboratory, NGL
(http://geodesy.unr.edu/):
* **Preferred:** Rapid data, 24 hours latency, 5 minutes sps, 1 ZIP file per year:
http://geodesy.unr.edu/NGLStationPages/RapidStationList
Archives:
http://geodesy.unr.edu/gps_timeseries/kenv/
Example for one station, 1 year:
http://geodesy.unr.edu/gps_timeseries/kenv/0ABI/0ABI.2022.kenv.zip
* Ultra-rapid data, 1 hour latency, 5 minutes sps (may have many gaps and outages):
http://geodesy.unr.edu/NGLStationPages/UltraStationList
Hourly upload:
http://geodesy.unr.edu/gps_timeseries/ultracombo/kenv/2022/141/
Archives:
http://geodesy.unr.edu/gps_timeseries/kenv/
* Final 24 h solutions from stations [stat]:
http://geodesy.unr.edu/gps_timeseries/txyz/IGS14/[stat].txyz2
Example: http://geodesy.unr.edu/gps_timeseries/txyz/IGS14/ARIS.txyz2
Fetch data
----------
.. note::
Data on the NGL server are zipped files, one file per year.
Geoffrey Blewitt why this is:
"We zip into yearly files which are updated every week. The actual day it is
updated varies depending on when other necessary inputs are ready, such as
JPL orbit files, and weather model files from TU Vienna, and ocean loading
files from Chalmers, Sweden. So, you would not want to check every day, but
I would say every Wednesday would typically work.
It’s also important to note that while we process Final data every week,
often we include newly discovered or late data that can go as far back as
1994. So, we are not just incrementing with a new week of files.
On the technical side, we zip the data into yearly files to reduce the number
of “inodes” (files) on our server, which can slow things down when attempting
to seek files, and in the past has overloaded the server with maximum number
of inodes.
Zipping files has speeded up our operations considerably and allows us to
process all the world’s GPS geodetic data (currently > 18,000 stations).
It also allows for efficient backup, which can take far too long with
individual files."
For fetching data from the server above adjust:
* *year* year to consider as set in the file name. Use "" for all.
* *url*
* *outPut*: target directory of fetched files
in the Python script provided with the source code:
:file:`ngl/plugin/tools/fetchFilesFromWeb.py` and use it:
.. code-block:: sh
python fetchFilesFromWeb.py
Change to the target directory and unpack the files:
.. code-block:: sh
cd [target]
unzip *
gunzip *
Running ngl2caps converts data to RAW and sends them to the CAPS server. The
assumed physical unit of the output data is **nanometer** and the original data
are assumed in units of **meter**. The conversion is applied by the :confval:`gain`
which is therefore **1000,000,000** by default.
**Make sure to generate the inventory with the correct gain and gainUnit.**
If data are stored in "nm" and gainUnit of the stream in the inventory is "m" (SI unit),
the gain in the inventory must be :math:`10^{-09}`. Example:
.. code-block:: sh
ngl2caps --debug -p [target directory] --gain 1000000000
Output streams
--------------
The plugin writes the following parameter to the header of the output data.
.. csv-table::
:widths: 2 2 1 5
:align: left
:delim: ;
:header: group, parameter, added, value/remark
network; code; x; from input parameter
network; start time; x; from input parameter
station; code; x; from input file
station; start time; x; same as network
station; coordinates; x; from input file
sensor location; code; x; from input parameter
sensor location; start; x; same as network
sensor location; elevation; x; same as station
stream; code; x; from band + [XYZ][ZNE]
; ; ; from band + [X][ZNE] : derived data, daily mean removed
; ; ; from band + [Y][ZNE] : raw data
; ; ; from band + [Z][ZNE] : sigma data
Inventory
=========
* Access to different stations lists and formats: http://geodesy.unr.edu/PlugNPlayPortal.php
* Station list (HTML): http://geodesy.unr.edu/NGLStationPages/GlobalStationList
* Station list (TXT, 1/300 sps, 24 hour latency):
http://geodesy.unr.edu/NGLStationPages/DataHoldingsRapid5min.txt
* Details about stations [stat]: http://geodesy.unr.edu/NGLStationPages/stations/[stat].sta
* Station table with coordinates:
http://plugandplay.unavco.org:8080/unrgsac/gsacapi/site/search#tabId3153-1
:ref:`table2inv` can be used for conversion of a station table to :term:`SCML`.
Module Configuration
====================
| :file:`etc/defaults/global.cfg`
| :file:`etc/defaults/ngl2caps.cfg`
| :file:`etc/global.cfg`
| :file:`etc/ngl2caps.cfg`
| :file:`~/.seiscomp/global.cfg`
| :file:`~/.seiscomp/ngl2caps.cfg`
ngl2caps inherits :ref:`global options<global-configuration>`.
.. note::
Modules/plugins may require a license file. The default path to license
files is :file:`@DATADIR@/licenses/` which can be overridden by global
configuration of the parameter :confval:`gempa.licensePath`. Example: ::
gempa.licensePath = @CONFIGDIR@/licenses
.. _input:
.. note::
**input.\***
*Parameters controlling the input of event information and*
*reception of GNSS data.*
.. confval:: input.readFrom
Type: *string*
Read input data from this file.
.. confval:: input.directory
Type: *string*
Watch this directory for incoming input files.
.. confval:: input.watchEvents
Default: ``close_write``
Type: *string*
Listen for specific inotify event\(s\).
If ommitted, close_write events are listened for. Events:
access \- file or directory contents were read,
modify \- file or directory contents were written,
attrib \- file or directory attributes changed,
close_write \- file or directory closed, after being opened in writable mode,
close_nowrite \- file or directory closed, after being opened in read\-only mode,
close \- file or directory closed, regardless of read\/write mode,
open \- file or directory opened,
moved_to \- file or directory moved to watched directory,
moved_from \- file or directory moved from watched directory,
move \- file or directory moved to or from watched directory,
create \- file or directory created within watched directory,
delete \- file or directory deleted within watched directory,
delete_self \- file or directory was deleted,
unmount \- file system containing file or directory unmounted.
.. confval:: input.watchPattern
Type: *string*
Process any events whose filename matches the specified regular expression
.. confval:: input.leapSecondsFile
Default: ``@DATADIR@/caps/plugins/ngl2caps/leapseconds.txt``
Type: *string*
Name of file with leap seconds.
.. _streams:
.. note::
**streams.\***
*Parameters controlling the processing of received data.*
.. confval:: streams.networkCode
Default: ``NG``
Type: *string*
Network code to use.
.. confval:: streams.locationCode
Type: *string*
Location code to use.
.. confval:: streams.bandCode
Default: ``U``
Type: *string*
Band code of streams to use. Streams will be formed as
[band][sensor type][component].
.. _output:
.. note::
**output.\***
*Parameters controlling the output of received data.*
.. confval:: output.host
Default: ``localhost``
Type: *string*
Data output host.
.. confval:: output.port
Default: ``18003``
Type: *int*
Data output port.
.. confval:: output.bufferSize
Default: ``1048576``
Type: *uint*
Size \(bytes\) of the packet buffer.
.. confval:: output.gain
Default: ``1000000``
Type: *float*
Apply given gain to samples.
Command-Line Options
====================
.. _Generic:
Generic
-------
.. option:: -h, --help
Show help message.
.. option:: -V, --version
Show version information.
.. option:: --config-file arg
Use alternative configuration file. When this option is
used the loading of all stages is disabled. Only the
given configuration file is parsed and used. To use
another name for the configuration create a symbolic
link of the application or copy it. Example:
scautopick \-> scautopick2.
.. _Verbosity:
Verbosity
---------
.. option:: --verbosity arg
Verbosity level [0..4]. 0:quiet, 1:error, 2:warning, 3:info,
4:debug.
.. option:: -v, --v
Increase verbosity level \(may be repeated, eg. \-vv\).
.. option:: -q, --quiet
Quiet mode: no logging output.
.. option:: --print-component arg
For each log entry print the component right after the
log level. By default the component output is enabled
for file output but disabled for console output.
.. option:: --component arg
Limit the logging to a certain component. This option can
be given more than once.
.. option:: -s, --syslog
Use syslog logging backend. The output usually goes to
\/var\/lib\/messages.
.. option:: -l, --lockfile arg
Path to lock file.
.. option:: --console arg
Send log output to stdout.
.. option:: --debug
Execute in debug mode.
Equivalent to \-\-verbosity\=4 \-\-console\=1 .
.. option:: --trace
Execute in trace mode.
Equivalent to \-\-verbosity\=4 \-\-console\=1 \-\-print\-component\=1
\-\-print\-context\=1 .
.. option:: --log-file arg
Use alternative log file.
.. _Input event:
Input event
-----------
.. option:: -d, --directory arg
Watch this directory for incoming event input files. By
default the current directory is watched.
.. option:: --watch-pattern arg
Process any event file which file name matches the specified
regular expression.
.. _Input waveforms:
Input waveforms
---------------
.. option:: -f, --file arg
Read kenv data directly from this given file.
.. option:: -p, --read-from arg
Read all kenv files from this directory path. Only
considered if file is not given.
.. option:: -l, --leap-file arg
Path to leap seconds file.
.. _Filter:
Filter
------
.. _Streams:
Streams
-------
.. option:: --network-code arg
Network code to use.
.. option:: --location-code arg
Sensor location code to use.
.. option:: --location-code arg
Band code to use. Streams will be formed as [band code][xyz][ZNE].
.. _Output:
Output
------
.. option:: -H, --host arg
Data output host.
.. option:: -P, --port arg
Data output port.
.. option:: -g, --gain arg
Gain value multiplied to the data for unit conversion to nm.
.. option:: -u, --gain-unit arg
Gain unit to write to data file.