552 lines
13 KiB
ReStructuredText
552 lines
13 KiB
ReStructuredText
.. highlight:: rst
|
||
|
||
.. _ngl2caps:
|
||
|
||
########
|
||
ngl2caps
|
||
########
|
||
|
||
**NGL CAPS plugin. Reads GNSS data in kenv format and sends it to CAPS.**
|
||
|
||
|
||
Description
|
||
===========
|
||
|
||
Read GNSS data in kenv format, convert to RAW and send to a CAPS server.
|
||
|
||
|
||
Waveform Data
|
||
=============
|
||
|
||
Format describtions
|
||
-------------------
|
||
|
||
* **kenv Format**:
|
||
Final 5 min rapid solutions from stations [stat]:
|
||
http://geodesy.unr.edu/gps_timeseries/rapids_5min/kenv/[stat]/
|
||
|
||
* Sample rate: 1/300 sps
|
||
* Example: http://geodesy.unr.edu/gps_timeseries/rapids_5min/kenv/0ALM/
|
||
* Format (http://geodesy.unr.edu/gps_timeseries/README_kenv.txt):
|
||
|
||
.. code-block:: properties
|
||
|
||
----------------------------------------
|
||
.kenv format (east,north,up time series)
|
||
----------------------------------------
|
||
Note: first line of .kenv contains header fields to help human interpretation.
|
||
|
||
Column Header Example Description
|
||
------ ----------- --------- ------------------------------------
|
||
1 site JPLM 4-character station ID
|
||
2 sec-J2000 3705278100 GPS seconds since 2000-01-01 12:00:00
|
||
3 __MJD 55833 Modified Julian Day for GPS day
|
||
4 year 2011 Year
|
||
5 mm 9 Month
|
||
6 dd 29 Day of month
|
||
7 doy 272 Day of year
|
||
8 s-day 10800 Seconds of the GPS day
|
||
9 ___e-ref(m) -0.202261 East from reference longitude in llh
|
||
10 ___n-ref(m) 0.079096 North from reference latitude in llh
|
||
11 ___v-ref(m) -0.025883 Up from reference height in llh
|
||
12 _e-mean(m) -0.015904 East from daily mean position
|
||
13 _n-mean(m) -0.000944 North from daily mean position
|
||
14 _v-mean(m) 0.000232 Up from daily mean position
|
||
15 sig_e(m) 0.005700 Sigma east
|
||
16 sig_n(m) 0.006875 Sigma north
|
||
17 sig_v(m) 0.021739 Sigma up
|
||
|
||
* RINEX data web server: http://geodesy.unr.edu/magnet/rinex/
|
||
|
||
|
||
Data sources
|
||
------------
|
||
|
||
kenv data are provided by Nevada Geodetic Laboratory, NGL
|
||
(http://geodesy.unr.edu/):
|
||
|
||
* **Preferred:** Rapid data, 24 hours latency, 5 minutes sps, 1 ZIP file per year:
|
||
http://geodesy.unr.edu/NGLStationPages/RapidStationList
|
||
|
||
Archives:
|
||
http://geodesy.unr.edu/gps_timeseries/kenv/
|
||
|
||
Example for one station, 1 year:
|
||
http://geodesy.unr.edu/gps_timeseries/kenv/0ABI/0ABI.2022.kenv.zip
|
||
|
||
* Ultra-rapid data, 1 hour latency, 5 minutes sps (may have many gaps and outages):
|
||
http://geodesy.unr.edu/NGLStationPages/UltraStationList
|
||
|
||
Hourly upload:
|
||
http://geodesy.unr.edu/gps_timeseries/ultracombo/kenv/2022/141/
|
||
|
||
Archives:
|
||
http://geodesy.unr.edu/gps_timeseries/kenv/
|
||
|
||
* Final 24 h solutions from stations [stat]:
|
||
http://geodesy.unr.edu/gps_timeseries/txyz/IGS14/[stat].txyz2
|
||
|
||
Example: http://geodesy.unr.edu/gps_timeseries/txyz/IGS14/ARIS.txyz2
|
||
|
||
|
||
Fetch data
|
||
----------
|
||
|
||
.. note::
|
||
|
||
Data on the NGL server are zipped files, one file per year.
|
||
Geoffrey Blewitt why this is:
|
||
|
||
"We zip into yearly files which are updated every week. The actual day it is
|
||
updated varies depending on when other necessary inputs are ready, such as
|
||
JPL orbit files, and weather model files from TU Vienna, and ocean loading
|
||
files from Chalmers, Sweden. So, you would not want to check every day, but
|
||
I would say every Wednesday would typically work.
|
||
It’s also important to note that while we process Final data every week,
|
||
often we include newly discovered or late data that can go as far back as
|
||
1994. So, we are not just incrementing with a new week of files.
|
||
|
||
On the technical side, we zip the data into yearly files to reduce the number
|
||
of “inodes” (files) on our server, which can slow things down when attempting
|
||
to seek files, and in the past has overloaded the server with maximum number
|
||
of inodes.
|
||
|
||
Zipping files has speeded up our operations considerably and allows us to
|
||
process all the world’s GPS geodetic data (currently > 18,000 stations).
|
||
It also allows for efficient backup, which can take far too long with
|
||
individual files."
|
||
|
||
For fetching data from the server above adjust:
|
||
|
||
* *year* year to consider as set in the file name. Use "" for all.
|
||
* *url*
|
||
* *outPut*: target directory of fetched files
|
||
|
||
in the Python script provided with the source code:
|
||
:file:`ngl/plugin/tools/fetchFilesFromWeb.py` and use it:
|
||
|
||
.. code-block:: sh
|
||
|
||
python fetchFilesFromWeb.py
|
||
|
||
Change to the target directory and unpack the files:
|
||
|
||
.. code-block:: sh
|
||
|
||
cd [target]
|
||
unzip *
|
||
gunzip *
|
||
|
||
Running ngl2caps converts data to RAW and sends them to the CAPS server. The
|
||
assumed physical unit of the output data is **nanometer** and the original data
|
||
are assumed in units of **meter**. The conversion is applied by the :confval:`gain`
|
||
which is therefore **1000,000,000** by default.
|
||
**Make sure to generate the inventory with the correct gain and gainUnit.**
|
||
If data are stored in "nm" and gainUnit of the stream in the inventory is "m" (SI unit),
|
||
the gain in the inventory must be :math:`10^{-09}`. Example:
|
||
|
||
.. code-block:: sh
|
||
|
||
ngl2caps --debug -p [target directory] --gain 1000000000
|
||
|
||
|
||
Output streams
|
||
--------------
|
||
|
||
The plugin writes the following parameter to the header of the output data.
|
||
|
||
.. csv-table::
|
||
:widths: 2 2 1 5
|
||
:align: left
|
||
:delim: ;
|
||
:header: group, parameter, added, value/remark
|
||
|
||
network; code; x; from input parameter
|
||
network; start time; x; from input parameter
|
||
|
||
station; code; x; from input file
|
||
station; start time; x; same as network
|
||
station; coordinates; x; from input file
|
||
|
||
sensor location; code; x; from input parameter
|
||
sensor location; start; x; same as network
|
||
sensor location; elevation; x; same as station
|
||
|
||
stream; code; x; from band + [XYZ][ZNE]
|
||
; ; ; from band + [X][ZNE] : derived data, daily mean removed
|
||
; ; ; from band + [Y][ZNE] : raw data
|
||
; ; ; from band + [Z][ZNE] : sigma data
|
||
|
||
|
||
Inventory
|
||
=========
|
||
|
||
* Access to different stations lists and formats: http://geodesy.unr.edu/PlugNPlayPortal.php
|
||
* Station list (HTML): http://geodesy.unr.edu/NGLStationPages/GlobalStationList
|
||
* Station list (TXT, 1/300 sps, 24 hour latency):
|
||
http://geodesy.unr.edu/NGLStationPages/DataHoldingsRapid5min.txt
|
||
* Details about stations [stat]: http://geodesy.unr.edu/NGLStationPages/stations/[stat].sta
|
||
|
||
* Station table with coordinates:
|
||
http://plugandplay.unavco.org:8080/unrgsac/gsacapi/site/search#tabId3153-1
|
||
|
||
:ref:`table2inv` can be used for conversion of a station table to :term:`SCML`.
|
||
|
||
|
||
|
||
Module Configuration
|
||
====================
|
||
|
||
| :file:`etc/defaults/global.cfg`
|
||
| :file:`etc/defaults/ngl2caps.cfg`
|
||
| :file:`etc/global.cfg`
|
||
| :file:`etc/ngl2caps.cfg`
|
||
| :file:`~/.seiscomp/global.cfg`
|
||
| :file:`~/.seiscomp/ngl2caps.cfg`
|
||
|
||
ngl2caps inherits :ref:`global options<global-configuration>`.
|
||
|
||
.. note::
|
||
|
||
Modules/plugins may require a license file. The default path to license
|
||
files is :file:`@DATADIR@/licenses/` which can be overridden by global
|
||
configuration of the parameter :confval:`gempa.licensePath`. Example: ::
|
||
|
||
gempa.licensePath = @CONFIGDIR@/licenses
|
||
|
||
|
||
|
||
.. _input:
|
||
|
||
|
||
.. note::
|
||
|
||
**input.\***
|
||
*Parameters controlling the input of event information and*
|
||
*reception of GNSS data.*
|
||
|
||
|
||
.. confval:: input.readFrom
|
||
|
||
Type: *string*
|
||
|
||
Read input data from this file.
|
||
|
||
|
||
.. confval:: input.directory
|
||
|
||
Type: *string*
|
||
|
||
Watch this directory for incoming input files.
|
||
|
||
|
||
.. confval:: input.watchEvents
|
||
|
||
Default: ``close_write``
|
||
|
||
Type: *string*
|
||
|
||
Listen for specific inotify event\(s\).
|
||
If ommitted, close_write events are listened for. Events:
|
||
|
||
access \- file or directory contents were read,
|
||
|
||
modify \- file or directory contents were written,
|
||
|
||
attrib \- file or directory attributes changed,
|
||
|
||
close_write \- file or directory closed, after being opened in writable mode,
|
||
|
||
close_nowrite \- file or directory closed, after being opened in read\-only mode,
|
||
|
||
close \- file or directory closed, regardless of read\/write mode,
|
||
|
||
open \- file or directory opened,
|
||
|
||
moved_to \- file or directory moved to watched directory,
|
||
|
||
moved_from \- file or directory moved from watched directory,
|
||
|
||
move \- file or directory moved to or from watched directory,
|
||
|
||
create \- file or directory created within watched directory,
|
||
|
||
delete \- file or directory deleted within watched directory,
|
||
|
||
delete_self \- file or directory was deleted,
|
||
|
||
unmount \- file system containing file or directory unmounted.
|
||
|
||
|
||
.. confval:: input.watchPattern
|
||
|
||
Type: *string*
|
||
|
||
Process any events whose filename matches the specified regular expression
|
||
|
||
|
||
.. confval:: input.leapSecondsFile
|
||
|
||
Default: ``@DATADIR@/caps/plugins/ngl2caps/leapseconds.txt``
|
||
|
||
Type: *string*
|
||
|
||
Name of file with leap seconds.
|
||
|
||
|
||
.. _streams:
|
||
|
||
|
||
.. note::
|
||
|
||
**streams.\***
|
||
*Parameters controlling the processing of received data.*
|
||
|
||
|
||
.. confval:: streams.networkCode
|
||
|
||
Default: ``NG``
|
||
|
||
Type: *string*
|
||
|
||
Network code to use.
|
||
|
||
|
||
.. confval:: streams.locationCode
|
||
|
||
Type: *string*
|
||
|
||
Location code to use.
|
||
|
||
|
||
.. confval:: streams.bandCode
|
||
|
||
Default: ``U``
|
||
|
||
Type: *string*
|
||
|
||
Band code of streams to use. Streams will be formed as
|
||
[band][sensor type][component].
|
||
|
||
|
||
.. _output:
|
||
|
||
|
||
.. note::
|
||
|
||
**output.\***
|
||
*Parameters controlling the output of received data.*
|
||
|
||
|
||
.. confval:: output.host
|
||
|
||
Default: ``localhost``
|
||
|
||
Type: *string*
|
||
|
||
Data output host.
|
||
|
||
|
||
.. confval:: output.port
|
||
|
||
Default: ``18003``
|
||
|
||
Type: *int*
|
||
|
||
Data output port.
|
||
|
||
|
||
.. confval:: output.bufferSize
|
||
|
||
Default: ``1048576``
|
||
|
||
Type: *uint*
|
||
|
||
Size \(bytes\) of the packet buffer.
|
||
|
||
|
||
.. confval:: output.gain
|
||
|
||
Default: ``1000000``
|
||
|
||
Type: *float*
|
||
|
||
Apply given gain to samples.
|
||
|
||
|
||
|
||
|
||
Command-Line Options
|
||
====================
|
||
|
||
|
||
.. _Generic:
|
||
|
||
|
||
Generic
|
||
-------
|
||
|
||
.. option:: -h, --help
|
||
|
||
Show help message.
|
||
|
||
.. option:: -V, --version
|
||
|
||
Show version information.
|
||
|
||
.. option:: --config-file arg
|
||
|
||
Use alternative configuration file. When this option is
|
||
used the loading of all stages is disabled. Only the
|
||
given configuration file is parsed and used. To use
|
||
another name for the configuration create a symbolic
|
||
link of the application or copy it. Example:
|
||
scautopick \-> scautopick2.
|
||
|
||
|
||
.. _Verbosity:
|
||
|
||
|
||
Verbosity
|
||
---------
|
||
|
||
.. option:: --verbosity arg
|
||
|
||
Verbosity level [0..4]. 0:quiet, 1:error, 2:warning, 3:info,
|
||
4:debug.
|
||
|
||
.. option:: -v, --v
|
||
|
||
Increase verbosity level \(may be repeated, eg. \-vv\).
|
||
|
||
.. option:: -q, --quiet
|
||
|
||
Quiet mode: no logging output.
|
||
|
||
.. option:: --print-component arg
|
||
|
||
For each log entry print the component right after the
|
||
log level. By default the component output is enabled
|
||
for file output but disabled for console output.
|
||
|
||
.. option:: --component arg
|
||
|
||
Limit the logging to a certain component. This option can
|
||
be given more than once.
|
||
|
||
.. option:: -s, --syslog
|
||
|
||
Use syslog logging backend. The output usually goes to
|
||
\/var\/lib\/messages.
|
||
|
||
.. option:: -l, --lockfile arg
|
||
|
||
Path to lock file.
|
||
|
||
.. option:: --console arg
|
||
|
||
Send log output to stdout.
|
||
|
||
.. option:: --debug
|
||
|
||
Execute in debug mode.
|
||
Equivalent to \-\-verbosity\=4 \-\-console\=1 .
|
||
|
||
.. option:: --trace
|
||
|
||
Execute in trace mode.
|
||
Equivalent to \-\-verbosity\=4 \-\-console\=1 \-\-print\-component\=1
|
||
\-\-print\-context\=1 .
|
||
|
||
.. option:: --log-file arg
|
||
|
||
Use alternative log file.
|
||
|
||
|
||
.. _Input event:
|
||
|
||
|
||
Input event
|
||
-----------
|
||
|
||
.. option:: -d, --directory arg
|
||
|
||
Watch this directory for incoming event input files. By
|
||
default the current directory is watched.
|
||
|
||
.. option:: --watch-pattern arg
|
||
|
||
Process any event file which file name matches the specified
|
||
regular expression.
|
||
|
||
|
||
.. _Input waveforms:
|
||
|
||
|
||
Input waveforms
|
||
---------------
|
||
|
||
.. option:: -f, --file arg
|
||
|
||
Read kenv data directly from this given file.
|
||
|
||
.. option:: -p, --read-from arg
|
||
|
||
Read all kenv files from this directory path. Only
|
||
considered if file is not given.
|
||
|
||
.. option:: -l, --leap-file arg
|
||
|
||
Path to leap seconds file.
|
||
|
||
|
||
.. _Filter:
|
||
|
||
|
||
Filter
|
||
------
|
||
|
||
|
||
.. _Streams:
|
||
|
||
|
||
Streams
|
||
-------
|
||
|
||
.. option:: --network-code arg
|
||
|
||
Network code to use.
|
||
|
||
.. option:: --location-code arg
|
||
|
||
Sensor location code to use.
|
||
|
||
.. option:: --location-code arg
|
||
|
||
Band code to use. Streams will be formed as [band code][xyz][ZNE].
|
||
|
||
|
||
.. _Output:
|
||
|
||
|
||
Output
|
||
------
|
||
|
||
.. option:: -H, --host arg
|
||
|
||
Data output host.
|
||
|
||
.. option:: -P, --port arg
|
||
|
||
Data output port.
|
||
|
||
.. option:: -g, --gain arg
|
||
|
||
Gain value multiplied to the data for unit conversion to nm.
|
||
|
||
.. option:: -u, --gain-unit arg
|
||
|
||
Gain unit to write to data file.
|
||
|
||
|