CLeanup

- Update plugin upgrade message
- Update api.login, update the connected_at field to the current time
- Update monitoring_history_fn to use custom user settings for metrics
- Update cron_alerts_fn to check for alerts, filters out empty strings (""), so they are not included in the result.
- Update process_pre_logbook_fn to detect and avoid logbook we more than 1000NM in less 15h

.gitignore

@@ -1,5 +1,6 @@
 .DS_Store
 .env
+docker-compose.mm.yml
 initdb/*.csv
 initdb/*.no
 initdb/*.jwk

README.md

@@ -24,6 +24,8 @@
     <a href="https://github.com/sponsors/xbgmsharp">Sponsors</a>
     .
     <a href="https://discord.gg/uuZrwz4dCS">Discord</a>
+    .
+    <a href="https://deepwiki.com/xbgmsharp/postgsail/">DeepWiki</a>
   </p>
 </p>
 
@@ -32,19 +34,24 @@
 [![issues - postgsail](https://img.shields.io/github/issues/xbgmsharp/postgsail)](https://github.com/xbgmsharp/postgsail/issues)
 [![PRs Welcome](https://img.shields.io/badge/PRs-welcome-brightgreen.svg?style=flat-square)](http://makeapullrequest.com)
 ![Contributors](https://img.shields.io/github/contributors/xbgmsharp/postgsail?color=dark-green) 
+[![GitHub Repo stars](https://img.shields.io/github/stars/xbgmsharp/postgsail?style=social)](https://github.com/xbgmsharp/postgsail/stargazers)
+[![Ask DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/xbgmsharp/postgsail)
 
 [![Test services db, api](https://github.com/xbgmsharp/postgsail/actions/workflows/db-test.yml/badge.svg)](https://github.com/xbgmsharp/postgsail/actions/workflows/db-test.yml)
 [![Test services db, api, web](https://github.com/xbgmsharp/postgsail/actions/workflows/frontend-test.yml/badge.svg)](https://github.com/xbgmsharp/postgsail/actions/workflows/frontend-test.yml)
 [![Test services db, grafana](https://github.com/xbgmsharp/postgsail/actions/workflows/grafana-test.yml/badge.svg)](https://github.com/xbgmsharp/postgsail/actions/workflows/grafana-test.yml)
 
 signalk-postgsail:
-[![GitHub Release](https://img.shields.io/github/release/xbgmsharp/signalk-postgsail.svg)](https://github.com/xbgmsharp/signalk-postgsail/releases/latest)
+[![GitHub Release](https://img.shields.io/github/release/xbgmsharp/signalk-postgsail.svg?color=blue)](https://github.com/xbgmsharp/signalk-postgsail/releases/latest)
+
+postgsail-backend:
+[![GitHub Release](https://img.shields.io/github/release/xbgmsharp/postgsail.svg?color=blue)](https://github.com/xbgmsharp/postgsail/releases/latest)
 
 postgsail-frontend:
-[![GitHub Release](https://img.shields.io/github/release/xbgmsharp/vuestic-postgsail.svg)](https://github.com/xbgmsharp/vuestic-postgsail/releases/latest)
+[![GitHub Release](https://img.shields.io/github/release/xbgmsharp/vuestic-postgsail.svg?color=blue)](https://github.com/xbgmsharp/vuestic-postgsail/releases/latest)
 
 postgsail-telegram-bot:
-[![GitHub Release](https://img.shields.io/github/release/xbgmsharp/postgsail-telegram-bot.svg)](https://github.com/xbgmsharp/postgsail-telegram-bot/releases/latest)
+[![GitHub Release](https://img.shields.io/github/release/xbgmsharp/postgsail-telegram-bot.svg?color=blue)](https://github.com/xbgmsharp/postgsail-telegram-bot/releases/latest)
 
 [![OpenSSF Best Practices](https://www.bestpractices.dev/projects/8124/badge)](https://www.bestpractices.dev/projects/8124)
 
@@ -55,7 +62,7 @@ postgsail-telegram-bot:
 - [About The Project](#about-the-project)
 - [Features](#features)
 - [Cloud-hosted PostgSail](#cloud-hosted-postgsail)
-- [On-Premise (for free)](#on-premise-for-free)
+- [On-Premise](#on-premise)
 - [Roadmap](#roadmap)
 - [Contributing](#contributing)
   - [Creating A Pull Request](#creating-a-pull-request)
@@ -104,10 +111,16 @@ To understand the why and how, you might want to read [Why.md](https://github.co
 
 Remove the hassle of running PostgSail yourself. Here you can skip the technical setup, the maintenance work and server costs by getting PostgSail on our reliable and secure PostgSail Cloud. Register and try for free at [iot.openplotter.cloud](https://iot.openplotter.cloud/).
 
-## On-Premise (for free)
+PostgSail Cloud is Open Source and free for personal use with a single vessel. If wish to manage multiple boats contact us.
+
+PostgSail is free to use, but is not free to make or host. The stability and accuracy of PostgSail depends on its volunteers and donations from its users. Please consider making an annual recurring gift to PostgSail.
+
+## On-Premise
 
 Self host postgSail where you want and how you want. There are no restrictions, you’re in full control. [Install Guide](https://github.com/xbgmsharp/postgsail/blob/main/docs/README.md)
 
+PostgSail is free to use, but is not free to make or host. The stability and accuracy of PostgSail depends on its volunteers and donations from its users. Please consider making an annual recurring gift to PostgSail.
+
 ## Roadmap
 
 See the [open issues](https://github.com/xbgmsharp/postgsail/issues) for a list of proposed features (and known issues).
@@ -142,5 +155,6 @@ An out of the box IoT platform using Docker (could be extend to K3 or K8) with t
 - [PostgreSQL, open source object-relational database system](https://postgresql.org)
 - [TimescaleDB, Time-series data extends PostgreSQL](https://www.timescale.com)
 - [PostGIS, a spatial database extender for PostgreSQL object-relational database.](https://postgis.net/)
+- [MobilityDB, An open source geospatial trajectory data management & analysis platform.](https://mobilitydb.com/)
 - [Grafana, open observability platform | Grafana Labs](https://grafana.com)
 - And many more

docker-compose.yml

@@ -1,5 +1,3 @@
-version: "3.9"
-
 services:
   db:
     image: xbgmsharp/timescaledb-postgis

docs/ERD/postgsail.md

@@ -13,31 +13,50 @@ erDiagram
         timestamp_with_time_zone _to_time 
         boolean active 
         double_precision avg_speed 
-        numeric distance "in NM"
-        interval duration "Best to use standard ISO 8601"
-        jsonb extra "computed signalk metrics of interest, runTime, currentLevel, etc"
+        numeric distance "Distance in nautical miles (NM)"
+        interval duration "Duration in ISO 8601 format"
+        jsonb extra "Computed SignalK metrics such as runtime, current level, etc."
         integer id "{NOT_NULL}"
         double_precision max_speed 
         double_precision max_wind_speed 
         text name 
         text notes 
-        geography track_geog "postgis geography type default SRID 4326 Unit: degres"
-        jsonb track_geojson "store generated geojson with track metrics data using with LineString and Point features, we can not depend api.metrics table"
-        geometry track_geom "postgis geometry type EPSG:4326 Unit: degres"
+        tgeogpoint trip "MobilityDB trajectory"
+        tfloat trip_batt_charge "Battery Charge"
+        tfloat trip_batt_voltage "Battery Voltage"
+        tfloat trip_cog "courseovergroundtrue"
+        tfloat trip_depth "Depth"
+        tfloat trip_heading "heading True"
+        tfloat trip_hum_out "Humidity outside"
+        ttext trip_notes 
+        tfloat trip_pres_out "Pressure outside"
+        tfloat trip_sog "speedoverground"
+        tfloat trip_solar_power "solar powerPanel"
+        tfloat trip_solar_voltage "solar voltage"
+        ttext trip_status 
+        tfloat trip_tank_level "Tank currentLevel"
+        tfloat trip_temp_out "Temperature outside"
+        tfloat trip_temp_water "Temperature water"
+        tfloat trip_twa "windspeedapparent"
+        tfloat trip_twd "truewinddirection"
+        tfloat trip_tws "truewindspeed"
         text vessel_id "{NOT_NULL}"
     }
 
     api_metadata {
         boolean active "trigger monitor online/offline"
         boolean active 
+        jsonb available_keys "Signalk paths with unit for custom mapping"
+        jsonb available_keys 
         double_precision beam 
-        text client_id 
-        text configuration 
+        jsonb configuration "Signalk path mapping for metrics"
+        jsonb configuration 
         timestamp_with_time_zone created_at "{NOT_NULL}"
         double_precision height 
-        integer id "{NOT_NULL}"
+        text ip "Store vessel ip address"
+        text ip 
         double_precision length 
-        numeric mmsi 
+        text mmsi 
         text name 
         text platform 
         text plugin_version "{NOT_NULL}"
@@ -49,9 +68,22 @@ erDiagram
         text vessel_id "{NOT_NULL}"
     }
 
+    api_metadata_ext {
+        timestamp_with_time_zone created_at "{NOT_NULL}"
+        bytea image "Store user boat image in bytea format"
+        text image_b64 
+        text image_type "Store user boat image type in text format"
+        timestamp_with_time_zone image_updated_at 
+        text image_url 
+        text make_model "Store user make & model in text format"
+        text polar "Store polar data in CSV notation as used on ORC sailboat data"
+        timestamp_with_time_zone polar_updated_at 
+        text vessel_id "{NOT_NULL}"
+    }
+
     api_metrics {
         double_precision anglespeedapparent 
-        text client_id 
+        text client_id "Deprecated client_id to be removed"
         double_precision courseovergroundtrue 
         double_precision latitude "With CONSTRAINT but allow NULL value to be ignored silently by trigger"
         double_precision longitude "With CONSTRAINT but allow NULL value to be ignored silently by trigger"
@@ -74,9 +106,7 @@ erDiagram
         jsonb nominatim 
         text notes 
         jsonb overpass 
-        integer reference_count 
         integer stay_code "Link api.stays_at with api.moorages via FOREIGN KEY and REFERENCES"
-        interval stay_duration "Best to use standard ISO 8601"
         text vessel_id "{NOT_NULL}"
     }
 
@@ -101,6 +131,17 @@ erDiagram
         integer stay_code "{NOT_NULL}"
     }
 
+    api_stays_ext {
+        timestamp_with_time_zone created_at "{NOT_NULL}"
+        bytea image "Store stays image in bytea format"
+        text image_b64 
+        text image_type "Store stays image type in text format"
+        timestamp_with_time_zone image_updated_at 
+        text image_url 
+        integer stay_id "{NOT_NULL}"
+        text vessel_id "{NOT_NULL}"
+    }
+
     auth_accounts {
         timestamp_with_time_zone connected_at "{NOT_NULL}"
         timestamp_with_time_zone created_at "{NOT_NULL}"
@@ -187,6 +228,13 @@ erDiagram
         numeric id 
     }
 
+    public_mobilitydb_opcache {
+        integer ltypnum 
+        oid opid 
+        integer opnum 
+        integer rtypnum 
+    }
+
     public_ne_10m_geography_marine_polys {
         text changed 
         text featurecla 
@@ -249,13 +297,18 @@ erDiagram
     api_logbook }o--|| api_metadata : ""
     api_logbook }o--|| api_moorages : ""
     api_logbook }o--|| api_moorages : ""
+    api_logbook }o--|| api_moorages : ""
+    api_logbook }o--|| api_moorages : ""
     api_metadata }o--|| auth_vessels : ""
+    api_metadata_ext |o--|| api_metadata : ""
     api_metrics }o--|| api_metadata : ""
     api_moorages }o--|| api_metadata : ""
     api_stays }o--|| api_metadata : ""
+    api_stays_ext }o--|| api_metadata : ""
     api_moorages }o--|| api_stays_at : ""
     api_stays }o--|| api_moorages : ""
     api_stays }o--|| api_stays_at : ""
+    api_stays_ext |o--|| api_stays : ""
     auth_otp |o--|| auth_accounts : ""
     auth_vessels }o--|| auth_accounts : ""
 ```

docs/README.md

@@ -115,16 +115,23 @@ Then connect to the web UI on port HTTP/5050.
 
 #### Step 3. Start frontend (web)
 
-Then launch the web frontend, execute:
+Last build and launch the web frontend, execute:
 
 ```bash
-$ docker compose up web
+docker compose build web
+docker compose up web
 ```
-This step can take some time as it will first do a build to generate the static website based on your settings.
+
+The first step can take some time as it will first run a build to generate the static website based on your settings.
+
+The frontend is a SPA (Single-Page Application). With SPA, the server provides the user with an empty HTML page and Javascript. The latter is where the magic happens. When the browser receives the HTML + Javascript, it loads the Javascript. Once loaded, the JS takes place and, through a set of operations in the DOM, renders the necessary components to the page. The routing is then handled by the browser itself, not hitting the server. 
 
 The frontend should be accessible via port HTTP/8080.
 
-Users are collaborating on an installation guide, [Self-hosted-installation-guide](https://github.com/xbgmsharp/postgsail/wiki/Self-hosted-installation-guide)
+Users are collaborating on two installation guide:
+- [Self-hosted-installation-guide](https://github.com/xbgmsharp/postgsail/blob/main/docs/install_guide.md)
+- [Self-hosted-installation-guide on AWS EC2](https://github.com/xbgmsharp/postgsail/blob/main/docs/Self%E2%80%90hosted-installation-guide%20on%20AWS.md)
+- [Self-hosted-installation-guide](https://github.com/xbgmsharp/postgsail/blob/main/docs/Self%E2%80%90hosted-installation-guide.md)
 
 ### SQL Configuration
 

docs/Self‐hosted-installation-guide on AWS.md

@@ -0,0 +1,288 @@
+## Self AWS cloud hosted setup example
+
+In this guide we install, setup and run a postgsail project on an AWS instance in the cloud.
+
+## On AWS Console
+***Launch an instance on AWS EC2***
+With the following settings:
++ Ubuntu
++ Instance type: t2.small
++ Create a new key pair: 
+    + key pair type: RSA
+    + Private key file format: .pem
++ The key file is stored for later use
+
++ Allow SSH traffic from: Anywhere
++ Allow HTTPS traffic from the internet
++ Allow HTTP traffic from the internet
+
+Configure storage:
+The standard storage of 8GiB is too small so change this to 16GiB.
+
+Create a new security group
++ Go to: EC2>Security groups>Create security group
+Add inbound rules for the following ports:443, 8080, 80, 3000, 5432, 22, 5050
++ Go to your instance>select your instance>Actions>security>change security group
++ And add the correct security group to the instance.
+
+## Connect to instance with SSH
+
++ Copy the key file in your default SSH configuration file location (the one VSCode will use)
++ In terminal, go to the folder and run this command to ensure your key is not publicly viewable: 
+```
+chmod 600 "privatekey.pem"
+```
+
+We are using VSCode to connect to the instance: 
++ Install the Remote - SSH Extension for VSCode
++ Open the Command Palette (Ctrl+Shift+P) and type Remote-SSH: Add New SSH Host:
+```
+ssh -i "privatekey.pem" ubuntu@ec2-111-22-33-44.eu-west-1.compute.amazonaws.com
+```
+When prompted, select the default SSH configuration file location.
+Open the config file and add the location:
+```
+xIdentityFile ~/.ssh/privatekey.pem
+```
+
+
+## Install Docker on your instance
+To install Docker on your new EC2 Ubuntu instance via SSH, follow these steps:
+
+Update your package list:
+```
+sudo apt-get update
+```
+Install required dependencies:
+```
+sudo apt-get install apt-transport-https ca-certificates curl software-properties-common
+```
+Add Docker's official GPG key:
+```
+curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg
+```
+Add Docker's official repository:
+```
+echo "deb [arch=amd64 signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
+```
+Update the package list again:
+```
+sudo apt-get update
+```
+Install Docker:
+```
+sudo apt-get install docker-ce docker-ce-cli containerd.io
+```
+Verify Docker installation:
+```
+sudo docker --version
+```
+Add your user to the docker group to run Docker without sudo:
+```
+sudo usermod -aG docker ubuntu
+```
+Then, log out and back in or use the following to apply the changes:
+```
+newgrp docker
+```
+
+
+
+## Install Postgsail 
+Git clone the postgsail repo:
+```
+git clone https://github.com/xbgmsharp/postgsail.git
+```
+
+## Edit environment variables
+Copy the example.env file and edit the environment variables:
+```
+cd postgsail
+cp .env.example .env
+nano .env
+```
+
++ POSTGRES_USER
+Come up with a unique username for the database user. This will be used in the docker image when it’s started up. Nothing beyond creating a unique username and password is required here.
+This environment variable is used in conjunction with `POSTGRES_PASSWORD` to set a user and its password. This variable will create the specified user with superuser power and a database with the same name.
+
+https://github.com/docker-library/docs/blob/master/postgres/README.md
+
++ POSTGRES_PASSWORD
+This should be a good password. It will be used for the postgres user above. Again this is used in the docker image.
+This environment variable is required for you to use the PostgreSQL image. It must not be empty or undefined. This environment variable sets the superuser password for PostgreSQL. The default superuser is defined by the POSTGRES_USER environment variable.
+
++ POSTGRES_DB
+This is the name of the database within postgres. You can leave it named postgres but give it a unique name if you like. The schema will be loaded into this database and all data will be stored within it. Since this is used inside the docker image the name really doesn’t matter. If you plan to run additional databases within the image, then you might care.
+This environment variable can be used to define a different name for the default database that is created when the image is first started. If it is not specified, then the value of `POSTGRES_USER` will be used.
+
++ PGSAIL_APP_URL
+This is the webapp (webui) entrypoint, typically the public DNS or IP
+```
+PGSAIL_APP_URL=http://localhost:8080
+```
+
+
++ PGSAIL_API_URL
+This is the URL to your API on your instance on port 3000:
+```
+PGSAIL_API_URL=PGSAIL_API_URL=http://localhost:3000
+```
+
++ PGSAIL_AUTHENTICATOR_PASSWORD
+This password is used as part of the database access configuration. It’s used as part of the access URI later on. (Put the same password in both lines.)
+
++ PGSAIL_GRAFANA_PASSWORD
+This password is used for the grafana service
+
++ PGSAIL_GRAFANA_AUTH_PASSWORD
+??This password is used for user authentication on grafana?
+
++ PGSAIL_EMAIL_FROM - PGSAIL_EMAIL_SERVER - PGSAIL_EMAIL_USER - PGSAIL_EMAIL_PASS Pgsail does not include a built in email service - only hooks to send email via an existing server.
+We use gmail as a third party email service:
+```
+PGSAIL_EMAIL_FROM=email@gmail.com
+PGSAIL_EMAIL_SERVER=smtp.gmail.com
+PGSAIL_EMAIL_USER=email@gmail.com
+```
+You need to get the PGSAIL_EMAIL_PASS from your gmail account security settings: it is not the account password, instead you need to make an "App password"
+
++ PGRST_JWT_SECRET
+This secret key must be at least 32 characters long, you can create a random key with the following command:
+```
+cat /dev/urandom | LC_ALL=C tr -dc 'a-zA-Z0-9' | fold -w 42 | head -n 1
+```
+
++ Other ENV variables
+```
+PGSAIL_PUSHOVER_APP_TOKEN
+PGSAIL_PUSHOVER_APP
+PGSAIL_TELEGRAM_BOT_TOKEN
+PGSAIL_AUTHENTICATOR_PASSWORD=password
+PGSAIL_GRAFANA_PASSWORD=password
+PGSAIL_GRAFANA_AUTH_PASSWORD=password
+#PGSAIL_PUSHOVER_APP_TOKEN= Comment if not used
+#PGSAIL_PUSHOVER_APP_URL= Comment if not used
+#PGSAIL_TELEGRAM_BOT_TOKEN= Comment if not used
+```
+
+## Run the project
+If needed, add your user to the docker group to run Docker without sudo:
+```
+sudo usermod -aG docker ubuntu
+```
+Then, log out and back in or use the following to apply the changes:
+```
+newgrp docker
+```
+
+
+Step 1. Import the SQL schema, execute:
+```
+docker compose up db
+```
+Step 2. Launch the full backend stack (db, api), execute:
+```
+docker compose up db api
+```
+Step 3. Launch the frontend webapp
+```
+docker compose up web
+```
+
+Open browser and navigate to your PGSAIL_APP_URL, you should see the postgsail login screen now:
+```
+http://ec2-11-234-567-890.eu-west-1.compute.amazonaws.com::8080
+```
+
+## Additional database setup
+Aditional setup will be required.
+There is no useraccount yet, also cronjobs need to be activated.
+We'll do that by using pgadmin.
+
+### Run Pgadmin & connect to database
+First add two more vars to your env. file:
+```
+PGADMIN_DEFAULT_EMAIL=setup@setup.com
+PGADMIN_DEFAULT_PASSWORD=123456
+```
+
+Pgadmin is defined in docker-compose.dev.yml so we need to start the service:
+```
+docker compose -f docker-compose.yml -f docker-compose.dev.yml up -d pgadmin
+```
+
+All services should be up now: api, db, web and pgadmin. Check all services running by:
+```
+docker ps
+```
+
+To open Pgadmin, navigate to your aws-url and port 5050:
+```
+http://ec2-11-234-567-890.eu-west-1.compute.amazonaws.com::5050
+```
+
+<p>
+You are now able to login with your credentials: PGADMIN_DEFAULT_EMAIL & PGADMIN_DEFAULT_PASSWORD.<br>
+</p>
+<p>
+In the right-side panel you will see "Servers(1)"; by clicking you'll see the Server: "PostgSail dev db"<br>
+</p>
+<p>
+**Warning:** A dialog box will open, prompting to input the password, but stating the wrong username (postgres) , you have to change this username by right-clicking on the server  "PostgSail dev db" > Properties > Connection > enter username: POSTGRES_USER > Save
+</p>
+<p>
+Now right-click and Connect to Server and enter your password: POSTGRES_PASSWORD
+</p>
+<p>
+You'll see 2 databases: "postgres" and "signalk"
+</p>
+
+### Enabling cron jobs by SQL query
+<p>
+Cron jobs are not active by default because if you don't have the correct settings set (for SMTP, PushOver, Telegram), you might enter in a loop with errors and you could be blocked or banned from the external services.
+</p>
+<p>
+Once you have setup the services correctly (entered credentials in .env file) you can activate the cron jobs. (We are only using the SMTP email service in this example) in the "postgres" database:
+</p>
++ Right-click on "postgres" database and select "Query Tool"
++ Execute the following SQL query:
+
+```
+UPDATE cron.job SET active = True;
+```
+
+### Adding a user by SQL query
+I was not able to create a new user through the web application (still figuring out what is going on). Therefore I added a new user by SQL in the "signalk" database.
++ Right-click on "signalk" database and select "Query Tool"
++ Check the current users in your database executing the query: 
+```
+SELECT * FROM auth.accounts;
+```
+
+
+
++ To add a new user executing the query:
+
+```
+INSERT INTO auth.accounts (
+email, first, last, pass, role) VALUES (
+'your.email@domain.com'::citext, 'Test'::text, 'your_username'::text, 'your_password'::text, 'user_role'::name)
+ returning email;
+```
+
+When SMTP is correctly setup, you will receive two emails: "Welcome" and "Email verification".
+<p>
+You will be able to login with these credentials on the web
+</p>
+<p>
+Each time you login, you will receive an email: "Email verification". This is the OTP process, you can bypass this process by updating the json key value of "Preferences":
+</p>
+
+```
+UPDATE auth.accounts
+SET preferences='{"email_valid": true}'::jsonb || preferences
+WHERE email='your.email@domain.com';
+```
+
+Now you are able to use PostGSail on the web on your own AWS server!

docs/Self‐hosted-installation-guide.md

@@ -0,0 +1,166 @@
+# Self hosted setup example environment:
+
+Virtual machine with Ubuntu 22.04 LTS minimal server installation.
+
+Install openssh, update and install docker-ce manually (ubuntu docker repo is lame)
+The following ports are exposed to the internet either using a static public IP address or port forwarding via your favorite firewall platform. (not need by default docker will expose all ports to all IPs)
+The base install uses ports 5432 (db) and 3000 (api) and 8080 (web).
+
+We’ll add https using Apache or Nginx proxy once everything is tested. At that point you’ll want to open 443 or whatever other port you want to use for secure communication.
+
+For docker-ce installation, this is a decent guide to installation:
+https://www.digitalocean.com/community/tutorials/how-to-install-and-use-docker-on-ubuntu-20-04
+
+Third party services and options:
+Emails
+For email notifications you may want to install a local email handler like postfix or use a third party service like gmail.
+
+Pushover
+Add more here
+
+Telegram Bot
+Add more here
+
+
+```
+$ git clone https://github.com/xbgmsharp/postgsail
+cd postgsail
+cp .env.example .env
+nano .env
+```
+
+Login to your docker host once it’s setup.
+Clone the repo to your user directory:
+
+Copy the example file and edit the environment variables
+
+The example has the following:
+```
+# POSTGRESQL ENV Settings
+POSTGRES_USER=username
+POSTGRES_PASSWORD=password
+POSTGRES_DB=postgres
+# PostgSail ENV Settings
+PGSAIL_AUTHENTICATOR_PASSWORD=password
+PGSAIL_GRAFANA_PASSWORD=password
+PGSAIL_GRAFANA_AUTH_PASSWORD=password
+# SMTP server settings
+PGSAIL_EMAIL_FROM=root@localhost
+PGSAIL_EMAIL_SERVER=localhost
+#PGSAIL_EMAIL_USER= Comment if not use
+#PGSAIL_EMAIL_PASS= Comment if not use
+# Pushover settings
+#PGSAIL_PUSHOVER_APP_TOKEN= Comment if not use
+#PGSAIL_PUSHOVER_APP_URL= Comment if not use
+# TELEGRAM BOT, ask BotFather
+#PGSAIL_TELEGRAM_BOT_TOKEN= Comment if not use
+# webapp entrypoint, typically the public DNS or IP
+PGSAIL_APP_URL=http://localhost:8080
+# API entrypoint from the webapp, typically the public DNS or IP
+PGSAIL_API_URL=http://localhost:3000
+# 
+POSTGREST ENV Settings
+PGRST_DB_URI=postgres://authenticator:${PGSAIL_AUTHENTICATOR_PASSWORD}@db:5432/signalk
+# % cat /dev/urandom | LC_ALL=C tr -dc 'a-zA-Z0-9' | fold -w 42 | head -n 1
+PGRST_JWT_SECRET=_at_least_32__char__long__random
+# Grafana ENV Settings
+GF_SECURITY_ADMIN_PASSWORD=password
+```
+
+All of these need to be configured.
+
+Step by step:
+
+## POSTGRESQL ENV Settings
+
+***POSTGRES_USER***
+Come up with a unique username for the database user. This will be used in the docker image when it’s started up. Nothing beyond creating a unique username and password is required here.
+This environment variable is used in conjunction with `POSTGRES_PASSWORD` to set a user and its password. This variable will create the specified user with superuser power and a database with the same name.
+
+https://github.com/docker-library/docs/blob/master/postgres/README.md
+
+***POSTGRES_PASSWORD***
+This should be a good password. It will be used for the postgres user above. Again this is used in the docker image.
+This environment variable is required for you to use the PostgreSQL image. It must not be empty or undefined. This environment variable sets the superuser password for PostgreSQL. The default superuser is defined by the POSTGRES_USER environment variable.
+
+***POSTGRES_DB***
+This is the name of the database within postgres. Give it a unique name if you like. The schema will be loaded into this database and all data will be stored within it. Since this is used inside the docker image the name really doesn’t matter. If you plan to run additional databases within the image, then you might care.
+This environment variable can be used to define a different name for the default database that is created when the image is first started. If it is not specified, then the value of `POSTGRES_USER` will be used.
+
+
+```
+# PostgSail ENV Settings
+PGSAIL_AUTHENTICATOR_PASSWORD=password
+PGSAIL_GRAFANA_PASSWORD=password
+PGSAIL_GRAFANA_AUTH_PASSWORD=password
+PGSAIL_EMAIL_FROM=root@localhost
+PGSAIL_EMAIL_SERVER=localhost
+#PGSAIL_EMAIL_USER= Comment if not use
+#PGSAIL_EMAIL_PASS= Comment if not use
+#PGSAIL_PUSHOVER_APP_TOKEN= Comment if not use
+#PGSAIL_PUSHOVER_APP_URL= Comment if not use
+#PGSAIL_TELEGRAM_BOT_TOKEN= Comment if not use
+PGSAIL_APP_URL=http://localhost:8080
+PGSAIL_API_URL=http://localhost:3000
+```
+
+PGSAIL_AUTHENTICATOR_PASSWORD
+This password is used as part of the database access configuration. It’s used as part of the access URI later on. (Put the same password in both lines.)
+
+PGSAIL_GRAFANA_PASSWORD
+This password is used for the grafana service
+
+PGSAIL_GRAFANA_AUTH_PASSWORD
+??This password is used for user authentication on grafana?
+
+PGSAIL_EMAIL_FROM
+PGSAIL_EMAIL_SERVER
+Pgsail does not include a built in email service - only hooks to send email via an existing server.
+You can install an email service on the ubuntu host or use a third party service like gmail. If you chose to use a local service, be aware that some email services will filter it as spam unless you’ve properly configured it.
+
+PGSAIL_PUSHOVER_APP_TOKEN
+PGSAIL_PUSHOVER_APP
+PGSAIL_TELEGRAM_BOT_TOKEN
+
+Add more info here
+PGSAIL_APP_URL
+This is the full url (with domain name or IP) that you access PGSAIL via. Once nginx ssl proxy is added this may need to be updated. (Service restart required after changing?)
+
+
+PGSAIL_API_URL
+This is the API URL that’s used for the boat and user access. Once apache or nginx ssl proxy is added this may need to be updated. (same restart?)
+
+Network configuration example:
+It is a docker question but in general no special network config should be need, docker created and assign one automatically. all images will be bind to all IPs on the host.
+The volume can be on disk or should be a docker volume prefer.
+```
+# docker compose -f docker-compose.yml -f docker-compose.dev.yml ps -a
+NAME       IMAGE                              COMMAND                                                                  SERVICE    CREATED        STATUS                  PORTS
+api        postgrest/postgrest                "/bin/postgrest"                                                         api        2 months ago   Up 2 months             0.0.0.0:3000->3000/tcp, :::3000->3000/tcp, 0.0.0.0:3003->3003/tcp, :::3003->3003/tcp
+app        grafana/grafana:latest             "/run.sh"                                                                app        3 months ago   Up 12 days              0.0.0.0:3001->3000/tcp, :::3001->3000/tcp
+db         xbgmsharp/timescaledb-postgis      "docker-entrypoint.sh postgres"                                          db         2 months ago   Up 2 months (healthy)   0.0.0.0:5432->5432/tcp, :::5432->5432/tcp
+```
+All services (db,api,web) will be accessible via localhost and others IPs, hence the default configuration.
+
+```bash
+# telnet localhost 5432
+```
+and 
+```bash
+# curl  localhost:3000
+```
+
+```bash
+# docker network ls
+NETWORK ID     NAME                DRIVER    SCOPE
+...
+14f30223ebf2   postgsail_default   bridge    local
+```
+
+Volumes:
+```bash
+% docker volume ls    
+DRIVER    VOLUME NAME
+local     postgsail_grafana-data
+local     postgsail_postgres-data
+```

docs/install_guide.md

@@ -0,0 +1,84 @@
+
+## Connect to the server
+```bash
+% ssh root@my.server.com
+```
+
+# Clone the git repo
+```bash
+% git clone https://github.com/xbgmsharp/postgsail
+Cloning into 'postgsail'...
+...
+```
+
+## Edit the configuration
+```bash
+% cd postgsail
+% cp .env.example .env
+% cat /dev/urandom | LC_ALL=C tr -dc 'a-zA-Z0-9' | fold -w 42 | head -n 1
+..
+% nano .env
+```
+
+## Install Docker
+From https://docs.docker.com/engine/install/ubuntu/
+```bash
+% apt-get update
+...
+% apt-get install -y ca-certificates curl
+...
+% install -m 0755 -d /etc/apt/keyrings
+% curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
+% chmod a+r /etc/apt/keyrings/docker.asc
+% echo \
+  "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu \
+  $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \
+  sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
+% apt-get update
+...
+% apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
+...
+```
+
+## Init the database
+```bash
+% docker compose up db
+...
+Gracefully stopping... (press Ctrl+C again to force)
+[+] Stopping 1/1
+ ✔ Container db  Stopped
+```
+
+## Start the db with the api
+```bash
+% docker compose pull api
+...
+% docker compose up -d db api
+```
+
+## Checks
+Making sure it works.
+```bash
+% telnet localhost 5432
+...
+telnet> quit
+Connection closed.
+% curl localhost:3000
+...
+% docker ps
+...
+% docker logs api
+...
+```
+
+# Run the web instance
+```bash
+% docker compose -f docker-compose.yml -f docker-compose.dev.yml build web (be patient)
+...
+
+% docker compose -f docker-compose.yml -f docker-compose.dev.yml up web (be patient)
+...
+web | 
+web  |   ➜  Local:   http://localhost:8080/
+web  |   ➜  Network: http://172.18.0.4:8080/
+```

initdb/99_migrations_202501.sql

@@ -0,0 +1,219 @@
+---------------------------------------------------------------------------
+-- Copyright 2021-2025 Francois Lacroix <xbgmsharp@gmail.com>
+-- This file is part of PostgSail which is released under Apache License, Version 2.0 (the "License").
+-- See file LICENSE or go to http://www.apache.org/licenses/LICENSE-2.0 for full license details.
+--
+-- Migration January-March 2025
+--
+-- List current database
+select current_database();
+
+-- connect to the DB
+\c signalk
+
+\echo 'Timing mode is enabled'
+\timing
+
+\echo 'Force timezone, just in case'
+set timezone to 'UTC';
+
+-- Update metadata table, mark client_id as deprecated
+COMMENT ON COLUMN api.metadata.client_id IS 'Deprecated client_id to be removed';
+-- Update metrics table, mark client_id as deprecated
+COMMENT ON COLUMN api.metrics.client_id IS 'Deprecated client_id to be removed';
+
+-- Update metadata table update configuration column type to jsonb and comment
+ALTER TABLE api.metadata ALTER COLUMN "configuration" TYPE jsonb USING "configuration"::jsonb;
+COMMENT ON COLUMN api.metadata.configuration IS 'Signalk path mapping for metrics';
+
+-- Update metadata table add new column available_keys and comment
+ALTER TABLE api.metadata ADD available_keys jsonb NULL;
+COMMENT ON COLUMN api.metadata.available_keys IS 'Signalk paths with unit for custom mapping';
+
+--DROP FUNCTION public.metadata_upsert_trigger_fn();
+-- Update metadata_upsert_trigger_fn to metadata table to support configuration and available_keys and deprecated client_id
+CREATE OR REPLACE FUNCTION public.metadata_upsert_trigger_fn()
+ RETURNS trigger
+ LANGUAGE plpgsql
+AS $function$
+    DECLARE
+        metadata_id integer;
+        metadata_active boolean;
+    BEGIN
+        -- Require Signalk plugin version 0.4.0
+        -- Set client_id to new value to allow RLS
+        --PERFORM set_config('vessel.client_id', NEW.client_id, false);
+        -- UPSERT - Insert vs Update for Metadata
+        --RAISE NOTICE 'metadata_upsert_trigger_fn';
+        --PERFORM set_config('vessel.id', NEW.vessel_id, true);
+        --RAISE WARNING 'metadata_upsert_trigger_fn [%] [%]', current_setting('vessel.id', true), NEW;
+        SELECT m.id,m.active INTO metadata_id, metadata_active
+            FROM api.metadata m
+            WHERE m.vessel_id IS NOT NULL AND m.vessel_id = current_setting('vessel.id', true);
+        --RAISE NOTICE 'metadata_id is [%]', metadata_id;
+        IF metadata_id IS NOT NULL THEN
+            -- send notification if boat is back online
+            IF metadata_active is False THEN
+                -- Add monitor online entry to process queue for later notification
+                INSERT INTO process_queue (channel, payload, stored, ref_id)
+                    VALUES ('monitoring_online', metadata_id, now(), current_setting('vessel.id', true));
+            END IF;
+            -- Update vessel metadata
+            UPDATE api.metadata
+                SET
+                    name = NEW.name,
+                    mmsi = NEW.mmsi,
+                    --client_id = NEW.client_id,
+                    length = NEW.length,
+                    beam = NEW.beam,
+                    height = NEW.height,
+                    ship_type = NEW.ship_type,
+                    plugin_version = NEW.plugin_version,
+                    signalk_version = NEW.signalk_version,
+                    platform = REGEXP_REPLACE(NEW.platform, '[^a-zA-Z0-9\(\) ]', '', 'g'),
+                    -- configuration = NEW.configuration, -- ignore configuration from vessel, it is manage by user
+                    -- time = NEW.time, ignore the time sent by the vessel as it is out of sync sometimes.
+                    time = NOW(), -- overwrite the time sent by the vessel
+                    available_keys = NEW.available_keys,
+                    active = true
+                WHERE id = metadata_id;
+            RETURN NULL; -- Ignore insert
+        ELSE
+            IF NEW.vessel_id IS NULL THEN
+                -- set vessel_id from jwt if not present in INSERT query
+                NEW.vessel_id := current_setting('vessel.id');
+            END IF;
+            -- Ignore and overwrite the time sent by the vessel
+            NEW.time := NOW();
+            -- Insert new vessel metadata
+            RETURN NEW; -- Insert new vessel metadata
+        END IF;
+    END;
+$function$
+;
+COMMENT ON FUNCTION public.metadata_upsert_trigger_fn() IS 'process metadata from vessel, upsert';
+
+-- Create or replace the function that will be executed by the trigger
+-- Add metadata table trigger for update_metadata_configuration
+CREATE OR REPLACE FUNCTION api.update_metadata_configuration()
+RETURNS TRIGGER AS $$
+BEGIN
+    -- Require Signalk plugin version 0.4.0
+    -- Update the configuration field with current date in ISO format
+    -- Using jsonb_set if configuration is already a JSONB field
+    IF NEW.configuration IS NOT NULL AND
+        jsonb_typeof(NEW.configuration) = 'object' THEN
+        NEW.configuration = jsonb_set(
+            NEW.configuration, 
+            '{update_at}', 
+            to_jsonb(to_char(NOW(), 'YYYY-MM-DD"T"HH24:MI:SS"Z"'))
+        );
+    END IF;
+    
+    RETURN NEW;
+END;
+$$ LANGUAGE plpgsql;
+COMMENT ON FUNCTION api.update_metadata_configuration() IS 'Update the configuration field with current date in ISO format';
+
+-- Create the trigger
+CREATE TRIGGER metadata_update_configuration_trigger
+BEFORE UPDATE ON api.metadata
+FOR EACH ROW
+EXECUTE FUNCTION api.update_metadata_configuration();
+
+-- Update api.export_logbook_geojson_linestring_trip_fn, add metadata properties
+CREATE OR REPLACE FUNCTION api.export_logbooks_geojson_linestring_trips_fn(
+    start_log integer DEFAULT NULL::integer,
+    end_log integer DEFAULT NULL::integer,
+    start_date text DEFAULT NULL::text,
+    end_date text DEFAULT NULL::text,
+    OUT geojson jsonb
+) RETURNS jsonb
+LANGUAGE plpgsql
+AS $function$
+DECLARE
+    logs_geojson jsonb;
+BEGIN
+    -- Normalize start and end values
+    IF start_log IS NOT NULL AND end_log IS NULL THEN end_log := start_log; END IF;
+    IF start_date IS NOT NULL AND end_date IS NULL THEN end_date := start_date; END IF;
+
+    WITH logbook_data AS (
+        -- get the logbook geometry and metadata, an array for each log
+        SELECT id, name,
+            starttimestamp(trip),
+            endtimestamp(trip),
+            --speed(trip_sog),
+            duration(trip),
+            --length(trip) as length, -- Meters
+            (length(trip) * 0.0005399568)::numeric as distance, -- NM
+            twavg(trip_sog) as avg_sog,
+            maxValue(trip_sog) as max_sog,
+            maxValue(trip_depth) as max_depth, -- Depth
+            maxValue(trip_batt_charge) as max_batt_charge, -- Battery Charge
+            maxValue(trip_batt_voltage) as max_batt_voltage, -- Battery Voltage
+            maxValue(trip_temp_water) as max_temp_water, -- Temperature water
+            maxValue(trip_temp_out) as max_temp_out, -- Temperature outside
+            maxValue(trip_pres_out) as max_pres_out, -- Pressure outside
+            maxValue(trip_hum_out) as max_hum_out, -- Humidity outside
+            twavg(trip_depth) as avg_depth, -- Depth
+            twavg(trip_batt_charge) as avg_batt_charge, -- Battery Charge
+            twavg(trip_batt_voltage) as avg_batt_voltage, -- Battery Voltage
+            twavg(trip_temp_water) as avg_temp_water, -- Temperature water
+            twavg(trip_temp_out) as avg_temp_out, -- Temperature outside
+            twavg(trip_pres_out) as avg_pres_out, -- Pressure outside
+            twavg(trip_hum_out) as avg_hum_out, -- Humidity outside
+            trajectory(l.trip)::geometry as track_geog -- extract trip to geography
+        FROM api.logbook l
+        WHERE (start_log IS NULL OR l.id >= start_log) AND
+              (end_log IS NULL OR l.id <= end_log) AND
+              (start_date IS NULL OR l._from_time >= start_date::TIMESTAMPTZ) AND
+              (end_date IS NULL OR l._to_time <= end_date::TIMESTAMPTZ + interval '23 hours 59 minutes') AND
+              l.trip IS NOT NULL
+        ORDER BY l._from_time ASC
+    ),
+    collect as (
+        SELECT ST_Collect(
+                        ARRAY(
+                            SELECT track_geog FROM logbook_data))
+    )
+    -- Create the GeoJSON response
+    SELECT jsonb_build_object(
+        'type', 'FeatureCollection',
+        'features', json_agg(ST_AsGeoJSON(logs.*)::json)) INTO geojson FROM logbook_data logs;
+END;
+$function$;
+-- Description
+COMMENT ON FUNCTION api.export_logbooks_geojson_linestring_trips_fn IS 'Generate geojson geometry LineString from trip with the corresponding properties';
+
+-- Add public.get_season, return the season based on the input date for logbook tag
+CREATE OR REPLACE FUNCTION public.get_season(input_date TIMESTAMPTZ)
+RETURNS TEXT AS $$
+BEGIN
+    CASE
+        WHEN (EXTRACT(MONTH FROM input_date) = 3 AND EXTRACT(DAY FROM input_date) >= 1) OR
+             (EXTRACT(MONTH FROM input_date) BETWEEN 4 AND 5) THEN
+            RETURN 'Spring';
+        WHEN (EXTRACT(MONTH FROM input_date) = 6 AND EXTRACT(DAY FROM input_date) >= 1) OR
+             (EXTRACT(MONTH FROM input_date) BETWEEN 7 AND 8) THEN
+            RETURN 'Summer';
+        WHEN (EXTRACT(MONTH FROM input_date) = 9 AND EXTRACT(DAY FROM input_date) >= 1) OR
+             (EXTRACT(MONTH FROM input_date) BETWEEN 10 AND 11) THEN
+            RETURN 'Fall';
+        ELSE
+            RETURN 'Winter';
+    END CASE;
+END;
+$$ LANGUAGE plpgsql IMMUTABLE;
+
+-- Refresh permissions
+GRANT SELECT ON TABLE api.metrics,api.metadata TO scheduler;
+GRANT INSERT, UPDATE, SELECT ON TABLE api.logbook,api.moorages,api.stays TO scheduler;
+GRANT EXECUTE ON ALL FUNCTIONS IN SCHEMA public TO scheduler;
+GRANT SELECT ON ALL TABLES IN SCHEMA public TO scheduler;
+GRANT SELECT, UPDATE ON TABLE public.process_queue TO scheduler;
+
+-- Update version
+UPDATE public.app_settings
+	SET value='0.9.0'
+	WHERE "name"='app.version';

initdb/99_migrations_202507.sql

@@ -0,0 +1,734 @@
+---------------------------------------------------------------------------
+-- Copyright 2021-2025 Francois Lacroix <xbgmsharp@gmail.com>
+-- This file is part of PostgSail which is released under Apache License, Version 2.0 (the "License").
+-- See file LICENSE or go to http://www.apache.org/licenses/LICENSE-2.0 for full license details.
+--
+-- Migration June/July 2025
+--
+-- List current database
+select current_database();
+
+-- connect to the DB
+\c signalk
+
+\echo 'Timing mode is enabled'
+\timing
+
+\echo 'Force timezone, just in case'
+set timezone to 'UTC';
+
+-- Update plugin upgrade message
+UPDATE public.email_templates
+	SET email_content='Hello __RECIPIENT__,
+Please upgrade your postgsail signalk plugin. Make sure you restart your Signalk instance after upgrading. Be sure to contact me if you encounter any issue.'
+	WHERE "name"='skplugin_upgrade';
+
+-- DROP FUNCTION api.login(text, text);
+-- Update api.login, update the connected_at field to the current time
+CREATE OR REPLACE FUNCTION api.login(email text, pass text)
+ RETURNS auth.jwt_token
+ LANGUAGE plpgsql
+ SECURITY DEFINER
+AS $function$
+declare
+  _role name;
+  result auth.jwt_token;
+  app_jwt_secret text;
+  _email_valid boolean := false;
+  _email text := email;
+  _user_id text := null;
+  _user_disable boolean := false;
+  headers   json := current_setting('request.headers', true)::json;
+  client_ip text := coalesce(headers->>'x-client-ip', NULL);
+begin
+  -- check email and password
+  select auth.user_role(email, pass) into _role;
+  if _role is null then
+    -- HTTP/403
+    --raise invalid_password using message = 'invalid user or password';
+    -- HTTP/401
+    raise insufficient_privilege using message = 'invalid user or password';
+  end if;
+
+  -- Check if user is disable due to abuse
+  SELECT preferences['disable'],user_id INTO _user_disable,_user_id
+              FROM auth.accounts a
+              WHERE a.email = _email;
+  IF _user_disable is True then
+  	-- due to the raise, the insert is never committed.
+    --INSERT INTO process_queue (channel, payload, stored, ref_id)
+    --  VALUES ('account_disable', _email, now(), _user_id);
+    RAISE sqlstate 'PT402' using message = 'Account disable, contact us',
+            detail = 'Quota exceeded',
+            hint = 'Upgrade your plan';
+  END IF;
+
+  -- Check email_valid and generate OTP
+  SELECT preferences['email_valid'],user_id INTO _email_valid,_user_id
+              FROM auth.accounts a
+              WHERE a.email = _email;
+  IF _email_valid is null or _email_valid is False THEN
+    INSERT INTO process_queue (channel, payload, stored, ref_id)
+      VALUES ('email_otp', _email, now(), _user_id);
+  END IF;
+
+  -- Track IP per user to avoid abuse
+  --RAISE WARNING 'api.login debug: [%],[%]', client_ip, login.email;
+  IF client_ip IS NOT NULL THEN
+    UPDATE auth.accounts a SET 
+        preferences = jsonb_recursive_merge(a.preferences, jsonb_build_object('ip', client_ip)),
+        connected_at = NOW()
+        WHERE a.email = login.email;
+  END IF;
+
+  -- Get app_jwt_secret
+  SELECT value INTO app_jwt_secret
+    FROM app_settings
+    WHERE name = 'app.jwt_secret';
+
+  --RAISE WARNING 'api.login debug: [%],[%],[%]', app_jwt_secret, _role, login.email;
+  -- Generate jwt
+  select jwt.sign(
+  --    row_to_json(r), ''
+  --    row_to_json(r)::json, current_setting('app.jwt_secret')::text
+      row_to_json(r)::json, app_jwt_secret
+    ) as token
+    from (
+      select _role as role, login.email as email,  -- TODO replace with user_id
+    --  select _role as role, user_id as uid, -- add support in check_jwt
+         extract(epoch from now())::integer + 60*60 as exp
+    ) r
+    into result;
+  return result;
+end;
+$function$
+;
+-- Description
+COMMENT ON FUNCTION api.login IS 'Handle user login, returns a JWT token with user role and email.';
+
+-- DROP FUNCTION api.monitoring_history_fn(in text, out jsonb);
+-- Update monitoring_history_fn to use custom user settings for metrics
+CREATE OR REPLACE FUNCTION api.monitoring_history_fn(time_interval text DEFAULT '24'::text, OUT history_metrics jsonb)
+ RETURNS jsonb
+ LANGUAGE plpgsql
+AS $function$
+    DECLARE
+        bucket_interval interval := '5 minutes';
+    BEGIN
+        RAISE NOTICE '-> monitoring_history_fn';
+        SELECT CASE time_interval
+            WHEN '24' THEN '5 minutes'
+            WHEN '48' THEN '2 hours'
+            WHEN '72' THEN '4 hours'
+            WHEN '168' THEN '7 hours'
+            ELSE '5 minutes'
+            END bucket INTO bucket_interval;
+        RAISE NOTICE '-> monitoring_history_fn % %', time_interval, bucket_interval;
+        WITH history_table AS (
+            SELECT time_bucket(bucket_interval::INTERVAL, mt.time) AS time_bucket,
+                avg(-- Water Temperature
+                    COALESCE(
+                        mt.metrics->'water'->>'temperature',
+                        mt.metrics->>(md.configuration->>'waterTemperatureKey'),
+                        mt.metrics->>'environment.water.temperature'
+                    )::FLOAT) AS waterTemperature,
+                avg(-- Inside Temperature
+                    COALESCE(
+                        mt.metrics->'temperature'->>'inside',
+                        mt.metrics->>(md.configuration->>'insideTemperatureKey'),
+                        mt.metrics->>'environment.inside.temperature'
+                    )::FLOAT) AS insideTemperature,
+                avg(-- Outside Temperature
+                    COALESCE(
+                        mt.metrics->'temperature'->>'outside',
+                        mt.metrics->>(md.configuration->>'outsideTemperatureKey'),
+                        mt.metrics->>'environment.outside.temperature'
+                    )::FLOAT) AS outsideTemperature,
+                avg(-- Wind Speed True
+                    COALESCE(
+                        mt.metrics->'wind'->>'speed',
+                        mt.metrics->>(md.configuration->>'windSpeedKey'),
+                        mt.metrics->>'environment.wind.speedTrue'
+                    )::FLOAT) AS windSpeedOverGround,
+                avg(-- Inside Humidity
+                    COALESCE(
+                        mt.metrics->'humidity'->>'inside',
+                        mt.metrics->>(md.configuration->>'insideHumidityKey'),
+                        mt.metrics->>'environment.inside.relativeHumidity',
+                        mt.metrics->>'environment.inside.humidity'
+                    )::FLOAT) AS insideHumidity,
+                avg(-- Outside Humidity
+                    COALESCE(
+                        mt.metrics->'humidity'->>'outside',
+                        mt.metrics->>(md.configuration->>'outsideHumidityKey'),
+                        mt.metrics->>'environment.outside.relativeHumidity',
+                        mt.metrics->>'environment.outside.humidity'
+                    )::FLOAT) AS outsideHumidity,
+                avg(-- Outside Pressure
+                    COALESCE(
+                        mt.metrics->'pressure'->>'outside',
+                        mt.metrics->>(md.configuration->>'outsidePressureKey'),
+                        mt.metrics->>'environment.outside.pressure'
+                    )::FLOAT) AS outsidePressure,
+                avg(--Inside Pressure
+                    COALESCE(
+                        mt.metrics->'pressure'->>'inside',
+                        mt.metrics->>(md.configuration->>'insidePressureKey'),
+                        mt.metrics->>'environment.inside.pressure'
+                    )::FLOAT) AS insidePressure,
+                avg(-- Battery Charge (State of Charge)
+                    COALESCE(
+                        mt.metrics->'battery'->>'charge',
+                        mt.metrics->>(md.configuration->>'stateOfChargeKey'),
+                        mt.metrics->>'electrical.batteries.House.capacity.stateOfCharge'
+                    )::FLOAT) AS batteryCharge,
+                avg(-- Battery Voltage
+                    COALESCE(
+                        mt.metrics->'battery'->>'voltage',
+                        mt.metrics->>(md.configuration->>'voltageKey'),
+                        mt.metrics->>'electrical.batteries.House.voltage'
+                    )::FLOAT) AS batteryVoltage,
+                avg(-- Water Depth
+                    COALESCE(
+                        mt.metrics->'water'->>'depth',
+                        mt.metrics->>(md.configuration->>'depthKey'),
+                        mt.metrics->>'environment.depth.belowTransducer'
+                    )::FLOAT) AS depth
+                FROM api.metrics mt
+				JOIN api.metadata md ON md.vessel_id = mt.vessel_id
+                WHERE mt.time > (NOW() AT TIME ZONE 'UTC' - INTERVAL '1 hours' * time_interval::NUMERIC)
+                GROUP BY time_bucket
+                ORDER BY time_bucket asc
+        )
+        SELECT jsonb_agg(history_table) INTO history_metrics FROM history_table;
+    END
+$function$
+;
+-- Description
+COMMENT ON FUNCTION api.monitoring_history_fn(in text, out jsonb) IS 'Export metrics from a time period 24h, 48h, 72h, 7d';
+
+-- DROP FUNCTION public.cron_alerts_fn();
+-- Update cron_alerts_fn to check for alerts, filters out empty strings (""), so they are not included in the result.
+CREATE OR REPLACE FUNCTION public.cron_alerts_fn()
+ RETURNS void
+ LANGUAGE plpgsql
+AS $function$
+DECLARE
+    alert_rec record;
+    default_last_metric TIMESTAMPTZ := NOW() - interval '1 day';
+    last_metric TIMESTAMPTZ;
+    metric_rec record;
+    app_settings JSONB;
+    user_settings JSONB;
+    alerting JSONB;
+    _alarms JSONB;
+    alarms TEXT;
+    alert_default JSONB := '{
+        "low_pressure_threshold": 990,
+        "high_wind_speed_threshold": 30,
+        "low_water_depth_threshold": 1,
+        "min_notification_interval": 6,
+        "high_pressure_drop_threshold": 12,
+        "low_battery_charge_threshold": 90,
+        "low_battery_voltage_threshold": 12.5,
+        "low_water_temperature_threshold": 10,
+        "low_indoor_temperature_threshold": 7,
+        "low_outdoor_temperature_threshold": 3
+    }';
+BEGIN
+    -- Check for new event notification pending update
+    RAISE NOTICE 'cron_alerts_fn';
+    FOR alert_rec in
+        SELECT
+            a.user_id,a.email,v.vessel_id,
+            COALESCE((a.preferences->'alert_last_metric')::TEXT, default_last_metric::TEXT) as last_metric,
+            (alert_default || ( -- Filters out empty strings (""), so they are not included in the result.
+							    SELECT jsonb_object_agg(key, value)
+							    FROM jsonb_each(a.preferences->'alerting') 
+							    WHERE value <> '""'
+							  )) as alerting,
+            (a.preferences->'alarms')::JSONB as alarms,
+            m.configuration as config
+            FROM auth.accounts a
+            LEFT JOIN auth.vessels AS v ON v.owner_email = a.email
+            LEFT JOIN api.metadata AS m ON m.vessel_id = v.vessel_id
+            WHERE (a.preferences->'alerting'->'enabled')::boolean = True
+                AND m.active = True
+        LOOP
+        RAISE NOTICE '-> cron_alerts_fn for [%]', alert_rec;
+        PERFORM set_config('vessel.id', alert_rec.vessel_id, false);
+        PERFORM set_config('user.email', alert_rec.email, false);
+        --RAISE WARNING 'public.cron_process_alert_rec_fn() scheduler vessel.id %, user.id', current_setting('vessel.id', false), current_setting('user.id', false);
+        -- Gather user settings
+        user_settings := get_user_settings_from_vesselid_fn(alert_rec.vessel_id::TEXT);
+        RAISE NOTICE '-> cron_alerts_fn checking user_settings [%]', user_settings;
+        -- Get all metrics from the last last_metric avg by 5 minutes
+        FOR metric_rec in
+            SELECT time_bucket('5 minutes', m.time) AS time_bucket,
+                    avg(-- Inside Temperature
+                        COALESCE(
+                            mt.metrics->'temperature'->>'inside',
+                            mt.metrics->>(md.configuration->>'insideTemperatureKey'),
+                            mt.metrics->>'environment.inside.temperature'
+                        )::FLOAT) AS intemp,
+                    avg(-- Wind Speed True
+                        COALESCE(
+                            mt.metrics->'wind'->>'speed',
+                            mt.metrics->>(md.configuration->>'windSpeedKey'),
+                            mt.metrics->>'environment.wind.speedTrue'
+                        )::FLOAT) AS wind,
+                    avg(-- Water Depth
+                        COALESCE(
+                            mt.metrics->'water'->>'depth',
+                            mt.metrics->>(md.configuration->>'depthKey'),
+                            mt.metrics->>'environment.depth.belowTransducer'
+                        )::FLOAT) AS watdepth,
+                    avg(-- Outside Temperature
+                        COALESCE(
+                            m.metrics->'temperature'->>'outside',
+                            m.metrics->>(alert_rec.config->>'outsideTemperatureKey'),
+                            m.metrics->>'environment.outside.temperature'
+                        )::NUMERIC) AS outtemp,
+                    avg(-- Water Temperature
+                        COALESCE(
+                            m.metrics->'water'->>'temperature',
+                            m.metrics->>(alert_rec.config->>'waterTemperatureKey'),
+                            m.metrics->>'environment.water.temperature'
+                        )::NUMERIC) AS wattemp,
+                    avg(-- Outside Pressure
+                        COALESCE(
+                            m.metrics->'pressure'->>'outside',
+                            m.metrics->>(alert_rec.config->>'outsidePressureKey'),
+                            m.metrics->>'environment.outside.pressure'
+                        )::NUMERIC) AS pressure,
+                    avg(-- Battery Voltage
+                        COALESCE(
+                            m.metrics->'battery'->>'voltage',
+                            m.metrics->>(alert_rec.config->>'voltageKey'),
+                            m.metrics->>'electrical.batteries.House.voltage'
+                        )::NUMERIC) AS voltage,
+                    avg(-- Battery Charge (State of Charge)
+                        COALESCE(
+                            m.metrics->'battery'->>'charge',
+                            m.metrics->>(alert_rec.config->>'stateOfChargeKey'),
+                            m.metrics->>'electrical.batteries.House.capacity.stateOfCharge'
+                        )::NUMERIC) AS charge
+                FROM api.metrics m
+                WHERE vessel_id = alert_rec.vessel_id
+                    AND m.time >= alert_rec.last_metric::TIMESTAMPTZ
+                GROUP BY time_bucket
+                ORDER BY time_bucket ASC LIMIT 100
+        LOOP
+            RAISE NOTICE '-> cron_alerts_fn checking metrics [%]', metric_rec;
+            RAISE NOTICE '-> cron_alerts_fn checking alerting [%]', alert_rec.alerting;
+            --RAISE NOTICE '-> cron_alerts_fn checking debug [%] [%]', kelvinToCel(metric_rec.intemp), (alert_rec.alerting->'low_indoor_temperature_threshold');
+            IF metric_rec.intemp IS NOT NULL AND public.kelvintocel(metric_rec.intemp::NUMERIC) < (alert_rec.alerting->'low_indoor_temperature_threshold')::NUMERIC then
+                RAISE NOTICE '-> cron_alerts_fn checking debug indoor_temp [%]', (alert_rec.alarms->'low_indoor_temperature_threshold'->>'date')::TIMESTAMPTZ;
+                RAISE NOTICE '-> cron_alerts_fn checking debug indoor_temp [%]', metric_rec.time_bucket::TIMESTAMPTZ;
+                -- Get latest alarms
+                SELECT preferences->'alarms' INTO _alarms FROM auth.accounts a WHERE a.email = current_setting('user.email', false);
+                -- Is alarm in the min_notification_interval time frame
+                IF (
+                    ((_alarms->'low_indoor_temperature_threshold'->>'date') IS NULL) OR
+                    (((_alarms->'low_indoor_temperature_threshold'->>'date')::TIMESTAMPTZ
+                    + ((interval '1 hour') * (alert_rec.alerting->>'min_notification_interval')::NUMERIC))
+                    < metric_rec.time_bucket::TIMESTAMPTZ)
+                    ) THEN
+                    -- Add alarm
+                    alarms := '{"low_indoor_temperature_threshold": {"value": '|| kelvinToCel(metric_rec.intemp) ||', "date":"' || metric_rec.time_bucket || '"}}';
+                    -- Merge alarms
+                    SELECT public.jsonb_recursive_merge(_alarms::jsonb, alarms::jsonb) into _alarms;
+                    -- Update alarms for user
+                    PERFORM api.update_user_preferences_fn('{alarms}'::TEXT, _alarms::TEXT);
+                    -- Gather user settings
+                    user_settings := get_user_settings_from_vesselid_fn(current_setting('vessel.id', false));
+                    SELECT user_settings::JSONB || ('{"alert": "low_outdoor_temperature_threshold value:'|| kelvinToCel(metric_rec.intemp) ||' date:'|| metric_rec.time_bucket ||' "}'::text)::JSONB into user_settings;
+                    -- Send notification
+                    PERFORM send_notification_fn('alert'::TEXT, user_settings::JSONB);
+                    -- DEBUG
+                    RAISE NOTICE '-> cron_alerts_fn checking debug low_indoor_temperature_threshold +interval';
+                END IF;
+                RAISE NOTICE '-> cron_alerts_fn checking debug low_indoor_temperature_threshold';
+            END IF;
+            IF metric_rec.outtemp IS NOT NULL AND public.kelvintocel(metric_rec.outtemp::NUMERIC) < (alert_rec.alerting->>'low_outdoor_temperature_threshold')::NUMERIC then
+                RAISE NOTICE '-> cron_alerts_fn checking debug outdoor_temp [%]', (alert_rec.alarms->'low_outdoor_temperature_threshold'->>'date')::TIMESTAMPTZ;
+                RAISE NOTICE '-> cron_alerts_fn checking debug outdoor_temp [%]', metric_rec.time_bucket::TIMESTAMPTZ;
+                -- Get latest alarms
+                SELECT preferences->'alarms' INTO _alarms FROM auth.accounts a WHERE a.email = current_setting('user.email', false);
+                -- Is alarm in the min_notification_interval time frame
+                IF (
+                    ((_alarms->'low_outdoor_temperature_threshold'->>'date') IS NULL) OR
+                    (((_alarms->'low_outdoor_temperature_threshold'->>'date')::TIMESTAMPTZ
+                    + ((interval '1 hour') * (alert_rec.alerting->>'min_notification_interval')::NUMERIC))
+                    < metric_rec.time_bucket::TIMESTAMPTZ)
+                    ) THEN
+                    -- Add alarm
+                    alarms := '{"low_outdoor_temperature_threshold": {"value": '|| kelvinToCel(metric_rec.outtemp) ||', "date":"' || metric_rec.time_bucket || '"}}';
+                    -- Merge alarms
+                    SELECT public.jsonb_recursive_merge(_alarms::jsonb, alarms::jsonb) into _alarms;
+                    -- Update alarms for user
+                    PERFORM api.update_user_preferences_fn('{alarms}'::TEXT, _alarms::TEXT);
+                    -- Gather user settings
+                    user_settings := get_user_settings_from_vesselid_fn(current_setting('vessel.id', false));
+                    SELECT user_settings::JSONB || ('{"alert": "low_outdoor_temperature_threshold value:'|| kelvinToCel(metric_rec.outtemp) ||' date:'|| metric_rec.time_bucket ||' "}'::text)::JSONB into user_settings;
+                    -- Send notification
+                    PERFORM send_notification_fn('alert'::TEXT, user_settings::JSONB);
+                    -- DEBUG
+                    RAISE NOTICE '-> cron_alerts_fn checking debug low_outdoor_temperature_threshold +interval';
+                END IF;
+                RAISE NOTICE '-> cron_alerts_fn checking debug low_outdoor_temperature_threshold';
+            END IF;
+            IF metric_rec.wattemp IS NOT NULL AND public.kelvintocel(metric_rec.wattemp::NUMERIC) < (alert_rec.alerting->>'low_water_temperature_threshold')::NUMERIC then
+                RAISE NOTICE '-> cron_alerts_fn checking debug water_temp [%]', (alert_rec.alarms->'low_water_temperature_threshold'->>'date')::TIMESTAMPTZ;
+                RAISE NOTICE '-> cron_alerts_fn checking debug water_temp [%]', metric_rec.time_bucket::TIMESTAMPTZ;
+                -- Get latest alarms
+                SELECT preferences->'alarms' INTO _alarms FROM auth.accounts a WHERE a.email = current_setting('user.email', false);
+                -- Is alarm in the min_notification_interval time frame
+                IF (
+                    ((_alarms->'low_water_temperature_threshold'->>'date') IS NULL) OR
+                    (((_alarms->'low_water_temperature_threshold'->>'date')::TIMESTAMPTZ
+                    + ((interval '1 hour') * (alert_rec.alerting->>'min_notification_interval')::NUMERIC))
+                    < metric_rec.time_bucket::TIMESTAMPTZ)
+                    ) THEN
+                    -- Add alarm
+                    alarms := '{"low_water_temperature_threshold": {"value": '|| kelvinToCel(metric_rec.wattemp) ||', "date":"' || metric_rec.time_bucket || '"}}';
+                    -- Merge alarms
+                    SELECT public.jsonb_recursive_merge(_alarms::jsonb, alarms::jsonb) into _alarms;
+                    -- Update alarms for user
+                    PERFORM api.update_user_preferences_fn('{alarms}'::TEXT, _alarms::TEXT);
+                    -- Gather user settings
+                    user_settings := get_user_settings_from_vesselid_fn(current_setting('vessel.id', false));
+                    SELECT user_settings::JSONB || ('{"alert": "low_water_temperature_threshold value:'|| kelvinToCel(metric_rec.wattemp) ||' date:'|| metric_rec.time_bucket ||' "}'::text)::JSONB into user_settings;
+                    -- Send notification
+                    PERFORM send_notification_fn('alert'::TEXT, user_settings::JSONB);
+                    -- DEBUG
+                    RAISE NOTICE '-> cron_alerts_fn checking debug low_water_temperature_threshold +interval';
+                END IF;
+                RAISE NOTICE '-> cron_alerts_fn checking debug low_water_temperature_threshold';
+            END IF;
+            IF metric_rec.watdepth IS NOT NULL AND metric_rec.watdepth::NUMERIC < (alert_rec.alerting->'low_water_depth_threshold')::NUMERIC then
+                RAISE NOTICE '-> cron_alerts_fn checking debug water_depth [%]', (alert_rec.alarms->'low_water_depth_threshold'->>'date')::TIMESTAMPTZ;
+                RAISE NOTICE '-> cron_alerts_fn checking debug water_depth [%]', metric_rec.time_bucket::TIMESTAMPTZ;
+                -- Get latest alarms
+                SELECT preferences->'alarms' INTO _alarms FROM auth.accounts a WHERE a.email = current_setting('user.email', false);
+                -- Is alarm in the min_notification_interval time frame
+                IF (
+                    ((_alarms->'low_water_depth_threshold'->>'date') IS NULL) OR
+                    (((_alarms->'low_water_depth_threshold'->>'date')::TIMESTAMPTZ
+                    + ((interval '1 hour') * (alert_rec.alerting->>'min_notification_interval')::NUMERIC))
+                    < metric_rec.time_bucket::TIMESTAMPTZ)
+                    ) THEN
+                    -- Add alarm
+                    alarms := '{"low_water_depth_threshold": {"value": '|| metric_rec.watdepth ||', "date":"' || metric_rec.time_bucket || '"}}';
+                    -- Merge alarms
+                    SELECT public.jsonb_recursive_merge(_alarms::jsonb, alarms::jsonb) into _alarms;
+                    -- Update alarms for user
+                    PERFORM api.update_user_preferences_fn('{alarms}'::TEXT, _alarms::TEXT);
+                    -- Gather user settings
+                    user_settings := get_user_settings_from_vesselid_fn(current_setting('vessel.id', false));
+                    SELECT user_settings::JSONB || ('{"alert": "low_water_depth_threshold value:'|| ROUND(metric_rec.watdepth,2) ||' date:'|| metric_rec.time_bucket ||' "}'::text)::JSONB into user_settings;
+                    -- Send notification
+                    PERFORM send_notification_fn('alert'::TEXT, user_settings::JSONB);
+                    -- DEBUG
+                    RAISE NOTICE '-> cron_alerts_fn checking debug low_water_depth_threshold +interval';
+                END IF;
+                RAISE NOTICE '-> cron_alerts_fn checking debug low_water_depth_threshold';
+            END IF;
+            if metric_rec.pressure IS NOT NULL AND metric_rec.pressure::NUMERIC < (alert_rec.alerting->'high_pressure_drop_threshold')::NUMERIC then
+                RAISE NOTICE '-> cron_alerts_fn checking debug pressure [%]', (alert_rec.alarms->'high_pressure_drop_threshold'->>'date')::TIMESTAMPTZ;
+                RAISE NOTICE '-> cron_alerts_fn checking debug pressure [%]', metric_rec.time_bucket::TIMESTAMPTZ;
+                -- Get latest alarms
+                SELECT preferences->'alarms' INTO _alarms FROM auth.accounts a WHERE a.email = current_setting('user.email', false);
+                -- Is alarm in the min_notification_interval time frame
+                IF (
+                    ((_alarms->'high_pressure_drop_threshold'->>'date') IS NULL) OR
+                    (((_alarms->'high_pressure_drop_threshold'->>'date')::TIMESTAMPTZ
+                    + ((interval '1 hour') * (alert_rec.alerting->>'min_notification_interval')::NUMERIC))
+                    < metric_rec.time_bucket::TIMESTAMPTZ)
+                    ) THEN
+                    -- Add alarm
+                    alarms := '{"high_pressure_drop_threshold": {"value": '|| metric_rec.pressure ||', "date":"' || metric_rec.time_bucket || '"}}';
+                    -- Merge alarms
+                    SELECT public.jsonb_recursive_merge(_alarms::jsonb, alarms::jsonb) into _alarms;
+                    -- Update alarms for user
+                    PERFORM api.update_user_preferences_fn('{alarms}'::TEXT, _alarms::TEXT);
+                    -- Gather user settings
+                    user_settings := get_user_settings_from_vesselid_fn(current_setting('vessel.id', false));
+                    SELECT user_settings::JSONB || ('{"alert": "high_pressure_drop_threshold value:'|| ROUND(metric_rec.pressure,2) ||' date:'|| metric_rec.time_bucket ||' "}'::text)::JSONB into user_settings;
+                    -- Send notification
+                    PERFORM send_notification_fn('alert'::TEXT, user_settings::JSONB);
+                    -- DEBUG
+                    RAISE NOTICE '-> cron_alerts_fn checking debug high_pressure_drop_threshold +interval';
+                END IF;
+                RAISE NOTICE '-> cron_alerts_fn checking debug high_pressure_drop_threshold';
+            END IF;
+            IF metric_rec.wind IS NOT NULL AND metric_rec.wind::NUMERIC > (alert_rec.alerting->'high_wind_speed_threshold')::NUMERIC then
+                RAISE NOTICE '-> cron_alerts_fn checking debug wind [%]', (alert_rec.alarms->'high_wind_speed_threshold'->>'date')::TIMESTAMPTZ;
+                RAISE NOTICE '-> cron_alerts_fn checking debug wind [%]', metric_rec.time_bucket::TIMESTAMPTZ;
+                -- Get latest alarms
+                SELECT preferences->'alarms' INTO _alarms FROM auth.accounts a WHERE a.email = current_setting('user.email', false);
+                -- Is alarm in the min_notification_interval time frame
+                IF (
+                    ((_alarms->'high_wind_speed_threshold'->>'date') IS NULL) OR
+                    (((_alarms->'high_wind_speed_threshold'->>'date')::TIMESTAMPTZ
+                    + ((interval '1 hour') * (alert_rec.alerting->>'min_notification_interval')::NUMERIC))
+                    < metric_rec.time_bucket::TIMESTAMPTZ)
+                    ) THEN
+                    -- Add alarm
+                    alarms := '{"high_wind_speed_threshold": {"value": '|| metric_rec.wind ||', "date":"' || metric_rec.time_bucket || '"}}';
+                    -- Merge alarms
+                    SELECT public.jsonb_recursive_merge(_alarms::jsonb, alarms::jsonb) into _alarms;
+                    -- Update alarms for user
+                    PERFORM api.update_user_preferences_fn('{alarms}'::TEXT, _alarms::TEXT);
+                    -- Gather user settings
+                    user_settings := get_user_settings_from_vesselid_fn(current_setting('vessel.id', false));
+                    SELECT user_settings::JSONB || ('{"alert": "high_wind_speed_threshold value:'|| ROUND(metric_rec.wind,2) ||' date:'|| metric_rec.time_bucket ||' "}'::text)::JSONB into user_settings;
+                    -- Send notification
+                    PERFORM send_notification_fn('alert'::TEXT, user_settings::JSONB);
+                    -- DEBUG
+                    RAISE NOTICE '-> cron_alerts_fn checking debug high_wind_speed_threshold +interval';
+                END IF;
+                RAISE NOTICE '-> cron_alerts_fn checking debug high_wind_speed_threshold';
+            END IF;
+            IF metric_rec.voltage IS NOT NULL AND metric_rec.voltage::NUMERIC < (alert_rec.alerting->'low_battery_voltage_threshold')::NUMERIC then
+                RAISE NOTICE '-> cron_alerts_fn checking debug voltage [%]', (alert_rec.alarms->'low_battery_voltage_threshold'->>'date')::TIMESTAMPTZ;
+                RAISE NOTICE '-> cron_alerts_fn checking debug voltage [%]', metric_rec.time_bucket::TIMESTAMPTZ;
+                -- Get latest alarms
+                SELECT preferences->'alarms' INTO _alarms FROM auth.accounts a WHERE a.email = current_setting('user.email', false);
+                -- Is alarm in the min_notification_interval time frame
+                IF (
+                    ((_alarms->'low_battery_voltage_threshold'->>'date') IS NULL) OR
+                    (((_alarms->'low_battery_voltage_threshold'->>'date')::TIMESTAMPTZ
+                    + ((interval '1 hour') * (alert_rec.alerting->>'min_notification_interval')::NUMERIC))
+                    < metric_rec.time_bucket::TIMESTAMPTZ)
+                    ) THEN
+                    -- Add alarm
+                    alarms := '{"low_battery_voltage_threshold": {"value": '|| metric_rec.voltage ||', "date":"' || metric_rec.time_bucket || '"}}';
+                    -- Merge alarms
+                    SELECT public.jsonb_recursive_merge(_alarms::jsonb, alarms::jsonb) into _alarms;
+                    -- Update alarms for user
+                    PERFORM api.update_user_preferences_fn('{alarms}'::TEXT, _alarms::TEXT);
+                    -- Gather user settings
+                    user_settings := get_user_settings_from_vesselid_fn(current_setting('vessel.id', false));
+                    SELECT user_settings::JSONB || ('{"alert": "low_battery_voltage_threshold value:'|| ROUND(metric_rec.voltage,2) ||' date:'|| metric_rec.time_bucket ||' "}'::text)::JSONB into user_settings;
+                    -- Send notification
+                    PERFORM send_notification_fn('alert'::TEXT, user_settings::JSONB);
+                    -- DEBUG
+                    RAISE NOTICE '-> cron_alerts_fn checking debug low_battery_voltage_threshold +interval';
+                END IF;
+                RAISE NOTICE '-> cron_alerts_fn checking debug low_battery_voltage_threshold';
+            END IF;
+            IF metric_rec.charge IS NOT NULL AND (metric_rec.charge::NUMERIC*100) < (alert_rec.alerting->'low_battery_charge_threshold')::NUMERIC then
+                RAISE NOTICE '-> cron_alerts_fn checking debug [%]', (alert_rec.alarms->'low_battery_charge_threshold'->>'date')::TIMESTAMPTZ;
+                RAISE NOTICE '-> cron_alerts_fn checking debug [%]', metric_rec.time_bucket::TIMESTAMPTZ;
+                -- Get latest alarms
+                SELECT preferences->'alarms' INTO _alarms FROM auth.accounts a WHERE a.email = current_setting('user.email', false);
+                -- Is alarm in the min_notification_interval time frame
+                IF (
+                    ((_alarms->'low_battery_charge_threshold'->>'date') IS NULL) OR
+                    (((_alarms->'low_battery_charge_threshold'->>'date')::TIMESTAMPTZ
+                    + ((interval '1 hour') * (alert_rec.alerting->>'min_notification_interval')::NUMERIC))
+                    < metric_rec.time_bucket::TIMESTAMPTZ)
+                    ) THEN
+                    -- Add alarm
+                    alarms := '{"low_battery_charge_threshold": {"value": '|| (metric_rec.charge*100) ||', "date":"' || metric_rec.time_bucket || '"}}';
+                    -- Merge alarms
+                    SELECT public.jsonb_recursive_merge(_alarms::jsonb, alarms::jsonb) into _alarms;
+                    -- Update alarms for user
+                    PERFORM api.update_user_preferences_fn('{alarms}'::TEXT, _alarms::TEXT);
+                    -- Gather user settings
+                    user_settings := get_user_settings_from_vesselid_fn(current_setting('vessel.id', false));
+                    SELECT user_settings::JSONB || ('{"alert": "low_battery_charge_threshold value:'|| ROUND(metric_rec.charge::NUMERIC*100,2) ||' date:'|| metric_rec.time_bucket ||' "}'::text)::JSONB into user_settings;
+                    -- Send notification
+                    PERFORM send_notification_fn('alert'::TEXT, user_settings::JSONB);
+                    -- DEBUG
+                    RAISE NOTICE '-> cron_alerts_fn checking debug low_battery_charge_threshold +interval';
+                END IF;
+                RAISE NOTICE '-> cron_alerts_fn checking debug low_battery_charge_threshold';
+            END IF;
+            -- Record last metrics time
+            SELECT metric_rec.time_bucket INTO last_metric;
+        END LOOP;
+        PERFORM api.update_user_preferences_fn('{alert_last_metric}'::TEXT, last_metric::TEXT);
+    END LOOP;
+END;
+$function$
+;
+-- Description
+COMMENT ON FUNCTION public.cron_alerts_fn() IS 'init by pg_cron to check for alerts';
+
+-- DROP FUNCTION public.process_pre_logbook_fn(int4);
+-- Update process_pre_logbook_fn to detect and avoid logbook we more than 1000NM in less 15h
+CREATE OR REPLACE FUNCTION public.process_pre_logbook_fn(_id integer)
+ RETURNS void
+ LANGUAGE plpgsql
+AS $function$
+    DECLARE
+        logbook_rec record;
+        avg_rec record;
+        geo_rec record;
+        _invalid_time boolean;
+        _invalid_interval boolean;
+        _invalid_distance boolean;
+        _invalid_ratio boolean;
+        count_metric numeric;
+        previous_stays_id numeric;
+        current_stays_departed text;
+        current_stays_id numeric;
+        current_stays_active boolean;
+        timebucket boolean;
+    BEGIN
+        -- If _id is not NULL
+        IF _id IS NULL OR _id < 1 THEN
+            RAISE WARNING '-> process_pre_logbook_fn invalid input %', _id;
+            RETURN;
+        END IF;
+        -- Get the logbook record with all necessary fields exist
+        SELECT * INTO logbook_rec
+            FROM api.logbook
+            WHERE active IS false
+                AND id = _id
+                AND _from_lng IS NOT NULL
+                AND _from_lat IS NOT NULL
+                AND _to_lng IS NOT NULL
+                AND _to_lat IS NOT NULL;
+        -- Ensure the query is successful
+        IF logbook_rec.vessel_id IS NULL THEN
+            RAISE WARNING '-> process_pre_logbook_fn invalid logbook %', _id;
+            RETURN;
+        END IF;
+
+        PERFORM set_config('vessel.id', logbook_rec.vessel_id, false);
+        --RAISE WARNING 'public.process_logbook_queue_fn() scheduler vessel.id %, user.id', current_setting('vessel.id', false), current_setting('user.id', false);
+
+        -- Check if all metrics are within 50meters base on geo loc
+        count_metric := logbook_metrics_dwithin_fn(logbook_rec._from_time::TEXT, logbook_rec._to_time::TEXT, logbook_rec._from_lng::NUMERIC, logbook_rec._from_lat::NUMERIC);
+        RAISE NOTICE '-> process_pre_logbook_fn logbook_metrics_dwithin_fn count:[%]', count_metric;
+
+        -- Calculate logbook data average and geo
+        -- Update logbook entry with the latest metric data and calculate data
+        avg_rec := logbook_update_avg_fn(logbook_rec.id, logbook_rec._from_time::TEXT, logbook_rec._to_time::TEXT);
+        geo_rec := logbook_update_geom_distance_fn(logbook_rec.id, logbook_rec._from_time::TEXT, logbook_rec._to_time::TEXT);
+
+        -- Avoid/ignore/delete logbook stationary movement or time sync issue
+        -- Check time start vs end
+        SELECT logbook_rec._to_time::TIMESTAMPTZ < logbook_rec._from_time::TIMESTAMPTZ INTO _invalid_time;
+        -- Is distance is less than 0.010
+        SELECT geo_rec._track_distance < 0.010 INTO _invalid_distance;
+        -- Is duration is less than 100sec
+        SELECT (logbook_rec._to_time::TIMESTAMPTZ - logbook_rec._from_time::TIMESTAMPTZ) < (100::text||' secs')::interval INTO _invalid_interval;
+        -- If we have more than 800NM in less 15h
+        IF geo_rec._track_distance >= 800 AND (logbook_rec._to_time::TIMESTAMPTZ - logbook_rec._from_time::TIMESTAMPTZ) < (15::text||' hours')::interval THEN
+            _invalid_distance := True;
+            _invalid_interval := True;
+            --RAISE NOTICE '-> process_pre_logbook_fn invalid logbook data id [%], _invalid_distance [%], _invalid_interval [%]', logbook_rec.id, _invalid_distance, _invalid_interval;
+        END IF;
+        -- If we have less than 20 metrics or less than 0.5NM or less than avg 0.5knts
+        -- Is within metrics represent more or equal than 60% of the total entry
+        IF count_metric::NUMERIC <= 20 OR geo_rec._track_distance < 0.5 OR avg_rec.avg_speed < 0.5 THEN
+            SELECT (count_metric::NUMERIC / avg_rec.count_metric::NUMERIC) >= 0.60 INTO _invalid_ratio;
+        END IF;
+        -- if stationary fix data metrics,logbook,stays,moorage
+        IF _invalid_time IS True OR _invalid_distance IS True
+            OR _invalid_interval IS True OR count_metric = avg_rec.count_metric
+            OR _invalid_ratio IS True
+            OR avg_rec.count_metric <= 3 THEN
+            RAISE NOTICE '-> process_pre_logbook_fn invalid logbook data id [%], _invalid_time [%], _invalid_distance [%], _invalid_interval [%], count_metric_in_zone [%], count_metric_log [%], _invalid_ratio [%]',
+                logbook_rec.id, _invalid_time, _invalid_distance, _invalid_interval, count_metric, avg_rec.count_metric, _invalid_ratio;
+            -- Update metrics status to moored
+            UPDATE api.metrics
+                SET status = 'moored'
+                WHERE time >= logbook_rec._from_time::TIMESTAMPTZ
+                    AND time <= logbook_rec._to_time::TIMESTAMPTZ
+                    AND vessel_id = current_setting('vessel.id', false);
+            -- Update logbook
+            UPDATE api.logbook
+                SET notes = 'invalid logbook data, stationary need to fix metrics?'
+                WHERE vessel_id = current_setting('vessel.id', false)
+                    AND id = logbook_rec.id;
+            -- Get related stays
+            SELECT id,departed,active INTO current_stays_id,current_stays_departed,current_stays_active
+                FROM api.stays s
+                WHERE s.vessel_id = current_setting('vessel.id', false)
+                    AND s.arrived = logbook_rec._to_time::TIMESTAMPTZ;
+            -- Update related stays
+            UPDATE api.stays s
+                SET notes = 'invalid stays data, stationary need to fix metrics?'
+                WHERE vessel_id = current_setting('vessel.id', false)
+                    AND arrived = logbook_rec._to_time::TIMESTAMPTZ;
+            -- Find previous stays
+            SELECT id INTO previous_stays_id
+				FROM api.stays s
+                WHERE s.vessel_id = current_setting('vessel.id', false)
+                    AND s.arrived < logbook_rec._to_time::TIMESTAMPTZ
+                    ORDER BY s.arrived DESC LIMIT 1;
+            -- Update previous stays with the departed time from current stays
+            --  and set the active state from current stays
+            UPDATE api.stays
+                SET departed = current_stays_departed::TIMESTAMPTZ,
+                    active = current_stays_active
+                WHERE vessel_id = current_setting('vessel.id', false)
+                    AND id = previous_stays_id;
+            -- Clean up, remove invalid logbook and stay entry
+            DELETE FROM api.logbook WHERE id = logbook_rec.id;
+            RAISE WARNING '-> process_pre_logbook_fn delete invalid logbook [%]', logbook_rec.id;
+            DELETE FROM api.stays WHERE id = current_stays_id;
+            RAISE WARNING '-> process_pre_logbook_fn delete invalid stays [%]', current_stays_id;
+            RETURN;
+        END IF;
+
+        --IF (logbook_rec.notes IS NULL) THEN -- run one time only
+        --    -- If duration is over 24h or number of entry is over 400, check for stays and potential multiple logs with stationary location
+        --    IF (logbook_rec._to_time::TIMESTAMPTZ - logbook_rec._from_time::TIMESTAMPTZ) > INTERVAL '24 hours'
+        --        OR avg_rec.count_metric > 400 THEN
+        --        timebucket := public.logbook_metrics_timebucket_fn('15 minutes'::TEXT, logbook_rec.id, logbook_rec._from_time::TIMESTAMPTZ, logbook_rec._to_time::TIMESTAMPTZ);
+        --        -- If true exit current process as the current logbook need to be re-process.
+        --        IF timebucket IS True THEN
+        --            RETURN;
+        --        END IF;
+        --    ELSE
+        --        timebucket := public.logbook_metrics_timebucket_fn('5 minutes'::TEXT, logbook_rec.id, logbook_rec._from_time::TIMESTAMPTZ, logbook_rec._to_time::TIMESTAMPTZ);
+        --        -- If true exit current process as the current logbook need to be re-process.
+        --        IF timebucket IS True THEN
+        --            RETURN;
+        --        END IF;
+        --    END IF;
+        --END IF;
+
+        -- Add logbook entry to process queue for later processing
+        INSERT INTO process_queue (channel, payload, stored, ref_id)
+            VALUES ('new_logbook', logbook_rec.id, NOW(), current_setting('vessel.id', true));
+
+    END;
+$function$
+;
+
+COMMENT ON FUNCTION public.process_pre_logbook_fn(int4) IS 'Detect/Avoid/ignore/delete logbook stationary movement or time sync issue';
+
+-- Revoke security definer
+--ALTER FUNCTION api.update_logbook_observations_fn(_id integer, observations text) SECURITY INVOKER;
+--ALTER FUNCTION api.delete_logbook_fn(_id integer) SECURITY INVOKER;
+ALTER FUNCTION api.merge_logbook_fn(_id integer, _id integer) SECURITY INVOKER;
+
+GRANT DELETE ON TABLE public.process_queue TO user_role;
+GRANT SELECT ON ALL TABLES IN SCHEMA api TO user_role;
+GRANT EXECUTE ON ALL FUNCTIONS IN SCHEMA api TO user_role;
+GRANT EXECUTE ON ALL FUNCTIONS IN SCHEMA public TO user_role;
+
+GRANT UPDATE (status) ON api.metrics TO user_role;
+GRANT UPDATE ON api.logbook TO user_role;
+
+DROP POLICY IF EXISTS api_user_role ON api.metrics;
+CREATE POLICY api_user_role ON api.metrics TO user_role
+    USING (vessel_id = current_setting('vessel.id', false))
+    WITH CHECK (vessel_id = current_setting('vessel.id', false));
+
+-- Update version
+UPDATE public.app_settings
+	SET value='0.9.3'
+	WHERE "name"='app.version';
+
+--\c postgres
+--UPDATE cron.job SET username = 'scheduler'; --  Update to scheduler
+--UPDATE cron.job SET username = current_user WHERE jobname = 'cron_vacuum'; -- Update to superuser for vacuum permissions
+--UPDATE cron.job SET username = current_user WHERE jobname = 'job_run_details_cleanup';

initdb/PGSAIL_VERSION

@@ -1 +1 @@
-0.7.8
+0.9.3

tests/README.md

@@ -1,5 +1,5 @@
 # PostgSail Unit Tests
-The Unit Tests allow to automatically validate api workflow.
+The Unit Tests allow to automatically validate SQL and API workflow.
 
 ## A global overview
 Based on `mocha` & `psql`

tests/index.js

@@ -27,6 +27,7 @@ const metrics_aava = require('./metrics_sample_aava.json');
 
 const fs = require('fs');
 
+let configtime = new Date().toISOString();
 
 // CNAMEs Array
 [
@@ -39,7 +40,7 @@ const fs = require('fs');
     vessel_metadata: {
             name: "kapla",
             mmsi: "123456789",
-            client_id: "vessels.urn:mrn:signalk:uuid:5b4f7543-7153-4840-b139-761310b242fd",
+            //client_id: "vessels.urn:mrn:signalk:uuid:5b4f7543-7153-4840-b139-761310b242fd",
             length: "12",
             beam: "10",
             height: "24",
@@ -59,8 +60,8 @@ const fs = require('fs');
     user_views: [
         // not processed yet, { url: '/stays_view', res_body_length: 1},
         // not processed yet, { url: '/moorages_view', res_body_length: 1},
-        { url: '/logs_view', res_body_length: 0},
-        { url: '/log_view', res_body_length: 2},
+        { url: '/logs_view', res_body_length: 0}, // not processed yet so empty
+        { url: '/log_view', res_body_length: 0}, // not processed yet so empty
         //{ url: '/stats_view', res_body_length: 1},
         { url: '/vessels_view', res_body_length: 1},
     ],
@@ -89,7 +90,7 @@ const fs = require('fs');
       */
     ],
     user_fn: [
-      { url: '/rpc/timelapse_fn',
+      { url: '/rpc/export_logbooks_geojson_point_trips_fn',
         payload: {
               start_log: 1
             },
@@ -97,7 +98,7 @@ const fs = require('fs');
           obj_name: 'geojson'
         }
       },
-      { url: '/rpc/export_logbook_geojson_fn',
+      { url: '/rpc/export_logbook_geojson_trip_fn',
         payload: {
               _id: 1
             },
@@ -105,7 +106,15 @@ const fs = require('fs');
           obj_name: 'geojson'
         }
       },
-      { url: '/rpc/export_logbook_gpx_fn',
+      { url: '/rpc/export_logbook_gpx_trip_fn',
+        payload: {
+              _id: 1
+            },
+        res: {
+          obj_name: null
+        }
+      },
+      { url: '/rpc/export_logbook_kml_trip_fn',
         payload: {
               _id: 1
             },
@@ -169,6 +178,18 @@ const fs = require('fs');
             obj_name: 'settings'
           }
       }
+    ],
+    config_fn: [
+      { url: '/metadata?select=configuration',
+        res: {
+          obj_name: 'configuration'
+        }
+      },
+      { url: `/metadata?select=configuration&configuration->>update_at=gt.${configtime}`,
+        res: {
+          obj_name: 'settings'
+        }
+      },
     ]
   },
   { cname: process.env.PGSAIL_API_URI, name: "PostgSail unit test, aava",
@@ -178,8 +199,8 @@ const fs = require('fs');
     preferences: { key: '{email_notifications}', value: false }, /* Disable email_notifications */
     vessel_metadata: {
             name: "aava",
-            mmsi: "787654321",
-            client_id: "vessels.urn:mrn:imo:mmsi:787654321",
+            mmsi: "n/a",
+            //client_id: "vessels.urn:mrn:imo:mmsi:787654321",
             length: "12",
             beam: "10",
             height: "24",
@@ -198,8 +219,8 @@ const fs = require('fs');
     user_views: [
         // not processed yet, { url: '/stays_view', res_body_length: 1},
         // not processed yet, { url: '/moorages_view', res_body_length: 1},
-        { url: '/logs_view', res_body_length: 0},
-        { url: '/log_view', res_body_length: 1},
+        { url: '/logs_view', res_body_length: 0}, // not processed yet so empty
+        { url: '/log_view', res_body_length: 0}, // not processed yet so empty
         //{ url: '/stats_view', res_body_length: 1},
         { url: '/vessels_view', res_body_length: 1},
     ],
@@ -228,15 +249,15 @@ const fs = require('fs');
       */
     ],
     user_fn: [
-      { url: '/rpc/timelapse_fn',
+      { url: '/rpc/export_logbooks_geojson_point_trips_fn',
         payload: {
-              start_log: 3
+              start_log: 1
             },
         res: {
           obj_name: 'geojson'
         }
       },
-      { url: '/rpc/export_logbook_geojson_fn',
+      { url: '/rpc/export_logbook_geojson_trip_fn',
         payload: {
               _id: 3
             },
@@ -244,7 +265,15 @@ const fs = require('fs');
           obj_name: 'geojson'
         }
       },
-      { url: '/rpc/export_logbook_gpx_fn',
+      { url: '/rpc/export_logbook_gpx_trip_fn',
+        payload: {
+              _id: 3
+            },
+        res: {
+          obj_name: null
+        }
+      },
+      { url: '/rpc/export_logbook_kml_trip_fn',
         payload: {
               _id: 3
             },
@@ -302,6 +331,30 @@ const fs = require('fs');
           obj_name: 'settings'
         }
       },
+    ],
+    config_fn: [
+      { url: '/metadata?select=configuration',
+        res: {
+          obj_name: 'configuration'
+        }
+      },
+      { url: `/metadata?select=configuration&configuration->>update_at=gt.${configtime}`,
+        res: {
+          obj_name: 'settings'
+        }
+      },
+    ],
+    meta_ext_fn: [
+      { url: '/metadata_ext?',
+        res: {
+          obj_name: 'configuration'
+        }
+      },
+      { url: `/metadata_ext?`,
+        res: {
+          obj_name: 'image'
+        }
+      },
     ]
   }
 ].forEach( function(test){
@@ -580,7 +633,7 @@ request.set('User-Agent', 'PostgSail unit tests');
         .set('Authorization', `Bearer ${vessel_jwt}`)
         .set('Accept', 'application/json')
         .set('Content-Type', 'application/json')
-        .set('Prefer', 'return=headers-only,resolution=merge-duplicates')
+        .set('Prefer', 'missing=default,return=headers-only,resolution=merge-duplicates')
         .end(function(err,res){
           res.status.should.equal(201);
           //console.log(res.header);
@@ -595,14 +648,15 @@ request.set('User-Agent', 'PostgSail unit tests');
   describe("Vessel POST metrics, JWT vessel_role", function(){
 
     let data = [];
-    //console.log(vessel_metrics['metrics'][0]);
+    //console.log(test.vessel_metrics['metrics'][0]);
     let i;
     for (i = 0; i < test.vessel_metrics['metrics'].length; i++) {
       data[i] = test.vessel_metrics['metrics'][i];
       // Override time, -2h to allow to new data later without delay.
-      data[i]['time'] = moment.utc().subtract(2, 'hours').add(i, 'minutes').format();
+      data[i]['time'] = moment.utc().subtract(1, 'day').add(i, 'minutes').format();
       // Override client_id
-      data[i]['client_id'] = test.vessel_metadata.client_id;
+      //data[i]['client_id'] = test.vessel_metadata.client_id;
+      data[i]['client_id'] = null;
     }
     // The last entry are invalid and should be ignore.
     // - Invalid status
@@ -611,6 +665,11 @@ request.set('User-Agent', 'PostgSail unit tests');
     // Force last valid entry to be back in time from previous, it should be ignore silently
     data.at(-1).time = moment.utc(data.at(-3).time).subtract(1, 'minutes').format();
     //console.log(data[0]);
+    // Force the -2 entry to be in the future add 1 year, it should be ignore silently
+    data.splice(i-2, 1, data.at(-2))
+    data.at(-3).time = moment.utc(data.at(-3).time).add(1, 'year').format();
+    //console.log(data.at(-2));
+    //console.log(data.at(-1));
 
     it('/metrics?select=time', function(done) {
       request = supertest.agent(test.cname);
@@ -629,6 +688,7 @@ request.set('User-Agent', 'PostgSail unit tests');
           res.header['content-type'].should.match(new RegExp('json','g'));
           res.header['server'].should.match(new RegExp('postgrest','g'));
           should.exist(res.body);
+          //console.log(res.body);
           res.body.length.should.match(test.vessel_metrics['metrics'].length-3);
           done(err);
         });
@@ -821,6 +881,37 @@ request.set('User-Agent', 'PostgSail unit tests');
   }); // Function OTP endpoint
   */
 
+  describe("Function Metadata configuration endpoint, JWT vessel_role", function(){
+
+    let otp = null;
+    test.config_fn.forEach(function (subtest) {
+      it(`${subtest.url}`, function(done) {
+        try {
+          //console.log(`${subtest.url} ${subtest.res}`);
+          // Reset agent so we do not save cookies
+          request = supertest.agent(test.cname);
+          request
+            .get(subtest.url)
+            .set('Authorization', `Bearer ${vessel_jwt}`)
+            .set('Accept', 'application/json')
+            .end(function(err,res){
+              res.status.should.equal(200);
+              should.exist(res.header['content-type']);
+              should.exist(res.header['server']);
+              res.header['content-type'].should.match(new RegExp('json','g'));
+              res.header['server'].should.match(new RegExp('postgrest','g'));
+              console.log(res.body);
+              should.exist(res.body);
+              done(err);
+            });
+        }
+        catch (error) {
+          done();
+        }
+      });
+    });
+  }); // Function metadata configuration endpoint
+
 }); // OpenAPI description
 
 }); // CNAMEs Array

tests/index2.js

@@ -6,7 +6,7 @@
 *
 * npm install supertest should mocha mochawesome moment
 * alias mocha="./node_modules/mocha/bin/_mocha"
-* mocha index.js --reporter mochawesome --reporter-options reportDir=/mnt/postgsail/,reportFilename=report_api.html
+* mocha index2.js --reporter mochawesome --reporter-options reportDir=/mnt/postgsail/,reportFilename=report_api.html
 *
 */
 
@@ -28,14 +28,15 @@ const metrics_simulator = require('./metrics_sample_simulator.json');
     vessel_metadata: {
             name: "aava",
             mmsi: "787654321",
-            client_id: "vessels.urn:mrn:imo:mmsi:787654321",
+            //client_id: "vessels.urn:mrn:imo:mmsi:787654321",
             length: "12",
             beam: "10",
             height: "24",
             ship_type: "37",
             plugin_version: "1.0.2",
             signalk_version: "1.20.0",
-            time: moment().subtract(69, 'minutes').format()
+            time: moment().subtract(69, 'minutes').format(),
+            available_keys: [],
           },
     vessel_metrics: metrics_simulator,
     user_tables: [
@@ -48,7 +49,7 @@ const metrics_simulator = require('./metrics_sample_simulator.json');
         // not processed yet, { url: '/stays_view', res_body_length: 1},
         // not processed yet, { url: '/moorages_view', res_body_length: 1},
         { url: '/logs_view', res_body_length: 1},
-        { url: '/log_view', res_body_length: 2},
+        { url: '/log_view', res_body_length: 0},  // not processed yet so empty
         //{ url: '/stats_view', res_body_length: 1},
         { url: '/vessels_view', res_body_length: 1},
     ],
@@ -77,7 +78,7 @@ const metrics_simulator = require('./metrics_sample_simulator.json');
       */
     ],
     user_fn: [
-      { url: '/rpc/timelapse_fn',
+      { url: '/rpc/export_logbooks_geojson_point_trips_fn',
         payload: {
               start_log: 4
             },
@@ -85,7 +86,7 @@ const metrics_simulator = require('./metrics_sample_simulator.json');
           obj_name: 'geojson'
         }
       },
-      { url: '/rpc/export_logbook_geojson_fn',
+      { url: '/rpc/export_logbook_geojson_trip_fn',
         payload: {
               _id: 4
             },
@@ -93,7 +94,15 @@ const metrics_simulator = require('./metrics_sample_simulator.json');
           obj_name: 'geojson'
         }
       },
-      { url: '/rpc/export_logbook_gpx_fn',
+      { url: '/rpc/export_logbook_gpx_trip_fn',
+        payload: {
+              _id: 4
+            },
+        res: {
+          obj_name: null
+        }
+      },
+      { url: '/rpc/export_logbook_kml_trip_fn',
         payload: {
               _id: 4
             },
@@ -404,18 +413,19 @@ request.set('User-Agent', 'PostgSail unit tests');
 
   describe("Vessel POST metadata, JWT vessel_role", function(){
 
-    it('/metadata', function(done) {
+    it('/metadata?on_conflict=vessel_id', function(done) {
       request = supertest.agent(test.cname);
       request
-        .post('/metadata')
+        .post('/metadata?on_conflict=vessel_id')
         .send(test.vessel_metadata)
         .set('Authorization', `Bearer ${vessel_jwt}`)
         .set('Accept', 'application/json')
         .set('Content-Type', 'application/json')
-        .set('Prefer', 'return=headers-only')
+        .set('Prefer', 'missing=default,return=headers-only,resolution=merge-duplicates')
         .end(function(err,res){
-          res.status.should.equal(201);
+          //console.log(res.body);
           //console.log(res.header);
+          res.status.should.equal(200);
           should.exist(res.header['server']);
           res.header['server'].should.match(new RegExp('postgrest','g'));
           done(err);
@@ -432,11 +442,12 @@ request.set('User-Agent', 'PostgSail unit tests');
     for (i = 0; i < test.vessel_metrics['metrics'].length; i++) {
       data[i] = test.vessel_metrics['metrics'][i];
       // Override time, +1h because previous sample include 47 entry.
-      data[i]['time'] = moment().add(1, 'hour').add(i, 'minutes').format();
+      data[i]['time'] = moment.utc().subtract(2, 'hours').add(i, 'minutes').format();
       // Override client_id
-      data[i]['client_id'] = test.vessel_metadata.client_id;
+      //data[i]['client_id'] = test.vessel_metadata.client_id;
+      data[i]['client_id'] = null;
     }
-    console.log(data[0]);
+    //console.log(data[0]);
 
     it('/metrics?select=time', function(done) {
       request = supertest.agent(test.cname);

tests/index3.js

@@ -6,7 +6,7 @@
 *
 * npm install supertest should mocha mochawesome moment
 * alias mocha="./node_modules/mocha/bin/_mocha"
-* mocha index.js --reporter mochawesome --reporter-options reportDir=/mnt/postgsail/,reportFilename=report_api.html
+* mocha index3.js --reporter mochawesome --reporter-options reportDir=/mnt/postgsail/,reportFilename=report_api.html
 *
 */
 
@@ -31,7 +31,7 @@ var moment = require('moment');
     vessel_metadata: {
             name: "kapla",
             mmsi: "123456789",
-            client_id: "vessels.urn:mrn:imo:mmsi:123456789",
+            //client_id: "vessels.urn:mrn:imo:mmsi:123456789",
             length: "12",
             beam: "10",
             height: "24",
@@ -79,7 +79,8 @@ var moment = require('moment');
       }
     ],
     user_fn: [
-      { url: '/rpc/timelapse_fn',
+      { //url: '/rpc/timelapse_fn',
+        url: '/rpc/export_logbooks_geojson_linestring_trips_fn',
         payload: {
               start_log: 2
             },
@@ -87,7 +88,17 @@ var moment = require('moment');
           obj_name: 'geojson'
         }
       },
-      { url: '/rpc/export_logbook_geojson_fn',
+      { //url: '/rpc/timelapse_fn',
+        url: '/rpc/export_logbooks_geojson_point_trips_fn',
+        payload: {
+              start_log: 2
+            },
+        res: {
+          obj_name: 'geojson'
+        }
+      },
+      { //url: '/rpc/export_logbook_geojson_fn',
+        url: '/rpc/export_logbook_geojson_trip_fn',
         payload: {
               _id: 2
             },
@@ -95,7 +106,8 @@ var moment = require('moment');
           obj_name: 'geojson'
         }
       },
-      { url: '/rpc/export_logbook_gpx_fn',
+      { //url: '/rpc/export_logbook_gpx_fn',
+        url: '/rpc/export_logbook_kml_trip_fn',
         payload: {
               _id: 2
             },
@@ -103,7 +115,8 @@ var moment = require('moment');
           obj_name: null
         }
       },
-      { url: '/rpc/export_logbook_kml_fn',
+      { //url: '/rpc/export_logbook_kml_fn',
+        url: '/rpc/export_logbook_kml_trip_fn',
         payload: {
               _id: 2
             },
@@ -123,6 +136,12 @@ var moment = require('moment');
           obj_name: null
         }
       },
+      { url: '/rpc/export_moorages_kml_fn',
+        payload: {},
+        res: {
+          obj_name: null
+        }
+      },
       { url: '/rpc/find_log_from_moorage_fn',
         payload: {
           _id: 2
@@ -230,7 +249,7 @@ var moment = require('moment');
     vessel_metadata: {
             name: "aava",
             mmsi: "787654321",
-            client_id: "vessels.urn:mrn:imo:mmsi:787654321",
+            //client_id: "vessels.urn:mrn:imo:mmsi:787654321",
             length: "12",
             beam: "10",
             height: "24",
@@ -277,7 +296,8 @@ var moment = require('moment');
       }
     ],
     user_fn: [
-      { url: '/rpc/timelapse_fn',
+      { //url: '/rpc/timelapse_fn',
+        url: '/rpc/export_logbooks_geojson_linestring_trips_fn',
         payload: {
               start_log: 4
             },
@@ -285,7 +305,17 @@ var moment = require('moment');
           obj_name: 'geojson'
         }
       },
-      { url: '/rpc/export_logbook_geojson_fn',
+      { //url: '/rpc/timelapse_fn',
+        url: '/rpc/export_logbooks_geojson_point_trips_fn',
+        payload: {
+              start_log: 4
+            },
+        res: {
+          obj_name: 'geojson'
+        }
+      },
+      { //url: '/rpc/export_logbook_geojson_fn',
+        url: '/rpc/export_logbook_geojson_trip_fn',
         payload: {
               _id: 4
             },
@@ -293,7 +323,8 @@ var moment = require('moment');
           obj_name: 'geojson'
         }
       },
-      { url: '/rpc/export_logbook_gpx_fn',
+      { //url: '/rpc/export_logbook_gpx_fn',
+        url: '/rpc/export_logbook_gpx_trip_fn',
         payload: {
               _id: 4
             },
@@ -301,7 +332,8 @@ var moment = require('moment');
           obj_name: null
         }
       },
-      { url: '/rpc/export_logbook_kml_fn',
+      { //url: '/rpc/export_logbook_kml_fn',
+        url: '/rpc/export_logbook_kml_trip_fn',
         payload: {
               _id: 4
             },
@@ -309,7 +341,8 @@ var moment = require('moment');
           obj_name: null
         }
       },
-      { url: '/rpc/export_logbooks_gpx_fn',
+      { //url: '/rpc/export_logbooks_gpx_fn',
+        url: '/rpc/export_logbooks_kml_trips_fn',
         payload: {
             start_log: 3,
             end_log: 4
@@ -318,7 +351,8 @@ var moment = require('moment');
           obj_name: null
         }
       },
-      { url: '/rpc/export_logbooks_kml_fn',
+      { //url: '/rpc/export_logbooks_kml_fn',
+        url: '/rpc/export_logbooks_kml_trips_fn',
         payload: {
             start_log: 3,
             end_log: 4
@@ -339,6 +373,12 @@ var moment = require('moment');
           obj_name: null
         }
       },
+      { url: '/rpc/export_moorages_kml_fn',
+        payload: {},
+        res: {
+          obj_name: null
+        }
+      },
       { url: '/rpc/find_log_from_moorage_fn',
         payload: {
           _id: 4

tests/index4.js

@@ -163,6 +163,10 @@ var moment = require("moment");
         url: "/rpc/update_user_preferences_fn",
         payload: { key: "{public_monitoring}", value: true },
       },
+      {
+        url: "/rpc/update_user_preferences_fn",
+        payload: { key: "{public_timelapse}", value: true },
+      },
     ],
   },
   {
@@ -684,8 +688,8 @@ var moment = require("moment");
             should.exist(res.body);
             let event = res.body;
             //console.log(event);
-            // minimum events log for kapla & aava 13 + 4 email_otp = 17
-            event.length.should.be.aboveOrEqual(13);
+            // minimum events log per users 6 + 4 logs + OTP one per login
+            event.length.should.be.aboveOrEqual(11);
             done(err);
           });
       });

tests/index5.js

@@ -45,13 +45,22 @@ var moment = require("moment");
       res: {},
     },
     timelapse: {
-      url: "/rpc/timelapse_fn",
+      //url: "/rpc/timelapse_fn",
+      url: '/rpc/export_logbooks_geojson_linestring_trips_fn',
       header: { name: "x-is-public", value: btoa("kapla,public_timelapse,1") },
       payload: null,
       res: {},
     },
     timelapse_full: {
-      url: "/rpc/timelapse_fn",
+      //url: "/rpc/timelapse_fn",
+      url: '/rpc/export_logbooks_geojson_linestring_trips_fn',
+      header: { name: "x-is-public", value: btoa("kapla,public_timelapse,0") },
+      payload: null,
+      res: {},
+    },
+    replay_full: {
+      //url: "/rpc/timelapse_fn",
+      url: '/rpc/export_logbooks_geojson_point_trips_fn',
       header: { name: "x-is-public", value: btoa("kapla,public_timelapse,0") },
       payload: null,
       res: {},
@@ -69,7 +78,7 @@ var moment = require("moment");
       res: {},
     },
     export_gpx: {
-      url: "/rpc/export_logbook_gpx_fn",
+      url: "/rpc/export_logbook_gpx_trip_fn",
       header: { name: "x-is-public", value: btoa("kapla,public_logs,0") },
       payload: null,
       res: {},
@@ -97,13 +106,21 @@ var moment = require("moment");
       res: {},
     },
     timelapse: {
-      url: "/rpc/timelapse_fn",
+      //url: "/rpc/timelapse_fn",
+      url: '/rpc/export_logbooks_geojson_linestring_trips_fn',
       header: { name: "x-is-public", value: btoa("aava,public_timelapse,3") },
       payload: null,
       res: {},
     },
     timelapse_full: {
-      url: "/rpc/timelapse_fn",
+      //url: "/rpc/timelapse_fn",
+      url: '/rpc/export_logbooks_geojson_linestring_trips_fn',
+      header: { name: "x-is-public", value: btoa("aava,public_timelapse,0") },
+      payload: null,
+      res: {},
+    },
+    replay_full: {
+      url: '/rpc/export_logbooks_geojson_point_trips_fn',
       header: { name: "x-is-public", value: btoa("aava,public_timelapse,0") },
       payload: null,
       res: {},
@@ -121,7 +138,7 @@ var moment = require("moment");
       res: {},
     },
     export_gpx: {
-      url: "/rpc/export_logbook_gpx_fn",
+      url: "/rpc/export_logbook_gpx_trip_fn",
       header: { name: "x-is-public", value: btoa("aava,public_logs,0") },
       payload: null,
       res: {},
@@ -134,7 +151,7 @@ var moment = require("moment");
     request.set("User-Agent", "PostgSail unit tests");
 
     describe("With no JWT as api_anonymous", function () {
-      it("/logs_view, api_anonymous no jwt token", function (done) {
+      it("/logs_view, api_anonymous no jwt token, x-is-public header", function (done) {
         // Reset agent so we do not save cookies
         request = supertest.agent(test.cname);
         request
@@ -150,7 +167,7 @@ var moment = require("moment");
             done(err);
           });
       });
-      it("/log_view, api_anonymous no jwt token", function (done) {
+      it("/log_view, api_anonymous no jwt token, x-is-public header", function (done) {
         // Reset agent so we do not save cookies
         request = supertest.agent(test.cname);
         request
@@ -166,7 +183,7 @@ var moment = require("moment");
             done(err);
           });
       });
-      it("/monitoring_view, api_anonymous no jwt token", function (done) {
+      it("/monitoring_view, api_anonymous no jwt token, x-is-public header", function (done) {
         // Reset agent so we do not save cookies
         request = supertest.agent(test.cname);
         request
@@ -183,7 +200,7 @@ var moment = require("moment");
             done(err);
           });
       });
-      it("/rpc/timelapse_fn, api_anonymous no jwt token", function (done) {
+      it("/rpc/export_logbooks_geojson_linestring_trips_fn, api_anonymous no jwt token, x-is-public header", function (done) {
         // Reset agent so we do not save cookies
         request = supertest.agent(test.cname);
         request
@@ -197,6 +214,39 @@ var moment = require("moment");
             should.exist(res.header["server"]);
             res.header["content-type"].should.match(new RegExp("json", "g"));
             res.header["server"].should.match(new RegExp("postgrest", "g"));
+            should.exist(res.body.geojson);
+            /*
+            if (res.body.geojson.features == null) { // aava
+              //res.body.geojson.features.should.not.be.ok();
+              done(err);
+            }
+            res.body.geojson.features.length.should.be.equal(4);
+            */
+            done(err);
+          });
+      });
+      it("/rpc/export_logbooks_geojson_point_trips_fn, api_anonymous no jwt token, x-is-public header", function (done) {
+        // Reset agent so we do not save cookies
+        request = supertest.agent(test.cname);
+        request
+          .post(test.replay_full.url)
+          .set(test.replay_full.header.name, test.replay_full.header.value)
+          .set("Accept", "application/json")
+          .end(function (err, res) {
+            console.log(res.text);
+            res.status.should.equal(200); // return 404 as it is not enable in user settings.
+            should.exist(res.header["content-type"]);
+            should.exist(res.header["server"]);
+            res.header["content-type"].should.match(new RegExp("json", "g"));
+            res.header["server"].should.match(new RegExp("postgrest", "g"));
+            should.exist(res.body.geojson);
+            /*
+            if (res.body.geojson.features == null) { // aava
+              //res.body.geojson.features.should.not.be.ok();
+              done(err);
+            }
+            res.body.geojson.features.length.should.be.equal(53);
+            */
             done(err);
           });
       });

tests/index6.js

@@ -1,13 +1,13 @@
 "use strict";
 /*
- * Unit test #5
+ * Unit test #6
  * Public/Anonymous access
  *
  * process.env.PGSAIL_API_URI = from inside the docker
  *
  * npm install supertest should mocha mochawesome moment
  * alias mocha="./node_modules/mocha/bin/_mocha"
- * mocha index5.js --reporter mochawesome --reporter-options reportDir=/mnt/postgsail/,reportFilename=report_api.html
+ * mocha index6.js --reporter mochawesome --reporter-options reportDir=/mnt/postgsail/,reportFilename=report_api.html
  *
  */
 
@@ -52,12 +52,17 @@ var moment = require("moment");
       res: {},
     },
     timelapse: {
-      url: "/rpc/timelapse_fn",
+      url: "/rpc/export_logbooks_geojson_linestring_trips_fn",
       payload: null,
       res: {},
     },
     timelapse_full: {
-      url: "/rpc/timelapse_fn",
+      url: "/rpc/export_logbooks_geojson_linestring_trips_fn",
+      payload: null,
+      res: {},
+    },
+    replay_full: {
+      url: "/rpc/export_logbooks_geojson_point_trips_fn",
       payload: null,
       res: {},
     },
@@ -72,7 +77,7 @@ var moment = require("moment");
       res: {},
     },
     export_gpx: {
-      url: "/rpc/export_logbook_gpx_fn",
+      url: "/rpc/export_logbook_gpx_trip_fn",
       payload: null,
       res: {},
     },
@@ -165,7 +170,7 @@ var moment = require("moment");
             done(err);
           });
       });
-      it("/rpc/timelapse_fn, api_anonymous no jwt token", function (done) {
+      it("/rpc/export_logbooks_geojson_linestring_trips_fn, api_anonymous no jwt token", function (done) {
         // Reset agent so we do not save cookies
         request = supertest.agent(test.cname);
         request
@@ -178,10 +183,28 @@ var moment = require("moment");
             should.exist(res.header["server"]);
             res.header["content-type"].should.match(new RegExp("json", "g"));
             res.header["server"].should.match(new RegExp("postgrest", "g"));
+            should.exist(res.body.geojson);
             done(err);
           });
       });
-      it("/rpc/export_logbook_gpx_fn, api_anonymous no jwt token", function (done) {
+      it("/rpc/export_logbooks_geojson_point_trips_fn, api_anonymous no jwt token", function (done) {
+        // Reset agent so we do not save cookies
+        request = supertest.agent(test.cname);
+        request
+          .post(test.replay_full.url)
+          .set("Accept", "application/json")
+          .end(function (err, res) {
+            console.log(res.body);
+            res.status.should.equal(200);
+            should.exist(res.header["content-type"]);
+            should.exist(res.header["server"]);
+            res.header["content-type"].should.match(new RegExp("json", "g"));
+            res.header["server"].should.match(new RegExp("postgrest", "g"));
+            should.exist(res.body.geojson);
+            done(err);
+          });
+      });
+      it("/rpc/export_logbook_gpx_trip_fn, api_anonymous no jwt token", function (done) {
         // Reset agent so we do not save cookies
         request = supertest.agent(test.cname);
         request

tests/sql/anonymous.sql

@@ -18,3 +18,10 @@ SELECT api.ispublic_fn('kapla', 'public_logs', 1);
 SELECT api.ispublic_fn('kapla', 'public_logs', 3);
 SELECT api.ispublic_fn('kapla', 'public_monitoring');
 SELECT api.ispublic_fn('kapla', 'public_timelapse');
+
+SELECT api.ispublic_fn('aava', 'public_test');
+SELECT api.ispublic_fn('aava', 'public_logs_list');
+SELECT api.ispublic_fn('aava', 'public_logs', 1);
+SELECT api.ispublic_fn('aava', 'public_logs', 3);
+SELECT api.ispublic_fn('aava', 'public_monitoring');
+SELECT api.ispublic_fn('aava', 'public_timelapse');

tests/sql/anonymous.sql.output

@@ -21,6 +21,24 @@ ispublic_fn | f
 -[ RECORD 1 ]--
 ispublic_fn | t
 
+-[ RECORD 1 ]--
+ispublic_fn | t
+
+-[ RECORD 1 ]--
+ispublic_fn | f
+
+-[ RECORD 1 ]--
+ispublic_fn | f
+
+-[ RECORD 1 ]--
+ispublic_fn | f
+
+-[ RECORD 1 ]--
+ispublic_fn | t
+
+-[ RECORD 1 ]--
+ispublic_fn | t
+
 -[ RECORD 1 ]--
 ispublic_fn | f
 

tests/sql/badges.sql

@@ -17,14 +17,19 @@ SELECT set_config('vessel.id', :'vessel_id', false) IS NOT NULL as vessel_id;
 
 \echo 'Insert new api.logbook for badges'
 INSERT INTO api.logbook
-    (id, active, "name", "_from", "_from_lat", "_from_lng", "_to", "_to_lat", "_to_lng", track_geom, track_geog, track_geojson, "_from_time", "_to_time", distance, duration, avg_speed, max_speed, max_wind_speed, notes, vessel_id)
+    (id, active, "name", "_from", "_from_lat", "_from_lng", "_to", "_to_lat", "_to_lng", trip, "_from_time", "_to_time", distance, duration, avg_speed, max_speed, max_wind_speed, notes, vessel_id)
     OVERRIDING SYSTEM VALUE VALUES
-    (nextval('api.logbook_id_seq'), false, 'Tropics Zone', NULL, NULL, NULL, NULL, NULL, NULL, 'SRID=4326;LINESTRING (-63.151124640791096 14.01074681627324, -77.0912026418618 12.870995731013664)'::public.geometry, NULL, NULL, NOW(), NOW(), 123, NULL, NULL, NULL, NULL, NULL, current_setting('vessel.id', false)),
-    (nextval('api.logbook_id_seq'), false, 'Alaska Zone', NULL, NULL, NULL, NULL, NULL, NULL, 'SRID=4326;LINESTRING (-143.5773697471158 59.4404631255976, -152.35402122385003 56.58243132943173)'::public.geometry, NULL, NULL, NOW(), NOW(), 1234, NULL, NULL, NULL, NULL, NULL, current_setting('vessel.id', false));
+    (nextval('api.logbook_id_seq'), false, 'Tropics Zone', NULL, NULL, NULL, NULL, NULL, NULL, 'SRID=4326;[Point(-63.151124640791096 14.01074681627324)@2025-01-01, Point(-77.0912026418618 12.870995731013664)@2025-01-02]'::public.tgeogpoint, NOW(), NOW(), 123, NULL, NULL, NULL, NULL, NULL, current_setting('vessel.id', false)),
+    (nextval('api.logbook_id_seq'), false, 'Alaska Zone', NULL, NULL, NULL, NULL, NULL, NULL, 'SRID=4326;[Point(-143.5773697471158 59.4404631255976)@2025-01-01, Point(-152.35402122385003 56.58243132943173)@2025-01-02]'::public.tgeogpoint, NOW(), NOW(), 1234, NULL, NULL, NULL, NULL, NULL, current_setting('vessel.id', false));
+
+-- Transform static geometry LINESTRING to mobilitydb
+-- 'SRID=4326;LINESTRING (-63.151124640791096 14.01074681627324, -77.0912026418618 12.870995731013664)'::public.geometry
+-- 'SRID=4326;LINESTRING (-143.5773697471158 59.4404631255976, -152.35402122385003 56.58243132943173)'::public.geometry
+--SELECT ST_AsGeoJSON('SRID=4326;LINESTRING (-63.151124640791096 14.01074681627324, -77.0912026418618 12.870995731013664)'::public.geometry);
+--SELECT ST_AsGeoJSON(trajectory('SRID=4326;[Point(-63.151124640791096 14.01074681627324)@2025-01-01, Point(-77.0912026418618 12.870995731013664)@2025-01-02]'::public.tgeogpoint));
 
 \echo 'Set config'
 SELECT set_config('user.email', 'demo+kapla@openplotter.cloud', false);
---SELECT set_config('vessel.client_id', 'vessels.urn:mrn:imo:mmsi:123456789', false);
 
 \echo 'Process badge'
 SELECT badges_logbook_fn(5,NOW()::TEXT);
@@ -32,11 +37,10 @@ SELECT badges_logbook_fn(6,NOW()::TEXT);
 SELECT badges_geom_fn(5,NOW()::TEXT);
 SELECT badges_geom_fn(6,NOW()::TEXT);
 
-\echo 'Check badges for user'
+\echo 'Check badges for all users'
 SELECT jsonb_object_keys ( a.preferences->'badges' ) FROM auth.accounts a;
 
 \echo 'Check details from vessel_id kapla'
---SELECT get_user_settings_from_vesselid_fn('vessels.urn:mrn:imo:mmsi:123456789'::TEXT);
 SELECT 
     json_build_object( 
             'boat', v.name,
@@ -53,10 +57,10 @@ SELECT
 
 \echo 'Insert new api.moorages for badges'
 INSERT INTO api.moorages
-    (id,"name",country,stay_code,stay_duration,reference_count,latitude,longitude,geog,home_flag,notes,vessel_id)
+    (id,"name",country,stay_code,latitude,longitude,geog,home_flag,notes,vessel_id)
     OVERRIDING SYSTEM VALUE VALUES
-    (8,'Badge Mooring Pro',NULL,3,'11 days 00:39:56.418',1,NULL,NULL,NULL,false,'Badge Mooring Pro',current_setting('vessel.id', false)),
-    (9,'Badge Anchormaster',NULL,2,'26 days 00:49:56.418',1,NULL,NULL,NULL,false,'Badge Anchormaster',current_setting('vessel.id', false));
+    (8,'Badge Mooring Pro',NULL,3,NULL,NULL,NULL,false,'Badge Mooring Pro',current_setting('vessel.id', false)),
+    (9,'Badge Anchormaster',NULL,2,NULL,NULL,NULL,false,'Badge Anchormaster',current_setting('vessel.id', false));
 
 \echo 'Set config'
 SELECT set_config('user.email', 'demo+aava@openplotter.cloud', false);
@@ -68,6 +72,9 @@ SELECT set_config('vessel.id', :'vessel_id', false) IS NOT NULL as vessel_id;
 \echo 'Process badge'
 SELECT badges_moorages_fn();
 
+\echo 'Check badges for all users'
+SELECT jsonb_object_keys ( a.preferences->'badges' ) FROM auth.accounts a;
+
 \echo 'Check details from vessel_id aava'
 --SELECT get_user_settings_from_vesselid_fn('vessels.urn:mrn:imo:mmsi:787654321'::TEXT);
 SELECT 

tests/sql/badges.sql.output

@@ -27,7 +27,7 @@ badges_geom_fn |
 -[ RECORD 1 ]--+-
 badges_geom_fn | 
 
-Check badges for user
+Check badges for all users
 -[ RECORD 1 ]-----+------------------
 jsonb_object_keys | Helmsman
 -[ RECORD 2 ]-----+------------------
@@ -76,6 +76,38 @@ Process badge
 -[ RECORD 1 ]------+-
 badges_moorages_fn | 
 
+Check badges for all users
+-[ RECORD 1 ]-----+------------------
+jsonb_object_keys | Helmsman
+-[ RECORD 2 ]-----+------------------
+jsonb_object_keys | Wake Maker
+-[ RECORD 3 ]-----+------------------
+jsonb_object_keys | Balearic Sea
+-[ RECORD 4 ]-----+------------------
+jsonb_object_keys | Stormtrooper
+-[ RECORD 5 ]-----+------------------
+jsonb_object_keys | Gulf of Finland
+-[ RECORD 6 ]-----+------------------
+jsonb_object_keys | Helmsman
+-[ RECORD 7 ]-----+------------------
+jsonb_object_keys | Wake Maker
+-[ RECORD 8 ]-----+------------------
+jsonb_object_keys | Club Alaska
+-[ RECORD 9 ]-----+------------------
+jsonb_object_keys | Stormtrooper
+-[ RECORD 10 ]----+------------------
+jsonb_object_keys | Captain Award
+-[ RECORD 11 ]----+------------------
+jsonb_object_keys | Caribbean Sea
+-[ RECORD 12 ]----+------------------
+jsonb_object_keys | Gulf of Alaska
+-[ RECORD 13 ]----+------------------
+jsonb_object_keys | Gulf of Finland
+-[ RECORD 14 ]----+------------------
+jsonb_object_keys | Navigator Award
+-[ RECORD 15 ]----+------------------
+jsonb_object_keys | Tropical Traveler
+
 Check details from vessel_id aava
 -[ RECORD 1 ]-+--------------------------------------------------------------------------------------------------------------
 user_settings | {"boat" : "aava", "recipient" : "first_aava", "email" : "demo+aava@openplotter.cloud", "pushover_key" : null}

tests/sql/cron_post_jobs.sql

@@ -21,11 +21,15 @@ SELECT v.vessel_id as "vessel_id" FROM auth.vessels v WHERE v.owner_email = 'dem
 --\echo :"vessel_id"
 SELECT set_config('vessel.id', :'vessel_id', false) IS NOT NULL as vessel_id;
 
+-- user_role
+SET ROLE user_role;
+
 -- Test logbook for user
 \echo 'logbook'
 SELECT count(*) FROM api.logbook WHERE vessel_id = current_setting('vessel.id', false);
 \echo 'logbook'
-SELECT name,_from_time IS NOT NULL AS _from_time,_to_time IS NOT NULL AS _to_time, track_geojson IS NOT NULL AS track_geojson, track_geom, distance,duration,avg_speed,max_speed,max_wind_speed,notes,extra FROM api.logbook WHERE vessel_id = current_setting('vessel.id', false);
+--SELECT name,_from_time IS NOT NULL AS _from_time,_to_time IS NOT NULL AS _to_time, track_geojson IS NOT NULL AS track_geojson, trajectory(trip)::geometry as track_geom, distance,duration,round(avg_speed::NUMERIC,6),max_speed,max_wind_speed,notes,extra FROM api.logbook WHERE vessel_id = current_setting('vessel.id', false);
+SELECT name,_from_time IS NOT NULL AS _from_time,_to_time IS NOT NULL AS _to_time, api.export_logbook_geojson_trip_fn(id) IS NOT NULL AS track_geojson, trajectory(trip)::geometry as track_geom, distance,duration,round(avg_speed::NUMERIC,6),max_speed,max_wind_speed,notes,extra FROM api.logbook WHERE vessel_id = current_setting('vessel.id', false);
 
 -- Test stays for user
 \echo 'stays'
@@ -37,7 +41,7 @@ SELECT active,name IS NOT NULL AS name,geog,stay_code FROM api.stays WHERE vesse
 \echo 'eventlogs_view'
 SELECT count(*) from api.eventlogs_view;
 
--- Test event logs view for user
+-- Test stats logs view for user
 \echo 'stats_logs_fn'
 SELECT api.stats_logs_fn(null, null) INTO stats_jsonb;
 SELECT stats_logs_fn->'name' AS name,
@@ -69,17 +73,28 @@ SELECT extra FROM api.logbook l WHERE id = 1 AND vessel_id = current_setting('ve
 SELECT api.update_logbook_observations_fn(1, '{"tags": ["tag_name"]}'::TEXT);
 SELECT extra FROM api.logbook l WHERE id = 1 AND vessel_id = current_setting('vessel.id', false);
 
-\echo 'Check numbers of geojson properties'
-SELECT jsonb_object_keys(jsonb_path_query(track_geojson, '$.features[0].properties'))
-    FROM api.logbook where id = 1 AND vessel_id = current_setting('vessel.id', false);
-SELECT jsonb_object_keys(jsonb_path_query(track_geojson, '$.features[1].properties'))
-    FROM api.logbook where id = 1 AND vessel_id = current_setting('vessel.id', false);
+\echo 'Check logbook geojson LineString properties'
+WITH logbook_tbl AS (
+    SELECT api.logbook_update_geojson_trip_fn(id) AS geojson
+    FROM api.logbook WHERE id = 1 AND vessel_id = current_setting('vessel.id', false)
+)
+SELECT jsonb_object_keys(jsonb_path_query(geojson, '$.features[0].properties'))
+    FROM logbook_tbl;
+\echo 'Check logbook geojson Point properties'
+WITH logbook_tbl AS (
+    SELECT api.logbook_update_geojson_trip_fn(id) AS geojson
+    FROM api.logbook WHERE id = 1 AND vessel_id = current_setting('vessel.id', false)
+)
+SELECT jsonb_object_keys(jsonb_path_query(geojson, '$.features[1].properties'))
+    FROM logbook_tbl;
 
 -- Check export
---\echo 'check logbook export fn'
+\echo 'Check logbook export fn'
 --SELECT api.export_logbook_geojson_fn(1);
 --SELECT api.export_logbook_gpx_fn(1);
 --SELECT api.export_logbook_kml_fn(1);
+SELECT api.export_logbook_gpx_trip_fn(1) IS NOT NULL AS gpx_trip;
+SELECT api.export_logbook_kml_trip_fn(1) IS NOT NULL AS kml_trip;
 
 -- Check history
 --\echo 'monitoring history fn'

tests/sql/cron_post_jobs.sql.output

@@ -11,33 +11,34 @@ user_id | t
 -[ RECORD 1 ]
 vessel_id | t
 
+SET
 logbook
 -[ RECORD 1 ]
 count | 2
 
 logbook
--[ RECORD 1 ]--+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+-[ RECORD 1 ]--+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
 name           | Pojoviken to Norra hamnen
 _from_time     | t
 _to_time       | t
 track_geojson  | t
-track_geom     | 0102000020E61000001C000000B0DEBBE0E68737404DA938FBF0094E40B0DEBBE0E68737404DA938FBF0094E4020D26F5F0786374030BB270F0B094E400C6E7ED60F843740AA60545227084E40D60FC48C03823740593CE27D42074E407B39D9F322803740984C158C4A064E4091ED7C3F357E3740898BB63D54054E40A8A1208B477C37404BA3DC9059044E404C5CB4EDA17A3740C4F856115B034E40A9A44E4013793740D8F0F44A59024E40E4839ECDAA773740211FF46C56014E405408D147067637408229F03B73004E40787AA52C43743740F90FE9B7AFFF4D40F8098D4D18723740C217265305FF4D4084E82303537037409A2D464AA0FE4D4022474DCE636F37402912396A72FE4D408351499D806E374088CFB02B40FE4D4076711B0DE06D3740B356C7040FFE4D404EAC66B0BC6E374058A835CD3BFE4D40D7A3703D0A6F3740D3E10EC15EFE4D4087602F277B6E3740A779C7293AFE4D4087602F277B6E3740A779C7293AFE4D402063EE5A426E3740B5A679C729FE4D40381DEE10EC6D37409ECA7C1A0AFE4D40E2C46A06CB6B37400A43F7BF36FD4D4075931804566E3740320BDAD125FD4D409A2D464AA06E37404A5658830AFD4D40029A081B9E6E37404A5658830AFD4D40
+track_geom     | 0102000020E61000001A000000B0DEBBE0E68737404DA938FBF0094E4020D26F5F0786374030BB270F0B094E400C6E7ED60F843740AA60545227084E40D60FC48C03823740593CE27D42074E407B39D9F322803740984C158C4A064E4091ED7C3F357E3740898BB63D54054E40A8A1208B477C37404BA3DC9059044E404C5CB4EDA17A3740C4F856115B034E40A9A44E4013793740D8F0F44A59024E40E4839ECDAA773740211FF46C56014E405408D147067637408229F03B73004E40787AA52C43743740F90FE9B7AFFF4D40F8098D4D18723740C217265305FF4D4084E82303537037409A2D464AA0FE4D4022474DCE636F37402912396A72FE4D408351499D806E374088CFB02B40FE4D4076711B0DE06D3740B356C7040FFE4D404EAC66B0BC6E374058A835CD3BFE4D40D7A3703D0A6F3740D3E10EC15EFE4D4087602F277B6E3740A779C7293AFE4D402063EE5A426E3740B5A679C729FE4D40381DEE10EC6D37409ECA7C1A0AFE4D40E2C46A06CB6B37400A43F7BF36FD4D4075931804566E3740320BDAD125FD4D409A2D464AA06E37404A5658830AFD4D40029A081B9E6E37404A5658830AFD4D40
 distance       | 7.6447
 duration       | PT27M
-avg_speed      | 3.6357142857142852
+round          | 3.635714
 max_speed      | 6.1
 max_wind_speed | 22.1
 notes          | 
 extra          | {"metrics": {"propulsion.main.runTime": "PT10S"}, "observations": {"seaState": -1, "visibility": -1, "cloudCoverage": -1}, "avg_wind_speed": 14.549999999999999}
--[ RECORD 2 ]--+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+-[ RECORD 2 ]--+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
 name           | Norra hamnen to Ekenäs
 _from_time     | t
 _to_time       | t
 track_geojson  | t
-track_geom     | 0102000020E610000015000000029A081B9E6E37404A5658830AFD4D40029A081B9E6E37404A5658830AFD4D404806A6C0EF6C3740DA1B7C6132FD4D40FE65F7E461693740226C787AA5FC4D407DD3E10EC1663740B29DEFA7C6FB4D40898BB63D5465374068479724BCFA4D409A5271F6E1633740B6847CD0B3F94D40431CEBE236623740E9263108ACF84D402C6519E2585F37407E678EBFC7F74D4096218E75715B374027C5B45C23F74D402AA913D044583740968DE1C46AF64D405AF5B9DA8A5537407BEF829B9FF54D407449C2ABD253374086C954C1A8F44D407D1A0AB278543740F2B0506B9AF34D409D11A5BDC15737406688635DDCF24D4061C3D32B655937402CAF6F3ADCF14D408988888888583740B3319C58CDF04D4021FAC8C0145837408C94405DB7EF4D40B8F9593F105B37403DC0804BEDEE4D40DE4C5FE2A25D3740AE47E17A14EE4D40DE4C5FE2A25D3740AE47E17A14EE4D40
+track_geom     | 0102000020E610000013000000029A081B9E6E37404A5658830AFD4D404806A6C0EF6C3740DA1B7C6132FD4D40FE65F7E461693740226C787AA5FC4D407DD3E10EC1663740B29DEFA7C6FB4D40898BB63D5465374068479724BCFA4D409A5271F6E1633740B6847CD0B3F94D40431CEBE236623740E9263108ACF84D402C6519E2585F37407E678EBFC7F74D4096218E75715B374027C5B45C23F74D402AA913D044583740968DE1C46AF64D405AF5B9DA8A5537407BEF829B9FF54D407449C2ABD253374086C954C1A8F44D407D1A0AB278543740F2B0506B9AF34D409D11A5BDC15737406688635DDCF24D4061C3D32B655937402CAF6F3ADCF14D408988888888583740B3319C58CDF04D4021FAC8C0145837408C94405DB7EF4D40B8F9593F105B37403DC0804BEDEE4D40DE4C5FE2A25D3740AE47E17A14EE4D40
 distance       | 8.8968
 duration       | PT20M
-avg_speed      | 5.4523809523809526
+round          | 5.452381
 max_speed      | 6.5
 max_wind_speed | 37.2
 notes          | 
@@ -66,23 +67,23 @@ stay_code | 4
 
 eventlogs_view
 -[ RECORD 1 ]
-count | 11
+count | 12
 
 stats_logs_fn
 SELECT 1
--[ RECORD 1 ]+----------
+-[ RECORD 1 ]+--------
 name         | "kapla"
-count        | 4
-max_speed    | 9.5
-max_distance | 68.8677
-max_duration | "PT1H11M"
-?column?     | 3
-?column?     | 90.6030
-?column?     | "PT2H44M"
-?column?     | 44.2
-?column?     | 3
-?column?     | 4
-?column?     | 4
+count        | 2
+max_speed    | 6.5
+max_distance | 8.8968
+max_duration | "PT27M"
+?column?     | 2
+?column?     | 16.5415
+?column?     | "PT47M"
+?column?     | 37.2
+?column?     | 2
+?column?     | 1
+?column?     | 2
 first_date   | t
 last_date    | t
 
@@ -110,54 +111,64 @@ update_logbook_observations_fn | t
 -[ RECORD 1 ]--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
 extra | {"tags": ["tag_name"], "metrics": {"propulsion.main.runTime": "PT10S"}, "observations": {"seaState": -1, "visibility": -1, "cloudCoverage": 1}, "avg_wind_speed": 14.549999999999999}
 
-Check numbers of geojson properties
+Check logbook geojson LineString properties
 -[ RECORD 1 ]-----+-----------------
 jsonb_object_keys | id
 -[ RECORD 2 ]-----+-----------------
-jsonb_object_keys | name
+jsonb_object_keys | _to
 -[ RECORD 3 ]-----+-----------------
-jsonb_object_keys | notes
+jsonb_object_keys | name
 -[ RECORD 4 ]-----+-----------------
-jsonb_object_keys | _to_time
+jsonb_object_keys | _from
 -[ RECORD 5 ]-----+-----------------
-jsonb_object_keys | distance
+jsonb_object_keys | notes
 -[ RECORD 6 ]-----+-----------------
-jsonb_object_keys | duration
+jsonb_object_keys | times
 -[ RECORD 7 ]-----+-----------------
-jsonb_object_keys | avg_speed
+jsonb_object_keys | _to_time
 -[ RECORD 8 ]-----+-----------------
-jsonb_object_keys | max_speed
+jsonb_object_keys | distance
 -[ RECORD 9 ]-----+-----------------
-jsonb_object_keys | _from_time
+jsonb_object_keys | duration
 -[ RECORD 10 ]----+-----------------
-jsonb_object_keys | _to_moorage_id
+jsonb_object_keys | avg_speed
 -[ RECORD 11 ]----+-----------------
-jsonb_object_keys | avg_wind_speed
+jsonb_object_keys | max_speed
 -[ RECORD 12 ]----+-----------------
-jsonb_object_keys | max_wind_speed
+jsonb_object_keys | _from_time
 -[ RECORD 13 ]----+-----------------
+jsonb_object_keys | _to_moorage_id
+-[ RECORD 14 ]----+-----------------
+jsonb_object_keys | avg_wind_speed
+-[ RECORD 15 ]----+-----------------
+jsonb_object_keys | max_wind_speed
+-[ RECORD 16 ]----+-----------------
 jsonb_object_keys | _from_moorage_id
 
--[ RECORD 1 ]-----+---------------------
+Check logbook geojson Point properties
+-[ RECORD 1 ]-----+-------
+jsonb_object_keys | cog
+-[ RECORD 2 ]-----+-------
+jsonb_object_keys | sog
+-[ RECORD 3 ]-----+-------
+jsonb_object_keys | twa
+-[ RECORD 4 ]-----+-------
+jsonb_object_keys | twd
+-[ RECORD 5 ]-----+-------
+jsonb_object_keys | tws
+-[ RECORD 6 ]-----+-------
 jsonb_object_keys | time
--[ RECORD 2 ]-----+---------------------
+-[ RECORD 7 ]-----+-------
 jsonb_object_keys | trip
--[ RECORD 3 ]-----+---------------------
+-[ RECORD 8 ]-----+-------
 jsonb_object_keys | notes
--[ RECORD 4 ]-----+---------------------
+-[ RECORD 9 ]-----+-------
 jsonb_object_keys | status
--[ RECORD 5 ]-----+---------------------
-jsonb_object_keys | latitude
--[ RECORD 6 ]-----+---------------------
-jsonb_object_keys | longitude
--[ RECORD 7 ]-----+---------------------
-jsonb_object_keys | truewindspeed
--[ RECORD 8 ]-----+---------------------
-jsonb_object_keys | speedoverground
--[ RECORD 9 ]-----+---------------------
-jsonb_object_keys | truewinddirection
--[ RECORD 10 ]----+---------------------
-jsonb_object_keys | windspeedapparent
--[ RECORD 11 ]----+---------------------
-jsonb_object_keys | courseovergroundtrue
+
+Check logbook export fn
+-[ RECORD 1 ]
+gpx_trip | t
+
+-[ RECORD 1 ]
+kml_trip | t
 

tests/sql/cron_run_jobs.sql

@@ -15,7 +15,12 @@ select current_database();
 \echo 'Check the number of process pending'
 -- Should be 24
 SELECT count(*) as jobs from public.process_queue pq where pq.processed is null;
---set role scheduler
+-- Switch to the scheduler role
+--\echo 'Switch to the scheduler role'
+--SET ROLE scheduler;
+-- Should be 24
+SELECT count(*) as jobs from public.process_queue pq where pq.processed is null;
+-- Run the cron jobs
 SELECT public.run_cron_jobs();
 -- Check any pending job
 SELECT count(*) as any_pending_jobs from public.process_queue pq where pq.processed is null;

tests/sql/cron_run_jobs.sql.output

@@ -7,7 +7,10 @@ You are now connected to database "signalk" as user "username".
 Expanded display is on.
 Check the number of process pending
 -[ RECORD 1 ]
-jobs | 24
+jobs | 26
+
+-[ RECORD 1 ]
+jobs | 26
 
 -[ RECORD 1 ]-+-
 run_cron_jobs | 

tests/sql/grafana.sql

@@ -15,21 +15,21 @@ select current_database();
 -- grafana_auth
 SET ROLE grafana_auth;
 \echo 'ROLE grafana_auth current_setting'
-SELECT current_user, current_setting('user.email', true), current_setting('vessel.client_id', true), current_setting('vessel.id', true);
+SELECT current_user, current_setting('user.email', true), current_setting('vessel.id', true);
 
 --SELECT a.pass,v.name,m.client_id FROM auth.accounts a JOIN auth.vessels v ON a.email = 'demo+kapla@openplotter.cloud' AND a.role = 'user_role' AND cast(a.preferences->>'email_valid' as Boolean) = True AND v.owner_email = a.email JOIN api.metadata m ON m.vessel_id = v.vessel_id;
 --SELECT a.pass,v.name,m.client_id FROM auth.accounts a JOIN auth.vessels v ON a.email = 'demo+kapla@openplotter.cloud' AND a.role = 'user_role' AND v.owner_email = a.email JOIN api.metadata m ON m.vessel_id = v.vessel_id;
 \echo 'link vessel and user based on current_setting'
-SELECT v.name,m.client_id FROM auth.accounts a JOIN auth.vessels v ON a.role = 'user_role' AND v.owner_email = a.email JOIN api.metadata m ON m.vessel_id = v.vessel_id;
+SELECT v.name, m.vessel_id IS NOT NULL AS vessel_id FROM auth.accounts a JOIN auth.vessels v ON a.role = 'user_role' AND v.owner_email = a.email JOIN api.metadata m ON m.vessel_id = v.vessel_id ORDER BY a.id DESC;
 
 \echo 'auth.accounts details'
-SELECT a.user_id IS NOT NULL AS user_id, a.email, a.first, a.last, a.pass IS NOT NULL AS pass, a.role, a.preferences->'telegram'->'chat' AS telegram, a.preferences->'pushover_user_key' AS pushover_user_key FROM auth.accounts AS a;
+SELECT a.user_id IS NOT NULL AS user_id, a.email, a.first, a.last, a.pass IS NOT NULL AS pass, a.role, a.preferences->'telegram'->'chat' AS telegram, a.preferences->'pushover_user_key' AS pushover_user_key FROM auth.accounts AS a ORDER BY a.id DESC;
 \echo 'auth.vessels details'
 --SELECT 'SELECT ' || STRING_AGG('v.' || column_name, ', ') || ' FROM auth.vessels AS v' FROM information_schema.columns WHERE table_name = 'vessels' AND table_schema = 'auth' AND column_name NOT IN ('created_at', 'updated_at');
 SELECT v.vessel_id IS NOT NULL AS vessel_id, v.owner_email, v.mmsi, v.name, v.role FROM auth.vessels AS v;
 \echo 'api.metadata details'
 --
-SELECT m.id, m.name, m.mmsi, m.client_id, m.length, m.beam, m.height, m.ship_type, m.plugin_version, m.signalk_version, m.time IS NOT NULL AS time, m.active FROM api.metadata AS m;
+SELECT vessel_id IS NOT NULL AS vessel_id_not_null, m.name, m.mmsi, m.length, m.beam, m.height, m.ship_type, m.plugin_version, m.signalk_version, m.time IS NOT NULL AS time, m.active, configuration IS NOT NULL AS configuration_not_null, available_keys FROM api.metadata AS m ORDER BY m.name DESC;
 
 --
 -- grafana
@@ -46,16 +46,16 @@ SELECT v.vessel_id as "vessel_id" FROM auth.vessels v WHERE v.owner_email = 'dem
 SELECT set_config('vessel.id', :'vessel_id', false) IS NOT NULL as vessel_id;
 
 --SELECT current_user, current_setting('user.email', true), current_setting('vessel.client_id', true), current_setting('vessel.id', true);
-SELECT current_user, current_setting('user.email', true), current_setting('vessel.client_id', true);
+SELECT current_user, current_setting('user.email', true);
 
-SELECT v.name AS __text, m.client_id AS __value FROM auth.vessels v JOIN api.metadata m ON v.owner_email = 'demo+kapla@openplotter.cloud' and m.vessel_id = v.vessel_id;
+SELECT v.name AS __text, m.vessel_id IS NOT NULL AS __value FROM auth.vessels v JOIN api.metadata m ON v.owner_email = 'demo+kapla@openplotter.cloud' and m.vessel_id = v.vessel_id;
 
 \echo 'auth.vessels details'
 --SELECT * FROM auth.vessels v;
 SELECT v.vessel_id IS NOT NULL AS vessel_id, v.owner_email, v.mmsi, v.name, v.role FROM auth.vessels AS v;
 --SELECT * FROM api.metadata m;
 \echo 'api.metadata details'
-SELECT m.id, m.name, m.mmsi, m.client_id, m.length, m.beam, m.height, m.ship_type, m.plugin_version, m.signalk_version, m.time IS NOT NULL AS time, m.active FROM api.metadata AS m;
+SELECT vessel_id IS NOT NULL AS vessel_id_not_null, m.name, m.mmsi, m.length, m.beam, m.height, m.ship_type, m.plugin_version, m.signalk_version, m.time IS NOT NULL AS time, m.active, configuration IS NOT NULL AS configuration_not_null, available_keys FROM api.metadata AS m;
 
 \echo 'api.logs_view'
 --SELECT * FROM api.logbook l;
@@ -67,13 +67,18 @@ SELECT l.id, l.name, l.from, l.to, l.distance, l.duration, l._from_moorage_id, l
 --SELECT * FROM api.stays s;
 SELECT m.id, m.vessel_id IS NOT NULL AS vessel_id, m.moorage_id, m.active, m.name IS NOT NULL AS name, m.latitude, m.longitude, m.geog, m.arrived IS NOT NULL AS arrived, m.departed IS NOT NULL AS departed, m.duration, m.stay_code, m.notes FROM api.stays AS m;
 
-\echo 'stays_view'
+\echo 'api.stays_view'
 --SELECT * FROM api.stays_view s;
 SELECT m.id, m.name IS NOT NULL AS name, m.moorage, m.moorage_id, m.duration, m.stayed_at, m.stayed_at_id, m.arrived IS NOT NULL AS arrived, m.departed IS NOT NULL AS departed, m.notes FROM api.stays_view AS m;
 
 \echo 'api.moorages'
 --SELECT * FROM api.moorages m;
-SELECT m.id, m.vessel_id IS NOT NULL AS vessel_id, m.name, m.country, m.stay_code, m.stay_duration, m.reference_count, m.latitude, m.longitude, m.geog, m.home_flag, m.notes FROM api.moorages AS m;
+--SELECT m.id, m.vessel_id IS NOT NULL AS vessel_id, m.name, m.country, m.stay_code, m.stay_duration, m.reference_count, m.latitude, m.longitude, m.geog, m.home_flag, m.notes FROM api.moorages AS m;
+SELECT m.id, m.vessel_id IS NOT NULL AS vessel_id, m.name, m.country, m.stay_code, m.latitude, m.longitude, m.geog, m.home_flag, m.notes FROM api.moorages AS m;
 
 \echo 'api.moorages_view'
-SELECT * FROM api.moorages_view s;
+SELECT * FROM api.moorages_view;
+
+\echo 'api.moorage_view'
+--SELECT * FROM api.moorage_view;
+SELECT m.id, m.name, default_stay, m.latitude, m.longitude, m.geog, m.home, m.notes, logs_count, stays_count, stay_first_seen_id, stay_last_seen_id, stay_first_seen IS NOT NULL AS stay_first_seen, stay_last_seen IS NOT NULL AS stay_last_seen FROM api.moorage_view m;

tests/sql/grafana.sql.output

@@ -11,15 +11,14 @@ ROLE grafana_auth current_setting
 current_user    | grafana_auth
 current_setting | 
 current_setting | 
-current_setting | 
 
 link vessel and user based on current_setting
--[ RECORD 1 ]----------------------------------------------------------------
+-[ RECORD 1 ]----
 name      | aava
-client_id | vessels.urn:mrn:imo:mmsi:787654321
--[ RECORD 2 ]----------------------------------------------------------------
+vessel_id | t
+-[ RECORD 2 ]----
 name      | kapla
-client_id | vessels.urn:mrn:signalk:uuid:5b4f7543-7153-4840-b139-761310b242fd
+vessel_id | t
 
 auth.accounts details
 -[ RECORD 1 ]-----+-----------------------------
@@ -56,32 +55,34 @@ name        | aava
 role        | vessel_role
 
 api.metadata details
--[ RECORD 1 ]---+------------------------------------------------------------------
-id              | 1
-name            | kapla
-mmsi            | 123456789
-client_id       | vessels.urn:mrn:signalk:uuid:5b4f7543-7153-4840-b139-761310b242fd
-length          | 12
-beam            | 10
-height          | 24
-ship_type       | 36
-plugin_version  | 0.0.1
-signalk_version | signalk_version
-time            | t
-active          | t
--[ RECORD 2 ]---+------------------------------------------------------------------
-id              | 2
-name            | aava
-mmsi            | 787654321
-client_id       | vessels.urn:mrn:imo:mmsi:787654321
-length          | 12
-beam            | 10
-height          | 24
-ship_type       | 37
-plugin_version  | 1.0.2
-signalk_version | 1.20.0
-time            | t
-active          | t
+-[ RECORD 1 ]----------+----------------
+vessel_id_not_null     | t
+name                   | kapla
+mmsi                   | 123456789
+length                 | 12
+beam                   | 10
+height                 | 24
+ship_type              | 36
+plugin_version         | 0.0.1
+signalk_version        | signalk_version
+time                   | t
+active                 | t
+configuration_not_null | t
+available_keys         | 
+-[ RECORD 2 ]----------+----------------
+vessel_id_not_null     | t
+name                   | aava
+mmsi                   | 787654321
+length                 | 12
+beam                   | 10
+height                 | 24
+ship_type              | 37
+plugin_version         | 1.0.2
+signalk_version        | 1.20.0
+time                   | t
+active                 | t
+configuration_not_null | t
+available_keys         | []
 
 SET
 ROLE grafana current_setting
@@ -93,11 +94,10 @@ vessel_id | t
 -[ RECORD 1 ]---+-----------------------------
 current_user    | grafana
 current_setting | demo+kapla@openplotter.cloud
-current_setting | 
 
--[ RECORD 1 ]--------------------------------------------------------------
+-[ RECORD 1 ]--
 __text  | kapla
-__value | vessels.urn:mrn:signalk:uuid:5b4f7543-7153-4840-b139-761310b242fd
+__value | t
 
 auth.vessels details
 -[ RECORD 1 ]-----------------------------
@@ -108,19 +108,20 @@ name        | kapla
 role        | vessel_role
 
 api.metadata details
--[ RECORD 1 ]---+------------------------------------------------------------------
-id              | 1
-name            | kapla
-mmsi            | 123456789
-client_id       | vessels.urn:mrn:signalk:uuid:5b4f7543-7153-4840-b139-761310b242fd
-length          | 12
-beam            | 10
-height          | 24
-ship_type       | 36
-plugin_version  | 0.0.1
-signalk_version | signalk_version
-time            | t
-active          | t
+-[ RECORD 1 ]----------+----------------
+vessel_id_not_null     | t
+name                   | kapla
+mmsi                   | 123456789
+length                 | 12
+beam                   | 10
+height                 | 24
+ship_type              | 36
+plugin_version         | 0.0.1
+signalk_version        | signalk_version
+time                   | t
+active                 | t
+configuration_not_null | t
+available_keys         | 
 
 api.logs_view
 -[ RECORD 1 ]----+-----------------------
@@ -186,7 +187,7 @@ duration   | PT2M
 stay_code  | 4
 notes      | 
 
-stays_view
+api.stays_view
 -[ RECORD 1 ]+---------------------
 id           | 2
 name         | t
@@ -211,45 +212,39 @@ departed     | t
 notes        | new stay note 3
 
 api.moorages
--[ RECORD 1 ]---+---------------------------------------------------
-id              | 1
-vessel_id       | t
-name            | patch moorage name 3
-country         | fi
-stay_code       | 2
-stay_duration   | PT1M
-reference_count | 1
-latitude        | 60.0776666666667
-longitude       | 23.5308666666667
-geog            | 0101000020E6100000B9DEBBE0E687374052A938FBF0094E40
-home_flag       | t
-notes           | new moorage note 3
--[ RECORD 2 ]---+---------------------------------------------------
-id              | 2
-vessel_id       | t
-name            | Norra hamnen
-country         | fi
-stay_code       | 4
-stay_duration   | PT2M
-reference_count | 2
-latitude        | 59.9768833333333
-longitude       | 23.4321
-geog            | 0101000020E6100000029A081B9E6E3740455658830AFD4D40
-home_flag       | f
-notes           | 
--[ RECORD 3 ]---+---------------------------------------------------
-id              | 3
-vessel_id       | t
-name            | Ekenäs
-country         | fi
-stay_code       | 1
-stay_duration   | PT0S
-reference_count | 1
-latitude        | 59.86
-longitude       | 23.3657666666667
-geog            | 0101000020E6100000E84C5FE2A25D3740AE47E17A14EE4D40
-home_flag       | f
-notes           | 
+-[ RECORD 1 ]-------------------------------------------------
+id        | 1
+vessel_id | t
+name      | patch moorage name 3
+country   | fi
+stay_code | 2
+latitude  | 60.0776666666667
+longitude | 23.5308666666667
+geog      | 0101000020E6100000B9DEBBE0E687374052A938FBF0094E40
+home_flag | t
+notes     | new moorage note 3
+-[ RECORD 2 ]-------------------------------------------------
+id        | 2
+vessel_id | t
+name      | Norra hamnen
+country   | fi
+stay_code | 4
+latitude  | 59.9768833333333
+longitude | 23.4321
+geog      | 0101000020E6100000029A081B9E6E3740455658830AFD4D40
+home_flag | f
+notes     | 
+-[ RECORD 3 ]-------------------------------------------------
+id        | 3
+vessel_id | t
+name      | Ekenäs
+country   | fi
+stay_code | 1
+latitude  | 59.86
+longitude | 23.3657666666667
+geog      | 0101000020E6100000E84C5FE2A25D3740AE47E17A14EE4D40
+home_flag | f
+notes     | 
 
 api.moorages_view
 -[ RECORD 1 ]-------+---------------------
@@ -257,23 +252,67 @@ id                  | 2
 moorage             | Norra hamnen
 default_stay        | Dock
 default_stay_id     | 4
-total_stay          | 0
-total_duration      | PT2M
 arrivals_departures | 2
+total_duration      | PT2M
 -[ RECORD 2 ]-------+---------------------
 id                  | 1
 moorage             | patch moorage name 3
 default_stay        | Anchor
 default_stay_id     | 2
-total_stay          | 0
-total_duration      | PT1M
 arrivals_departures | 1
+total_duration      | PT1M
 -[ RECORD 3 ]-------+---------------------
 id                  | 3
 moorage             | Ekenäs
 default_stay        | Unknown
 default_stay_id     | 1
-total_stay          | 0
-total_duration      | PT0S
 arrivals_departures | 1
+total_duration      | PT0S
+
+api.moorage_view
+-[ RECORD 1 ]------+---------------------------------------------------
+id                 | 3
+name               | Ekenäs
+default_stay       | Unknown
+latitude           | 59.86
+longitude          | 23.3657666666667
+geog               | 0101000020E6100000E84C5FE2A25D3740AE47E17A14EE4D40
+home               | f
+notes              | 
+logs_count         | 1
+stays_count        | 0
+stay_first_seen_id | 
+stay_last_seen_id  | 
+stay_first_seen    | f
+stay_last_seen     | f
+-[ RECORD 2 ]------+---------------------------------------------------
+id                 | 2
+name               | Norra hamnen
+default_stay       | Dock
+latitude           | 59.9768833333333
+longitude          | 23.4321
+geog               | 0101000020E6100000029A081B9E6E3740455658830AFD4D40
+home               | f
+notes              | 
+logs_count         | 2
+stays_count        | 1
+stay_first_seen_id | 2
+stay_last_seen_id  | 2
+stay_first_seen    | t
+stay_last_seen     | t
+-[ RECORD 3 ]------+---------------------------------------------------
+id                 | 1
+name               | patch moorage name 3
+default_stay       | Anchor
+latitude           | 60.0776666666667
+longitude          | 23.5308666666667
+geog               | 0101000020E6100000B9DEBBE0E687374052A938FBF0094E40
+home               | t
+notes              | new moorage note 3
+logs_count         | 1
+stays_count        | 1
+stay_first_seen_id | 1
+stay_last_seen_id  | 1
+stay_first_seen    | t
+stay_last_seen     | t
 

tests/sql/logbook.sql

@@ -22,11 +22,26 @@ SELECT v.vessel_id as "vessel_id" FROM auth.vessels v WHERE v.owner_email = 'dem
 --\echo :"vessel_id"
 SELECT set_config('vessel.id', :'vessel_id', false) IS NOT NULL as vessel_id;
 
--- Delete logbook for user
+-- Count logbook for user
 \echo 'logbook'
 SELECT count(*) FROM api.logbook WHERE vessel_id = current_setting('vessel.id', false);
 \echo 'logbook'
-SELECT name,_from_time IS NOT NULL AS _from_time,_to_time IS NOT NULL AS _to_time, track_geojson IS NOT NULL AS track_geojson, track_geom, distance,duration,avg_speed,max_speed,max_wind_speed,notes,extra FROM api.logbook WHERE vessel_id = current_setting('vessel.id', false);
+-- track_geom and track_geojson are now dynamic from mobilitydb
+SELECT name,_from_time IS NOT NULL AS _from_time_not_null, _to_time IS NOT NULL AS _to_time_not_null, trajectory(trip) AS track_geom, distance,duration,avg_speed,max_speed,max_wind_speed,notes,extra FROM api.logbook WHERE vessel_id = current_setting('vessel.id', false) ORDER BY id ASC;
+
+--
+-- user_role
+SET ROLE user_role;
+\echo 'ROLE user_role current_setting'
+
+SELECT set_config('vessel.id', :'vessel_id', false) IS NOT NULL as vessel_id;
+
+-- Count logbook for user
+\echo 'logbook'
+SELECT count(*) FROM api.logbook;
+\echo 'logbook'
+-- track_geom and track_geojson are now dynamic from mobilitydb
+SELECT name,_from_time IS NOT NULL AS _from_time_not_null, _to_time IS NOT NULL AS _to_time_not_null, trajectory(trip) AS track_geom, distance,duration,avg_speed,max_speed,max_wind_speed,notes,extra FROM api.logbook ORDER BY id ASC;
 
 -- Delete logbook for user
 \echo 'Delete logbook for user kapla'

tests/sql/logbook.sql.output

@@ -17,58 +17,113 @@ logbook
 count | 4
 
 logbook
--[ RECORD 1 ]--+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
-name           | patch log name 3
-_from_time     | t
-_to_time       | t
-track_geojson  | t
-track_geom     | 0102000020E61000001C000000B0DEBBE0E68737404DA938FBF0094E40B0DEBBE0E68737404DA938FBF0094E4020D26F5F0786374030BB270F0B094E400C6E7ED60F843740AA60545227084E40D60FC48C03823740593CE27D42074E407B39D9F322803740984C158C4A064E4091ED7C3F357E3740898BB63D54054E40A8A1208B477C37404BA3DC9059044E404C5CB4EDA17A3740C4F856115B034E40A9A44E4013793740D8F0F44A59024E40E4839ECDAA773740211FF46C56014E405408D147067637408229F03B73004E40787AA52C43743740F90FE9B7AFFF4D40F8098D4D18723740C217265305FF4D4084E82303537037409A2D464AA0FE4D4022474DCE636F37402912396A72FE4D408351499D806E374088CFB02B40FE4D4076711B0DE06D3740B356C7040FFE4D404EAC66B0BC6E374058A835CD3BFE4D40D7A3703D0A6F3740D3E10EC15EFE4D4087602F277B6E3740A779C7293AFE4D4087602F277B6E3740A779C7293AFE4D402063EE5A426E3740B5A679C729FE4D40381DEE10EC6D37409ECA7C1A0AFE4D40E2C46A06CB6B37400A43F7BF36FD4D4075931804566E3740320BDAD125FD4D409A2D464AA06E37404A5658830AFD4D40029A081B9E6E37404A5658830AFD4D40
-distance       | 7.6447
-duration       | PT27M
-avg_speed      | 3.6357142857142852
-max_speed      | 6.1
-max_wind_speed | 22.1
-notes          | new log note 3
-extra          | {"tags": ["tag_name"], "metrics": {"propulsion.main.runTime": "PT10S"}, "observations": {"seaState": -1, "visibility": -1, "cloudCoverage": 1}, "avg_wind_speed": 14.549999999999999}
--[ RECORD 2 ]--+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
-name           | Norra hamnen to Ekenäs
-_from_time     | t
-_to_time       | t
-track_geojson  | t
-track_geom     | 0102000020E610000015000000029A081B9E6E37404A5658830AFD4D40029A081B9E6E37404A5658830AFD4D404806A6C0EF6C3740DA1B7C6132FD4D40FE65F7E461693740226C787AA5FC4D407DD3E10EC1663740B29DEFA7C6FB4D40898BB63D5465374068479724BCFA4D409A5271F6E1633740B6847CD0B3F94D40431CEBE236623740E9263108ACF84D402C6519E2585F37407E678EBFC7F74D4096218E75715B374027C5B45C23F74D402AA913D044583740968DE1C46AF64D405AF5B9DA8A5537407BEF829B9FF54D407449C2ABD253374086C954C1A8F44D407D1A0AB278543740F2B0506B9AF34D409D11A5BDC15737406688635DDCF24D4061C3D32B655937402CAF6F3ADCF14D408988888888583740B3319C58CDF04D4021FAC8C0145837408C94405DB7EF4D40B8F9593F105B37403DC0804BEDEE4D40DE4C5FE2A25D3740AE47E17A14EE4D40DE4C5FE2A25D3740AE47E17A14EE4D40
-distance       | 8.8968
-duration       | PT20M
-avg_speed      | 5.4523809523809526
-max_speed      | 6.5
-max_wind_speed | 37.2
-notes          | 
-extra          | {"metrics": {"propulsion.main.runTime": "PT11S"}, "observations": {"seaState": -1, "visibility": -1, "cloudCoverage": -1}, "avg_wind_speed": 10.476190476190478}
--[ RECORD 3 ]--+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
-name           | Tropics Zone
-_from_time     | t
-_to_time       | t
-track_geojson  | f
-track_geom     | 0102000020E610000002000000A4E85E0D58934FC000DC509B80052C40BC069B43D64553C090510727F3BD2940
-distance       | 123
-duration       | 
-avg_speed      | 
-max_speed      | 
-max_wind_speed | 
-notes          | 
-extra          | 
--[ RECORD 4 ]--+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
-name           | Alaska Zone
-_from_time     | t
-_to_time       | t
-track_geojson  | f
-track_geom     | 0102000020E610000002000000FDB11ED079F261C090C47F1861B84D40D3505124540B63C09C091C1C8D4A4C40
-distance       | 1234
-duration       | 
-avg_speed      | 
-max_speed      | 
-max_wind_speed | 
-notes          | 
-extra          | 
+-[ RECORD 1 ]-------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+name                | patch log name 3
+_from_time_not_null | t
+_to_time_not_null   | t
+track_geom          | 0102000020E61000001A000000B0DEBBE0E68737404DA938FBF0094E4020D26F5F0786374030BB270F0B094E400C6E7ED60F843740AA60545227084E40D60FC48C03823740593CE27D42074E407B39D9F322803740984C158C4A064E4091ED7C3F357E3740898BB63D54054E40A8A1208B477C37404BA3DC9059044E404C5CB4EDA17A3740C4F856115B034E40A9A44E4013793740D8F0F44A59024E40E4839ECDAA773740211FF46C56014E405408D147067637408229F03B73004E40787AA52C43743740F90FE9B7AFFF4D40F8098D4D18723740C217265305FF4D4084E82303537037409A2D464AA0FE4D4022474DCE636F37402912396A72FE4D408351499D806E374088CFB02B40FE4D4076711B0DE06D3740B356C7040FFE4D404EAC66B0BC6E374058A835CD3BFE4D40D7A3703D0A6F3740D3E10EC15EFE4D4087602F277B6E3740A779C7293AFE4D402063EE5A426E3740B5A679C729FE4D40381DEE10EC6D37409ECA7C1A0AFE4D40E2C46A06CB6B37400A43F7BF36FD4D4075931804566E3740320BDAD125FD4D409A2D464AA06E37404A5658830AFD4D40029A081B9E6E37404A5658830AFD4D40
+distance            | 7.6447
+duration            | PT27M
+avg_speed           | 3.6357142857142852
+max_speed           | 6.1
+max_wind_speed      | 22.1
+notes               | new log note 3
+extra               | {"tags": ["tag_name"], "metrics": {"propulsion.main.runTime": "PT10S"}, "observations": {"seaState": -1, "visibility": -1, "cloudCoverage": 1}, "avg_wind_speed": 14.549999999999999}
+-[ RECORD 2 ]-------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+name                | Norra hamnen to Ekenäs
+_from_time_not_null | t
+_to_time_not_null   | t
+track_geom          | 0102000020E610000013000000029A081B9E6E37404A5658830AFD4D404806A6C0EF6C3740DA1B7C6132FD4D40FE65F7E461693740226C787AA5FC4D407DD3E10EC1663740B29DEFA7C6FB4D40898BB63D5465374068479724BCFA4D409A5271F6E1633740B6847CD0B3F94D40431CEBE236623740E9263108ACF84D402C6519E2585F37407E678EBFC7F74D4096218E75715B374027C5B45C23F74D402AA913D044583740968DE1C46AF64D405AF5B9DA8A5537407BEF829B9FF54D407449C2ABD253374086C954C1A8F44D407D1A0AB278543740F2B0506B9AF34D409D11A5BDC15737406688635DDCF24D4061C3D32B655937402CAF6F3ADCF14D408988888888583740B3319C58CDF04D4021FAC8C0145837408C94405DB7EF4D40B8F9593F105B37403DC0804BEDEE4D40DE4C5FE2A25D3740AE47E17A14EE4D40
+distance            | 8.8968
+duration            | PT20M
+avg_speed           | 5.4523809523809526
+max_speed           | 6.5
+max_wind_speed      | 37.2
+notes               | 
+extra               | {"metrics": {"propulsion.main.runTime": "PT11S"}, "observations": {"seaState": -1, "visibility": -1, "cloudCoverage": -1}, "avg_wind_speed": 10.476190476190478}
+-[ RECORD 3 ]-------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+name                | Tropics Zone
+_from_time_not_null | t
+_to_time_not_null   | t
+track_geom          | 0102000020E610000002000000A4E85E0D58934FC000DC509B80052C40BC069B43D64553C090510727F3BD2940
+distance            | 123
+duration            | 
+avg_speed           | 
+max_speed           | 
+max_wind_speed      | 
+notes               | 
+extra               | 
+-[ RECORD 4 ]-------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+name                | Alaska Zone
+_from_time_not_null | t
+_to_time_not_null   | t
+track_geom          | 0102000020E610000002000000FDB11ED079F261C090C47F1861B84D40D3505124540B63C09C091C1C8D4A4C40
+distance            | 1234
+duration            | 
+avg_speed           | 
+max_speed           | 
+max_wind_speed      | 
+notes               | 
+extra               | 
+
+SET
+ROLE user_role current_setting
+-[ RECORD 1 ]
+vessel_id | t
+
+logbook
+-[ RECORD 1 ]
+count | 4
+
+logbook
+-[ RECORD 1 ]-------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+name                | patch log name 3
+_from_time_not_null | t
+_to_time_not_null   | t
+track_geom          | 0102000020E61000001A000000B0DEBBE0E68737404DA938FBF0094E4020D26F5F0786374030BB270F0B094E400C6E7ED60F843740AA60545227084E40D60FC48C03823740593CE27D42074E407B39D9F322803740984C158C4A064E4091ED7C3F357E3740898BB63D54054E40A8A1208B477C37404BA3DC9059044E404C5CB4EDA17A3740C4F856115B034E40A9A44E4013793740D8F0F44A59024E40E4839ECDAA773740211FF46C56014E405408D147067637408229F03B73004E40787AA52C43743740F90FE9B7AFFF4D40F8098D4D18723740C217265305FF4D4084E82303537037409A2D464AA0FE4D4022474DCE636F37402912396A72FE4D408351499D806E374088CFB02B40FE4D4076711B0DE06D3740B356C7040FFE4D404EAC66B0BC6E374058A835CD3BFE4D40D7A3703D0A6F3740D3E10EC15EFE4D4087602F277B6E3740A779C7293AFE4D402063EE5A426E3740B5A679C729FE4D40381DEE10EC6D37409ECA7C1A0AFE4D40E2C46A06CB6B37400A43F7BF36FD4D4075931804566E3740320BDAD125FD4D409A2D464AA06E37404A5658830AFD4D40029A081B9E6E37404A5658830AFD4D40
+distance            | 7.6447
+duration            | PT27M
+avg_speed           | 3.6357142857142852
+max_speed           | 6.1
+max_wind_speed      | 22.1
+notes               | new log note 3
+extra               | {"tags": ["tag_name"], "metrics": {"propulsion.main.runTime": "PT10S"}, "observations": {"seaState": -1, "visibility": -1, "cloudCoverage": 1}, "avg_wind_speed": 14.549999999999999}
+-[ RECORD 2 ]-------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+name                | Norra hamnen to Ekenäs
+_from_time_not_null | t
+_to_time_not_null   | t
+track_geom          | 0102000020E610000013000000029A081B9E6E37404A5658830AFD4D404806A6C0EF6C3740DA1B7C6132FD4D40FE65F7E461693740226C787AA5FC4D407DD3E10EC1663740B29DEFA7C6FB4D40898BB63D5465374068479724BCFA4D409A5271F6E1633740B6847CD0B3F94D40431CEBE236623740E9263108ACF84D402C6519E2585F37407E678EBFC7F74D4096218E75715B374027C5B45C23F74D402AA913D044583740968DE1C46AF64D405AF5B9DA8A5537407BEF829B9FF54D407449C2ABD253374086C954C1A8F44D407D1A0AB278543740F2B0506B9AF34D409D11A5BDC15737406688635DDCF24D4061C3D32B655937402CAF6F3ADCF14D408988888888583740B3319C58CDF04D4021FAC8C0145837408C94405DB7EF4D40B8F9593F105B37403DC0804BEDEE4D40DE4C5FE2A25D3740AE47E17A14EE4D40
+distance            | 8.8968
+duration            | PT20M
+avg_speed           | 5.4523809523809526
+max_speed           | 6.5
+max_wind_speed      | 37.2
+notes               | 
+extra               | {"metrics": {"propulsion.main.runTime": "PT11S"}, "observations": {"seaState": -1, "visibility": -1, "cloudCoverage": -1}, "avg_wind_speed": 10.476190476190478}
+-[ RECORD 3 ]-------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+name                | Tropics Zone
+_from_time_not_null | t
+_to_time_not_null   | t
+track_geom          | 0102000020E610000002000000A4E85E0D58934FC000DC509B80052C40BC069B43D64553C090510727F3BD2940
+distance            | 123
+duration            | 
+avg_speed           | 
+max_speed           | 
+max_wind_speed      | 
+notes               | 
+extra               | 
+-[ RECORD 4 ]-------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+name                | Alaska Zone
+_from_time_not_null | t
+_to_time_not_null   | t
+track_geom          | 0102000020E610000002000000FDB11ED079F261C090C47F1861B84D40D3505124540B63C09C091C1C8D4A4C40
+distance            | 1234
+duration            | 
+avg_speed           | 
+max_speed           | 
+max_wind_speed      | 
+notes               | 
+extra               | 
 
 Delete logbook for user kapla
 -[ RECORD 1 ]-----+--

tests/sql/metadata.sql

@@ -0,0 +1,84 @@
+---------------------------------------------------------------------------
+-- Listing
+--
+
+-- List current database
+select current_database();
+
+-- connect to the DB
+\c signalk
+
+-- output display format
+\x on
+
+SELECT count(*) as count_eq_2 FROM api.metadata m;
+
+SELECT v.vessel_id as "vessel_id" FROM auth.vessels v WHERE v.owner_email = 'demo+kapla@openplotter.cloud' \gset
+--\echo :"vessel_id"
+SELECT set_config('vessel.id', :'vessel_id', false) IS NOT NULL as vessel_id;
+
+-- user_role
+SET ROLE user_role;
+
+\echo 'api.metadata details'
+SELECT vessel_id IS NOT NULL AS vessel_id_not_null, m.name, m.mmsi, m.length, m.beam, m.height, m.ship_type, m.plugin_version, m.signalk_version, m.time IS NOT NULL AS time, m.active, configuration, available_keys FROM api.metadata AS m ORDER BY m.name ASC;
+
+\echo 'api.metadata get configuration'
+select configuration from api.metadata; --WHERE vessel_id = current_setting('vessel.id', false);
+
+\echo 'api.metadata update configuration'
+UPDATE api.metadata SET configuration = '{ "depthKey": "environment.depth.belowTransducer" }'; --WHERE vessel_id = current_setting('vessel.id', false);
+
+\echo 'api.metadata get configuration with new value'
+select configuration->'depthKey' AS depthKey, configuration->'update_at' IS NOT NULL AS update_at_not_null from api.metadata; --WHERE vessel_id = current_setting('vessel.id', false);
+
+\echo 'api.metadata get configuration base on update_at value'
+select configuration->'depthKey' AS depthKey, configuration->'update_at' IS NOT NULL AS update_at_not_null from api.metadata WHERE vessel_id = current_setting('vessel.id', false) AND configuration->>'update_at' <= to_char(NOW(), 'YYYY-MM-DD"T"HH24:MI:SS"Z"');
+
+-- Upsert make_model on metadata_ext table
+\echo 'api.metadata_ext set make_model'
+INSERT INTO api.metadata_ext (vessel_id, make_model)
+    VALUES (current_setting('vessel.id', false), 'my super yacht')
+    ON CONFLICT (vessel_id) DO UPDATE
+    SET make_model = EXCLUDED.make_model;
+
+-- Upsert polar on metadata_ext table
+\echo 'api.metadata_ext set polar'
+INSERT INTO api.metadata_ext (vessel_id, polar)
+    VALUES (current_setting('vessel.id', false), 'twa/tws;4;6;8;10;12;14;16;20;24\n0;0;0;0;0;0;0;0;0;0')
+    ON CONFLICT (vessel_id) DO UPDATE
+    SET polar = EXCLUDED.polar;
+
+-- Upsert image on metadata_ext table
+\echo 'api.metadata_ext set image/image_b64'
+INSERT INTO api.metadata_ext (vessel_id, image_b64)
+    VALUES (current_setting('vessel.id', false), 'iVBORw0KGgoAAAANSUhEUgAAAMgAAAAyCAIAAACWMwO2AAABNklEQVR4nO3bwY6CMBiF0XYy7//KzIKk6VBjiMMNk59zVljRIH6WsrBv29bgal93HwA1CYsIYREhLCKERYSwiBAWEcIiQlhECIsIYREhLCKERYSwiBAWEcIiQlhECIsIYREhLCK+7z6A/6j33lq75G8m')
+    ON CONFLICT (vessel_id) DO UPDATE
+    SET image_b64 = EXCLUDED.image_b64;
+
+-- Ensure make_model on metadata_ext table is updated
+\echo 'api.metadata_ext get make_model'
+SELECT make_model FROM api.metadata_ext; --WHERE vessel_id = current_setting('vessel.id', false);
+
+-- Ensure polar_updated_at on metadata_ext table is updated by trigger
+\echo 'api.metadata_ext get polar_updated_at'
+SELECT polar,polar_updated_at IS NOT NULL AS polar_updated_at_not_null FROM api.metadata_ext; --WHERE vessel_id = current_setting('vessel.id', false);
+
+-- Ensure image_updated_at on metadata_ext table is updated by trigger
+\echo 'api.metadata_ext get image_updated_at'
+SELECT image_b64 IS NULL AS image_b64_is_null,image IS NOT NULL AS image_not_null,image_updated_at IS NOT NULL AS image_updated_at_not_null FROM api.metadata_ext; --WHERE vessel_id = current_setting('vessel.id', false);
+
+-- vessel_role
+SET ROLE vessel_role;
+
+\echo 'api.metadata get configuration with new value as vessel'
+select configuration->'depthKey' AS depthKey, configuration->'update_at' IS NOT NULL AS update_at_not_null from api.metadata; -- WHERE vessel_id = current_setting('vessel.id', false);
+
+\echo 'api.metadata get configuration base on update_at value as vessel'
+select configuration->'depthKey' AS depthKey, configuration->'update_at' IS NOT NULL AS update_at_not_null from api.metadata WHERE vessel_id = current_setting('vessel.id', false) AND configuration->>'update_at' <= to_char(NOW(), 'YYYY-MM-DD"T"HH24:MI:SS"Z"');
+
+-- api_anonymous
+SET ROLE api_anonymous;
+
+\echo 'api_anonymous get vessel image'
+SELECT api.vessel_image(current_setting('vessel.id', false)) IS NOT NULL AS vessel_image_not_null;

tests/sql/metadata.sql.output

@@ -0,0 +1,83 @@
+ current_database 
+------------------
+ signalk
+(1 row)
+
+You are now connected to database "signalk" as user "username".
+Expanded display is on.
+-[ RECORD 1 ]-
+count_eq_2 | 2
+
+-[ RECORD 1 ]
+vessel_id | t
+
+SET
+api.metadata details
+-[ RECORD 1 ]------+----------------
+vessel_id_not_null | t
+name               | kapla
+mmsi               | 123456789
+length             | 12
+beam               | 10
+height             | 24
+ship_type          | 36
+plugin_version     | 0.0.1
+signalk_version    | signalk_version
+time               | t
+active             | t
+configuration      | 
+available_keys     | 
+
+api.metadata get configuration
+-[ RECORD 1 ]-+-
+configuration | 
+
+api.metadata update configuration
+UPDATE 1
+api.metadata get configuration with new value
+-[ RECORD 1 ]------+------------------------------------
+depthkey           | "environment.depth.belowTransducer"
+update_at_not_null | t
+
+api.metadata get configuration base on update_at value
+-[ RECORD 1 ]------+------------------------------------
+depthkey           | "environment.depth.belowTransducer"
+update_at_not_null | t
+
+api.metadata_ext set make_model
+INSERT 0 1
+api.metadata_ext set polar
+INSERT 0 1
+api.metadata_ext set image/image_b64
+INSERT 0 1
+api.metadata_ext get make_model
+-[ RECORD 1 ]--------------
+make_model | my super yacht
+
+api.metadata_ext get polar_updated_at
+-[ RECORD 1 ]-------------+-----------------------------------------------------
+polar                     | twa/tws;4;6;8;10;12;14;16;20;24\n0;0;0;0;0;0;0;0;0;0
+polar_updated_at_not_null | t
+
+api.metadata_ext get image_updated_at
+-[ RECORD 1 ]-------------+--
+image_b64_is_null         | f
+image_not_null            | t
+image_updated_at_not_null | t
+
+SET
+api.metadata get configuration with new value as vessel
+-[ RECORD 1 ]------+------------------------------------
+depthkey           | "environment.depth.belowTransducer"
+update_at_not_null | t
+
+api.metadata get configuration base on update_at value as vessel
+-[ RECORD 1 ]------+------------------------------------
+depthkey           | "environment.depth.belowTransducer"
+update_at_not_null | t
+
+SET
+api_anonymous get vessel image
+-[ RECORD 1 ]---------+--
+vessel_image_not_null | t
+

tests/sql/mobilitydb.sql

@@ -0,0 +1,96 @@
+---------------------------------------------------------------------------
+-- Listing
+--
+
+-- List current database
+select current_database();
+
+-- connect to the DB
+\c signalk
+
+-- output display format
+\x on
+
+-- Assign vessel_id var
+SELECT v.vessel_id as "vessel_id_kapla" FROM auth.vessels v WHERE v.owner_email = 'demo+kapla@openplotter.cloud' \gset
+SELECT v.vessel_id as "vessel_id_aava" FROM auth.vessels v WHERE v.owner_email = 'demo+aava@openplotter.cloud' \gset
+
+-- user_role
+SET ROLE user_role;
+-- Switch user as aava
+SELECT set_config('vessel.id', :'vessel_id_aava', false) IS NOT NULL as vessel_id;
+
+-- Update notes
+\echo 'Add a note for an entry from a trip'
+-- Get original value, should be empty
+SELECT numInstants(trip), valueAtTimestamp(trip_notes,timestampN(trip,13)) from api.logbook where id = 3;
+-- Create the string
+SELECT concat('["fishing"@', timestampN(trip,13),',""@',timestampN(trip,14),']') as to_be_update FROM api.logbook where id = 3 \gset
+--\echo :to_be_update
+-- Update the notes
+SELECT api.update_trip_notes_fn(3, :'to_be_update');
+-- Compare with previous value, should include "fishing"
+SELECT valueAtTimestamp(trip_notes,timestampN(trip,13)) from api.logbook where id = 3;
+
+-- Delete notes
+\echo 'Delete an entry from a trip'
+-- Get original value, should be 45
+SELECT numInstants(trip), jsonb_array_length(api.export_logbook_geojson_point_trip_fn(id)->'features') from api.logbook where id = 3;
+-- Extract the timestamps of the invalid coords
+--SELECT timestampN(trip,14) as "to_be_delete" FROM api.logbook where id = 3 \gset
+SELECT concat('[', timestampN(trip,14),',',timestampN(trip,15),')') as to_be_delete FROM api.logbook where id = 3 \gset
+--\echo :to_be_delete
+-- Delete the entry for all trip sequence
+SELECT api.delete_trip_entry_fn(3, :'to_be_delete');
+-- Compare with previous value, should be 44
+SELECT numInstants(trip), jsonb_array_length(api.export_logbook_geojson_point_trip_fn(id)->'features') from api.logbook where id = 3;
+
+-- Export PostGIS geography from a trip
+\echo 'Export PostGIS geography from trajectory'
+--SELECT ST_IsValid(trajectory(trip)::geometry) IS TRUE FROM api.logbook WHERE vessel_id = current_setting('vessel.id', false);
+SELECT trajectory(trip)::geometry FROM api.logbook WHERE id = 3;
+
+-- Export GeoJSON from a trip
+\echo 'Export GeoJSON with properties from a trip'
+SELECT jsonb_array_length(api.export_logbook_geojson_point_trip_fn(3)->'features');
+
+-- Export GPX from a trip
+\echo 'Export GPX from a trip'
+SELECT api.export_logbook_gpx_trip_fn(3) IS NOT NULL;
+
+-- Export KML from a trip
+\echo 'Export KML from a trip'
+SELECT api.export_logbook_kml_trip_fn(3) IS NOT NULL;
+
+-- Switch user as kapla
+SELECT set_config('vessel.id', :'vessel_id_kapla', false) IS NOT NULL as vessel_id;
+
+-- Export timelapse as Geometry LineString from a trip
+\echo 'Export timelapse as Geometry LineString from a trip'
+--SELECT api.export_logbooks_geojson_linestring_trips_fn(1,2) FROM api.logbook WHERE vessel_id = current_setting('vessel.id', false);
+-- Test geometry_type and num_properties
+-- propoerties include endtimestamp and starttimestamp
+WITH geojson_output AS (
+    SELECT api.export_logbooks_geojson_linestring_trips_fn(1, 2) AS geojson
+        FROM api.logbook
+        WHERE vessel_id = current_setting('vessel.id', false)
+)
+SELECT
+    --geojson
+    geojson->'features'->0->'geometry'->>'type' AS geometry_type, 
+    --jsonb_array_length(jsonb_object_keys(geojson->'features'->0->'properties')::JSONB),
+    --jsonb_array_length(jsonb_object_keys(geojson->'features')) AS num_geometry,
+    (SELECT COUNT(*) FROM jsonb_object_keys(geojson->'features'->0->'properties')) AS num_properties
+FROM geojson_output;
+
+-- Export timelapse as Geometry Point from a trip
+\echo 'Export timelapse as Geometry Point from a trip'
+SELECT api.export_logbooks_geojson_point_trips_fn(1,2) IS NOT NULL FROM api.logbook WHERE vessel_id = current_setting('vessel.id', false);
+
+-- Export GPX from trips
+\echo 'Export GPX from trips'
+SELECT api.export_logbooks_gpx_trips_fn(1,2) IS NOT NULL FROM api.logbook WHERE vessel_id = current_setting('vessel.id', false);
+
+-- Export KML from trips
+\echo 'Export KML from trips'
+SELECT api.export_logbooks_kml_trips_fn(1,2) IS NOT NULL FROM api.logbook WHERE vessel_id = current_setting('vessel.id', false);

tests/sql/mobilitydb.sql.output

@@ -0,0 +1,70 @@
+ current_database 
+------------------
+ signalk
+(1 row)
+
+You are now connected to database "signalk" as user "username".
+Expanded display is on.
+SET
+-[ RECORD 1 ]
+vessel_id | t
+
+Add a note for an entry from a trip
+-[ RECORD 1 ]----+---
+numinstants      | 45
+valueattimestamp | 
+
+-[ RECORD 1 ]--------+-
+update_trip_notes_fn | 
+
+-[ RECORD 1 ]----+--------
+valueattimestamp | fishing
+
+Delete an entry from a trip
+-[ RECORD 1 ]------+---
+numinstants        | 45
+jsonb_array_length | 45
+
+-[ RECORD 1 ]--------+-
+delete_trip_entry_fn | 
+
+-[ RECORD 1 ]------+---
+numinstants        | 44
+jsonb_array_length | 44
+
+Export PostGIS geography from trajectory
+-[ RECORD 1 ]----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+trajectory | 0102000020E61000002B0000001CACA4BA25BC3840217B18B556DC4D40569FABADD8BB384012EA33B10ADC4D408A65E9F989BB384018A023A8D0DB4D40B174F4AE30BB38409951E2299ADB4D407C348B06DFBA3840443A973D64DB4D40FE4465C39ABA38409313927131DB4D402C9CA4F963BA384029E6C52EF6DA4D4010559D7A49BA384019E25817B7DA4D401BF0F96184BC3840F868160DBEDC4D4095B7239C16BC38401DDB7C6D47DC4D40559FABADD8BB384012EA33B10ADC4D408A65E9F989BB384018A023A8D0DB4D40B274F4AE30BB38409951E2299ADB4D407C348B06DFBA3840443A973D64DB4D40FE4465C39ABA38409313927131DB4D402C9CA4F963BA384029E6C52EF6DA4D4010559D7A49BA384019E25817B7DA4D401BF0F96184BC3840F868160DBEDC4D4098B1B2C755BC38404D52F41B81DC4D4095B7239C16BC38401DDB7C6D47DC4D407448C55AD7BB38406CABFEAD09DC4D40D8367B5688BB38400D54C6BFCFDB4D40B274F4AE30BB38409951E2299ADB4D401256BEC2DDBA3840EB22E06B63DB4D40FE4465C39ABA38409313927131DB4D402C9CA4F963BA384029E6C52EF6DA4D40407D152A49BA3840CDA66D0DB6DA4D40BDBFE6C182BC3840F9269710BDDC4D4098B1B2C755BC38404D52F41B81DC4D4024308CAA15BC3840B85DC36746DC4D407448C55AD7BB38406CABFEAD09DC4D40D8367B5688BB38400D54C6BFCFDB4D408224A24E2FBB384070BEC74F99DB4D4028B0A5EC99BA3840F90C4D7E30DB4D402C4080B163BA38402FA93528F5DA4D40407D152A49BA3840CDA66D0DB6DA4D40BDBFE6C182BC3840F9269710BDDC4D4099B1B2C755BC38404D52F41B81DC4D4024308CAA15BC3840B85DC36746DC4D407448C55AD7BB38406CABFEAD09DC4D40D8367B5688BB38400D54C6BFCFDB4D408224A24E2FBB384070BEC74F99DB4D401156BEC2DDBA3840EB22E06B63DB4D40
+
+Export GeoJSON with properties from a trip
+-[ RECORD 1 ]------+---
+jsonb_array_length | 44
+
+Export GPX from a trip
+-[ RECORD 1 ]
+?column? | t
+
+Export KML from a trip
+-[ RECORD 1 ]
+?column? | t
+
+-[ RECORD 1 ]
+vessel_id | t
+
+Export timelapse as Geometry LineString from a trip
+-[ RECORD 1 ]--+-----------
+geometry_type  | LineString
+num_properties | 35
+
+Export timelapse as Geometry Point from a trip
+-[ RECORD 1 ]
+?column? | t
+
+Export GPX from trips
+-[ RECORD 1 ]
+?column? | t
+
+Export KML from trips
+-[ RECORD 1 ]
+?column? | t
+

tests/sql/qgis.sql

@@ -49,4 +49,5 @@ SELECT public.qgis_bbox_trip_py_fn(CONCAT(:'vessel_id_aava'::TEXT, '_', 3, '_',
 --SELECT set_config('vessel.id', :'vessel_id_kapla', false) IS NOT NULL as vessel_id;
 -- SQL request from QGIS to fetch the necessary data base on vessel_id
 --SELECT id, vessel_id, name as logname, ST_Transform(track_geom, 3857) as track_geom, ROUND(distance, 2) as distance, ROUND(EXTRACT(epoch FROM duration)/3600,2) as duration,_from_time,_to_time FROM api.logbook where track_geom is not null and _to_time is not null ORDER BY _from_time DESC;
-SELECT count(*) FROM api.logbook where track_geom is not null and _to_time is not null;
+--SELECT count(*) FROM api.logbook WHERE track_geom IS NOT NULL AND _to_time iIS NOT NULL;
+SELECT count(*) FROM api.logbook WHERE trip IS NOT NULL AND _to_time IS NOT NULL;

tests/sql/qgis.sql.output

@@ -11,35 +11,35 @@ Get BBOX Extent from SQL query for a log: "^/log_(w+)_(d+).png$"
 qgis_bbox_py_fn | 2556155.0636042403,8365608,2660086.9363957597,8420076
 
 -[ RECORD 1 ]---+----------------------------------------------------
-qgis_bbox_py_fn | 2745681,8303937.662962963,2871529,8369891.337037037
+qgis_bbox_py_fn | 2749398.035335689,8334944,2756917.964664311,8338885
 
 Get BBOX Extent from SQL query for a log as line: "^/log_(w+)_(d+)_line.png$"
 -[ RECORD 1 ]---+-------------------------------------------------------------------------
 qgis_bbox_py_fn | 2570800.6277114027,8368634.173700442,2645441.4677270483,8417049.85371059
 
--[ RECORD 1 ]---+-----------------------------------------------------------------------
-qgis_bbox_py_fn | 2752672.6236475753,8300633.73408079,2864537.04561218,8373194.440219993
+-[ RECORD 1 ]---+--------------------------------------------------------------------------
+qgis_bbox_py_fn | 2750457.4431765806,8335162.530580978,2755858.0759322727,8338665.643719805
 
 Get BBOX Extent from SQL query for all logs by vessel_id: "^/logs_(w+)_(d+).png$"
 -[ RECORD 1 ]---+------------------------------------------------------
 qgis_bbox_py_fn | 2556155.0636042403,8365608,2660086.9363957597,8420076
 
 -[ RECORD 1 ]---+------------------------------------------------------
-qgis_bbox_py_fn | -1950837.4558303887,4864146,5068977.455830389,8543049
+qgis_bbox_py_fn | -2006284.4558303887,4864146,5013530.455830389,8543049
 
 Get BBOX Extent from SQL query for a trip by vessel_id: "^/trip_(w+)_(d+)_(d+).png$"
 -[ RECORD 1 ]---+-------------------------------------
 qgis_bbox_py_fn | 2595383,4787988.0,2620859,11997696.0
 
 -[ RECORD 1 ]---+---------------------------------------
-qgis_bbox_py_fn | 90420,-201110377.5,3027720,214517572.5
+qgis_bbox_py_fn | 97351,-192283890.5,2909895,205691085.5
 
 Get BBOX Extent from SQL query for a trip by vessel_id: "^/trip_((w+)_(d+)_(d+)).png$"
 -[ RECORD 1 ]--------+------------------------------------------------------
 qgis_bbox_trip_py_fn | 2556155.0636042403,8365608,2660086.9363957597,8420076
 
 -[ RECORD 1 ]--------+------------------------------------------------------
-qgis_bbox_trip_py_fn | -1950837.4558303887,4864146,5068977.455830389,8543049
+qgis_bbox_trip_py_fn | -2006284.4558303887,4864146,5013530.455830389,8543049
 
 -[ RECORD 1 ]
 count | 3

tests/sql/stays_ext.sql

@@ -0,0 +1,47 @@
+---------------------------------------------------------------------------
+-- Listing
+--
+
+-- List current database
+select current_database();
+
+-- connect to the DB
+\c signalk
+
+-- output display format
+\x on
+
+SELECT count(*) as count_eq_0 FROM api.stays_ext m;
+
+SELECT v.vessel_id as "vessel_id" FROM auth.vessels v WHERE v.owner_email = 'demo+kapla@openplotter.cloud' \gset
+--\echo :"vessel_id"
+SELECT set_config('vessel.id', :'vessel_id', false) IS NOT NULL as vessel_id;
+
+-- user_role
+SET ROLE user_role;
+
+\echo 'api.stays details'
+SELECT vessel_id IS NOT NULL AS vessel_id_not_null, m.name IS NOT NULL AS name_not_null FROM api.stays AS m WHERE active IS False ORDER BY m.name ASC;
+
+-- Upsert image on stays_ext table
+\echo 'api.stays_ext set image/image_b64'
+INSERT INTO api.stays_ext (vessel_id, stay_id, image_b64)
+    VALUES (current_setting('vessel.id', false), 1, 'iVBORw0KGgoAAAANSUhEUgAAAMgAAAAyCAIAAACWMwO2AAABNklEQVR4nO3bwY6CMBiF0XYy7//KzIKk6VBjiMMNk59zVljRIH6WsrBv29bgal93HwA1CYsIYREhLCKERYSwiBAWEcIiQlhECIsIYREhLCKERYSwiBAWEcIiQlhECIsIYREhLCK+7z6A/6j33lq75G8m')
+    ON CONFLICT (stay_id) DO UPDATE
+    SET image_b64 = EXCLUDED.image_b64;
+
+-- Ensure image_updated_at on metadata_ext table is updated by trigger
+\echo 'api.stays_ext get image_updated_at'
+SELECT image_b64 IS NULL AS image_b64_is_null,image IS NOT NULL AS image_not_null,image_updated_at IS NOT NULL AS image_updated_at_not_null FROM api.metadata_ext; --WHERE vessel_id = current_setting('vessel.id', false);
+
+-- vessel_role
+SET ROLE vessel_role;
+
+\echo 'api.stays_ext'
+SELECT vessel_id IS NOT NULL AS vessel_id_not_null, stay_id FROM api.stays_ext;
+
+-- api_anonymous
+SET ROLE api_anonymous;
+
+\echo 'api_anonymous get stays image'
+SELECT api.stays_image(current_setting('vessel.id', false), 1) IS NOT NULL AS stays_image_not_null;

tests/sql/stays_ext.sql.output

@@ -0,0 +1,37 @@
+ current_database 
+------------------
+ signalk
+(1 row)
+
+You are now connected to database "signalk" as user "username".
+Expanded display is on.
+-[ RECORD 1 ]-
+count_eq_0 | 0
+
+-[ RECORD 1 ]
+vessel_id | t
+
+SET
+api.stays details
+-[ RECORD 1 ]------+--
+vessel_id_not_null | t
+name_not_null      | t
+-[ RECORD 2 ]------+--
+vessel_id_not_null | t
+name_not_null      | t
+
+api.stays_ext set image/image_b64
+INSERT 0 1
+api.stays_ext get image_updated_at
+-[ RECORD 1 ]-------------+--
+image_b64_is_null         | f
+image_not_null            | t
+image_updated_at_not_null | t
+
+SET
+api.stays_ext
+SET
+api_anonymous get stays image
+-[ RECORD 1 ]--------+--
+stays_image_not_null | t
+

tests/sql/summary.sql.output

@@ -5,11 +5,11 @@
 
 You are now connected to database "signalk" as user "username".
 Expanded display is on.
--[ RECORD 1 ]--+-------------------------------
-server_version | 16.4 (Debian 16.4-1.pgdg120+2)
+-[ RECORD 1 ]--+--------------------------------
+server_version | 16.10 (Debian 16.10-1.pgdg12+1)
 
--[ RECORD 1 ]--------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
-postgis_full_version | POSTGIS="3.5.0 d2c3ca4" [EXTENSION] PGSQL="160" GEOS="3.11.1-CAPI-1.17.1" PROJ="9.1.1 NETWORK_ENABLED=OFF URL_ENDPOINT=https://cdn.proj.org USER_WRITABLE_DIRECTORY=/var/lib/postgresql/.local/share/proj DATABASE_PATH=/usr/share/proj/proj.db" (compiled against PROJ 9.11.1) LIBXML="2.9.14" LIBJSON="0.16" LIBPROTOBUF="1.4.1" WAGYU="0.5.0 (Internal)"
+-[ RECORD 1 ]--------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+postgis_full_version | POSTGIS="3.5.3 aab5f55" [EXTENSION] PGSQL="160" GEOS="3.11.1-CAPI-1.17.1" PROJ="9.1.1 NETWORK_ENABLED=OFF URL_ENDPOINT=https://cdn.proj.org USER_WRITABLE_DIRECTORY=/var/lib/postgresql/.local/share/proj DATABASE_PATH=/usr/share/proj/proj.db" (compiled against PROJ 9.1.1) LIBXML="2.9.14" LIBJSON="0.16" LIBPROTOBUF="1.4.1" WAGYU="0.5.0 (Internal)"
 
 -[ RECORD 1 ]--------------------------------------------------------------------------------------
 Name        | citext
@@ -22,41 +22,51 @@ Version     | 1.0
 Schema      | public
 Description | transform between jsonb and plpython3u
 -[ RECORD 3 ]--------------------------------------------------------------------------------------
+Name        | mobilitydb
+Version     | 1.2.0
+Schema      | public
+Description | MobilityDB geospatial trajectory data management & analysis platform
+-[ RECORD 4 ]--------------------------------------------------------------------------------------
 Name        | moddatetime
 Version     | 1.0
 Schema      | public
 Description | functions for tracking last modification time
--[ RECORD 4 ]--------------------------------------------------------------------------------------
+-[ RECORD 5 ]--------------------------------------------------------------------------------------
 Name        | pg_stat_statements
 Version     | 1.10
 Schema      | public
 Description | track planning and execution statistics of all SQL statements executed
--[ RECORD 5 ]--------------------------------------------------------------------------------------
+-[ RECORD 6 ]--------------------------------------------------------------------------------------
 Name        | pgcrypto
 Version     | 1.3
 Schema      | public
 Description | cryptographic functions
--[ RECORD 6 ]--------------------------------------------------------------------------------------
+-[ RECORD 7 ]--------------------------------------------------------------------------------------
 Name        | plpgsql
 Version     | 1.0
 Schema      | pg_catalog
 Description | PL/pgSQL procedural language
--[ RECORD 7 ]--------------------------------------------------------------------------------------
+-[ RECORD 8 ]--------------------------------------------------------------------------------------
 Name        | plpython3u
 Version     | 1.0
 Schema      | pg_catalog
 Description | PL/Python3U untrusted procedural language
--[ RECORD 8 ]--------------------------------------------------------------------------------------
+-[ RECORD 9 ]--------------------------------------------------------------------------------------
 Name        | postgis
-Version     | 3.5.0
+Version     | 3.5.3
 Schema      | public
 Description | PostGIS geometry and geography spatial types and functions
--[ RECORD 9 ]--------------------------------------------------------------------------------------
+-[ RECORD 10 ]-------------------------------------------------------------------------------------
 Name        | timescaledb
-Version     | 2.17.1
+Version     | 2.21.3
 Schema      | public
 Description | Enables scalable inserts and complex queries for time-series data (Community Edition)
--[ RECORD 10 ]-------------------------------------------------------------------------------------
+-[ RECORD 11 ]-------------------------------------------------------------------------------------
+Name        | timescaledb_toolkit
+Version     | 1.21.0
+Schema      | public
+Description | Library of analytical hyperfunctions, time-series pipelining, and other SQL utilities
+-[ RECORD 12 ]-------------------------------------------------------------------------------------
 Name        | uuid-ossp
 Version     | 1.1
 Schema      | public
@@ -106,14 +116,14 @@ laninline     | 13566
 lanvalidator  | 13567
 lanacl        | 
 -[ RECORD 5 ]-+-----------
-oid           | 18190
+oid           | 18251
 lanname       | plpython3u
 lanowner      | 10
 lanispl       | t
 lanpltrusted  | t
-lanplcallfoid | 18187
-laninline     | 18188
-lanvalidator  | 18189
+lanplcallfoid | 18248
+laninline     | 18249
+lanvalidator  | 18250
 lanacl        | 
 
 -[ RECORD 1 ]+-----------
@@ -180,42 +190,51 @@ Type   | table
 Owner  | username
 -[ RECORD 8 ]---------------------------------
 Schema | public
-Name   | ne_10m_geography_marine_polys
+Name   | mobilitydb_opcache
 Type   | table
 Owner  | username
 -[ RECORD 9 ]---------------------------------
 Schema | public
+Name   | ne_10m_geography_marine_polys
+Type   | table
+Owner  | username
+-[ RECORD 10 ]--------------------------------
+Schema | public
 Name   | ne_10m_geography_marine_polys_gid_seq
 Type   | sequence
 Owner  | username
--[ RECORD 10 ]--------------------------------
+-[ RECORD 11 ]--------------------------------
 Schema | public
 Name   | process_queue
 Type   | table
 Owner  | username
--[ RECORD 11 ]--------------------------------
+-[ RECORD 12 ]--------------------------------
 Schema | public
 Name   | process_queue_id_seq
 Type   | sequence
 Owner  | username
--[ RECORD 12 ]--------------------------------
+-[ RECORD 13 ]--------------------------------
 Schema | public
 Name   | spatial_ref_sys
 Type   | table
 Owner  | username
 
--[ RECORD 1 ]--------
+-[ RECORD 1 ]------------
 schema_api | logbook
--[ RECORD 2 ]--------
+-[ RECORD 2 ]------------
 schema_api | metadata
--[ RECORD 3 ]--------
+-[ RECORD 3 ]------------
+schema_api | metadata_ext
+-[ RECORD 4 ]------------
 schema_api | metrics
--[ RECORD 4 ]--------
+-[ RECORD 5 ]------------
 schema_api | moorages
--[ RECORD 5 ]--------
+-[ RECORD 6 ]------------
 schema_api | stays
--[ RECORD 6 ]--------
+-[ RECORD 7 ]------------
 schema_api | stays_at
+-[ RECORD 8 ]------------
+schema_api | stays_ext
 
 -[ RECORD 1 ]-+------------------------------
 schema_public | aistypes
@@ -232,10 +251,12 @@ schema_public | iso3166
 -[ RECORD 7 ]-+------------------------------
 schema_public | mid
 -[ RECORD 8 ]-+------------------------------
-schema_public | ne_10m_geography_marine_polys
+schema_public | mobilitydb_opcache
 -[ RECORD 9 ]-+------------------------------
-schema_public | process_queue
+schema_public | ne_10m_geography_marine_polys
 -[ RECORD 10 ]+------------------------------
+schema_public | process_queue
+-[ RECORD 11 ]+------------------------------
 schema_public | spatial_ref_sys
 
 -[ RECORD 1 ]---------
@@ -262,31 +283,13 @@ with_check | true
 -[ RECORD 2 ]------------------------------------------------------------------------------------------------------------------------------
 schemaname | api
 tablename  | metadata
-policyname | api_vessel_role
-permissive | PERMISSIVE
-roles      | {vessel_role}
-cmd        | ALL
-qual       | (vessel_id = current_setting('vessel.id'::text, false))
-with_check | true
--[ RECORD 3 ]------------------------------------------------------------------------------------------------------------------------------
-schemaname | api
-tablename  | metadata
-policyname | api_user_role
-permissive | PERMISSIVE
-roles      | {user_role}
-cmd        | ALL
-qual       | (vessel_id = current_setting('vessel.id'::text, true))
-with_check | (vessel_id = current_setting('vessel.id'::text, false))
--[ RECORD 4 ]------------------------------------------------------------------------------------------------------------------------------
-schemaname | api
-tablename  | metadata
 policyname | api_scheduler_role
 permissive | PERMISSIVE
 roles      | {scheduler}
 cmd        | ALL
 qual       | (vessel_id = current_setting('vessel.id'::text, false))
 with_check | (vessel_id = current_setting('vessel.id'::text, false))
--[ RECORD 5 ]------------------------------------------------------------------------------------------------------------------------------
+-[ RECORD 3 ]------------------------------------------------------------------------------------------------------------------------------
 schemaname | api
 tablename  | metadata
 policyname | grafana_role
@@ -295,7 +298,7 @@ roles      | {grafana}
 cmd        | ALL
 qual       | (vessel_id = current_setting('vessel.id'::text, false))
 with_check | false
--[ RECORD 6 ]------------------------------------------------------------------------------------------------------------------------------
+-[ RECORD 4 ]------------------------------------------------------------------------------------------------------------------------------
 schemaname | api
 tablename  | metadata
 policyname | grafana_proxy_role
@@ -304,52 +307,34 @@ roles      | {grafana_auth}
 cmd        | ALL
 qual       | true
 with_check | false
+-[ RECORD 5 ]------------------------------------------------------------------------------------------------------------------------------
+schemaname | api
+tablename  | metrics
+policyname | admin_all
+permissive | PERMISSIVE
+roles      | {username}
+cmd        | ALL
+qual       | true
+with_check | true
+-[ RECORD 6 ]------------------------------------------------------------------------------------------------------------------------------
+schemaname | auth
+tablename  | vessels
+policyname | grafana_proxy_role
+permissive | PERMISSIVE
+roles      | {grafana_auth}
+cmd        | ALL
+qual       | true
+with_check | false
 -[ RECORD 7 ]------------------------------------------------------------------------------------------------------------------------------
 schemaname | api
 tablename  | metrics
-policyname | admin_all
-permissive | PERMISSIVE
-roles      | {username}
-cmd        | ALL
-qual       | true
-with_check | true
--[ RECORD 8 ]------------------------------------------------------------------------------------------------------------------------------
-schemaname | api
-tablename  | metrics
-policyname | api_vessel_role
-permissive | PERMISSIVE
-roles      | {vessel_role}
-cmd        | ALL
-qual       | (vessel_id = current_setting('vessel.id'::text, false))
-with_check | true
--[ RECORD 9 ]------------------------------------------------------------------------------------------------------------------------------
-schemaname | auth
-tablename  | vessels
-policyname | grafana_proxy_role
-permissive | PERMISSIVE
-roles      | {grafana_auth}
-cmd        | ALL
-qual       | true
-with_check | false
--[ RECORD 10 ]-----------------------------------------------------------------------------------------------------------------------------
-schemaname | api
-tablename  | metrics
-policyname | api_user_role
-permissive | PERMISSIVE
-roles      | {user_role}
-cmd        | ALL
-qual       | (vessel_id = current_setting('vessel.id'::text, true))
-with_check | (vessel_id = current_setting('vessel.id'::text, false))
--[ RECORD 11 ]-----------------------------------------------------------------------------------------------------------------------------
-schemaname | api
-tablename  | metrics
 policyname | api_scheduler_role
 permissive | PERMISSIVE
 roles      | {scheduler}
 cmd        | ALL
 qual       | (vessel_id = current_setting('vessel.id'::text, false))
 with_check | (vessel_id = current_setting('vessel.id'::text, false))
--[ RECORD 12 ]-----------------------------------------------------------------------------------------------------------------------------
+-[ RECORD 8 ]------------------------------------------------------------------------------------------------------------------------------
 schemaname | api
 tablename  | metrics
 policyname | grafana_role
@@ -358,7 +343,7 @@ roles      | {grafana}
 cmd        | ALL
 qual       | (vessel_id = current_setting('vessel.id'::text, false))
 with_check | false
--[ RECORD 13 ]-----------------------------------------------------------------------------------------------------------------------------
+-[ RECORD 9 ]------------------------------------------------------------------------------------------------------------------------------
 schemaname | api
 tablename  | metrics
 policyname | api_anonymous_role
@@ -367,7 +352,7 @@ roles      | {api_anonymous}
 cmd        | ALL
 qual       | (vessel_id = current_setting('vessel.id'::text, false))
 with_check | false
--[ RECORD 14 ]-----------------------------------------------------------------------------------------------------------------------------
+-[ RECORD 10 ]-----------------------------------------------------------------------------------------------------------------------------
 schemaname | api
 tablename  | logbook
 policyname | admin_all
@@ -376,7 +361,7 @@ roles      | {username}
 cmd        | ALL
 qual       | true
 with_check | true
--[ RECORD 15 ]-----------------------------------------------------------------------------------------------------------------------------
+-[ RECORD 11 ]-----------------------------------------------------------------------------------------------------------------------------
 schemaname | api
 tablename  | logbook
 policyname | api_vessel_role
@@ -385,7 +370,7 @@ roles      | {vessel_role}
 cmd        | ALL
 qual       | (vessel_id = current_setting('vessel.id'::text, false))
 with_check | true
--[ RECORD 16 ]-----------------------------------------------------------------------------------------------------------------------------
+-[ RECORD 12 ]-----------------------------------------------------------------------------------------------------------------------------
 schemaname | auth
 tablename  | accounts
 policyname | admin_all
@@ -394,7 +379,7 @@ roles      | {username}
 cmd        | ALL
 qual       | true
 with_check | true
--[ RECORD 17 ]-----------------------------------------------------------------------------------------------------------------------------
+-[ RECORD 13 ]-----------------------------------------------------------------------------------------------------------------------------
 schemaname | api
 tablename  | logbook
 policyname | api_user_role
@@ -403,7 +388,7 @@ roles      | {user_role}
 cmd        | ALL
 qual       | (vessel_id = current_setting('vessel.id'::text, true))
 with_check | (vessel_id = current_setting('vessel.id'::text, false))
--[ RECORD 18 ]-----------------------------------------------------------------------------------------------------------------------------
+-[ RECORD 14 ]-----------------------------------------------------------------------------------------------------------------------------
 schemaname | api
 tablename  | logbook
 policyname | api_scheduler_role
@@ -412,7 +397,7 @@ roles      | {scheduler}
 cmd        | ALL
 qual       | (vessel_id = current_setting('vessel.id'::text, false))
 with_check | (vessel_id = current_setting('vessel.id'::text, false))
--[ RECORD 19 ]-----------------------------------------------------------------------------------------------------------------------------
+-[ RECORD 15 ]-----------------------------------------------------------------------------------------------------------------------------
 schemaname | api
 tablename  | logbook
 policyname | grafana_role
@@ -421,7 +406,7 @@ roles      | {grafana}
 cmd        | ALL
 qual       | (vessel_id = current_setting('vessel.id'::text, false))
 with_check | false
--[ RECORD 20 ]-----------------------------------------------------------------------------------------------------------------------------
+-[ RECORD 16 ]-----------------------------------------------------------------------------------------------------------------------------
 schemaname | api
 tablename  | logbook
 policyname | api_anonymous_role
@@ -430,7 +415,7 @@ roles      | {api_anonymous}
 cmd        | ALL
 qual       | (vessel_id = current_setting('vessel.id'::text, false))
 with_check | false
--[ RECORD 21 ]-----------------------------------------------------------------------------------------------------------------------------
+-[ RECORD 17 ]-----------------------------------------------------------------------------------------------------------------------------
 schemaname | api
 tablename  | stays
 policyname | admin_all
@@ -439,7 +424,7 @@ roles      | {username}
 cmd        | ALL
 qual       | true
 with_check | true
--[ RECORD 22 ]-----------------------------------------------------------------------------------------------------------------------------
+-[ RECORD 18 ]-----------------------------------------------------------------------------------------------------------------------------
 schemaname | api
 tablename  | stays
 policyname | api_vessel_role
@@ -448,7 +433,7 @@ roles      | {vessel_role}
 cmd        | ALL
 qual       | (vessel_id = current_setting('vessel.id'::text, false))
 with_check | true
--[ RECORD 23 ]-----------------------------------------------------------------------------------------------------------------------------
+-[ RECORD 19 ]-----------------------------------------------------------------------------------------------------------------------------
 schemaname | api
 tablename  | logbook
 policyname | logbook_qgis_role
@@ -457,7 +442,7 @@ roles      | {qgis_role}
 cmd        | ALL
 qual       | true
 with_check | false
--[ RECORD 24 ]-----------------------------------------------------------------------------------------------------------------------------
+-[ RECORD 20 ]-----------------------------------------------------------------------------------------------------------------------------
 schemaname | public
 tablename  | process_queue
 policyname | public_maplapse_role
@@ -466,7 +451,7 @@ roles      | {maplapse_role}
 cmd        | ALL
 qual       | true
 with_check | true
--[ RECORD 25 ]-----------------------------------------------------------------------------------------------------------------------------
+-[ RECORD 21 ]-----------------------------------------------------------------------------------------------------------------------------
 schemaname | api
 tablename  | stays
 policyname | api_user_role
@@ -475,7 +460,7 @@ roles      | {user_role}
 cmd        | ALL
 qual       | (vessel_id = current_setting('vessel.id'::text, true))
 with_check | (vessel_id = current_setting('vessel.id'::text, false))
--[ RECORD 26 ]-----------------------------------------------------------------------------------------------------------------------------
+-[ RECORD 22 ]-----------------------------------------------------------------------------------------------------------------------------
 schemaname | api
 tablename  | stays
 policyname | api_scheduler_role
@@ -484,7 +469,7 @@ roles      | {scheduler}
 cmd        | ALL
 qual       | (vessel_id = current_setting('vessel.id'::text, false))
 with_check | (vessel_id = current_setting('vessel.id'::text, false))
--[ RECORD 27 ]-----------------------------------------------------------------------------------------------------------------------------
+-[ RECORD 23 ]-----------------------------------------------------------------------------------------------------------------------------
 schemaname | api
 tablename  | stays
 policyname | grafana_role
@@ -493,7 +478,7 @@ roles      | {grafana}
 cmd        | ALL
 qual       | (vessel_id = current_setting('vessel.id'::text, false))
 with_check | false
--[ RECORD 28 ]-----------------------------------------------------------------------------------------------------------------------------
+-[ RECORD 24 ]-----------------------------------------------------------------------------------------------------------------------------
 schemaname | api
 tablename  | stays
 policyname | api_anonymous_role
@@ -502,7 +487,7 @@ roles      | {api_anonymous}
 cmd        | ALL
 qual       | (vessel_id = current_setting('vessel.id'::text, false))
 with_check | false
--[ RECORD 29 ]-----------------------------------------------------------------------------------------------------------------------------
+-[ RECORD 25 ]-----------------------------------------------------------------------------------------------------------------------------
 schemaname | api
 tablename  | moorages
 policyname | admin_all
@@ -511,7 +496,7 @@ roles      | {username}
 cmd        | ALL
 qual       | true
 with_check | true
--[ RECORD 30 ]-----------------------------------------------------------------------------------------------------------------------------
+-[ RECORD 26 ]-----------------------------------------------------------------------------------------------------------------------------
 schemaname | api
 tablename  | moorages
 policyname | api_vessel_role
@@ -520,7 +505,16 @@ roles      | {vessel_role}
 cmd        | ALL
 qual       | (vessel_id = current_setting('vessel.id'::text, false))
 with_check | true
--[ RECORD 31 ]-----------------------------------------------------------------------------------------------------------------------------
+-[ RECORD 27 ]-----------------------------------------------------------------------------------------------------------------------------
+schemaname | api
+tablename  | stays_ext
+policyname | admin_all
+permissive | PERMISSIVE
+roles      | {username}
+cmd        | ALL
+qual       | true
+with_check | true
+-[ RECORD 28 ]-----------------------------------------------------------------------------------------------------------------------------
 schemaname | api
 tablename  | moorages
 policyname | api_user_role
@@ -529,7 +523,7 @@ roles      | {user_role}
 cmd        | ALL
 qual       | (vessel_id = current_setting('vessel.id'::text, true))
 with_check | (vessel_id = current_setting('vessel.id'::text, false))
--[ RECORD 32 ]-----------------------------------------------------------------------------------------------------------------------------
+-[ RECORD 29 ]-----------------------------------------------------------------------------------------------------------------------------
 schemaname | api
 tablename  | moorages
 policyname | api_scheduler_role
@@ -538,7 +532,7 @@ roles      | {scheduler}
 cmd        | ALL
 qual       | (vessel_id = current_setting('vessel.id'::text, false))
 with_check | (vessel_id = current_setting('vessel.id'::text, false))
--[ RECORD 33 ]-----------------------------------------------------------------------------------------------------------------------------
+-[ RECORD 30 ]-----------------------------------------------------------------------------------------------------------------------------
 schemaname | api
 tablename  | moorages
 policyname | grafana_role
@@ -547,7 +541,7 @@ roles      | {grafana}
 cmd        | ALL
 qual       | (vessel_id = current_setting('vessel.id'::text, false))
 with_check | false
--[ RECORD 34 ]-----------------------------------------------------------------------------------------------------------------------------
+-[ RECORD 31 ]-----------------------------------------------------------------------------------------------------------------------------
 schemaname | api
 tablename  | moorages
 policyname | api_anonymous_role
@@ -556,7 +550,7 @@ roles      | {api_anonymous}
 cmd        | ALL
 qual       | (vessel_id = current_setting('vessel.id'::text, false))
 with_check | false
--[ RECORD 35 ]-----------------------------------------------------------------------------------------------------------------------------
+-[ RECORD 32 ]-----------------------------------------------------------------------------------------------------------------------------
 schemaname | auth
 tablename  | vessels
 policyname | admin_all
@@ -565,7 +559,7 @@ roles      | {username}
 cmd        | ALL
 qual       | true
 with_check | true
--[ RECORD 36 ]-----------------------------------------------------------------------------------------------------------------------------
+-[ RECORD 33 ]-----------------------------------------------------------------------------------------------------------------------------
 schemaname | auth
 tablename  | vessels
 policyname | api_user_role
@@ -574,7 +568,7 @@ roles      | {user_role}
 cmd        | ALL
 qual       | ((vessel_id = current_setting('vessel.id'::text, true)) AND ((owner_email)::text = current_setting('user.email'::text, true)))
 with_check | ((vessel_id = current_setting('vessel.id'::text, true)) AND ((owner_email)::text = current_setting('user.email'::text, true)))
--[ RECORD 37 ]-----------------------------------------------------------------------------------------------------------------------------
+-[ RECORD 34 ]-----------------------------------------------------------------------------------------------------------------------------
 schemaname | auth
 tablename  | vessels
 policyname | grafana_role
@@ -583,7 +577,7 @@ roles      | {grafana}
 cmd        | ALL
 qual       | ((owner_email)::text = current_setting('user.email'::text, true))
 with_check | false
--[ RECORD 38 ]-----------------------------------------------------------------------------------------------------------------------------
+-[ RECORD 35 ]-----------------------------------------------------------------------------------------------------------------------------
 schemaname | auth
 tablename  | accounts
 policyname | api_user_role
@@ -592,7 +586,7 @@ roles      | {user_role}
 cmd        | ALL
 qual       | ((email)::text = current_setting('user.email'::text, true))
 with_check | ((email)::text = current_setting('user.email'::text, true))
--[ RECORD 39 ]-----------------------------------------------------------------------------------------------------------------------------
+-[ RECORD 36 ]-----------------------------------------------------------------------------------------------------------------------------
 schemaname | auth
 tablename  | accounts
 policyname | api_scheduler_role
@@ -601,7 +595,7 @@ roles      | {scheduler}
 cmd        | ALL
 qual       | ((email)::text = current_setting('user.email'::text, true))
 with_check | ((email)::text = current_setting('user.email'::text, true))
--[ RECORD 40 ]-----------------------------------------------------------------------------------------------------------------------------
+-[ RECORD 37 ]-----------------------------------------------------------------------------------------------------------------------------
 schemaname | auth
 tablename  | accounts
 policyname | grafana_proxy_role
@@ -610,7 +604,7 @@ roles      | {grafana_auth}
 cmd        | ALL
 qual       | true
 with_check | false
--[ RECORD 41 ]-----------------------------------------------------------------------------------------------------------------------------
+-[ RECORD 38 ]-----------------------------------------------------------------------------------------------------------------------------
 schemaname | public
 tablename  | process_queue
 policyname | admin_all
@@ -619,7 +613,7 @@ roles      | {username}
 cmd        | ALL
 qual       | true
 with_check | true
--[ RECORD 42 ]-----------------------------------------------------------------------------------------------------------------------------
+-[ RECORD 39 ]-----------------------------------------------------------------------------------------------------------------------------
 schemaname | public
 tablename  | process_queue
 policyname | api_vessel_role
@@ -628,7 +622,7 @@ roles      | {vessel_role}
 cmd        | ALL
 qual       | ((ref_id = current_setting('user.id'::text, true)) OR (ref_id = current_setting('vessel.id'::text, true)))
 with_check | true
--[ RECORD 43 ]-----------------------------------------------------------------------------------------------------------------------------
+-[ RECORD 40 ]-----------------------------------------------------------------------------------------------------------------------------
 schemaname | public
 tablename  | process_queue
 policyname | api_user_role
@@ -637,7 +631,7 @@ roles      | {user_role}
 cmd        | ALL
 qual       | ((ref_id = current_setting('user.id'::text, true)) OR (ref_id = current_setting('vessel.id'::text, true)))
 with_check | ((ref_id = current_setting('user.id'::text, true)) OR (ref_id = current_setting('vessel.id'::text, true)))
--[ RECORD 44 ]-----------------------------------------------------------------------------------------------------------------------------
+-[ RECORD 41 ]-----------------------------------------------------------------------------------------------------------------------------
 schemaname | public
 tablename  | process_queue
 policyname | api_scheduler_role
@@ -646,6 +640,105 @@ roles      | {scheduler}
 cmd        | ALL
 qual       | true
 with_check | false
+-[ RECORD 42 ]-----------------------------------------------------------------------------------------------------------------------------
+schemaname | api
+tablename  | stays_ext
+policyname | api_user_role
+permissive | PERMISSIVE
+roles      | {user_role}
+cmd        | ALL
+qual       | (vessel_id = current_setting('vessel.id'::text, false))
+with_check | (vessel_id = current_setting('vessel.id'::text, false))
+-[ RECORD 43 ]-----------------------------------------------------------------------------------------------------------------------------
+schemaname | api
+tablename  | stays_ext
+policyname | api_anonymous_role
+permissive | PERMISSIVE
+roles      | {api_anonymous}
+cmd        | ALL
+qual       | true
+with_check | false
+-[ RECORD 44 ]-----------------------------------------------------------------------------------------------------------------------------
+schemaname | api
+tablename  | stays_ext
+policyname | api_vessel_role
+permissive | PERMISSIVE
+roles      | {vessel_role}
+cmd        | ALL
+qual       | false
+with_check | false
+-[ RECORD 45 ]-----------------------------------------------------------------------------------------------------------------------------
+schemaname | api
+tablename  | metadata_ext
+policyname | admin_all
+permissive | PERMISSIVE
+roles      | {username}
+cmd        | ALL
+qual       | true
+with_check | true
+-[ RECORD 46 ]-----------------------------------------------------------------------------------------------------------------------------
+schemaname | api
+tablename  | metadata_ext
+policyname | api_user_role
+permissive | PERMISSIVE
+roles      | {user_role}
+cmd        | ALL
+qual       | (vessel_id = current_setting('vessel.id'::text, false))
+with_check | (vessel_id = current_setting('vessel.id'::text, false))
+-[ RECORD 47 ]-----------------------------------------------------------------------------------------------------------------------------
+schemaname | api
+tablename  | metadata_ext
+policyname | api_anonymous_role
+permissive | PERMISSIVE
+roles      | {api_anonymous}
+cmd        | ALL
+qual       | true
+with_check | false
+-[ RECORD 48 ]-----------------------------------------------------------------------------------------------------------------------------
+schemaname | api
+tablename  | metadata_ext
+policyname | api_vessel_role
+permissive | PERMISSIVE
+roles      | {vessel_role}
+cmd        | ALL
+qual       | false
+with_check | false
+-[ RECORD 49 ]-----------------------------------------------------------------------------------------------------------------------------
+schemaname | api
+tablename  | metrics
+policyname | api_vessel_role
+permissive | PERMISSIVE
+roles      | {vessel_role}
+cmd        | ALL
+qual       | (vessel_id = current_setting('vessel.id'::text, false))
+with_check | (vessel_id = current_setting('vessel.id'::text, false))
+-[ RECORD 50 ]-----------------------------------------------------------------------------------------------------------------------------
+schemaname | api
+tablename  | metadata
+policyname | api_vessel_role
+permissive | PERMISSIVE
+roles      | {vessel_role}
+cmd        | ALL
+qual       | (vessel_id = current_setting('vessel.id'::text, false))
+with_check | (vessel_id = current_setting('vessel.id'::text, false))
+-[ RECORD 51 ]-----------------------------------------------------------------------------------------------------------------------------
+schemaname | api
+tablename  | metadata
+policyname | api_user_role
+permissive | PERMISSIVE
+roles      | {user_role}
+cmd        | ALL
+qual       | (vessel_id = current_setting('vessel.id'::text, false))
+with_check | (vessel_id = current_setting('vessel.id'::text, false))
+-[ RECORD 52 ]-----------------------------------------------------------------------------------------------------------------------------
+schemaname | api
+tablename  | metrics
+policyname | api_user_role
+permissive | PERMISSIVE
+roles      | {user_role}
+cmd        | ALL
+qual       | (vessel_id = current_setting('vessel.id'::text, false))
+with_check | (vessel_id = current_setting('vessel.id'::text, false))
 
 Test nominatim reverse_geocode_py_fn
 -[ RECORD 1 ]---------+----------------------------------------
@@ -653,22 +746,22 @@ reverse_geocode_py_fn | {"name": "Spain", "country_code": "es"}
 
 Test geoip reverse_geoip_py_fn
 Test opverpass API overpass_py_fn
--[ RECORD 1 ]--+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
-overpass_py_fn | {"fee": "yes", "vhf": "09", "name": "Port Olímpic", "image": "https://i.imgur.com/1KQVeFV.jpeg", "phone": "+34 933561016", "leisure": "marina", "website": "https://portolimpic.barcelona/", "wikidata": "Q171204", "wikipedia": "ca:Port Olímpic de Barcelona", "check_date": "2024-09-16", "addr:street": "Moll de Xaloc", "power_supply": "yes", "seamark:type": "harbour", "addr:postcode": "08005", "internet_access": "wlan", "wikimedia_commons": "Category:Port Olímpic (Barcelona)", "sanitary_dump_station": "yes", "seamark:harbour:category": "marina"}
+-[ RECORD 1 ]--+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+overpass_py_fn | {"fee": "yes", "vhf": "09", "name": "Port Olímpic", "phone": "+34 933561016", "leisure": "marina", "website": "https://portolimpic.barcelona/", "wikidata": "Q171204", "panoramax": "b637d864-4a5f-4d56-a68e-6c81d2c70292", "wikipedia": "ca:Port Olímpic de Barcelona", "check_date": "2024-09-16", "addr:street": "Moll de Xaloc", "power_supply": "yes", "seamark:type": "harbour", "addr:postcode": "08005", "internet_access": "wlan", "wikimedia_commons": "Category:Port Olímpic (Barcelona)", "sanitary_dump_station": "yes", "seamark:harbour:category": "marina"}
 
 -[ RECORD 1 ]--+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
 overpass_py_fn | {"name": "Port de la Ginesta", "type": "multipolygon", "leisure": "marina", "name:ca": "Port de la Ginesta", "wikidata": "Q16621038", "wikipedia": "ca:Port Ginesta", "check_date": "2024-08-23"}
 
--[ RECORD 1 ]--+----------------------------------------------
-overpass_py_fn | {"name": "Norra hamnen", "leisure": "marina"}
+-[ RECORD 1 ]--+---------------------------------------------------------------------------------------------------------------
+overpass_py_fn | {"name": "Norra hamnen", "leisure": "marina", "seamark:type": "harbour", "seamark:harbour:category": "marina"}
 
--[ RECORD 1 ]----------------------------------------------------------------------------------------------------------------------------------------------
-versions_fn | {"api_version" : "0.7.8", "sys_version" : "PostgreSQL 16.4", "timescaledb" : "2.17.1", "postgis" : "3.5.0", "postgrest" : "PostgREST 12.2.3"}
+-[ RECORD 1 ]-----------------------------------------------------------------------------------------------------------------------------------------------------------------------
+versions_fn | {"api_version" : "0.9.3", "sys_version" : "PostgreSQL 16.10", "mobilitydb" : "1.2.0", "timescaledb" : "2.21.3", "postgis" : "3.5.3", "postgrest" : "PostgREST 13.0.5"}
 
 -[ RECORD 1 ]-----------------
-api_version | 0.7.8
-sys_version | PostgreSQL 16.4
-timescaledb | 2.17.1
-postgis     | 3.5.0
-postgrest   | PostgREST 12.2.3
+api_version | 0.9.3
+sys_version | PostgreSQL 16.10
+timescaledb | 2.21.3
+postgis     | 3.5.3
+postgrest   | PostgREST 13.0.5
 

tests/tests.sh

@@ -1,4 +1,4 @@
-# PostgSail Unit test 
+# PostgSail Unit test
 
 if [[ -z "${PGSAIL_DB_URI}" ]]; then
   echo "PGSAIL_DB_URI is undefined"
@@ -14,14 +14,6 @@ if [[ ! -x "/usr/bin/psql" ]]; then
     apt update && apt -y install postgresql-client
 fi
 
-# go install
-if [[ ! -x "/usr/bin/go" || ! -x "/root/go/bin/mermerd" ]]; then
-    #wget -q https://go.dev/dl/go1.21.4.linux-arm64.tar.gz && \
-    #rm -rf /usr/local/go && tar -C /usr/local -xzf go1.21.4.linux-arm64.tar.gz && \
-    apt update && apt -y install golang && \
-    go install github.com/KarnerTh/mermerd@latest
-fi
-
 # pnpm install
 if [[ ! -x "/usr/local/bin/pnpm" ]]; then
     npm install -g pnpm
@@ -48,6 +40,19 @@ else
     exit 1
 fi
 
+# metadata and vessel configuration unit tests
+psql ${PGSAIL_DB_URI} < sql/metadata.sql > output/metadata.sql.output
+diff sql/metadata.sql.output output/metadata.sql.output > /dev/null
+#diff -u sql/metadata.sql.output output/metadata.sql.output | wc -l
+#echo 0
+if [ $? -eq 0 ]; then
+    echo OK
+else
+    echo SQL metadata.sql FAILED
+    diff -u sql/metadata.sql.output output/metadata.sql.output
+    exit 1
+fi
+
 # https://www.postgresql.org/docs/current/app-psql.html
 # run cron jobs
 #psql -U ${POSTGRES_USER} -h 172.30.0.1 signalk < sql/cron_run_jobs.sql > output/cron_run_jobs.sql.output
@@ -125,6 +130,19 @@ else
     exit
 fi
 
+# Stays extended unit tests
+psql ${PGSAIL_DB_URI} < sql/stays_ext.sql > output/stays_ext.sql.output
+diff sql/stays_ext.sql.output output/stays_ext.sql.output > /dev/null
+#diff -u sql/stays_ext.sql.output output/stays_ext.sql.output | wc -l
+#echo 0
+if [ $? -eq 0 ]; then
+    echo OK
+else
+    echo SQL stays_ext.sql FAILED
+    diff -u sql/stays_ext.sql.output output/stays_ext.sql.output
+    exit 1
+fi
+
 # Summary unit tests
 psql ${PGSAIL_DB_URI} < sql/summary.sql > output/summary.sql.output
 diff sql/summary.sql.output output/summary.sql.output > /dev/null
@@ -203,6 +221,32 @@ else
     exit 1
 fi
 
+# Stats SQL unit tests
+psql ${PGSAIL_DB_URI} < sql/stats.sql > output/stats.sql.output
+diff sql/stats.sql.output output/stats.sql.output > /dev/null
+#diff -u sql/stats.sql.output output/stats.sql.output | wc -l
+#echo 0
+if [ $? -eq 0 ]; then
+    echo SQL stats.sql OK
+else
+    echo SQL stats.sql FAILED
+    diff -u sql/stats.sql.output output/stats.sql.output
+    exit 1
+fi
+
+# MobilityDB SQL unit tests
+psql ${PGSAIL_DB_URI} < sql/mobilitydb.sql > output/mobilitydb.sql.output
+diff sql/mobilitydb.sql.output output/mobilitydb.sql.output > /dev/null
+#diff -u sql/mobilitydb.sql.output output/mobilitydb.sql.output | wc -l
+#echo 0
+if [ $? -eq 0 ]; then
+    echo SQL mobilitydb.sql OK
+else
+    echo SQL mobilitydb.sql FAILED
+    diff -u sql/mobilitydb.sql.output output/mobilitydb.sql.output
+    exit 1
+fi
+
 # qgis SQL unit tests
 psql ${PGSAIL_DB_URI} < sql/qgis.sql > output/qgis.sql.output
 diff sql/qgis.sql.output output/qgis.sql.output > /dev/null
@@ -239,17 +283,3 @@ else
     echo openapi.json FAILED
     exit 1
 fi
-
-# Generate and update mermaid schema documentation
-/root/go/bin/mermerd --runConfig ../docs/ERD/mermerdConfig.yaml
-#echo $?
-echo 0 # not working in github-actions
-if [ $? -eq 0 ]; then
-    cp postgsail.md ../docs/ERD/postgsail.md
-    echo postgsail.md OK
-else
-    echo postgsail.md FAILED
-    exit 1
-fi
-
-#npm i -D schemalint && npx schemalint