75 Commits

Author SHA1 Message Date
xbgmsharp
670aa2e43a Update openapi documentation 2024-05-01 22:22:18 +02:00
xbgmsharp
1a8790f2a0 Update frontend to 0.1.0-beta7 2024-05-01 22:20:06 +02:00
xbgmsharp
6d97bb1e32 Update the 202404 migration.
Add anonymous check check for timelapse2
Add new tip properties in geojson
2024-04-29 18:13:24 +02:00
xbgmsharp
779cee21ec Update docker compose, make storage as a docker volume 2024-04-29 18:05:26 +02:00
xbgmsharp
e48192e609 Update README 2024-04-29 18:02:05 +02:00
xbgmsharp
f2beead3a7 Update README 2024-04-28 21:29:06 +02:00
xbgmsharp
5df3e1dbd3 Update process_logbook_queue_fn, moorage name part2 2024-04-27 18:13:06 +02:00
xbgmsharp
ed555c1f32 Update process_logbook_queue_fn, add moorage name as note inside in the geojson. this allow to display it during the timelapse replay. 2024-04-27 17:28:52 +02:00
xbgmsharp
02dd68f2d8 Update telemetry 2024-04-27 17:27:58 +02:00
xbgmsharp
d9329705ba Update frontend to 0.1.0-beta6 2024-04-25 21:20:45 +02:00
xbgmsharp
54af136682 Update api.timelapse2_fn, generate a geojson with onlygeometry point to include all the properties 2024-04-25 17:22:34 +02:00
xbgmsharp
6f96a070b8 Update README 2024-04-23 22:20:55 +02:00
xbgmsharp
e79086c4a2 Update README 2024-04-23 22:18:17 +02:00
xbgmsharp
ed417a4c5d Update README 2024-04-23 22:03:20 +02:00
xbgmsharp
546274ce29 Add Funding 2024-04-23 21:41:00 +02:00
xbgmsharp
d5b6072273 Update README 2024-04-23 20:26:03 +02:00
xbgmsharp
e8addd2e9c Add CHANGELOG CODE_OF_CONDUCT CONTRIBUTING 2024-04-23 20:25:45 +02:00
xbgmsharp
f843a6a1f3 Update README 2024-04-23 18:52:39 +02:00
xbgmsharp
478bbf5529 Update README 2024-04-23 16:58:36 +02:00
xbgmsharp
c9523e2f6f Update README 2024-04-23 16:55:25 +02:00
xbgmsharp
5fa85821de Update README 2024-04-23 16:37:28 +02:00
xbgmsharp
7ccef80904 Update README 2024-04-23 16:35:12 +02:00
xbgmsharp
04cc7de245 Update README 2024-04-23 16:27:28 +02:00
xbgmsharp
a75ba105df Update telemetry 2024-04-19 12:37:08 +02:00
xbgmsharp
7eddeefa47 Update frontend 2024-04-19 12:28:41 +02:00
xbgmsharp
314cdc71c7 Update tests, bump version 0.7.2 2024-04-19 12:27:54 +02:00
xbgmsharp
d508ac1662 Update migration 202404.
- Update cron_windy_fn, improve parameters check
- Update email_templates, improve windy error message
- Update delete_logbook_fn, avoid variable not found in subplan target list
- Update tables permissions on api.moorages and api.logbook
- Draft api.timelapse2_fn, new timelapse per point vs linestring
2024-04-19 11:42:27 +02:00
xbgmsharp
6f9956ee46 Add telemetry 2024-04-19 11:41:18 +02:00
xbgmsharp
0b854374ff Update README 2024-04-09 09:36:38 +02:00
xbgmsharp
9b647c9a49 Update README 2024-04-09 09:34:28 +02:00
xbgmsharp
800f0c83e3 Update frontend to 0.1.0-beta5 2024-04-08 19:59:57 +02:00
xbgmsharp
7424fbbe49 Update README 2024-04-07 21:32:45 +02:00
xbgmsharp
e183530435 Update .env.example clarification comments 2024-04-07 21:31:40 +02:00
xbgmsharp
4444f73919 Merge remote-tracking branch 'origin/main' 2024-04-07 15:44:26 +02:00
xbgmsharp
241c70fcb5 Update doc README 2024-04-07 15:41:11 +02:00
xbgmsharp
327324d7c3 Merge pull request #6 from xbgmsharp/dependabot/github_actions/actions/checkout-4
Bump actions/checkout from 3 to 4
2024-04-07 10:20:42 +02:00
xbgmsharp
28ccc9e2da Revert test update 2024-04-07 10:20:11 +02:00
xbgmsharp
966846c93d Update test, fix actually check mermaid exit code 2024-04-07 10:09:15 +02:00
dependabot[bot]
d966d34a29 Bump actions/checkout from 3 to 4
Bumps [actions/checkout](https://github.com/actions/checkout) from 3 to 4.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v3...v4)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-04-07 07:20:12 +00:00
xbgmsharp
8bd7594357 Add dependabot 2024-04-07 09:19:41 +02:00
xbgmsharp
4de47fd011 Update github-actions versions 2024-04-07 09:19:05 +02:00
xbgmsharp
cf0f178d13 Update README 2024-03-25 22:00:31 +01:00
xbgmsharp
b224374e65 Update frontend to 0.1.0-beta4 2024-03-25 21:57:09 +01:00
xbgmsharp
1c441af48c Update delete_account_fn and delete_vessel_fn 2024-03-25 21:56:16 +01:00
xbgmsharp
e5287d74f6 Fix cron_process_no_activity_fn 2024-03-11 09:28:40 +01:00
xbgmsharp
9cf334125a update public.cron_process_no_activity_fn, delete old data 2024-03-10 21:50:05 +01:00
xbgmsharp
1342dba298 Update public.cron_windy_fn, first notifition on first notification
Update delete_vessel_fn, return a jsonb with details
2024-03-09 22:24:09 +01:00
xbgmsharp
7e734c412b Bump up frontend v0.1.0-beta2 2024-03-03 23:09:44 +01:00
xbgmsharp
9ae10b0519 Fix migration, metadata_upsert_trigger_fn 2024-03-03 23:09:18 +01:00
xbgmsharp
ab1b4c7076 Update front-end tests, build web and web_dev 2024-03-03 23:08:46 +01:00
xbgmsharp
edb7bc7dd8 Update test, Bump up version 2024-03-03 12:55:24 +01:00
xbgmsharp
e124bdea44 Update README 2024-03-03 12:41:23 +01:00
xbgmsharp
b944c39507 Update tests, log runTime is now store as ISO duration format. 2024-03-03 12:40:36 +01:00
xbgmsharp
b4791f059c Update logbook_update_extra_json_fn: convert log extra json fields runtime in ISO format and log in NM
Update metadata_upsert_trigger_fn: strip special char from platform metadata
Drop deprecated fn
2024-03-03 12:21:38 +01:00
xbgmsharp
78ff78a6b1 Update OpenAPI 2024-03-02 11:24:26 +01:00
xbgmsharp
b6be05753c Update test, Bump up version 2024-03-02 11:22:58 +01:00
xbgmsharp
30e3797854 Update tests, add convertion output 2024-03-02 11:22:36 +01:00
xbgmsharp
e9dea3b124 Add migration 202403, Add new conversion metersToKnots, Update logbook_update_geojson_fn expose data in nautical units, Update process_lat_lon_fn process moorage to always set a country 2024-03-02 11:19:42 +01:00
xbgmsharp
1a08acda85 Update moorage process to always set country colunm 2024-03-02 11:18:40 +01:00
xbgmsharp
633e73b29d Update tests, add convertion unit test radiantToDegrees, valToPercent, metersToKnots 2024-03-02 11:17:40 +01:00
xbgmsharp
a383d0db53 Bump up timescaledb 2024-02-29 22:30:12 +01:00
xbgmsharp
0116ea4feb Bump up timescaledb 2024-02-29 22:24:57 +01:00
xbgmsharp
92bcf0ffa6 Update public.logbook_update_geojson_fn, add TWD and TWS and navigation.state into log GeoJSON 2024-02-29 22:17:37 +01:00
xbgmsharp
414736909b Update README 2024-02-23 23:49:17 +01:00
xbgmsharp
36173658a0 Update REAME 2024-02-23 21:30:34 +01:00
xbgmsharp
984a7c14da Add Grafana URL 2024-02-23 21:08:22 +01:00
xbgmsharp
48be759e4d Update README 2024-02-23 21:07:29 +01:00
xbgmsharp
e473baa5a0 Update docker, Add build arg 2024-02-22 22:53:50 +01:00
xbgmsharp
9ba79123ec Update README 2024-02-21 16:36:29 +01:00
xbgmsharp
82e7056b0c Update web container, build locally instead of cloud version 2024-02-21 16:28:12 +01:00
xbgmsharp
4829d0b848 Update test, trim users vessel name 2024-02-21 16:25:00 +01:00
xbgmsharp
9069b11f71 Update migrations files. Add license header
Update windy PWS, ensure the require path are present
Update send_notification_fn, fix pushover notificaiton if missing pushover key
Update vessel name, trim name
2024-02-21 15:24:20 +01:00
xbgmsharp
bf3eb3c806 Bump up frontend v0.1.0-beta 2024-02-19 21:44:18 +01:00
xbgmsharp
6e863ca355 Update migration 202402, add cron_prune_otp_fn 2024-02-19 09:32:18 +01:00
xbgmsharp
51128112f3 Update api_version 2024-02-18 18:51:46 +01:00
30 changed files with 1864 additions and 219 deletions

View File

@@ -6,17 +6,23 @@ POSTGRES_DB=postgres
PGSAIL_AUTHENTICATOR_PASSWORD=password
PGSAIL_GRAFANA_PASSWORD=password
PGSAIL_GRAFANA_AUTH_PASSWORD=password
# SMTP server settings
PGSAIL_EMAIL_FROM=root@localhost
PGSAIL_EMAIL_SERVER=localhost
#PGSAIL_EMAIL_USER= Comment if not use
#PGSAIL_EMAIL_PASS= Comment if not use
# Pushover settings
#PGSAIL_PUSHOVER_APP_TOKEN= Comment if not use
#PGSAIL_PUSHOVER_APP_URL= Comment if not use
# TELEGRAM BOT, ask BotFather
#PGSAIL_TELEGRAM_BOT_TOKEN= Comment if not use
# webapp entrypoint, typically the public DNS or IP
PGSAIL_APP_URL=http://localhost:8080
# API entrypoint from the webapp, typically the public DNS or IP
PGSAIL_API_URL=http://localhost:3000
# POSTGREST ENV Settings
PGRST_DB_URI=postgres://authenticator:${PGSAIL_AUTHENTICATOR_PASSWORD}@db:5432/signalk
# % cat /dev/urandom | LC_ALL=C tr -dc 'a-zA-Z0-9' | fold -w 42 | head -n 1
PGRST_JWT_SECRET=_at_least_32__char__long__random
# Grafana ENV Settings
GF_SECURITY_ADMIN_PASSWORD=password

14
.github/FUNDING.yml vendored Normal file
View File

@@ -0,0 +1,14 @@
# These are supported funding model platforms
github: [xbgmsharp]
patreon: # Replace with a single Patreon username
open_collective: # Replace with a single Open Collective username
ko_fi: # Replace with a single Ko-fi username
tidelift: # Replace with a single Tidelift platform-name/package-name e.g., npm/babel
community_bridge: # Replace with a single Community Bridge project-name e.g., cloud-foundry
liberapay: # Replace with a single Liberapay username
issuehunt: # Replace with a single IssueHunt username
lfx_crowdfunding: # Replace with a single LFX Crowdfunding project-name e.g., cloud-foundry
polar: # Replace with a single Polar username
buy_me_a_coffee: # Replace with a single Buy Me a Coffee username
custom: # Replace with up to 4 custom sponsorship URLs e.g., ['link1', 'link2']

14
.github/dependabot.yml vendored Normal file
View File

@@ -0,0 +1,14 @@
version: 2
updates:
# Enable version updates for Docker
- package-ecosystem: "docker"
# Look for a `Dockerfile` in the `root` directory
directory: "/"
# Check for updates once a week
schedule:
interval: "weekly"
- package-ecosystem: github-actions
directory: "/"
schedule:
interval: weekly

View File

@@ -21,7 +21,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Check out the source
uses: actions/checkout@v3
uses: actions/checkout@v4
- name: Set env
run: cp .env.example .env

View File

@@ -23,7 +23,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Check out the source
uses: actions/checkout@v3
uses: actions/checkout@v4
- name: Set env
run: cp .env.example .env

View File

@@ -20,7 +20,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v3
uses: actions/checkout@v4
with:
submodules: 'true'
@@ -31,7 +31,11 @@ jobs:
run: docker compose -f docker-compose.dev.yml -f docker-compose.yml pull db api web_tests
- name: Build Docker images
run: docker compose -f docker-compose.dev.yml -f docker-compose.yml build web_dev
run: |
set -eu
source .env
docker compose -f docker-compose.dev.yml -f docker-compose.yml build web_dev
docker compose -f docker-compose.dev.yml -f docker-compose.yml build web
- name: Run PostgSail Web tests
# Environment variables

View File

@@ -20,7 +20,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v3
uses: actions/checkout@v4
- name: Set env
run: cp .env.example .env

1
CHANGELOG.md Normal file
View File

@@ -0,0 +1 @@
## Please see [Releases](https://github.com/xbgmsharp/postgsail/releases) for the release notes.

45
CODE_OF_CONDUCT.md Normal file
View File

@@ -0,0 +1,45 @@
# Contributor Covenant Code of Conduct
## Our Pledge
In the interest of fostering an open and welcoming environment, we as contributors and maintainers pledge to make participation in our project and our community a harassment-free experience for everyone, regardless of age, body size, disability, ethnicity, sex characteristics, gender identity and expression, level of experience, education, socio-economic status, nationality, personal appearance, race, religion, or sexual identity and orientation.
## Our Standards
Examples of behavior that contributes to creating a positive environment include:
* Using welcoming and inclusive language
* Being respectful of differing viewpoints and experiences
* Gracefully accepting constructive criticism
* Focusing on what is best for the community
* Showing empathy towards other community members
Examples of unacceptable behavior by participants include:
* The use of sexualized language or imagery and unwelcome sexual attention or advances
* Trolling, insulting/derogatory comments, and personal or political attacks
* Public or private harassment
* Publishing others' private information, such as a physical or electronic address, without explicit permission
* Other conduct which could reasonably be considered inappropriate in a professional setting
## Our Responsibilities
Project maintainers are responsible for clarifying the standards of acceptable behavior and are expected to take appropriate and fair corrective action in response to any instances of unacceptable behavior.
Project maintainers have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct, or to ban temporarily or permanently any contributor for other behaviors that they deem inappropriate, threatening, offensive, or harmful.
## Scope
This Code of Conduct applies within all project spaces, and it also applies when an individual is representing the project or its community in public spaces. Examples of representing a project or community include using an official project e-mail address, posting via an official social media account, or acting as an appointed representative at an online or offline event. Representation of a project may be further defined and clarified by project maintainers.
## Enforcement
Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by contacting the project maintainer using any of the [private contact addresses](https://github.com/dec0dOS/amazing-github-template#support). All complaints will be reviewed and investigated and will result in a response that is deemed necessary and appropriate to the circumstances. The project team is obligated to maintain confidentiality with regard to the reporter of an incident. Further details of specific enforcement policies may be posted separately.
Project maintainers who do not follow or enforce the Code of Conduct in good faith may face temporary or permanent repercussions as determined by other members of the project's leadership.
## Attribution
This Code of Conduct is adapted from the [Contributor Covenant](https://www.contributor-covenant.org), version 1.4, available at <https://www.contributor-covenant.org/version/1/4/code-of-conduct.html>
For answers to common questions about this code of conduct, see <https://www.contributor-covenant.org/faq>

3
CONTRIBUTING.md Normal file
View File

@@ -0,0 +1,3 @@
Styleguides
Ensure you code is in lint formatting.

251
README.md
View File

@@ -1,11 +1,37 @@
# PostgSail
<br/>
<p align="center">
<a href="https://github.com/xbgmsharp/postgsail">
<img src="https://iot.openplotter.cloud/android-chrome-192x192.png" alt="Logo" width="80" height="80">
</a>
Effortless cloud based solution for storing and sharing your SignalK data. Allow you to effortlessly log your sails and monitor your boat with historical data.
<h3 align="center">PostgSail</h3>
<p align="center">
PostgSail is an open-source alternative to traditional vessel data management!
<br/>
<br/>
<a href="https://github.com/xbgmsharp/postgsail/blob/main/docs/README.md"><strong>Explore the docs »</strong></a>
<br/>
<br/>
<a href="#about-the-project">View Demo</a>
.
<a href="https://github.com/xbgmsharp/postgsail/issues">Report Bug</a>
.
<a href="https://github.com/xbgmsharp/postgsail/issues">Request Feature</a>
.
<a href="https://xbgmsharp.github.io/postgsail/">Website</a>
.
<a href="https://github.com/sponsors/xbgmsharp">Sponsors</a>
.
<a href="https://discord.gg/uuZrwz4dCS">Discord</a>
</p>
</p>
[![release](https://img.shields.io/github/release/xbgmsharp/postgsail?include_prereleases=&sort=semver&color=blue)](https://github.com/xbgmsharp/postgsail/releases/latest)
[![License](https://img.shields.io/github/license/xbgmsharp/postgsail)](#license)
[![issues - postgsail](https://img.shields.io/github/issues/xbgmsharp/postgsail)](https://github.com/xbgmsharp/postgsail/issues)
[![PRs Welcome](https://img.shields.io/badge/PRs-welcome-brightgreen.svg?style=flat-square)](http://makeapullrequest.com)
![Contributors](https://img.shields.io/github/contributors/xbgmsharp/postgsail?color=dark-green)
[![Test services db, api](https://github.com/xbgmsharp/postgsail/actions/workflows/db-test.yml/badge.svg)](https://github.com/xbgmsharp/postgsail/actions/workflows/db-test.yml)
[![Test services db, api, web](https://github.com/xbgmsharp/postgsail/actions/workflows/frontend-test.yml/badge.svg)](https://github.com/xbgmsharp/postgsail/actions/workflows/frontend-test.yml)
@@ -22,6 +48,35 @@ postgsail-telegram-bot:
[![OpenSSF Best Practices](https://www.bestpractices.dev/projects/8124/badge)](https://www.bestpractices.dev/projects/8124)
## Table Of Contents
- [Table Of Contents](#table-of-contents)
- [About The Project](#about-the-project)
- [Features](#features)
- [Cloud-hosted PostgSail](#cloud-hosted-postgsail)
- [On-Premise (for free)](#on-premise-for-free)
- [Roadmap](#roadmap)
- [Contributing](#contributing)
- [Creating A Pull Request](#creating-a-pull-request)
- [License](#license)
- [Acknowledgements](#acknowledgements)
## About The Project
https://github.com/xbgmsharp/signalk-postgsail/assets/1498985/b2669c39-11ad-4a50-9f91-9397f9057ee8
Effortless cloud based solution for storing and sharing your SignalK data. Allow you to effortlessly log your sails and monitor your boat with historical data.
Here's how:
It is all about SQL, object-relational, time-series, spatial databases with a bit of python.
PostgSail is an open-source alternative to traditional vessel data management.
It is based on a well known open-source technology stack, Signalk, PostgreSQL, TimescaleDB, PostGIS, PostgREST. It does perfectly integrate with standard monitoring tool stack like Grafana.
To understand the why and how, you might want to read [Why.md](https://github.com/xbgmsharp/postgsail/blob/main/Why.md)
## Features
- Automatically log your voyages without manually starting or stopping a trip.
@@ -40,191 +95,51 @@ postgsail-telegram-bot:
- Awesome statistics and graphs.
- Create and manage your own dashboards.
- Windy PWS (Personal Weather Station).
- Engine Hours Logger.
- Polar performance.
- Anything missing? just ask!
## Context
## Cloud-hosted PostgSail
It is all about SQL, object-relational, time-series, spatial databases with a bit of python.
Remove the hassle of running PostgSail yourself. Here you can skip the technical setup, the maintenance work and server costs by getting PostgSail on our reliable and secure PostgSail Cloud. Register and try for free at [iot.openplotter.cloud](https://iot.openplotter.cloud/).
PostgSail is an open-source alternative to traditional vessel data management.
It is based on a well known open-source technology stack, Signalk, PostgreSQL, TimescaleDB, PostGIS, PostgREST. It does perfectly integrate with standard monitoring tool stack like Grafana.
## On-Premise (for free)
To understand the why and how, you might want to read [Why.md](https://github.com/xbgmsharp/postgsail/tree/main/Why.md)
Self host postgSail where you want and how you want. There are no restrictions, youre in full control. [Install Guide](https://github.com/xbgmsharp/postgsail/blob/main/docs/README.md)
## Architecture
A simple scalable architecture:
## Roadmap
![Architecture overview](https://raw.githubusercontent.com/xbgmsharp/postgsail/main/PostgSail.png "Architecture overview")
See the [open issues](https://github.com/xbgmsharp/postgsail/issues) for a list of proposed features (and known issues).
For more clarity and visibility the complete [Entity-Relationship Diagram (ERD)](https://github.com/xbgmsharp/postgsail/tree/main/ERD/README.md) is export as PNG and SVG file.
Join the community, Get support and exchange on [Discord](https://discord.gg/uuZrwz4dCS). Missing a feature? just ask!
## Cloud
## Contributing
If you prefer not to install or administer your instance of PostgSail, hosted versions of PostgSail are available in the cloud of your choice.
Contributions are what make the open source community such an amazing place to be learn, inspire, and create. Any contributions you make are **greatly appreciated**.
* If you have suggestions for features, feel free to [open an issue](https://github.com/xbgmsharp/postgsail/issues/new) to discuss it, or directly create a pull request with necessary changes.
* Please make sure you check your spelling and grammar.
* Create individual PR for each suggestion.
* Please also read through the [Code Of Conduct](https://github.com/xbgmsharp/postgsail/blob/main/CODE_OF_CONDUCT.md) before posting your first idea as well.
### The cloud advantage.
### Creating A Pull Request
Hosted and fullymanaged options for PostgSail, designed for all your deployment and business needs. Register and try for free at https://iot.openplotter.cloud/.
1. Fork the Project
2. Create your Feature Branch (`git checkout -b feature/AmazingFeature`)
3. Commit your Changes (`git commit -m 'Add some AmazingFeature'`)
4. Push to the Branch (`git push origin feature/AmazingFeature`)
5. Open a Pull Request
## Using PostgSail
## License
A full-featured development environment.
Distributed under the Apache License Version 2.0. See [LICENSE](https://github.com/xbgmsharp/postgsail/blob/main/LICENSE) for more information.
#### With CodeSandbox
## Acknowledgements
- Develop on [![CodeSandbox Ready-to-Code](https://img.shields.io/badge/CodeSandbox-Ready--to--Code-blue?logo=codesandbox)](https://codesandbox.io/p/github/xbgmsharp/postgsail/main)
- or via [direct link](https://codesandbox.io/p/github/xbgmsharp/postgsail/main)
#### With DevPod
- [![Open in DevPod!](https://devpod.sh/assets/open-in-devpod.svg)](https://devpod.sh/open#https://github.com/xbgmsharp/postgsail/&workspace=postgsail&provider=docker&ide=openvscode)
- or via [direct link](https://devpod.sh/open#https://github.com/xbgmsharp/postgsail&workspace=postgsail&provider=docker&ide=openvscode)
#### With Docker Dev Environments
- [Open in Docker dev-envs!](https://open.docker.com/dashboard/dev-envs?url=https://github.com/xbgmsharp/postgsail/)
### pre-deploy configuration
To get these running, copy `.env.example` and rename to `.env` then set the value accordingly.
```bash
# cp .env.example .env
```
Notice, that `PGRST_JWT_SECRET` must be at least 32 characters long.
`$ head /dev/urandom | tr -dc A-Za-z0-9 | head -c 42 ; echo ''`
```bash
# nano .env
```
### Deploy
By default there is no network set and all data are store in a docker volume.
You can update the default settings by editing `docker-compose.yml` and `docker-compose.dev.yml` to your need.
First let's initialize the database.
#### Step 1. Initialize database
First let's import the SQL schema, execute:
```bash
$ docker-compose up db
```
#### Step 2. Start backend (db, api)
Then launch the full stack (db, api) backend, execute:
```bash
$ docker-compose up db api
```
The API should be accessible via port HTTP/3000.
The database should be accessible via port TCP/5432.
You can connect to the database via a web gui like [pgadmin](https://www.pgadmin.org/) or you can use a client [dbeaver](https://dbeaver.io/).
### SQL Configuration
Check and update your postgsail settings via SQL in the table `app_settings`:
```sql
SELECT * FROM app_settings;
```
```sql
UPDATE app_settings
SET
value = 'new_value'
WHERE name = 'app.email_server';
```
### Ingest data
Next, to ingest data from signalk, you need to install [signalk-postgsail](https://github.com/xbgmsharp/signalk-postgsail) plugin on your signalk server instance.
Also, if you like, you can import saillogger data using the postgsail helpers, [postgsail-helpers](https://github.com/xbgmsharp/postgsail-helpers).
You might want to import your influxdb1 data as well, [outflux](https://github.com/timescale/outflux).
For InfluxDB 2.x and 3.x. You will need to enable the 1.x APIs to use them. Consult the InfluxDB documentation for more details.
Last, if you like, you can import the sample data from Signalk NMEA Plaka by running the tests.
If everything goes well all tests pass successfully and you should receive a few notifications by email or PushOver or Telegram.
[End-to-End (E2E) Testing.](https://github.com/xbgmsharp/postgsail/blob/main/tests/)
```
$ docker-compose up tests
```
### API Documentation
The OpenAPI description output depends on the permissions of the role that is contained in the JWT role claim.
Other applications can also use the [PostgSAIL API](https://petstore.swagger.io/?url=https://raw.githubusercontent.com/xbgmsharp/postgsail/main/openapi.json).
API anonymous:
```
$ curl http://localhost:3000/
```
API user_role:
```
$ curl http://localhost:3000/ -H 'Authorization: Bearer my_token_from_login_or_signup_fn'
```
API vessel_role:
```
$ curl http://localhost:3000/ -H 'Authorization: Bearer my_token_from_register_vessel_fn'
```
#### API main workflow
Check the [End-to-End (E2E) test sample](https://github.com/xbgmsharp/postgsail/blob/main/tests/).
### Docker dependencies
`docker-compose` is used to start environment dependencies. Dependencies consist of 3 containers:
- `timescaledb-postgis` alias `db`, PostgreSQL with TimescaleDB extension along with the PostGIS extension.
- `postgrest` alias `api`, Standalone web server that turns your PostgreSQL database directly into a RESTful API.
- `grafana` alias `app`, visualize and monitor your data
### Optional docker images
- [pgAdmin](https://hub.docker.com/r/dpage/pgadmin4), web UI to monitor and manage multiple PostgreSQL
- [Swagger](https://hub.docker.com/r/swaggerapi/swagger-ui), web UI to visualize documentation from PostgREST
```
docker-compose -f docker-compose-optional.yml up
```
### Software reference
Out of the box iot platform using docker with the following software:
An out of the box IoT platform using Docker (could be extend to K3 or K8) with the following software:
- [Signal K server, a Free and Open Source universal marine data exchange format](https://signalk.org)
- [PostgreSQL, open source object-relational database system](https://postgresql.org)
- [TimescaleDB, Time-series data extends PostgreSQL](https://www.timescale.com)
- [PostGIS, a spatial database extender for PostgreSQL object-relational database.](https://postgis.net/)
- [Grafana, open observability platform | Grafana Labs](https://grafana.com)
### Support
To get support, please create new [issue](https://github.com/xbgmsharp/postgsail/issues).
There is more likely security flows and bugs.
### Contribution
I'm happy to accept Pull Requests!
Feel free to contribute.
### License
This is a free software, Apache License Version 2.0.
- And many more

View File

@@ -18,7 +18,7 @@ services:
ports:
- "5432:5432"
volumes:
- ./db-data:/var/lib/postgresql/data
- postgres-data:/var/lib/postgresql/data
- ./initdb:/docker-entrypoint-initdb.d
logging:
options:
@@ -39,6 +39,7 @@ services:
- "db:database"
ports:
- "3000:3000"
- "3003:3003"
env_file: .env
environment:
PGRST_DB_SCHEMA: api
@@ -70,8 +71,8 @@ services:
links:
- "db:database"
volumes:
- data:/var/lib/grafana
- data:/var/log/grafana
- grafana-data:/var/lib/grafana
- grafana-data:/var/log/grafana
- ./grafana:/etc/grafana
ports:
- "3001:3000"
@@ -112,18 +113,30 @@ services:
max-size: 10m
web:
image: xbgmsharp/postgsail-vuestic
image: vuestic-postgsail
build:
context: https://github.com/xbgmsharp/vuestic-postgsail.git#live
dockerfile: Dockerfile
args:
- VITE_PGSAIL_URL=${PGSAIL_API_URL}
- VITE_APP_INCLUDE_DEMOS=false
- VITE_APP_BUILD_VERSION=true
- VITE_APP_TITLE=${VITE_APP_TITLE}
- VITE_GRAFANA_URL=${VITE_GRAFANA_URL}
hostname: web
container_name: web
restart: unless-stopped
links:
- "api:postgrest"
ports:
- 8080:8080
env_file: .env
environment:
- VITE_PGSAIL_URL=${PGSAIL_API_URL}
- VITE_APP_INCLUDE_DEMOS=false
- VITE_APP_BUILD_VERSION=true
- VITE_APP_TITLE=${VITE_APP_TITLE}
- VITE_GRAFANA_URL=${VITE_GRAFANA_URL}
depends_on:
- db
- api
@@ -132,4 +145,5 @@ services:
max-size: 10m
volumes:
data: {}
grafana-data: {}
postgres-data: {}

View File

@@ -7,7 +7,7 @@ Auto generated Mermaid diagram using [mermerd](https://github.com/KarnerTh/merme
[PostgSail SQL Schema](https://github.com/xbgmsharp/postgsail/tree/main/docs/ERD/postgsail.md "PostgSail SQL Schema")
## Further
There is 3 main schemas:
There is 3 main schemas into the signalk database:
- API Schema:
- tables
- metrics
@@ -32,3 +32,77 @@ There is 3 main schemas:
- functions
- ...
## Overview
- Insert data into table metadata from API using PostgREST
- Insert data into table metrics from API using PostgREST
- TimescaleDB Hypertable to store signalk metrics
- pgsql functions to generate logbook, stays, moorages
- CRON functions to process logbook, stays, moorages
- python functions for geo reverse and send notification via email and/or pushover
- Views statistics, timelapse, monitoring, logs
- Always store time in UTC
## Ingest flowchart
```mermaid
graph LR
A[SignalK] -- HTTP POST --> B{PostgREST}
B -- SQL --> C{PostgreSQL}
C --> D((metadata trigger))
C --> E((metrics trigger))
D --> F{tbl.metadata}
E --> G{tbl.metrics}
E --> H{tbl.logs}
E --> I{tbl.stays}
```
## pg_cron flowchart
```mermaid
flowchart TD
A[pg_cron] --> B((cron_new_notification))
A --> C((cron_pre_logbook))
A --> D((cron_new_logbook))
A --> E((cron_new_stay))
A --> F((cron_monitor_offline))
A --> G((cron_monitor_online))
C --> K{Validate logbook details}
D --> L{Update logbook details}
E --> M{Update stay details}
L --> N{Update Moorages details}
M --> N{Update Moorages details}
B --> O{Update account,vessel,otp}
F --> P{Update metadata}
G --> P
```
cron job are not process by default because if you don't have the correct settings set (SMTP, PushOver, Telegram), you might enter in a loop with error and you could be blocked or banned from the external services.
Therefor by default they are no active job as it require external configuration settings (SMTP, PushOver, Telegram).
To activate all cron.job run the following SQL command:
```sql
UPDATE cron.job SET active = True;
```
Be sure to review your postgsail settings via SQL in the table `app_settings`:
```sql
SELECT * FROM app_settings;
```
### How to bypass OTP for a local install?
You can skip the otp, add or update json key value to the account preference.
"email_valid": true
OTP is created and sent by email using a cron in postgres/cron/job
accounts are store in table signalk/auth/accounts
You should have an history in table signalk/public/process_queue
```sql
select * from public.process_queue;
```
### How to turn off signups
If you just want to use this as a standalone application and don't want people to be able to sign up for an account.
```SQL
revoke execute on function api.signup(text,text,text,text) to api_anonymous;
```

View File

@@ -1,2 +1,200 @@
Simple and scalable architecture.
## Architecture
Efficient, simple and scalable architecture.
![Architecture overview](https://raw.githubusercontent.com/xbgmsharp/postgsail/main/PostgSail.png "Architecture overview")
For more clarity and visibility the complete [Entity-Relationship Diagram (ERD)](https://github.com/xbgmsharp/postgsail/blob/main/docs/ERD/README.md) is export as Mermaid, PNG and SVG file.
## Using PostgSail
### Development
A full-featured development environment.
#### With CodeSandbox
- Develop on [![CodeSandbox Ready-to-Code](https://img.shields.io/badge/CodeSandbox-Ready--to--Code-blue?logo=codesandbox)](https://codesandbox.io/p/github/xbgmsharp/postgsail/main)
- or via [direct link](https://codesandbox.io/p/github/xbgmsharp/postgsail/main)
#### With DevPod
- [![Open in DevPod!](https://devpod.sh/assets/open-in-devpod.svg)](https://devpod.sh/open#https://github.com/xbgmsharp/postgsail/&workspace=postgsail&provider=docker&ide=openvscode)
- or via [direct link](https://devpod.sh/open#https://github.com/xbgmsharp/postgsail&workspace=postgsail&provider=docker&ide=openvscode)
#### With Docker Dev Environments
- [Open in Docker dev-envs!](https://open.docker.com/dashboard/dev-envs?url=https://github.com/xbgmsharp/postgsail/)
### On-premise (self-hosted)
#### pre-deploy configuration
To get these running, copy `.env.example` and rename to `.env` then set the value accordingly.
```bash
# cp .env.example .env
```
```bash
# nano .env
```
Notice, that `PGRST_JWT_SECRET` must be at least 32 characters long.
`$ cat /dev/urandom | LC_ALL=C tr -dc 'a-zA-Z0-9' | fold -w 42 | head -n 1`
`PGSAIL_APP_URL` is the URL you connect to from your browser.
`PGSAIL_API_URL` is the URL where `PGSAIL_APP_URL` connect to.
`PGRST_DB_URI` is the URI where the `PGSAIL_API_URL` connect to.
To summarize:
```mermaid
flowchart LR
subgraph frontend
direction TB
A(PGSAIL_APP_URL)
B(PGSAIL_API_URL)
end
subgraph backend
direction TB
B(PGSAIL_API_URL) -- SQL --> C(PGRST_DB_URI)
end
%% ^ These subgraphs are identical, except for the links to them:
%% Link *to* subgraph1: subgraph1 direction is maintained
User -- HTTP --> A
User -- HTTP --> B
%% Link *within* subgraph2:
%% subgraph2 inherits the direction of the top-level graph (LR)
Boat -- HTTP --> B
```
### Deploy
There is two compose files used. You can update the default settings by editing `docker-compose.yml` and `docker-compose.dev.yml` to your need.
Now let's initialize the database.
#### Step 1. Initialize database
First let's import the SQL schema, execute:
```bash
$ docker compose up db
```
#### Step 2. Start backend (db, api)
Then launch the full backend stack (db, api), execute:
```bash
$ docker compose up db api
```
The API should be accessible via port HTTP/3000.
The database should be accessible via port TCP/5432.
You can connect to the database via a web gui like [pgadmin](https://www.pgadmin.org/) or you can use a client [dbeaver](https://dbeaver.io/).
```bash
$ docker compose -f docker-compose.yml -f docker-compose.dev.yml pgadmin
```
Then connect to the web UI on port HTTP/5050.
#### Step 3. Start frontend (web)
Then launch the web frontend, execute:
```bash
$ docker compose up web
```
This step can take some time as it will first do a build to generate the static website based on your settings.
The frontend should be accessible via port HTTP/8080.
Users are collaborating on an installation guide, [Self-hosted-installation-guide](https://github.com/xbgmsharp/postgsail/wiki/Self-hosted-installation-guide)
### SQL Configuration
Check and update your postgsail settings via SQL in the table `app_settings`:
```sql
SELECT * FROM app_settings;
```
```sql
UPDATE app_settings
SET
value = 'new_value'
WHERE name = 'app.email_server';
```
As it is all about SQL, [Read more](https://github.com/xbgmsharp/postgsail/blob/main/docs/ERD/README.md) about the database and explore your data.
### Ingest data
Next, to ingest data from signalk, you need to install [signalk-postgsail](https://github.com/xbgmsharp/signalk-postgsail) plugin on your signalk server instance.
Also, if you like, you can import saillogger data using the postgsail helpers, [postgsail-helpers](https://github.com/xbgmsharp/postgsail-helpers).
You might want to import your influxdb1 data as well, [outflux](https://github.com/timescale/outflux).
For InfluxDB 2.x and 3.x. You will need to enable the 1.x APIs to use them. Consult the InfluxDB documentation for more details.
Last, if you like, you can import the sample data from Signalk NMEA Plaka by running the tests.
If everything goes well all tests pass successfully and you should receive a few notifications by email or PushOver or Telegram.
[End-to-End (E2E) Testing.](https://github.com/xbgmsharp/postgsail/blob/main/tests/)
```
$ docker-compose up tests
```
### API Documentation
The OpenAPI description output depends on the permissions of the role that is contained in the JWT role claim.
Other applications can also use the [PostgSAIL API](https://petstore.swagger.io/?url=https://raw.githubusercontent.com/xbgmsharp/postgsail/main/openapi.json).
API anonymous:
```
$ curl http://localhost:3000/
```
API user_role:
```
$ curl http://localhost:3000/ -H 'Authorization: Bearer my_token_from_login_or_signup_fn'
```
API vessel_role:
```
$ curl http://localhost:3000/ -H 'Authorization: Bearer my_token_from_register_vessel_fn'
```
#### API main workflow
Check the [End-to-End (E2E) test sample](https://github.com/xbgmsharp/postgsail/blob/main/tests/).
### Docker dependencies
`docker-compose` is used to start environment dependencies. Dependencies consist of 3 containers:
- `timescaledb-postgis` alias `db`, PostgreSQL with TimescaleDB extension along with the PostGIS extension.
- `postgrest` alias `api`, Standalone web server that turns your PostgreSQL database directly into a RESTful API.
- `grafana` alias `app`, visualize and monitor your data
### Optional docker images
- [pgAdmin](https://hub.docker.com/r/dpage/pgadmin4), web UI to monitor and manage multiple PostgreSQL
- [Swagger](https://hub.docker.com/r/swaggerapi/swagger-ui), web UI to visualize documentation from PostgREST
```
docker-compose -f docker-compose-optional.yml up
```

View File

@@ -1,10 +1,10 @@
---------------------------------------------------------------------------
-- TODO
-- Copyright 2021-2024 Francois Lacroix <xbgmsharp@gmail.com>
-- This file is part of PostgSail which is released under Apache License, Version 2.0 (the "License").
-- See file LICENSE or go to http://www.apache.org/licenses/LICENSE-2.0 for full license details.
--
-- Migration January 2024
--
----------------------------------------
----- TODO --------------
----------------------------------------
-- List current database
select current_database();

View File

@@ -1,10 +1,10 @@
---------------------------------------------------------------------------
-- TODO
-- Copyright 2021-2024 Francois Lacroix <xbgmsharp@gmail.com>
-- This file is part of PostgSail which is released under Apache License, Version 2.0 (the "License").
-- See file LICENSE or go to http://www.apache.org/licenses/LICENSE-2.0 for full license details.
--
-- Migration February 2024
--
----------------------------------------
----- TODO --------------
----------------------------------------
-- List current database
select current_database();
@@ -21,7 +21,7 @@ set timezone to 'UTC';
--VALUES ('alert','PostgSail Alert',E'Hello __RECIPIENT__,\nWe detected an alert __ALERT__.\nSee more details at __APP_URL__\nStay safe.\nFrancois','PostgSail Alert!',E'Congratulations!\nWe detected an alert __ALERT__.\n');
INSERT INTO public.email_templates ("name",email_subject,email_content,pushover_title,pushover_message)
VALUES ('windy_error','PostgSail Windy Weather station Error','Hello __RECIPIENT__,\nSorry!We could not convert your boat to a Windy Personal Weather Station.\nWindy Personal Weather Station is now disable.','PostgSail Windy error!','Sorry!\nWe could not convert your boat to a Windy Personal Weather Station.');
VALUES ('windy_error','PostgSail Windy Weather station Error',E'Hello __RECIPIENT__,\nSorry!We could not convert your boat into a Windy Personal Weather Station due to missing data (temp or wind).\nWindy Personal Weather Station is now disable.','PostgSail Windy error!',E'Sorry!\nWe could not convert your boat into a Windy Personal Weather Station.');
-- Update app_settings
CREATE OR REPLACE FUNCTION public.get_app_settings_fn(OUT app_settings jsonb)
@@ -152,7 +152,7 @@ COMMENT ON FUNCTION
public.windy_pws_py_fn
IS 'Forward vessel data to Windy as a Personal Weather Station using plpython3u';
CREATE OR REPLACE FUNCTION public.cron_windy_fn() RETURNS void AS $$
CREATE OR REPLACE FUNCTION public.cron_windy_fn() RETURNS void AS $cron_windy$
DECLARE
windy_rec record;
default_last_metric TIMESTAMPTZ := NOW() - interval '1 day';
@@ -204,9 +204,17 @@ BEGIN
ORDER BY time_bucket ASC LIMIT 100
LOOP
RAISE NOTICE '-> cron_windy_fn checking metrics [%]', metric_rec;
IF metric_rec.wind IS NULL OR metric_rec.temperature IS NULL THEN
-- Ignore when there is no metrics
-- Send notification
PERFORM send_notification_fn('windy_error'::TEXT, user_settings::JSONB);
-- Disable windy
PERFORM api.update_user_preferences_fn('{public_windy}'::TEXT, 'false'::TEXT);
RETURN;
END IF;
-- https://community.windy.com/topic/8168/report-your-weather-station-data-to-windy
-- temp from kelvin to celcuis
-- winddir from radiant to degres
-- temp from Kelvin to Celsius
-- winddir from radiant to Degrees
-- rh from ratio to percentage
SELECT jsonb_build_object(
'dateutc', metric_rec.time_bucket,
@@ -237,7 +245,7 @@ BEGIN
PERFORM api.update_user_preferences_fn('{windy_last_metric}'::TEXT, last_metric::TEXT);
END LOOP;
END;
$$ language plpgsql;
$cron_windy$ language plpgsql;
-- Description
COMMENT ON FUNCTION
public.cron_windy_fn
@@ -697,6 +705,33 @@ COMMENT ON FUNCTION
public.cron_deactivated_fn
IS 'init by pg_cron, check for vessel with no activity for more than 1 year then send notification and delete data';
DROP FUNCTION IF EXISTS public.cron_prune_otp_fn;
CREATE OR REPLACE FUNCTION public.cron_prune_otp_fn() RETURNS void
AS $$
DECLARE
otp_rec record;
BEGIN
-- Purge OTP older than 15 minutes
RAISE NOTICE 'cron_prune_otp_fn';
FOR otp_rec in
SELECT *
FROM auth.otp
WHERE otp_timestamp < NOW() AT TIME ZONE 'UTC' - INTERVAL '15 MINUTES'
ORDER BY otp_timestamp desc
LOOP
RAISE NOTICE '-> cron_prune_otp_fn deleting expired otp for user [%]', otp_rec.user_email;
-- remove entry
DELETE FROM auth.otp
WHERE user_email = otp_rec.user_email;
RAISE NOTICE '-> cron_prune_otp_fn deleted expire otp for user [%]', otp_rec.user_email;
END LOOP;
END;
$$ language plpgsql;
-- Description
COMMENT ON FUNCTION
public.cron_prune_otp_fn
IS 'init by pg_cron to purge older than 15 minutes OTP token';
DROP FUNCTION IF EXISTS public.cron_process_prune_otp_fn();
DROP FUNCTION IF EXISTS public.cron_process_no_vessel_fn();
DROP FUNCTION IF EXISTS public.cron_process_no_metadata_fn();
@@ -706,10 +741,154 @@ DROP FUNCTION IF EXISTS public.cron_process_windy_fn();
DROP FUNCTION IF EXISTS public.cron_process_alerts_fn();
-- Remove deprecated fn
DROP FUNCTION public.cron_process_new_account_fn();
DROP FUNCTION public.cron_process_new_account_otp_validation_fn();
DROP FUNCTION public.cron_process_new_moorage_fn();
DROP FUNCTION public.cron_process_new_vessel_fn();
DROP FUNCTION IF EXISTS public.cron_process_new_account_fn();
DROP FUNCTION IF EXISTS public.cron_process_new_account_otp_validation_fn();
DROP FUNCTION IF EXISTS public.cron_process_new_moorage_fn();
DROP FUNCTION IF EXISTS public.cron_process_new_vessel_fn();
CREATE OR REPLACE FUNCTION send_notification_fn(
IN email_type TEXT,
IN user_settings JSONB) RETURNS VOID
AS $send_notification$
DECLARE
app_settings JSONB;
_email_notifications BOOLEAN := False;
_phone_notifications BOOLEAN := False;
_pushover_user_key TEXT := NULL;
pushover_settings JSONB := NULL;
_telegram_notifications BOOLEAN := False;
_telegram_chat_id TEXT := NULL;
telegram_settings JSONB := NULL;
_email TEXT := NULL;
BEGIN
-- TODO input check
--RAISE NOTICE '--> send_notification_fn type [%]', email_type;
-- Gather notification app settings, eg: email, pushover, telegram
app_settings := get_app_settings_fn();
--RAISE NOTICE '--> send_notification_fn app_settings [%]', app_settings;
--RAISE NOTICE '--> user_settings [%]', user_settings->>'email'::TEXT;
-- Gather notifications settings and merge with user settings
-- Send notification email
SELECT preferences['email_notifications'] INTO _email_notifications
FROM auth.accounts a
WHERE a.email = user_settings->>'email'::TEXT;
RAISE NOTICE '--> send_notification_fn email_notifications [%]', _email_notifications;
-- If email server app settings set and if email user settings set
IF app_settings['app.email_server'] IS NOT NULL AND _email_notifications IS True THEN
PERFORM send_email_py_fn(email_type::TEXT, user_settings::JSONB, app_settings::JSONB);
END IF;
-- Send notification pushover
SELECT preferences['phone_notifications'],preferences->>'pushover_user_key' INTO _phone_notifications,_pushover_user_key
FROM auth.accounts a
WHERE a.email = user_settings->>'email'::TEXT;
RAISE NOTICE '--> send_notification_fn phone_notifications [%]', _phone_notifications;
-- If pushover app settings set and if pushover user settings set
IF app_settings['app.pushover_app_token'] IS NOT NULL AND _phone_notifications IS True AND _pushover_user_key IS NOT NULL THEN
SELECT json_build_object('pushover_user_key', _pushover_user_key) into pushover_settings;
SELECT user_settings::JSONB || pushover_settings::JSONB into user_settings;
--RAISE NOTICE '--> send_notification_fn user_settings + pushover [%]', user_settings;
PERFORM send_pushover_py_fn(email_type::TEXT, user_settings::JSONB, app_settings::JSONB);
END IF;
-- Send notification telegram
SELECT (preferences->'telegram'->'chat'->'id') IS NOT NULL,preferences['telegram']['chat']['id'] INTO _telegram_notifications,_telegram_chat_id
FROM auth.accounts a
WHERE a.email = user_settings->>'email'::TEXT;
RAISE NOTICE '--> send_notification_fn telegram_notifications [%]', _telegram_notifications;
-- If telegram app settings set and if telegram user settings set
IF app_settings['app.telegram_bot_token'] IS NOT NULL AND _telegram_notifications IS True AND _phone_notifications IS True THEN
SELECT json_build_object('telegram_chat_id', _telegram_chat_id) into telegram_settings;
SELECT user_settings::JSONB || telegram_settings::JSONB into user_settings;
--RAISE NOTICE '--> send_notification_fn user_settings + telegram [%]', user_settings;
PERFORM send_telegram_py_fn(email_type::TEXT, user_settings::JSONB, app_settings::JSONB);
END IF;
END;
$send_notification$ LANGUAGE plpgsql;
-- fn to trim new vessel name
CREATE FUNCTION new_vessel_trim_fn() RETURNS trigger AS $new_vessel_trim_fn$
BEGIN
NEW.name := TRIM(NEW.name);
RETURN NEW;
END;
$new_vessel_trim_fn$ language plpgsql;
-- Description
COMMENT ON FUNCTION
public.new_vessel_trim_fn
IS 'Trim space vessel name';
-- Trigger trim new vessel name
CREATE TRIGGER new_vessel_trim BEFORE INSERT ON auth.vessels
FOR EACH ROW EXECUTE FUNCTION public.new_vessel_trim_fn();
-- Description
COMMENT ON TRIGGER new_vessel_trim
ON auth.vessels
IS 'Trim space vessel name';
CREATE or REPLACE FUNCTION public.logbook_update_geojson_fn(IN _id integer, IN _start text, IN _end text,
OUT _track_geojson JSON
) AS $logbook_geojson$
declare
log_geojson jsonb;
metrics_geojson jsonb;
_map jsonb;
begin
-- GeoJson Feature Logbook linestring
SELECT
ST_AsGeoJSON(log.*) into log_geojson
FROM
( SELECT
id,name,
distance,
duration,
avg_speed,
max_speed,
max_wind_speed,
_from_time,
_to_time
_from_moorage_id,
_to_moorage_id,
notes,
track_geom
FROM api.logbook
WHERE id = _id
) AS log;
-- GeoJson Feature Metrics point
SELECT
json_agg(ST_AsGeoJSON(t.*)::json) into metrics_geojson
FROM (
( SELECT
time,
courseovergroundtrue,
speedoverground,
windspeedapparent,
longitude,latitude,
'' AS notes,
coalesce(metrics->'environment.wind.speedTrue', null) as truewindspeed,
coalesce(metrics->'environment.wind.directionTrue', null) as truewinddirection,
coalesce(status, null) as status,
st_makepoint(longitude,latitude) AS geo_point
FROM api.metrics m
WHERE m.latitude IS NOT NULL
AND m.longitude IS NOT NULL
AND time >= _start::TIMESTAMPTZ
AND time <= _end::TIMESTAMPTZ
AND vessel_id = current_setting('vessel.id', false)
ORDER BY m.time ASC
)
) AS t;
-- Merge jsonb
SELECT log_geojson::jsonb || metrics_geojson::jsonb into _map;
-- output
SELECT
json_build_object(
'type', 'FeatureCollection',
'features', _map
) into _track_geojson;
END;
$logbook_geojson$ LANGUAGE plpgsql;
-- Update version
UPDATE public.app_settings

View File

@@ -0,0 +1,528 @@
---------------------------------------------------------------------------
-- Copyright 2021-2024 Francois Lacroix <xbgmsharp@gmail.com>
-- This file is part of PostgSail which is released under Apache License, Version 2.0 (the "License").
-- See file LICENSE or go to http://www.apache.org/licenses/LICENSE-2.0 for full license details.
--
-- Migration March 2024
--
-- List current database
select current_database();
-- connect to the DB
\c signalk
\echo 'Force timezone, just in case'
set timezone to 'UTC';
CREATE OR REPLACE FUNCTION public.process_lat_lon_fn(IN lon NUMERIC, IN lat NUMERIC,
OUT moorage_id INTEGER,
OUT moorage_type INTEGER,
OUT moorage_name TEXT,
OUT moorage_country TEXT
) AS $process_lat_lon$
DECLARE
stay_rec record;
--moorage_id INTEGER := NULL;
--moorage_type INTEGER := 1; -- Unknown
--moorage_name TEXT := NULL;
--moorage_country TEXT := NULL;
existing_rec record;
geo jsonb;
overpass jsonb;
BEGIN
RAISE NOTICE '-> process_lat_lon_fn';
IF lon IS NULL OR lat IS NULL THEN
RAISE WARNING '-> process_lat_lon_fn invalid input lon %, lat %', lon, lat;
--return NULL;
END IF;
-- Do we have an existing moorages within 300m of the new stay
FOR existing_rec in
SELECT
*
FROM api.moorages m
WHERE
m.latitude IS NOT NULL
AND m.longitude IS NOT NULL
AND m.geog IS NOT NULL
AND ST_DWithin(
Geography(ST_MakePoint(m.longitude, m.latitude)),
Geography(ST_MakePoint(lon, lat)),
300 -- in meters
)
AND m.vessel_id = current_setting('vessel.id', false)
ORDER BY id ASC
LOOP
-- found previous stay within 300m of the new moorage
IF existing_rec.id IS NOT NULL AND existing_rec.id > 0 THEN
RAISE NOTICE '-> process_lat_lon_fn found previous moorages within 300m %', existing_rec;
EXIT; -- exit loop
END IF;
END LOOP;
-- if with in 300m use existing name and stay_code
-- else insert new entry
IF existing_rec.id IS NOT NULL AND existing_rec.id > 0 THEN
RAISE NOTICE '-> process_lat_lon_fn found close by moorage using existing name and stay_code %', existing_rec;
moorage_id := existing_rec.id;
moorage_name := existing_rec.name;
moorage_type := existing_rec.stay_code;
ELSE
RAISE NOTICE '-> process_lat_lon_fn create new moorage';
-- query overpass api to guess moorage type
overpass := overpass_py_fn(lon::NUMERIC, lat::NUMERIC);
RAISE NOTICE '-> process_lat_lon_fn overpass name:[%] seamark:type:[%]', overpass->'name', overpass->'seamark:type';
moorage_type = 1; -- Unknown
IF overpass->>'seamark:type' = 'harbour' AND overpass->>'seamark:harbour:category' = 'marina' then
moorage_type = 4; -- Dock
ELSIF overpass->>'seamark:type' = 'mooring' AND overpass->>'seamark:mooring:category' = 'buoy' then
moorage_type = 3; -- Mooring Buoy
ELSIF overpass->>'seamark:type' ~ '(anchorage|anchor_berth|berth)' OR overpass->>'natural' ~ '(bay|beach)' then
moorage_type = 2; -- Anchor
ELSIF overpass->>'seamark:type' = 'mooring' then
moorage_type = 3; -- Mooring Buoy
ELSIF overpass->>'leisure' = 'marina' then
moorage_type = 4; -- Dock
END IF;
-- geo reverse _lng _lat
geo := reverse_geocode_py_fn('nominatim', lon::NUMERIC, lat::NUMERIC);
moorage_country := geo->>'country_code';
IF overpass->>'name:en' IS NOT NULL then
moorage_name = overpass->>'name:en';
ELSIF overpass->>'name' IS NOT NULL then
moorage_name = overpass->>'name';
ELSE
moorage_name := geo->>'name';
END IF;
RAISE NOTICE '-> process_lat_lon_fn output name:[%] type:[%]', moorage_name, moorage_type;
RAISE NOTICE '-> process_lat_lon_fn insert new moorage for [%] name:[%] type:[%]', current_setting('vessel.id', false), moorage_name, moorage_type;
-- Insert new moorage from stay
INSERT INTO api.moorages
(vessel_id, name, country, stay_code, reference_count, latitude, longitude, geog, overpass, nominatim)
VALUES (
current_setting('vessel.id', false),
coalesce(moorage_name, null),
coalesce(moorage_country, null),
moorage_type,
1,
lat,
lon,
Geography(ST_MakePoint(lon, lat)),
coalesce(overpass, null),
coalesce(geo, null)
) returning id into moorage_id;
-- Add moorage entry to process queue for reference
--INSERT INTO process_queue (channel, payload, stored, ref_id, processed)
-- VALUES ('new_moorage', moorage_id, now(), current_setting('vessel.id', true), now());
END IF;
--return json_build_object(
-- 'id', moorage_id,
-- 'name', moorage_name,
-- 'type', moorage_type
-- )::jsonb;
END;
$process_lat_lon$ LANGUAGE plpgsql;
CREATE or replace FUNCTION public.logbook_update_geojson_fn(IN _id integer, IN _start text, IN _end text,
OUT _track_geojson JSON
) AS $logbook_geojson$
declare
log_geojson jsonb;
metrics_geojson jsonb;
_map jsonb;
begin
-- GeoJson Feature Logbook linestring
SELECT
ST_AsGeoJSON(log.*) into log_geojson
FROM
( SELECT
id,name,
distance,
duration,
avg_speed,
max_speed,
max_wind_speed,
_from_time,
_to_time
_from_moorage_id,
_to_moorage_id,
notes,
track_geom
FROM api.logbook
WHERE id = _id
) AS log;
-- GeoJson Feature Metrics point
SELECT
json_agg(ST_AsGeoJSON(t.*)::json) into metrics_geojson
FROM (
( SELECT
time,
courseovergroundtrue,
speedoverground,
windspeedapparent,
longitude,latitude,
'' AS notes,
coalesce(metersToKnots((metrics->'environment.wind.speedTrue')::NUMERIC), null) as truewindspeed,
coalesce(radiantToDegrees((metrics->'environment.wind.directionTrue')::NUMERIC), null) as truewinddirection,
coalesce(status, null) as status,
st_makepoint(longitude,latitude) AS geo_point
FROM api.metrics m
WHERE m.latitude IS NOT NULL
AND m.longitude IS NOT NULL
AND time >= _start::TIMESTAMPTZ
AND time <= _end::TIMESTAMPTZ
AND vessel_id = current_setting('vessel.id', false)
ORDER BY m.time ASC
)
) AS t;
-- Merge jsonb
SELECT log_geojson::jsonb || metrics_geojson::jsonb into _map;
-- output
SELECT
json_build_object(
'type', 'FeatureCollection',
'features', _map
) into _track_geojson;
END;
$logbook_geojson$ LANGUAGE plpgsql;
CREATE OR REPLACE FUNCTION public.metersToKnots(IN meters NUMERIC)
RETURNS NUMERIC
AS $$
BEGIN
RETURN ROUND(((meters * 1.9438445) * 10) / 10, 2);
END
$$
LANGUAGE plpgsql IMMUTABLE;
-- Description
COMMENT ON FUNCTION
public.metersToKnots
IS 'convert speed meters/s To Knots';
CREATE OR REPLACE FUNCTION logbook_update_extra_json_fn(IN _id integer, IN _start text, IN _end text,
OUT _extra_json JSON
) AS $logbook_extra_json$
declare
obs_json jsonb default '{ "seaState": -1, "cloudCoverage": -1, "visibility": -1}'::jsonb;
log_json jsonb default '{}'::jsonb;
runtime_json jsonb default '{}'::jsonb;
metrics_json jsonb default '{}'::jsonb;
metric_rec record;
BEGIN
-- Calculate 'navigation.log' metrics
WITH
start_trip as (
-- Fetch 'navigation.log' start, first entry
SELECT key, value
FROM api.metrics m,
jsonb_each_text(m.metrics)
WHERE key ILIKE 'navigation.log'
AND time = _start::TIMESTAMPTZ
AND vessel_id = current_setting('vessel.id', false)
),
end_trip as (
-- Fetch 'navigation.log' end, last entry
SELECT key, value
FROM api.metrics m,
jsonb_each_text(m.metrics)
WHERE key ILIKE 'navigation.log'
AND time = _end::TIMESTAMPTZ
AND vessel_id = current_setting('vessel.id', false)
),
nm as (
-- calculate distance and convert meter to nautical miles
SELECT ((end_trip.value::NUMERIC - start_trip.value::numeric) * 0.00053996) as trip from start_trip,end_trip
)
-- Generate JSON
SELECT jsonb_build_object('navigation.log', trip) INTO log_json FROM nm;
RAISE NOTICE '-> logbook_update_extra_json_fn navigation.log: %', log_json;
-- Calculate engine hours from propulsion.%.runTime first entry
FOR metric_rec IN
SELECT key, value
FROM api.metrics m,
jsonb_each_text(m.metrics)
WHERE key ILIKE 'propulsion.%.runTime'
AND time = _start::TIMESTAMPTZ
AND vessel_id = current_setting('vessel.id', false)
LOOP
-- Engine Hours in seconds
RAISE NOTICE '-> logbook_update_extra_json_fn propulsion.*.runTime: %', metric_rec;
with
end_runtime AS (
-- Fetch 'propulsion.*.runTime' last entry
SELECT key, value
FROM api.metrics m,
jsonb_each_text(m.metrics)
WHERE key ILIKE metric_rec.key
AND time = _end::TIMESTAMPTZ
AND vessel_id = current_setting('vessel.id', false)
),
runtime AS (
-- calculate runTime Engine Hours as ISO duration
--SELECT (end_runtime.value::numeric - metric_rec.value::numeric) AS value FROM end_runtime
SELECT (((end_runtime.value::numeric - metric_rec.value::numeric) / 3600) * '1 hour'::interval)::interval as value FROM end_runtime
)
-- Generate JSON
SELECT jsonb_build_object(metric_rec.key, runtime.value) INTO runtime_json FROM runtime;
RAISE NOTICE '-> logbook_update_extra_json_fn key: %, value: %', metric_rec.key, runtime_json;
END LOOP;
-- Update logbook with extra value and return json
SELECT COALESCE(log_json::JSONB, '{}'::jsonb) || COALESCE(runtime_json::JSONB, '{}'::jsonb) INTO metrics_json;
SELECT jsonb_build_object('metrics', metrics_json, 'observations', obs_json) INTO _extra_json;
RAISE NOTICE '-> logbook_update_extra_json_fn log_json: %, runtime_json: %, _extra_json: %', log_json, runtime_json, _extra_json;
END;
$logbook_extra_json$ LANGUAGE plpgsql;
DROP FUNCTION IF EXISTS public.logbook_update_gpx_fn();
CREATE OR REPLACE FUNCTION metadata_upsert_trigger_fn() RETURNS trigger AS $metadata_upsert$
DECLARE
metadata_id integer;
metadata_active boolean;
BEGIN
-- Set client_id to new value to allow RLS
--PERFORM set_config('vessel.client_id', NEW.client_id, false);
-- UPSERT - Insert vs Update for Metadata
--RAISE NOTICE 'metadata_upsert_trigger_fn';
--PERFORM set_config('vessel.id', NEW.vessel_id, true);
--RAISE WARNING 'metadata_upsert_trigger_fn [%] [%]', current_setting('vessel.id', true), NEW;
SELECT m.id,m.active INTO metadata_id, metadata_active
FROM api.metadata m
WHERE m.vessel_id IS NOT NULL AND m.vessel_id = current_setting('vessel.id', true);
--RAISE NOTICE 'metadata_id is [%]', metadata_id;
IF metadata_id IS NOT NULL THEN
-- send notification if boat is back online
IF metadata_active is False THEN
-- Add monitor online entry to process queue for later notification
INSERT INTO process_queue (channel, payload, stored, ref_id)
VALUES ('monitoring_online', metadata_id, now(), current_setting('vessel.id', true));
END IF;
-- Update vessel metadata
UPDATE api.metadata
SET
name = NEW.name,
mmsi = NEW.mmsi,
client_id = NEW.client_id,
length = NEW.length,
beam = NEW.beam,
height = NEW.height,
ship_type = NEW.ship_type,
plugin_version = NEW.plugin_version,
signalk_version = NEW.signalk_version,
platform = REGEXP_REPLACE(NEW.platform, '[^a-zA-Z0-9\(\) ]', '', 'g'),
configuration = NEW.configuration,
-- time = NEW.time, ignore the time sent by the vessel as it is out of sync sometimes.
time = NOW(), -- overwrite the time sent by the vessel
active = true
WHERE id = metadata_id;
RETURN NULL; -- Ignore insert
ELSE
IF NEW.vessel_id IS NULL THEN
-- set vessel_id from jwt if not present in INSERT query
NEW.vessel_id := current_setting('vessel.id');
END IF;
-- Ignore and overwrite the time sent by the vessel
NEW.time := NOW();
-- Insert new vessel metadata
RETURN NEW; -- Insert new vessel metadata
END IF;
END;
$metadata_upsert$ LANGUAGE plpgsql;
CREATE OR REPLACE FUNCTION public.cron_windy_fn() RETURNS void AS $$
DECLARE
windy_rec record;
default_last_metric TIMESTAMPTZ := NOW() - interval '1 day';
last_metric TIMESTAMPTZ := NOW();
metric_rec record;
windy_metric jsonb;
app_settings jsonb;
user_settings jsonb;
windy_pws jsonb;
BEGIN
-- Check for new observations pending update
RAISE NOTICE 'cron_process_windy_fn';
-- Gather url from app settings
app_settings := get_app_settings_fn();
-- Find users with Windy active and with an active vessel
-- Map account id to Windy Station ID
FOR windy_rec in
SELECT
a.id,a.email,v.vessel_id,v.name,
COALESCE((a.preferences->'windy_last_metric')::TEXT, default_last_metric::TEXT) as last_metric
FROM auth.accounts a
LEFT JOIN auth.vessels AS v ON v.owner_email = a.email
LEFT JOIN api.metadata AS m ON m.vessel_id = v.vessel_id
WHERE (a.preferences->'public_windy')::boolean = True
AND m.active = True
LOOP
RAISE NOTICE '-> cron_process_windy_fn for [%]', windy_rec;
PERFORM set_config('vessel.id', windy_rec.vessel_id, false);
--RAISE WARNING 'public.cron_process_windy_rec_fn() scheduler vessel.id %, user.id', current_setting('vessel.id', false), current_setting('user.id', false);
-- Gather user settings
user_settings := get_user_settings_from_vesselid_fn(windy_rec.vessel_id::TEXT);
RAISE NOTICE '-> cron_process_windy_fn checking user_settings [%]', user_settings;
-- Get all metrics from the last windy_last_metric avg by 5 minutes
-- TODO json_agg to send all data in once, but issue with py jsonb transformation decimal.
FOR metric_rec in
SELECT time_bucket('5 minutes', m.time) AS time_bucket,
avg((m.metrics->'environment.outside.temperature')::numeric) AS temperature,
avg((m.metrics->'environment.outside.pressure')::numeric) AS pressure,
avg((m.metrics->'environment.outside.relativeHumidity')::numeric) AS rh,
avg((m.metrics->'environment.wind.directionTrue')::numeric) AS winddir,
avg((m.metrics->'environment.wind.speedTrue')::numeric) AS wind,
max((m.metrics->'environment.wind.speedTrue')::numeric) AS gust,
last(latitude, time) AS lat,
last(longitude, time) AS lng
FROM api.metrics m
WHERE vessel_id = windy_rec.vessel_id
AND m.time >= windy_rec.last_metric::TIMESTAMPTZ
GROUP BY time_bucket
ORDER BY time_bucket ASC LIMIT 100
LOOP
RAISE NOTICE '-> cron_process_windy_fn checking metrics [%]', metric_rec;
IF metric_rec.wind is null or metric_rec.temperature is null THEN
-- Ignore when there is no metrics.
-- Send notification
PERFORM send_notification_fn('windy_error'::TEXT, user_settings::JSONB);
-- Disable windy
PERFORM api.update_user_preferences_fn('{public_windy}'::TEXT, 'false'::TEXT);
RETURN;
END IF;
-- https://community.windy.com/topic/8168/report-your-weather-station-data-to-windy
-- temp from kelvin to celcuis
-- winddir from radiant to degres
-- rh from ratio to percentage
SELECT jsonb_build_object(
'dateutc', metric_rec.time_bucket,
'station', windy_rec.id,
'name', windy_rec.name,
'lat', metric_rec.lat,
'lon', metric_rec.lng,
'wind', metric_rec.wind,
'gust', metric_rec.gust,
'pressure', metric_rec.pressure,
'winddir', radiantToDegrees(metric_rec.winddir::numeric),
'temp', kelvinToCel(metric_rec.temperature::numeric),
'rh', valToPercent(metric_rec.rh::numeric)
) INTO windy_metric;
RAISE NOTICE '-> cron_process_windy_fn checking windy_metrics [%]', windy_metric;
SELECT windy_pws_py_fn(windy_metric, user_settings, app_settings) into windy_pws;
RAISE NOTICE '-> cron_process_windy_fn Windy PWS [%]', ((windy_pws->'header')::JSONB ? 'id');
IF NOT((user_settings->'settings')::JSONB ? 'windy') and ((windy_pws->'header')::JSONB ? 'id') then
RAISE NOTICE '-> cron_process_windy_fn new Windy PWS [%]', (windy_pws->'header')::JSONB->>'id';
-- Send metrics to Windy
PERFORM api.update_user_preferences_fn('{windy}'::TEXT, ((windy_pws->'header')::JSONB->>'id')::TEXT);
-- Send notification
PERFORM send_notification_fn('windy'::TEXT, user_settings::JSONB);
-- Refresh user settings after first success
user_settings := get_user_settings_from_vesselid_fn(windy_rec.vessel_id::TEXT);
END IF;
-- Record last metrics time
SELECT metric_rec.time_bucket INTO last_metric;
END LOOP;
PERFORM api.update_user_preferences_fn('{windy_last_metric}'::TEXT, last_metric::TEXT);
END LOOP;
END;
$$ language plpgsql;
DROP FUNCTION public.delete_vessel_fn;
CREATE OR REPLACE FUNCTION public.delete_vessel_fn(IN _vessel_id TEXT) RETURNS JSONB
AS $delete_vessel$
DECLARE
total_metrics INTEGER;
del_metrics INTEGER;
del_logs INTEGER;
del_stays INTEGER;
del_moorages INTEGER;
del_queue INTEGER;
out_json JSONB;
BEGIN
select count(*) INTO total_metrics from api.metrics m where vessel_id = _vessel_id;
WITH deleted AS (delete from api.metrics m where vessel_id = _vessel_id RETURNING *) SELECT count(*) INTO del_metrics FROM deleted;
WITH deleted AS (delete from api.logbook l where vessel_id = _vessel_id RETURNING *) SELECT count(*) INTO del_logs FROM deleted;
WITH deleted AS (delete from api.stays s where vessel_id = _vessel_id RETURNING *) SELECT count(*) INTO del_stays FROM deleted;
WITH deleted AS (delete from api.moorages m where vessel_id = _vessel_id RETURNING *) SELECT count(*) INTO del_moorages FROM deleted;
WITH deleted AS (delete from public.process_queue m where ref_id = _vessel_id RETURNING *) SELECT count(*) INTO del_queue FROM deleted;
SELECT jsonb_build_object('total_metrics', total_metrics,
'del_metrics', del_metrics,
'del_logs', del_logs,
'del_stays', del_stays,
'del_moorages', del_moorages,
'del_queue', del_queue) INTO out_json;
RETURN out_json;
END
$delete_vessel$ language plpgsql;
-- Description
COMMENT ON FUNCTION
public.delete_vessel_fn
IS 'Delete all vessel data (metrics,logbook,stays,moorages,process_queue) for a vessel_id';
DROP FUNCTION IF EXISTS public.cron_process_no_activity_fn();
CREATE OR REPLACE FUNCTION public.cron_process_no_activity_fn() RETURNS void AS $no_activity$
DECLARE
no_activity_rec record;
user_settings jsonb;
total_metrics INTEGER;
total_logs INTEGER;
del_metrics INTEGER;
out_json JSONB;
BEGIN
-- Check for vessel with no activity for more than 230 days
RAISE NOTICE 'cron_process_no_activity_fn';
FOR no_activity_rec in
SELECT
v.owner_email,m.name,m.vessel_id,m.time,a.first
FROM auth.accounts a
LEFT JOIN auth.vessels v ON v.owner_email = a.email
LEFT JOIN api.metadata m ON v.vessel_id = m.vessel_id
WHERE m.time < NOW() AT TIME ZONE 'UTC' - INTERVAL '230 DAYS'
AND v.owner_email <> 'demo@openplotter.cloud'
ORDER BY m.time DESC
LOOP
RAISE NOTICE '-> cron_process_no_activity_rec_fn for [%]', no_activity_rec;
SELECT json_build_object('email', no_activity_rec.owner_email, 'recipient', no_activity_rec.first) into user_settings;
RAISE NOTICE '-> debug cron_process_no_activity_rec_fn [%]', user_settings;
-- Send notification
PERFORM send_notification_fn('no_activity'::TEXT, user_settings::JSONB);
SELECT count(*) INTO total_metrics from api.metrics where vessel_id = no_activity_rec.vessel_id;
WITH deleted AS (delete from api.metrics m where vessel_id = no_activity_rec.vessel_id RETURNING *) SELECT count(*) INTO del_metrics FROM deleted;
SELECT count(*) INTO total_logs from api.logbook where vessel_id = no_activity_rec.vessel_id;
SELECT jsonb_build_object('total_metrics', total_metrics, 'total_logs', total_logs, 'del_metrics', del_metrics) INTO out_json;
RAISE NOTICE '-> debug cron_process_no_activity_rec_fn [%]', out_json;
END LOOP;
END;
$no_activity$ language plpgsql;
DROP FUNCTION public.delete_account_fn(text,text);
CREATE OR REPLACE FUNCTION public.delete_account_fn(IN _email TEXT, IN _vessel_id TEXT) RETURNS JSONB
AS $delete_account$
DECLARE
del_vessel_data JSONB;
del_meta INTEGER;
del_vessel INTEGER;
del_account INTEGER;
out_json JSONB;
BEGIN
SELECT public.delete_vessel_fn(_vessel_id) INTO del_vessel_data;
WITH deleted AS (delete from api.metadata where vessel_id = _vessel_id RETURNING *) SELECT count(*) INTO del_meta FROM deleted;
WITH deleted AS (delete from auth.vessels where vessel_id = _vessel_id RETURNING *) SELECT count(*) INTO del_vessel FROM deleted;
WITH deleted AS (delete from auth.accounts where email = _email RETURNING *) SELECT count(*) INTO del_account FROM deleted;
SELECT jsonb_build_object('del_metadata', del_meta,
'del_vessel', del_vessel,
'del_account', del_account) || del_vessel_data INTO out_json;
RETURN out_json;
END
$delete_account$ language plpgsql;
-- Description
COMMENT ON FUNCTION
public.delete_account_fn
IS 'Delete all data for a account by email and vessel_id';
-- Update version
UPDATE public.app_settings
SET value='0.7.1'
WHERE "name"='app.version';

View File

@@ -0,0 +1,625 @@
---------------------------------------------------------------------------
-- Copyright 2021-2024 Francois Lacroix <xbgmsharp@gmail.com>
-- This file is part of PostgSail which is released under Apache License, Version 2.0 (the "License").
-- See file LICENSE or go to http://www.apache.org/licenses/LICENSE-2.0 for full license details.
--
-- Migration April 2024
--
-- List current database
select current_database();
-- connect to the DB
\c signalk
\echo 'Timing mode is enabled'
\timing
\echo 'Force timezone, just in case'
set timezone to 'UTC';
UPDATE public.email_templates
SET email_content='Hello __RECIPIENT__,
Sorry!We could not convert your boat into a Windy Personal Weather Station due to missing data (temperature, wind or pressure).
Windy Personal Weather Station is now disable.'
WHERE "name"='windy_error';
CREATE OR REPLACE FUNCTION public.cron_windy_fn() RETURNS void AS $$
DECLARE
windy_rec record;
default_last_metric TIMESTAMPTZ := NOW() - interval '1 day';
last_metric TIMESTAMPTZ := NOW();
metric_rec record;
windy_metric jsonb;
app_settings jsonb;
user_settings jsonb;
windy_pws jsonb;
BEGIN
-- Check for new observations pending update
RAISE NOTICE 'cron_process_windy_fn';
-- Gather url from app settings
app_settings := get_app_settings_fn();
-- Find users with Windy active and with an active vessel
-- Map account id to Windy Station ID
FOR windy_rec in
SELECT
a.id,a.email,v.vessel_id,v.name,
COALESCE((a.preferences->'windy_last_metric')::TEXT, default_last_metric::TEXT) as last_metric
FROM auth.accounts a
LEFT JOIN auth.vessels AS v ON v.owner_email = a.email
LEFT JOIN api.metadata AS m ON m.vessel_id = v.vessel_id
WHERE (a.preferences->'public_windy')::boolean = True
AND m.active = True
LOOP
RAISE NOTICE '-> cron_process_windy_fn for [%]', windy_rec;
PERFORM set_config('vessel.id', windy_rec.vessel_id, false);
--RAISE WARNING 'public.cron_process_windy_rec_fn() scheduler vessel.id %, user.id', current_setting('vessel.id', false), current_setting('user.id', false);
-- Gather user settings
user_settings := get_user_settings_from_vesselid_fn(windy_rec.vessel_id::TEXT);
RAISE NOTICE '-> cron_process_windy_fn checking user_settings [%]', user_settings;
-- Get all metrics from the last windy_last_metric avg by 5 minutes
-- TODO json_agg to send all data in once, but issue with py jsonb transformation decimal.
FOR metric_rec in
SELECT time_bucket('5 minutes', m.time) AS time_bucket,
avg((m.metrics->'environment.outside.temperature')::numeric) AS temperature,
avg((m.metrics->'environment.outside.pressure')::numeric) AS pressure,
avg((m.metrics->'environment.outside.relativeHumidity')::numeric) AS rh,
avg((m.metrics->'environment.wind.directionTrue')::numeric) AS winddir,
avg((m.metrics->'environment.wind.speedTrue')::numeric) AS wind,
max((m.metrics->'environment.wind.speedTrue')::numeric) AS gust,
last(latitude, time) AS lat,
last(longitude, time) AS lng
FROM api.metrics m
WHERE vessel_id = windy_rec.vessel_id
AND m.time >= windy_rec.last_metric::TIMESTAMPTZ
GROUP BY time_bucket
ORDER BY time_bucket ASC LIMIT 100
LOOP
RAISE NOTICE '-> cron_process_windy_fn checking metrics [%]', metric_rec;
if metric_rec.wind is null or metric_rec.temperature is null
or metric_rec.pressure is null or metric_rec.rh is null then
-- Ignore when there is no metrics.
-- Send notification
PERFORM send_notification_fn('windy_error'::TEXT, user_settings::JSONB);
-- Disable windy
PERFORM api.update_user_preferences_fn('{public_windy}'::TEXT, 'false'::TEXT);
RETURN;
end if;
-- https://community.windy.com/topic/8168/report-your-weather-station-data-to-windy
-- temp from kelvin to Celsius
-- winddir from radiant to degrees
-- rh from ratio to percentage
SELECT jsonb_build_object(
'dateutc', metric_rec.time_bucket,
'station', windy_rec.id,
'name', windy_rec.name,
'lat', metric_rec.lat,
'lon', metric_rec.lng,
'wind', metric_rec.wind,
'gust', metric_rec.gust,
'pressure', metric_rec.pressure,
'winddir', radiantToDegrees(metric_rec.winddir::numeric),
'temp', kelvinToCel(metric_rec.temperature::numeric),
'rh', valToPercent(metric_rec.rh::numeric)
) INTO windy_metric;
RAISE NOTICE '-> cron_process_windy_fn checking windy_metrics [%]', windy_metric;
SELECT windy_pws_py_fn(windy_metric, user_settings, app_settings) into windy_pws;
RAISE NOTICE '-> cron_process_windy_fn Windy PWS [%]', ((windy_pws->'header')::JSONB ? 'id');
IF NOT((user_settings->'settings')::JSONB ? 'windy') and ((windy_pws->'header')::JSONB ? 'id') then
RAISE NOTICE '-> cron_process_windy_fn new Windy PWS [%]', (windy_pws->'header')::JSONB->>'id';
-- Send metrics to Windy
PERFORM api.update_user_preferences_fn('{windy}'::TEXT, ((windy_pws->'header')::JSONB->>'id')::TEXT);
-- Send notification
PERFORM send_notification_fn('windy'::TEXT, user_settings::JSONB);
-- Refresh user settings after first success
user_settings := get_user_settings_from_vesselid_fn(windy_rec.vessel_id::TEXT);
END IF;
-- Record last metrics time
SELECT metric_rec.time_bucket INTO last_metric;
END LOOP;
PERFORM api.update_user_preferences_fn('{windy_last_metric}'::TEXT, last_metric::TEXT);
END LOOP;
END;
$$ language plpgsql;
-- Add security definer, run this function as admin to avoid weird bug
-- ERROR: variable not found in subplan target list
CREATE OR REPLACE FUNCTION api.delete_logbook_fn(IN _id integer) RETURNS BOOLEAN AS $delete_logbook$
DECLARE
logbook_rec record;
previous_stays_id numeric;
current_stays_departed text;
current_stays_id numeric;
current_stays_active boolean;
BEGIN
-- If _id is not NULL
IF _id IS NULL OR _id < 1 THEN
RAISE WARNING '-> delete_logbook_fn invalid input %', _id;
RETURN FALSE;
END IF;
-- Get the logbook record with all necessary fields exist
SELECT * INTO logbook_rec
FROM api.logbook
WHERE id = _id;
-- Ensure the query is successful
IF logbook_rec.vessel_id IS NULL THEN
RAISE WARNING '-> delete_logbook_fn invalid logbook %', _id;
RETURN FALSE;
END IF;
-- Update logbook
UPDATE api.logbook l
SET notes = 'mark for deletion'
WHERE l.vessel_id = current_setting('vessel.id', false)
AND id = logbook_rec.id;
-- Update metrics status to moored
-- This generate an error when run as user_role "variable not found in subplan target list"
UPDATE api.metrics
SET status = 'moored'
WHERE time >= logbook_rec._from_time
AND time <= logbook_rec._to_time
AND vessel_id = current_setting('vessel.id', false);
-- Get related stays
SELECT id,departed,active INTO current_stays_id,current_stays_departed,current_stays_active
FROM api.stays s
WHERE s.vessel_id = current_setting('vessel.id', false)
AND s.arrived = logbook_rec._to_time;
-- Update related stays
UPDATE api.stays s
SET notes = 'mark for deletion'
WHERE s.vessel_id = current_setting('vessel.id', false)
AND s.arrived = logbook_rec._to_time;
-- Find previous stays
SELECT id INTO previous_stays_id
FROM api.stays s
WHERE s.vessel_id = current_setting('vessel.id', false)
AND s.arrived < logbook_rec._to_time
ORDER BY s.arrived DESC LIMIT 1;
-- Update previous stays with the departed time from current stays
-- and set the active state from current stays
UPDATE api.stays
SET departed = current_stays_departed::TIMESTAMPTZ,
active = current_stays_active
WHERE vessel_id = current_setting('vessel.id', false)
AND id = previous_stays_id;
-- Clean up, remove invalid logbook and stay entry
DELETE FROM api.logbook WHERE id = logbook_rec.id;
RAISE WARNING '-> delete_logbook_fn delete logbook [%]', logbook_rec.id;
DELETE FROM api.stays WHERE id = current_stays_id;
RAISE WARNING '-> delete_logbook_fn delete stays [%]', current_stays_id;
-- Clean up, Subtract (-1) moorages ref count
UPDATE api.moorages
SET reference_count = reference_count - 1
WHERE vessel_id = current_setting('vessel.id', false)
AND id = previous_stays_id;
RETURN TRUE;
END;
$delete_logbook$ LANGUAGE plpgsql security definer;
-- Allow users to update certain columns on specific TABLES on API schema add reference_count, when deleting a log
GRANT UPDATE (name, notes, stay_code, home_flag, reference_count) ON api.moorages TO user_role;
-- Allow users to update certain columns on specific TABLES on API schema add track_geojson
GRANT UPDATE (name, _from, _to, notes, track_geojson) ON api.logbook TO user_role;
DROP FUNCTION IF EXISTS api.timelapse2_fn;
CREATE OR REPLACE FUNCTION api.timelapse2_fn(
IN start_log INTEGER DEFAULT NULL,
IN end_log INTEGER DEFAULT NULL,
IN start_date TEXT DEFAULT NULL,
IN end_date TEXT DEFAULT NULL,
OUT geojson JSONB) RETURNS JSONB AS $timelapse2$
DECLARE
_geojson jsonb;
BEGIN
-- Using sub query to force id order by time
-- User can now directly edit the json to add comment or remove track point
-- Merge json track_geojson with Geometry Point into a single GeoJSON Points
--raise WARNING 'input % % %' , start_log, end_log, public.isnumeric(end_log::text);
IF start_log IS NOT NULL AND end_log IS NULL THEN
end_log := start_log;
END IF;
IF start_date IS NOT NULL AND end_date IS NULL THEN
end_date := start_date;
END IF;
--raise WARNING 'input % % %' , start_log, end_log, public.isnumeric(end_log::text);
IF start_log IS NOT NULL AND public.isnumeric(start_log::text) AND public.isnumeric(end_log::text) THEN
SELECT jsonb_agg(
jsonb_build_object('type', 'Feature',
'properties', f->'properties',
'geometry', jsonb_build_object( 'coordinates', f->'geometry'->'coordinates', 'type', 'Point'))
) INTO _geojson
FROM (
SELECT jsonb_array_elements(track_geojson->'features') AS f
FROM api.logbook l
WHERE l.id >= start_log
AND l.id <= end_log
AND l.track_geojson IS NOT NULL
ORDER BY l._from_time ASC
) AS sub
WHERE (f->'geometry'->>'type') = 'Point';
ELSIF start_date IS NOT NULL AND public.isdate(start_date::text) AND public.isdate(end_date::text) THEN
SELECT jsonb_agg(
jsonb_build_object('type', 'Feature',
'properties', f->'properties',
'geometry', jsonb_build_object( 'coordinates', f->'geometry'->'coordinates', 'type', 'Point'))
) INTO _geojson
FROM (
SELECT jsonb_array_elements(track_geojson->'features') AS f
FROM api.logbook l
WHERE l._from_time >= start_date::TIMESTAMPTZ
AND l._to_time <= end_date::TIMESTAMPTZ + interval '23 hours 59 minutes'
AND l.track_geojson IS NOT NULL
ORDER BY l._from_time ASC
) AS sub
WHERE (f->'geometry'->>'type') = 'Point';
ELSE
SELECT jsonb_agg(
jsonb_build_object('type', 'Feature',
'properties', f->'properties',
'geometry', jsonb_build_object( 'coordinates', f->'geometry'->'coordinates', 'type', 'Point'))
) INTO _geojson
FROM (
SELECT jsonb_array_elements(track_geojson->'features') AS f
FROM api.logbook l
WHERE l.track_geojson IS NOT NULL
ORDER BY l._from_time ASC
) AS sub
WHERE (f->'geometry'->>'type') = 'Point';
END IF;
-- Return a GeoJSON MultiLineString
-- result _geojson [null, null]
--RAISE WARNING 'result _geojson %' , _geojson;
SELECT jsonb_build_object(
'type', 'FeatureCollection',
'features', _geojson ) INTO geojson;
END;
$timelapse2$ LANGUAGE plpgsql;
-- Description
COMMENT ON FUNCTION
api.timelapse2_fn
IS 'Export all selected logs geojson `track_geojson` to a geojson as points including properties';
-- Allow timelapse2_fn execution for user_role and api_anonymous (public replay)
GRANT EXECUTE ON ALL FUNCTIONS IN SCHEMA api TO user_role;
GRANT EXECUTE ON FUNCTION api.timelapse2_fn TO api_anonymous;
DROP FUNCTION IF EXISTS public.process_logbook_queue_fn;
CREATE OR REPLACE FUNCTION public.process_logbook_queue_fn(IN _id integer) RETURNS void AS $process_logbook_queue$
DECLARE
logbook_rec record;
from_name text;
to_name text;
log_name text;
from_moorage record;
to_moorage record;
avg_rec record;
geo_rec record;
log_settings jsonb;
user_settings jsonb;
geojson jsonb;
extra_json jsonb;
trip_note jsonb;
from_moorage_note jsonb;
to_moorage_note jsonb;
BEGIN
-- If _id is not NULL
IF _id IS NULL OR _id < 1 THEN
RAISE WARNING '-> process_logbook_queue_fn invalid input %', _id;
RETURN;
END IF;
-- Get the logbook record with all necessary fields exist
SELECT * INTO logbook_rec
FROM api.logbook
WHERE active IS false
AND id = _id
AND _from_lng IS NOT NULL
AND _from_lat IS NOT NULL
AND _to_lng IS NOT NULL
AND _to_lat IS NOT NULL;
-- Ensure the query is successful
IF logbook_rec.vessel_id IS NULL THEN
RAISE WARNING '-> process_logbook_queue_fn invalid logbook %', _id;
RETURN;
END IF;
PERFORM set_config('vessel.id', logbook_rec.vessel_id, false);
--RAISE WARNING 'public.process_logbook_queue_fn() scheduler vessel.id %, user.id', current_setting('vessel.id', false), current_setting('user.id', false);
-- Calculate logbook data average and geo
-- Update logbook entry with the latest metric data and calculate data
avg_rec := logbook_update_avg_fn(logbook_rec.id, logbook_rec._from_time::TEXT, logbook_rec._to_time::TEXT);
geo_rec := logbook_update_geom_distance_fn(logbook_rec.id, logbook_rec._from_time::TEXT, logbook_rec._to_time::TEXT);
-- Do we have an existing moorage within 300m of the new log
-- generate logbook name, concat _from_location and _to_location from moorage name
from_moorage := process_lat_lon_fn(logbook_rec._from_lng::NUMERIC, logbook_rec._from_lat::NUMERIC);
to_moorage := process_lat_lon_fn(logbook_rec._to_lng::NUMERIC, logbook_rec._to_lat::NUMERIC);
SELECT CONCAT(from_moorage.moorage_name, ' to ' , to_moorage.moorage_name) INTO log_name;
-- Process `propulsion.*.runTime` and `navigation.log`
-- Calculate extra json
extra_json := logbook_update_extra_json_fn(logbook_rec.id, logbook_rec._from_time::TEXT, logbook_rec._to_time::TEXT);
RAISE NOTICE 'Updating valid logbook entry logbook id:[%] start:[%] end:[%]', logbook_rec.id, logbook_rec._from_time, logbook_rec._to_time;
UPDATE api.logbook
SET
duration = (logbook_rec._to_time::TIMESTAMPTZ - logbook_rec._from_time::TIMESTAMPTZ),
avg_speed = avg_rec.avg_speed,
max_speed = avg_rec.max_speed,
max_wind_speed = avg_rec.max_wind_speed,
_from = from_moorage.moorage_name,
_from_moorage_id = from_moorage.moorage_id,
_to_moorage_id = to_moorage.moorage_id,
_to = to_moorage.moorage_name,
name = log_name,
track_geom = geo_rec._track_geom,
distance = geo_rec._track_distance,
extra = extra_json,
notes = NULL -- reset pre_log process
WHERE id = logbook_rec.id;
-- GeoJSON require track_geom field
geojson := logbook_update_geojson_fn(logbook_rec.id, logbook_rec._from_time::TEXT, logbook_rec._to_time::TEXT);
UPDATE api.logbook
SET
track_geojson = geojson
WHERE id = logbook_rec.id;
-- Add trip details name as note for the first geometry point entry from the GeoJSON
SELECT format('{"trip": { "name": "%s", "duration": "%s", "distance": "%s" }}', logbook_rec.name, logbook_rec.duration, logbook_rec.distance) into trip_note;
-- Update the properties of the first feature
UPDATE api.logbook
SET track_geojson = jsonb_set(
track_geojson,
'{features, 1, properties}',
(track_geojson -> 'features' -> 1 -> 'properties' || trip_note)::jsonb
)
WHERE id = logbook_rec.id
and track_geojson -> 'features' -> 1 -> 'geometry' ->> 'type' = 'Point';
-- Add moorage name as note for the third and last entry of the GeoJSON
SELECT format('{"notes": "%s"}', from_moorage.moorage_name) into from_moorage_note;
-- Update the properties of the third feature, the second with geometry point
UPDATE api.logbook
SET track_geojson = jsonb_set(
track_geojson,
'{features, 2, properties}',
(track_geojson -> 'features' -> 2 -> 'properties' || from_moorage_note)::jsonb
)
WHERE id = logbook_rec.id
AND track_geojson -> 'features' -> 2 -> 'geometry' ->> 'type' = 'Point';
-- Update the note properties of the last feature with geometry point
SELECT format('{"notes": "%s"}', to_moorage.moorage_name) into to_moorage_note;
UPDATE api.logbook
SET track_geojson = jsonb_set(
track_geojson,
'{features, -1, properties}',
CASE
WHEN COALESCE((track_geojson -> 'features' -> -1 -> 'properties' ->> 'notes'), '') = '' THEN
(track_geojson -> 'features' -> -1 -> 'properties' || to_moorage_note)::jsonb
ELSE
track_geojson -> 'features' -> -1 -> 'properties'
END
)
WHERE id = logbook_rec.id
AND track_geojson -> 'features' -> -1 -> 'geometry' ->> 'type' = 'Point';
-- Prepare notification, gather user settings
SELECT json_build_object('logbook_name', log_name, 'logbook_link', logbook_rec.id) into log_settings;
user_settings := get_user_settings_from_vesselid_fn(logbook_rec.vessel_id::TEXT);
SELECT user_settings::JSONB || log_settings::JSONB into user_settings;
RAISE NOTICE '-> debug process_logbook_queue_fn get_user_settings_from_vesselid_fn [%]', user_settings;
RAISE NOTICE '-> debug process_logbook_queue_fn log_settings [%]', log_settings;
-- Send notification
PERFORM send_notification_fn('logbook'::TEXT, user_settings::JSONB);
-- Process badges
RAISE NOTICE '-> debug process_logbook_queue_fn user_settings [%]', user_settings->>'email'::TEXT;
PERFORM set_config('user.email', user_settings->>'email'::TEXT, false);
PERFORM badges_logbook_fn(logbook_rec.id, logbook_rec._to_time::TEXT);
PERFORM badges_geom_fn(logbook_rec.id, logbook_rec._to_time::TEXT);
END;
$process_logbook_queue$ LANGUAGE plpgsql;
-- Description
COMMENT ON FUNCTION
public.process_logbook_queue_fn
IS 'Update logbook details when completed, logbook_update_avg_fn, logbook_update_geom_distance_fn, reverse_geocode_py_fn';
-- Update the pre.check for the new timelapse function
CREATE OR REPLACE FUNCTION public.check_jwt() RETURNS void AS $$
-- Prevent unregister user or unregister vessel access
-- Allow anonymous access
-- Need to be refactor and simplify, specially the anonymous part.
DECLARE
_role name;
_email text;
anonymous record;
_path name;
_vid text;
_vname text;
boat TEXT;
_pid INTEGER := 0; -- public_id
_pvessel TEXT := NULL; -- public_type
_ptype TEXT := NULL; -- public_type
_ppath BOOLEAN := False; -- public_path
_pvalid BOOLEAN := False; -- public_valid
_pheader text := NULL; -- public_header
valid_public_type BOOLEAN := False;
account_rec record;
vessel_rec record;
BEGIN
-- Extract email and role from jwt token
--RAISE WARNING 'check_jwt jwt %', current_setting('request.jwt.claims', true);
SELECT current_setting('request.jwt.claims', true)::json->>'email' INTO _email;
PERFORM set_config('user.email', _email, false);
SELECT current_setting('request.jwt.claims', true)::json->>'role' INTO _role;
--RAISE WARNING 'jwt email %', current_setting('request.jwt.claims', true)::json->>'email';
--RAISE WARNING 'jwt role %', current_setting('request.jwt.claims', true)::json->>'role';
--RAISE WARNING 'cur_user %', current_user;
--TODO SELECT current_setting('request.jwt.uid', true)::json->>'uid' INTO _user_id;
--TODO RAISE WARNING 'jwt user_id %', current_setting('request.jwt.uid', true)::json->>'uid';
--TODO SELECT current_setting('request.jwt.vid', true)::json->>'vid' INTO _vessel_id;
--TODO RAISE WARNING 'jwt vessel_id %', current_setting('request.jwt.vid', true)::json->>'vid';
IF _role = 'user_role' THEN
-- Check the user exist in the accounts table
SELECT * INTO account_rec
FROM auth.accounts
WHERE auth.accounts.email = _email;
IF account_rec.email IS NULL THEN
RAISE EXCEPTION 'Invalid user'
USING HINT = 'Unknown user or password';
END IF;
-- Set session variables
PERFORM set_config('user.id', account_rec.user_id, false);
SELECT current_setting('request.path', true) into _path;
--RAISE WARNING 'req path %', current_setting('request.path', true);
-- Function allow without defined vessel like for anonymous role
IF _path ~ '^\/rpc\/(login|signup|recover|reset)$' THEN
RETURN;
END IF;
-- Function allow without defined vessel as user role
-- openapi doc, user settings, otp code and vessel registration
IF _path = '/rpc/settings_fn'
OR _path = '/rpc/register_vessel'
OR _path = '/rpc/update_user_preferences_fn'
OR _path = '/rpc/versions_fn'
OR _path = '/rpc/email_fn'
OR _path = '/' THEN
RETURN;
END IF;
-- Check a vessel and user exist
SELECT auth.vessels.* INTO vessel_rec
FROM auth.vessels, auth.accounts
WHERE auth.vessels.owner_email = auth.accounts.email
AND auth.accounts.email = _email;
-- check if boat exist yet?
IF vessel_rec.owner_email IS NULL THEN
-- Return http status code 551 with message
RAISE sqlstate 'PT551' using
message = 'Vessel Required',
detail = 'Invalid vessel',
hint = 'Unknown vessel';
--RETURN; -- ignore if not exist
END IF;
-- Redundant?
IF vessel_rec.vessel_id IS NULL THEN
RAISE EXCEPTION 'Invalid vessel'
USING HINT = 'Unknown vessel id';
END IF;
-- Set session variables
PERFORM set_config('vessel.id', vessel_rec.vessel_id, false);
PERFORM set_config('vessel.name', vessel_rec.name, false);
--RAISE WARNING 'public.check_jwt() user_role vessel.id [%]', current_setting('vessel.id', false);
--RAISE WARNING 'public.check_jwt() user_role vessel.name [%]', current_setting('vessel.name', false);
ELSIF _role = 'vessel_role' THEN
SELECT current_setting('request.path', true) into _path;
--RAISE WARNING 'req path %', current_setting('request.path', true);
-- Function allow without defined vessel like for anonymous role
IF _path ~ '^\/rpc\/(oauth_\w+)$' THEN
RETURN;
END IF;
-- Extract vessel_id from jwt token
SELECT current_setting('request.jwt.claims', true)::json->>'vid' INTO _vid;
-- Check the vessel and user exist
SELECT auth.vessels.* INTO vessel_rec
FROM auth.vessels, auth.accounts
WHERE auth.vessels.owner_email = auth.accounts.email
AND auth.accounts.email = _email
AND auth.vessels.vessel_id = _vid;
IF vessel_rec.owner_email IS NULL THEN
RAISE EXCEPTION 'Invalid vessel'
USING HINT = 'Unknown vessel owner_email';
END IF;
PERFORM set_config('vessel.id', vessel_rec.vessel_id, false);
PERFORM set_config('vessel.name', vessel_rec.name, false);
--RAISE WARNING 'public.check_jwt() user_role vessel.name %', current_setting('vessel.name', false);
--RAISE WARNING 'public.check_jwt() user_role vessel.id %', current_setting('vessel.id', false);
ELSIF _role = 'api_anonymous' THEN
--RAISE WARNING 'public.check_jwt() api_anonymous';
-- Check if path is a valid allow anonymous path
SELECT current_setting('request.path', true) ~ '^/(logs_view|log_view|rpc/timelapse_fn|rpc/timelapse2_fn|monitoring_view|stats_logs_view|stats_moorages_view|rpc/stats_logs_fn)$' INTO _ppath;
if _ppath is True then
-- Check is custom header is present and valid
SELECT current_setting('request.headers', true)::json->>'x-is-public' into _pheader;
RAISE WARNING 'public.check_jwt() api_anonymous _pheader [%]', _pheader;
if _pheader is null then
RAISE EXCEPTION 'Invalid public_header'
USING HINT = 'Stop being so evil and maybe you can log in';
end if;
SELECT convert_from(decode(_pheader, 'base64'), 'utf-8')
~ '\w+,public_(logs|logs_list|stats|timelapse|monitoring),\d+$' into _pvalid;
RAISE WARNING 'public.check_jwt() api_anonymous _pvalid [%]', _pvalid;
if _pvalid is null or _pvalid is False then
RAISE EXCEPTION 'Invalid public_valid'
USING HINT = 'Stop being so evil and maybe you can log in';
end if;
WITH regex AS (
SELECT regexp_match(
convert_from(
decode(_pheader, 'base64'), 'utf-8'),
'(\w+),(public_(logs|logs_list|stats|timelapse|monitoring)),(\d+)$') AS match
)
SELECT match[1], match[2], match[4] into _pvessel, _ptype, _pid
FROM regex;
RAISE WARNING 'public.check_jwt() api_anonymous [%] [%] [%]', _pvessel, _ptype, _pid;
if _pvessel is not null and _ptype is not null then
-- Everything seem fine, get the vessel_id base on the vessel name.
SELECT _ptype::name = any(enum_range(null::public_type)::name[]) INTO valid_public_type;
IF valid_public_type IS False THEN
-- Ignore entry if type is invalid
RAISE EXCEPTION 'Invalid public_type'
USING HINT = 'Stop being so evil and maybe you can log in';
END IF;
-- Check if boat name match public_vessel name
boat := '^' || _pvessel || '$';
IF _ptype ~ '^public_(logs|timelapse)$' AND _pid > 0 THEN
WITH log as (
SELECT vessel_id from api.logbook l where l.id = _pid
)
SELECT v.vessel_id, v.name into anonymous
FROM auth.accounts a, auth.vessels v, jsonb_each_text(a.preferences) as prefs, log l
WHERE v.vessel_id = l.vessel_id
AND a.email = v.owner_email
AND a.preferences->>'public_vessel'::text ~* boat
AND prefs.key = _ptype::TEXT
AND prefs.value::BOOLEAN = true;
RAISE WARNING '-> ispublic_fn public_logs output boat:[%], type:[%], result:[%]', _pvessel, _ptype, anonymous;
IF anonymous.vessel_id IS NOT NULL THEN
PERFORM set_config('vessel.id', anonymous.vessel_id, false);
PERFORM set_config('vessel.name', anonymous.name, false);
RETURN;
END IF;
ELSE
SELECT v.vessel_id, v.name into anonymous
FROM auth.accounts a, auth.vessels v, jsonb_each_text(a.preferences) as prefs
WHERE a.email = v.owner_email
AND a.preferences->>'public_vessel'::text ~* boat
AND prefs.key = _ptype::TEXT
AND prefs.value::BOOLEAN = true;
RAISE WARNING '-> ispublic_fn output boat:[%], type:[%], result:[%]', _pvessel, _ptype, anonymous;
IF anonymous.vessel_id IS NOT NULL THEN
PERFORM set_config('vessel.id', anonymous.vessel_id, false);
PERFORM set_config('vessel.name', anonymous.name, false);
RETURN;
END IF;
END IF;
RAISE sqlstate 'PT404' using message = 'unknown resource';
END IF; -- end anonymous path
END IF;
ELSIF _role <> 'api_anonymous' THEN
RAISE EXCEPTION 'Invalid role'
USING HINT = 'Stop being so evil and maybe you can log in';
END IF;
END
$$ language plpgsql security definer;
-- Description
COMMENT ON FUNCTION
public.check_jwt
IS 'PostgREST API db-pre-request check, set_config according to role (api_anonymous,vessel_role,user_role)';
GRANT EXECUTE ON FUNCTION public.check_jwt() TO api_anonymous;
-- Update version
UPDATE public.app_settings
SET value='0.7.2'
WHERE "name"='app.version';

View File

@@ -28,3 +28,5 @@ ALTER ROLE authenticator WITH PASSWORD '${PGSAIL_AUTHENTICATOR_PASSWORD}';
ALTER ROLE grafana WITH PASSWORD '${PGSAIL_GRAFANA_PASSWORD}';
ALTER ROLE grafana_auth WITH PASSWORD '${PGSAIL_GRAFANA_AUTH_PASSWORD}';
END
curl -s -XPOST -Hx-pgsail:${PGSAIL_VERSION} https://api.openplotter.cloud/rpc/telemetry_fn

View File

@@ -1 +1 @@
0.7.0
0.7.1

File diff suppressed because one or more lines are too long

View File

@@ -34,7 +34,7 @@ const fs = require('fs');
{ cname: process.env.PGSAIL_API_URI, name: "PostgSail unit test kapla",
signin: { email: 'demo+kapla@openplotter.cloud', pass: 'test', firstname:'First_kapla', lastname:'Last_kapla'},
login: { email: 'demo+kapla@openplotter.cloud', pass: 'test'},
vessel: { vessel_email: "demo+kapla@openplotter.cloud", vessel_mmsi: "test", vessel_name: "kapla"},
vessel: { vessel_email: "demo+kapla@openplotter.cloud", vessel_mmsi: "test", vessel_name: " kapla "},
preferences: { key: '{email_notifications}', value: false }, /* Disable email_notifications */
vessel_metadata: {
name: "kapla",
@@ -174,7 +174,7 @@ const fs = require('fs');
{ cname: process.env.PGSAIL_API_URI, name: "PostgSail unit test, aava",
signin: { email: 'demo+aava@openplotter.cloud', pass: 'test', firstname:'first_aava', lastname:'last_aava'},
login: { email: 'demo+aava@openplotter.cloud', pass: 'test'},
vessel: { vessel_email: "demo+aava@openplotter.cloud", vessel_mmsi: "787654321", vessel_name: "aava"},
vessel: { vessel_email: "demo+aava@openplotter.cloud", vessel_mmsi: "787654321", vessel_name: " aava "},
preferences: { key: '{email_notifications}', value: false }, /* Disable email_notifications */
vessel_metadata: {
name: "aava",

View File

@@ -347,7 +347,7 @@ var moment = require("moment");
res.header["content-type"].should.match(new RegExp("json", "g"));
res.header["server"].should.match(new RegExp("postgrest", "g"));
should.exist(res.body.token);
res.body.token.should.match(user_jwt);
//res.body.token.should.match(user_jwt);
console.log(user_jwt);
should.exist(user_jwt);
done(err);

View File

@@ -28,7 +28,7 @@ avg_speed | 3.6357142857142852
max_speed | 6.1
max_wind_speed | 22.1
notes |
extra | {"metrics": {"propulsion.main.runTime": 10}, "observations": {"seaState": -1, "visibility": -1, "cloudCoverage": -1}}
extra | {"metrics": {"propulsion.main.runTime": "PT10S"}, "observations": {"seaState": -1, "visibility": -1, "cloudCoverage": -1}}
-[ RECORD 2 ]--+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
name | Norra hamnen to Ekenäs
_from_time | t
@@ -41,7 +41,7 @@ avg_speed | 5.4523809523809526
max_speed | 6.5
max_wind_speed | 37.2
notes |
extra | {"metrics": {"propulsion.main.runTime": 11}, "observations": {"seaState": -1, "visibility": -1, "cloudCoverage": -1}}
extra | {"metrics": {"propulsion.main.runTime": "PT11S"}, "observations": {"seaState": -1, "visibility": -1, "cloudCoverage": -1}}
stays
-[ RECORD 1 ]
@@ -91,12 +91,12 @@ DROP TABLE
stats_logs_fn |
update_logbook_observations_fn
-[ RECORD 1 ]----------------------------------------------------------------------------------------------------------------
extra | {"metrics": {"propulsion.main.runTime": 10}, "observations": {"seaState": -1, "visibility": -1, "cloudCoverage": -1}}
-[ RECORD 1 ]---------------------------------------------------------------------------------------------------------------------
extra | {"metrics": {"propulsion.main.runTime": "PT10S"}, "observations": {"seaState": -1, "visibility": -1, "cloudCoverage": -1}}
-[ RECORD 1 ]------------------+--
update_logbook_observations_fn | t
-[ RECORD 1 ]---------------------------------------------------------------------------------------------------------------
extra | {"metrics": {"propulsion.main.runTime": 10}, "observations": {"seaState": -1, "visibility": -1, "cloudCoverage": 1}}
-[ RECORD 1 ]--------------------------------------------------------------------------------------------------------------------
extra | {"metrics": {"propulsion.main.runTime": "PT10S"}, "observations": {"seaState": -1, "visibility": -1, "cloudCoverage": 1}}

View File

@@ -215,7 +215,7 @@ api.moorages
id | 1
vessel_id | t
name | patch moorage name 3
country |
country | fi
stay_code | 2
stay_duration | PT1M
reference_count | 1
@@ -228,7 +228,7 @@ notes | new moorage note 3
id | 2
vessel_id | t
name | Norra hamnen
country |
country | fi
stay_code | 4
stay_duration | PT2M
reference_count | 2

View File

@@ -51,3 +51,12 @@ select count(*) from api.monitoring_temperatures;
-- Test monitoring for user
--select * from api.monitoring_humidity;
select count(*) from api.monitoring_humidity;
\echo 'Test metersToKnots'
select public.metersToKnots(1);
\echo 'Test radiantToDegrees'
select public.radiantToDegrees(1);
\echo 'Test valToPercent'
select public.valToPercent(1);

View File

@@ -36,3 +36,15 @@ Test monitoring_humidity for user
-[ RECORD 1 ]
count | 0
Test metersToKnots
-[ RECORD 1 ]-+-----
meterstoknots | 1.94
Test radiantToDegrees
-[ RECORD 1 ]----+---
radianttodegrees | 57
Test valToPercent
-[ RECORD 1 ]+----
valtopercent | 100

View File

@@ -53,7 +53,7 @@ Schema | public
Description | PostGIS geometry and geography spatial types and functions
-[ RECORD 9 ]--------------------------------------------------------------------------------------
Name | timescaledb
Version | 2.14.1
Version | 2.14.2
Schema | public
Description | Enables scalable inserts and complex queries for time-series data (Community Edition)
-[ RECORD 10 ]-------------------------------------------------------------------------------------
@@ -645,12 +645,12 @@ overpass_py_fn | {"name": "Port de la Ginesta", "type": "multipolygon", "leisure
overpass_py_fn | {"name": "Norra hamnen", "leisure": "marina"}
-[ RECORD 1 ]----------------------------------------------------------------------------------------------------------------------------------------------
versions_fn | {"api_version" : "0.6.1", "sys_version" : "PostgreSQL 16.2", "timescaledb" : "2.14.1", "postgis" : "3.4.2", "postgrest" : "PostgREST 12.0.2"}
versions_fn | {"api_version" : "0.7.2", "sys_version" : "PostgreSQL 16.2", "timescaledb" : "2.14.2", "postgis" : "3.4.2", "postgrest" : "PostgREST 12.0.2"}
-[ RECORD 1 ]-----------------
api_version | 0.7.0
api_version | 0.7.2
sys_version | PostgreSQL 16.2
timescaledb | 2.14.1
timescaledb | 2.14.2
postgis | 3.4.2
postgrest | PostgREST 12.0.2

View File

@@ -195,8 +195,8 @@ fi
# Generate and update mermaid schema documentation
/root/go/bin/mermerd --runConfig ../docs/ERD/mermerdConfig.yaml
echo $?
echo 0
#echo $?
echo 0 # not working in github-actions
if [ $? -eq 0 ]; then
cp postgsail.md ../docs/ERD/postgsail.md
echo postgsail.md OK
@@ -204,3 +204,5 @@ else
echo postgsail.md FAILED
exit 1
fi
#npm i -D schemalint && npx schemalint