mirrors/postgsail
1
0
Fork 0
mirror of https://github.com/xbgmsharp/postgsail.git synced 2025-09-17 03:07:47 +00:00
Code Issues Packages Projects Releases Wiki Activity

Merge pull request #22 from lokasoft/main

Browse Source 
Added self hosted update guide
This commit is contained in:
xbgmsharp
2025-08-31 09:20:05 +02:00
committed by GitHub
parent 883c875e39 5f25a57c3c
commit ee44420e30
1 changed files with 277 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

277
docs/Self-hosted-update-guide.md Normal file
View File

@@ -0,0 +1,277 @@
# Self hosted update guide
In this guide we are updating a self hosted installation version 0.7.2 to version 0.9.3. When updating from or to other versions principle remain the same.
The installation we are upgrading was installed in April 2024 using the installation instructions found on the pgsail github site. Platform is an Ubuntu 22.04 Virtual Machine.
Before the upgrade, around 120 trips were logged. Needless to say we don't want to loose our data.
Unfortunately, there is no automatic update path available, this may change but for now we had to follow the general update instuctions.
## General update instructions
- Make a backup
- Update the containers.
- Update possible extensions.
- Run database migrations.
- Additional data migration.
- Update SignalK client.
## Let's go
### Tools used
In addition to the tools that are already installed as part of Unbuntu and PostgSail, I used DBeaver to examine the database from my Windows desktop.
<https://dbeaver.io/download/>
### Make a backup
Start by making a backup of the database, the docker-compose.yml and .env files. Note that in my case the database was stored in a host folder, later versions are using a docker volume. To copy the database it neccesary the containers are stopped.
```bash
cd postgsail
mkdir backup
docker compose stop
cp .env docker-compose.yml backup/
docker compose cp -a db:/var/lib/postgresql/data backup/db-data
```
### Update the containers
Make a note of the last migration in the initdb folder, in my case this was 99_migrations_202404.sql. Because I used git clone, the migration file was a bit inbetween 0.7.1 and 0.7.2, therefore I decided 99_migrations_202404.sql was the first migration to run.
Remove the containers:
```bash
docker compose down
```
Get the latest PostgSail from github, we checkout a specific tag to ensure we have a stable release version. If you installed it from a binary release, just update from the latest binary release.
```bash
git pull remote main
git fetch --all --tags
git checkout tags/v0.9.3
```
```text
Note: switching to 'tags/v0.9.3'.
You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.
If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:
git switch -c <new-branch-name>
Or undo this operation with:
git switch -
Turn off this advice by setting config variable advice.detachedHead to false
HEAD is now at 12e4baf Release PostgSail 0.9.3
```
**Ensure new docker-compose.yml file matches your database folder or volume setting, adjust as needed.**
Get the latest containers.
```bash
docker compose pull
```
### Update possible extentions
Start database container.
```bash
docker compose up -d db
```
Excec psql shell in databse container.
```bash
docker compose exec db sh
psql --username "$POSTGRES_USER" --dbname "$POSTGRES_DB"
\c signalk;
```
Check extensions which can be updated, be sure to run from the signalk database:
```sql
SELECT name, default_version, installed_version FROM pg_available_extensions where default_version <> installed_version;
```
The postgis extention can be upgraded with this SQL query:
```sql
SELECT postgis_extensions_upgrade();
```
Updating the timescaledb requires running from a new session, use following commands (note the -X options, that is neccesary):
```bash
docker compose exec db sh
psql --username "$POSTGRES_USER" --dbname "$POSTGRES_DB" -X
```
Then run following SQL commands from the psql shell:
```sql
ALTER EXTENSION timescaledb UPDATE;
CREATE EXTENSION IF NOT EXISTS timescaledb_toolkit;
ALTER EXTENSION timescaledb_toolkit UPDATE;
```
For others, to be checked. In my case, the postgis extension was essential.
### Run datbabase migrations
Then run the migrations, adjust start and end for first and last migration file to execute.
```bash
start=202404; end=202507; for f in $(ls ./docker-entrypoint-initdb.d/99_migrations_*.sql | sort); do s=$(basename "$f" | sed -E 's/^99_migrations_([0-9]{6})\.sql$/\1/'); if [[ "$s" < "$start" || "$s" > "$end" ]]; then continue; fi; echo "Running $f"; psql --username "$POSTGRES_USER" --dbname "$POSTGRES_DB" < "$f"; done
```
Or line by line
```bash
psql --username "$POSTGRES_USER" --dbname "$POSTGRES_DB" < ./docker-entrypoint-initdb.d/99_migrations_202404.sql
psql --username "$POSTGRES_USER" --dbname "$POSTGRES_DB" < ./docker-entrypoint-initdb.d/99_migrations_202405.sql
psql --username "$POSTGRES_USER" --dbname "$POSTGRES_DB" < ./docker-entrypoint-initdb.d/99_migrations_202406.sql
psql --username "$POSTGRES_USER" --dbname "$POSTGRES_DB" < ./docker-entrypoint-initdb.d/99_migrations_202407.sql
psql --username "$POSTGRES_USER" --dbname "$POSTGRES_DB" < ./docker-entrypoint-initdb.d/99_migrations_202408.sql
psql --username "$POSTGRES_USER" --dbname "$POSTGRES_DB" < ./docker-entrypoint-initdb.d/99_migrations_202409.sql
psql --username "$POSTGRES_USER" --dbname "$POSTGRES_DB" < ./docker-entrypoint-initdb.d/99_migrations_202410.sql
psql --username "$POSTGRES_USER" --dbname "$POSTGRES_DB" < ./docker-entrypoint-initdb.d/99_migrations_202411.sql
psql --username "$POSTGRES_USER" --dbname "$POSTGRES_DB" < ./docker-entrypoint-initdb.d/99_migrations_202412.sql
psql --username "$POSTGRES_USER" --dbname "$POSTGRES_DB" < ./docker-entrypoint-initdb.d/99_migrations_202501.sql
psql --username "$POSTGRES_USER" --dbname "$POSTGRES_DB" < ./docker-entrypoint-initdb.d/99_migrations_202504.sql
psql --username "$POSTGRES_USER" --dbname "$POSTGRES_DB" < ./docker-entrypoint-initdb.d/99_migrations_202505.sql
psql --username "$POSTGRES_USER" --dbname "$POSTGRES_DB" < ./docker-entrypoint-initdb.d/99_migrations_202507.sql
```
Now rebuild the web app.
```bash
docker compose build web
```
Maybe need to run 99env.sh - check.
Then we can start the other containers.
```bash
docker compose up -d
```
After everything is started, the web site should be accesible.
### Additional data migration
Depending on the starting version, additional data migration may be needed.
If the old trips are visible, but the routes are not, we need to run an SQL Script to re-calculate the trip metadata.
```sql
DO $$
declare
-- Re calculate the trip metadata
logbook_rec record;
avg_rec record;
t_rec record;
batch_size INTEGER := 20;
offset_value INTEGER := 0;
done BOOLEAN := FALSE;
processed INTEGER := 0;
begin
WHILE NOT done LOOP
processed := 0;
FOR logbook_rec IN
SELECT *
FROM api.logbook
WHERE _from IS NOT NULL
AND _to IS NOT NULL
AND active IS FALSE
AND trip IS NULL
--AND trip_heading IS NULL
--AND vessel_id = '06b6d311ccfe'
ORDER BY id DESC
LIMIT batch_size -- OFFSET offset_value -- don's use offset as causes entries to skip
LOOP
processed := processed + 1;
-- Update logbook entry with the latest metric data and calculate data
PERFORM set_config('vessel.id', logbook_rec.vessel_id, false);
-- Calculate trip metadata
avg_rec := logbook_update_avg_fn(logbook_rec.id, logbook_rec._from_time::TEXT, logbook_rec._to_time::TEXT);
--UPDATE api.logbook
-- SET extra = jsonb_recursive_merge(extra, jsonb_build_object('avg_wind_speed', avg_rec.avg_wind_speed))
-- WHERE id = logbook_rec.id;
if avg_rec.count_metric IS NULL OR avg_rec.count_metric = 0 then
-- We don't have the orignal metrics, we should read the geojson
continue; -- return current row of SELECT
end if;
-- mobilitydb, add spaciotemporal sequence
-- reduce the numbers of metrics by skipping row or aggregate time-series
-- By default the signalk plugin report one entry every minute.
IF avg_rec.count_metric < 30 THEN -- if less ~20min trip we keep it all data
t_rec := logbook_update_metrics_short_fn(avg_rec.count_metric, logbook_rec._from_time, logbook_rec._to_time);
ELSIF avg_rec.count_metric < 2000 THEN -- if less ~33h trip we skip data
t_rec := logbook_update_metrics_fn(avg_rec.count_metric, logbook_rec._from_time, logbook_rec._to_time);
ELSE -- As we have too many data, we time-series aggregate data
t_rec := logbook_update_metrics_timebucket_fn(avg_rec.count_metric, logbook_rec._from_time, logbook_rec._to_time);
END IF;
--RAISE NOTICE 'mobilitydb [%]', t_rec;
IF t_rec.trajectory IS NULL THEN
RAISE WARNING '-> process_logbook_queue_fn, vessel_id [%], invalid mobilitydb data [%] [%]', logbook_rec.vessel_id, _id, t_rec;
RETURN;
END IF;
RAISE NOTICE '-> process_logbook_queue_fn, vessel_id [%], update entry logbook id:[%] start:[%] end:[%]', logbook_rec.vessel_id, logbook_rec.id, logbook_rec._from_time, logbook_rec._to_time;
UPDATE api.logbook
SET
trip = t_rec.trajectory,
trip_cog = t_rec.courseovergroundtrue,
trip_sog = t_rec.speedoverground,
trip_twa = t_rec.windspeedapparent,
trip_tws = t_rec.truewindspeed,
trip_twd = t_rec.truewinddirection,
trip_notes = t_rec.notes, -- don't overwrite existing user notes. **** Must set trip_notes otherwise replay is not working.
trip_status = t_rec.status,
trip_depth = t_rec.depth,
trip_batt_charge = t_rec.stateofcharge,
trip_batt_voltage = t_rec.voltage,
trip_temp_water = t_rec.watertemperature,
trip_temp_out = t_rec.outsidetemperature,
trip_pres_out = t_rec.outsidepressure,
trip_hum_out = t_rec.outsidehumidity,
trip_heading = t_rec.heading, -- heading True
trip_tank_level = t_rec.tankLevel, -- Tank currentLevel
trip_solar_voltage = t_rec.solarVoltage, -- solar voltage
trip_solar_power = t_rec.solarPower -- solar powerPanel
WHERE id = logbook_rec.id;
END LOOP;
RAISE NOTICE '-> Processed:[%]', processed;
IF processed = 0 THEN
done := TRUE;
ELSE
offset_value := offset_value + batch_size;
END IF;
END LOOP;
END $$;
```
### Update SignalK client
The SignalK client can be updated from the SignalK Web UI. After the migration we updated this to version v0.5.0
### Trouble shooting
During this migration, several issues came up, they eventually boiled down to an extension not updated and permissions issues.