From 55719010da8b9493b10725f1dfafaae4ab438e2c Mon Sep 17 00:00:00 2001 From: Jens Ahrensfeld Date: Tue, 26 May 2026 17:01:42 +0200 Subject: [PATCH] Docs: full README rewrite + gui_client.py module docstring update README now covers: - Updated diff-mode section (self-describing protocol, no _diff marker) - Full GUI section with Connector/Dashboard/Plot tab descriptions - Plot mouse interactions table (scroll, Shift+scroll, drag) - Module reference for all server/, utils/, and gui_client classes - --dry flag and all credential-file keys documented gui_client.py module docstring updated to list tabs and mouse interactions. Co-Authored-By: Claude Sonnet 4.6 --- README.md | 231 ++++++++++++++++++++++++++++++++++++++++---------- gui_client.py | 15 +++- 2 files changed, 198 insertions(+), 48 deletions(-) diff --git a/README.md b/README.md index 7023bbc..a733b3e 100644 --- a/README.md +++ b/README.md @@ -7,16 +7,18 @@ A PyQt5 GUI client visualises the data in real time. ## Repository layout ``` -collect.py — collector daemon (entry point) -gui_client.py — PyQt5 GUI client -client.py — minimal CLI test client +collect.py — collector daemon (entry point) +gui_client.py — PyQt5 GUI client +client.py — minimal CLI test client server/ - data_model.py — domain extractors, collect_snapshot() - network.py — TCP push server (accept loop + broadcast) - storage_helpers.py— load / save JSON store, midnight rotation helpers - we_connect.py — WeConnect login and vehicle selection - log_config.py — logging setup -credentials/ — JSON config files (gitignored) + data_model.py — domain extractors, collect_snapshot() + network.py — TCP push server (accept loop + broadcast) + storage_helpers.py — load / save JSON store, midnight rotation helpers + we_connect.py — WeConnect login and vehicle selection + log_config.py — logging setup +utils/ + jay_diff.py — JSON diff / merge library +credentials/ — JSON config files (gitignored) requirements.txt ``` @@ -32,11 +34,13 @@ Or use the installer which also sets up a systemd user service: ./install.sh ``` +--- + ## Collector (`collect.py`) Connects to WeConnect, polls selected domains at a configurable interval, and -appends records to a rotating JSON log. Simultaneously runs a TCP push server -so clients receive every new snapshot the moment it is collected. +appends timestamped records to a rotating JSON log. Simultaneously runs a TCP +push server so clients receive every new snapshot the moment it is collected. ### Quick start @@ -68,7 +72,8 @@ All connection and collection settings live in a single JSON file: "interval_s": 300, "host": "0.0.0.0", "port": 9999, - "log_dir": "./logs" + "log_dir": "./logs", + "diff": false } ``` @@ -82,11 +87,11 @@ All connection and collection settings live in a single JSON file: | `host` | `"0.0.0.0"` | Push server bind address | | `port` | `9999` | Push server TCP port | | `log_dir` | `"./logs"` | Directory for rotating JSON log files | -| `diff` | `false` | Enable diff mode for push server transmission | +| `diff` | `false` | Enable diff-mode compression on the push server | ### CLI reference -CLI flags override the corresponding credentials file value when given. +CLI flags override the corresponding credentials-file value when given. | Flag | Overrides | Description | |------|-----------|-------------| @@ -97,9 +102,9 @@ CLI flags override the corresponding credentials file value when given. | `--host HOST` | `host` | Push server bind address | | `--port PORT` | `port` | Push server TCP port | | `--log-dir DIR` | `log_dir` | Directory for rotating log files | -| `--max-records N` | — | Keep only the last N records per file | -| `--diff` | `diff` | Send diffs instead of full snapshots (must match client setting) | -| `--dry` | — | Skip WeConnect; run push server only (for testing) | +| `--max-records N` | — | Keep only the last N records per file (default: unlimited) | +| `--diff` | `diff` | Enable diff-mode on the push server | +| `--dry` | — | Skip WeConnect login; run push server only (useful for GUI testing) | | `-v` | — | Verbose / debug logging | | `--list-domains` | — | Print available domains and exit | @@ -136,24 +141,34 @@ Log files are named `YYYY_MM_DD_HH_MM_SS_.json` and contain: Physical values are always `{"value": …, "unit": …}` objects. +--- + ## Push server protocol -The server listens on the `host`:`port` from the credentials file (default `0.0.0.0:9999`). +The server listens on the `host`:`port` from the credentials file +(default `0.0.0.0:9999`). -* On connect: sends all records from the last 24 hours, sorted by timestamp. -* On each new poll: sends the new snapshot immediately. +* **On connect:** sends all records from the last 24 hours, sorted by timestamp. +* **On each new poll:** sends the new snapshot immediately. All messages are **newline-delimited JSON** (one record per line, `\n` terminated). ### Diff mode (`--diff`) -When enabled on the server, the history burst is compressed: the first -record is sent in full, subsequent records as `jay_diff_full` diffs -(`{"_diff": true, "update_add": {…}, "delete": {…}}`). Live snapshots -are also diffed against the previous broadcast. Enable `--diff` on the -client side (`client.py`) or check **Diff mode** in the GUI Connector -tab to reconstruct full snapshots from the stream. Both sides must -agree — a mismatch produces garbled output. +When diff mode is enabled the history burst and every live snapshot are +compressed using `jay_diff_full` before transmission. The wire format is +self-describing — diffs carry `update_add` / `delete` keys, full snapshots do +not — so no separate framing is needed. + +The first message sent to a new client is always `jay_diff_full({}, record[0])`, +i.e. an `update_add` containing every field of the first record. Subsequent +messages contain only the fields that changed. + +Enable `--diff` on the server **and** on the client (`client.py --diff` or the +**Diff mode** checkbox in the GUI Connector tab). Both sides must agree — a +mismatch produces garbled output. + +--- ## GUI client (`gui_client.py`) @@ -161,39 +176,167 @@ agree — a mismatch produces garbled output. python gui_client.py ``` -Dark theme. Three tabs: +Dark-theme PyQt5 application. Three tabs: **Connector**, **Dashboard**, +**Plot**. Connection settings, trace selections, and time windows are persisted +to `~/.config/we_monitor/plot_settings.json` and restored on the next launch. ### Connector tab -Enter host and port, then click **Connect**. Settings are saved automatically -and restored on the next launch. All plot data is cleared on each -connect so fresh server history is never mixed with stale local data. +| Control | Description | +|---------|-------------| +| **Host / Port** | Address of the `collect.py` push server | +| **Diff mode** checkbox | Must match the server's `--diff` setting | +| **Connect / Disconnect** | Opens or closes the TCP connection | +| **Status** | Shows `Connected`, `Disconnected`, or the error message | + +Clicking **Connect** saves the current host, port, and diff-mode setting. All +plot data is cleared on each connect so the fresh server history is never mixed +with stale local data. ### Dashboard tab -Live tree view of the latest snapshot — domain → status object → field, -showing value and unit. Expand state is preserved across updates. +Live tree view of the most recent snapshot. The hierarchy mirrors the JSON +structure: domain → status object → field, with value and unit shown inline. +Expand / collapse state is preserved across updates. ### Plot tab -8 plots (4 × 2) in a scrollable grid. Each plot has its own control bar: +Eight plots arranged in a 4 × 2 scrollable grid. Each plot is independent and +has its own control bar. -* **Time window spinner** — show the last 1–24 hours (per plot). -* **Dropdown + `+`** — select a signal and add it as a trace (max 4 per plot). -* **Colored chips (`name ×`)** — click to remove a trace. -* Zoom / pan with mouse; crosshair cursor tracks mouse position. -* When available data is less than the selected window, the x-axis adjusts - to the actual data extent — no empty gap on the left. +#### Control bar (per plot) -Trace selections, time windows, and connection settings are persisted to -`~/.config/we_monitor/plot_settings.json` and restored automatically on -the next launch. +| Control | Description | +|---------|-------------| +| **Signal dropdown** | Lists every numeric `{value, unit}` field received so far | +| **`+` button** | Adds the selected signal as a new trace (max 4 per plot) | +| **Colored chips** (`name ×`) | One chip per active trace; click `×` to remove | +| **Time window spinner** | Show the last 1–24 hours of data (per plot) | +| **Date label** (right-aligned) | Shows the calendar date(s) of the visible window — single date or `YYYY-MM-DD – YYYY-MM-DD` when the view spans midnight | + +#### Plot area + +* **X axis** — time of day in `HH:MM:SS` format; no locale-dependent date prefix. +* **Y axis** — auto-zooms to fit the visible data on the first draw after connect + or after a trace is added / removed. Manual zoom is preserved between data + updates. +* **Crosshair** — dashed vertical line tracks the mouse cursor. +* **Floating tooltip** — appears next to the cursor and shows the nearest + sampled value for every active trace, with a colored bullet and unit. +* **Legend** — trace names and colors, shown inside the plot area. + +#### Mouse interaction + +| Action | Effect | +|--------|--------| +| Scroll wheel | Zoom X axis in / out, centred on cursor | +| Shift + scroll wheel | Zoom Y axis in / out, centred on cursor | +| Click + drag | Pan the view | +| Right-click | pyqtgraph context menu (export, auto-range, …) | + +--- ## CLI test client (`client.py`) ```bash python client.py --host 192.168.1.10 --port 9999 +python client.py --diff # reconstruct from diff stream ``` Connects to the push server and prints every incoming snapshot as -pretty-printed JSON with a running record counter. \ No newline at end of file +pretty-printed JSON with a running record counter. + +| Flag | Description | +|------|-------------| +| `--host HOST` | Server host (default: `127.0.0.1`) | +| `--port PORT` | Server port (default: `9999`) | +| `--diff` | Reconstruct full snapshots from diff stream | + +--- + +## Module reference + +### `collect.py` + +Entry point for the collector daemon. Parses CLI / credentials-file arguments, +establishes the WeConnect session, starts the push server, and runs the main +poll loop. + +### `server/data_model.py` + +`collect_snapshot(vehicle, domains) → dict` +Iterates over the requested domains on a `weconnect.Vehicle` and extracts every +`Status` attribute into a flat-ish dict keyed by domain and status-object name. +Physical values are normalised to `{"value": …, "unit": …}`. Returns a record +with a top-level `"ts"` ISO-8601 timestamp. + +`ALL_DOMAINS` +Tuple of all domain names the collector knows about. + +### `server/network.py` + +`start_push_server(host, port) → (clients, lock, store_ref)` +Binds a TCP socket, starts the accept-loop thread (daemon), and returns the +shared state needed by `broadcast`. + +`broadcast(snapshot, clients, lock)` +Serialises `snapshot` to a JSON line and sends it to every connected client. +Dead connections are removed silently. + +`store_ref` dict keys: +| Key | Type | Description | +|-----|------|-------------| +| `"current"` | `dict` | The currently-open JSON store | +| `"preloaded"` | `list` | 24-h history records loaded at startup | +| `"diff_mode"` | `bool` | Whether to compress the history burst as diffs | + +### `server/storage_helpers.py` + +`load_store(path, vin, interval) → dict` +Loads an existing store file or creates a fresh one. + +`save_store(path, store, max_records)` +Writes the store to disk, optionally trimming to the last `max_records` entries. + +`auto_out_path(log_dir, vin) → Path` +Returns a timestamped file path for a new store. + +`load_last_24h_records(log_dir, vin) → list` +Scans the log directory for files belonging to `vin`, loads any records from +the last 24 hours, and returns them sorted by timestamp. + +### `server/we_connect.py` + +`connect(username, password) → WeConnect` +Authenticates and returns a connected `WeConnect` instance. + +`select_vehicle(wc, vin) → (vehicle, error)` +Returns the target vehicle (first available if `vin` is empty) or an error string. + +### `utils/jay_diff.py` + +Lightweight JSON diff / patch library. + +`jay_diff_full(old, new, combine_upd_add=True) → dict` +Computes a diff between two dicts. With `combine_upd_add=True` (the default +used everywhere in this project) the result uses `update_add` for all additions +and updates, plus a separate `delete` key. Returns `{}` when nothing changed. + +`jay_merge_full(old, diff) → dict` +Applies a `jay_diff_full` diff to `old`. The diff structure is self-describing: +if `diff` contains none of the known diff keys (`update`, `update_add`, `add`, +`delete`) and `old` is empty, `diff` is returned as-is (full-snapshot +passthrough for bootstrap). + +### `gui_client.py` — class overview + +| Class | Role | +|-------|------| +| `TcpReader` | Background thread; reads newline-delimited JSON from the socket, reconstructs full snapshots in diff mode via `jay_merge_full`, and emits each snapshot via the `message` Qt signal | +| `ConnectorTab` | Host / port / diff-mode form; emits `connect_requested` and `disconnect_requested` signals; persists settings | +| `DashboardTab` | Live `QTreeWidget` showing the latest snapshot; preserves expand state across updates | +| `PlotTab` | Eight independent time-series plots in a scrollable grid | +| `TimeAxisItem` | `pg.AxisItem` subclass; formats x-axis tick labels as `HH:MM:SS` | +| `PlotViewBox` | `pg.ViewBox` subclass; routes plain scroll to X zoom and Shift+scroll to Y zoom | +| `PlotTooltip` | Frameless child `QWidget` per plot; shows nearest sampled values for all active traces at the cursor position | +| `MainWindow` | `QMainWindow`; wires the three tabs together and owns the `TcpReader` | \ No newline at end of file diff --git a/gui_client.py b/gui_client.py index 3985340..6fdf503 100644 --- a/gui_client.py +++ b/gui_client.py @@ -2,13 +2,20 @@ """ WeConnect GUI client. -Connects to the collect.py TCP push server and displays live data. +Connects to the collect.py TCP push server and displays live vehicle data. Tabs ---- -Connector — server address, connect / disconnect, status -Dashboard — live tree view of the latest snapshot -Plot — time-series charts of all physical {value, unit} fields +Connector — host / port / diff-mode form; connect / disconnect +Dashboard — live tree view of the latest snapshot (domain → object → field) +Plot — 8 independent time-series plots in a scrollable 4×2 grid + +Plot interactions +----------------- +Scroll wheel — zoom X axis in/out, centred on cursor +Shift + scroll wheel — zoom Y axis in/out, centred on cursor +Click + drag — pan +Crosshair + tooltip — dashed vertical line + floating value readout per trace """ import json