jensandClaude Sonnet 4.6
a01ae714c8
README covers installation, collector CLI reference, push server protocol, GUI client usage, and output format. conversation_recap documents the full development history: features, bugs fixed, and design decisions. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
151 lines
6.2 KiB
Markdown
151 lines
6.2 KiB
Markdown
# Conversation Recap — we_monitor development
|
||
|
||
## Overview
|
||
|
||
Starting from a single `collect.py` script that polled a VW WeConnect vehicle
|
||
and wrote JSON to disk, the project was iteratively extended into a modular
|
||
Python package with a TCP push server and a PyQt5 / pyqtgraph GUI client.
|
||
|
||
---
|
||
|
||
## Feature development (in order)
|
||
|
||
### 1. Credentials file (`-c / --credentials`)
|
||
Added a JSON credentials file parameter alongside the existing `-u / -p` flags.
|
||
File supports `username`, `password`, `vin`, and `domains` keys. Domains may
|
||
be a JSON array or a comma-separated string. Explicit CLI flags and env vars
|
||
(`WC_USER` / `WC_PASS`) take precedence over the file.
|
||
|
||
### 2. `--list-domains`
|
||
Added a CLI flag that prints all available domain names and exits.
|
||
|
||
### 3. Physical value format
|
||
Refactored all numeric measurements to `{"value": x, "unit": "km"}` objects,
|
||
stripping unit suffixes from field names (e.g. `odometer_km` → `odometer`).
|
||
|
||
### 4. Default log path + auto-create directory
|
||
Output defaults to `logs/YYYY_MM_DD_HH_MM_SS_<VIN>.json`; the `logs/`
|
||
directory is created automatically if it does not exist.
|
||
|
||
### 5. Midnight UTC log rotation
|
||
When using the default `logs/` directory the collector starts a new file after
|
||
midnight UTC. The old file is kept intact; the new file is auto-named with the
|
||
current date stamp.
|
||
|
||
### 6. TCP push server
|
||
Added `--host` / `--port` flags (default `0.0.0.0:9999`). The server pushes
|
||
every new snapshot to all connected clients as newline-delimited JSON
|
||
immediately after each poll. The last snapshot is sent to newly connecting
|
||
clients so they have something to display right away.
|
||
|
||
### 7. CLI test client (`client.py`)
|
||
Connects to the push server, pretty-prints every incoming JSON record, and
|
||
prints a running record counter after each reception.
|
||
|
||
### 8. Refactor into modules
|
||
`collect.py` was split into focused server-side modules:
|
||
|
||
| Module | Responsibility |
|
||
|--------|---------------|
|
||
| `server/network.py` | TCP accept loop, broadcast |
|
||
| `server/data_model.py` | Domain extractors, `collect_snapshot()` |
|
||
| `server/storage_helpers.py` | Load / save JSON store, `load_today_records()` |
|
||
| `server/we_connect.py` | WeConnect login and vehicle selection |
|
||
| `server/log_config.py` | `logging.basicConfig` setup |
|
||
|
||
`collect.py` remains in the project root as the sole entry point.
|
||
|
||
### 9. PyQt5 GUI client (`gui_client.py`)
|
||
Three-tab application:
|
||
|
||
* **Connector** — host / port, Connect / Disconnect, status label.
|
||
* **Dashboard** — `QTreeWidget` showing domain → object → field/value/unit;
|
||
expand state preserved across live updates.
|
||
* **Plot** — time-series charts of all physical fields using pyqtgraph.
|
||
|
||
### 10. Full-day history on connect
|
||
On client connect the server now sends all records from the current day instead
|
||
of only the most recent snapshot. Records are loaded from disk at startup into
|
||
`push_store_ref["preloaded"]` and merged with live in-memory records (dedup by
|
||
`ts`, sorted).
|
||
|
||
### 11. Plot tab — user-selectable traces (pyqtgraph)
|
||
Replaced static matplotlib plots with four pyqtgraph `PlotWidget`s. Users
|
||
select up to 4 traces per plot from a dropdown; zoom/pan via mouse;
|
||
crosshair cursor with nearest-value readout.
|
||
|
||
### 12. 8 plots with scrolling
|
||
Extended to 8 plots in a 4 × 2 scrollable grid (`QScrollArea`). Per-plot
|
||
control bars (add-combo + `+` button + colored chip buttons) sit directly
|
||
above each plot.
|
||
|
||
### 13. Abbreviated trace names + label mapping
|
||
`_TRACE_LABELS` dict maps full dotted keys to short human-readable labels
|
||
(e.g. `charging.batteryStatus.currentSOC` → `"SOC"`). Combo boxes, chip
|
||
buttons, legends, and cursor readout all use the short label; the full key is
|
||
stored as item data.
|
||
|
||
### 14. Persist plot settings
|
||
Trace selections saved to `~/.config/we_monitor/plot_settings.json` on every
|
||
add/remove. Restored silently on the next launch once matching signals arrive
|
||
from the server.
|
||
|
||
### 15. Status bar record counter
|
||
The main window status bar shows `Last snapshot: <ts> | Records: <N>`.
|
||
Counter resets on each new connection.
|
||
|
||
### 16. Dark theme (nmea_client palette)
|
||
Color palette copied from `nmea_client/gui/plot_widget.py`:
|
||
|
||
| Role | Hex |
|
||
|------|-----|
|
||
| Main background | `#1a1a2e` |
|
||
| Plot / input background | `#12121e` |
|
||
| Borders | `#2a2a4a` / `#3a3a5e` |
|
||
| Normal text | `#aaaacc` |
|
||
| Bright text | `#ddddee` |
|
||
| Connected | `#4CAF50` |
|
||
| Error | `#F44336` |
|
||
|
||
Trace colors use the Material palette from nmea_client
|
||
(`#2196F3`, `#F44336`, `#4CAF50`, `#FF9800`, …).
|
||
|
||
---
|
||
|
||
## Bugs fixed
|
||
|
||
### Socket read timeout (gui_client spurious disconnect)
|
||
`socket.create_connection(timeout=5)` sets a permanent 5-second read timeout.
|
||
After the first message, if the collection interval > 5 s, `recv()` raised
|
||
`socket.timeout` and the GUI reported "Connection lost".
|
||
**Fix:** `self._sock.settimeout(None)` immediately after connect.
|
||
|
||
### `load_today()` never called after first poll
|
||
The lazy-load condition `if not records` was never true once the first
|
||
WeConnect poll had written a record into memory, so disk history was never
|
||
loaded for late-connecting clients.
|
||
**Fix:** Eager loading — `load_today_records()` called at startup, result
|
||
stored in `push_store_ref["preloaded"]`; merged with live records on every
|
||
connect.
|
||
|
||
### Erratic axis scaling on mouse hover
|
||
pyqtgraph's `InfiniteLine` (cursor crosshair) was included in auto-range
|
||
calculations. Moving the cursor triggered view rescaling on every mouse event.
|
||
**Fix:** `pw.addItem(vl, ignoreBounds=True)` in both the initial setup and
|
||
after each `_redraw()` call.
|
||
|
||
---
|
||
|
||
## Key design decisions
|
||
|
||
* **`store_ref` mutable dict** — shared between the main thread and the accept
|
||
loop thread under a single `threading.Lock`. Keys: `"current"` (live store)
|
||
and `"preloaded"` (today's disk records).
|
||
* **NDJSON protocol** — one JSON object per line over TCP; trivial to consume
|
||
with `nc`, `python client.py`, or the GUI.
|
||
* **pyqtgraph over matplotlib** — chosen for native Qt integration, live
|
||
update performance, and built-in zoom/pan without extra toolbars.
|
||
* **`_TRACE_LABELS` dict** — single place to maintain display names; falls
|
||
back to `key.split(".")[-1]` for unknown keys.
|
||
* **`QComboBox` item data** — short label displayed, full key stored via
|
||
`addItem(label, key)` / `currentData()`; no reverse-lookup dict needed. |