diff --git a/README.md b/README.md new file mode 100644 index 0000000..6f97796 --- /dev/null +++ b/README.md @@ -0,0 +1,160 @@ +# we_monitor + +Periodically polls a VW WeConnect vehicle via the `weconnect` library, stores +timestamped records as JSON, and pushes live data to connected TCP clients. +A PyQt5 GUI client visualises the data in real time. + +## Repository layout + +``` +collect.py — collector daemon (entry point) +gui_client.py — PyQt5 GUI client +client.py — minimal CLI test client +server/ + data_model.py — domain extractors, collect_snapshot() + network.py — TCP push server (accept loop + broadcast) + storage_helpers.py— load / save JSON store, midnight rotation helpers + we_connect.py — WeConnect login and vehicle selection + log_config.py — logging setup +requirements.txt +``` + +## Requirements + +``` +pip install weconnect PyQt5 pyqtgraph +``` + +## Collector (`collect.py`) + +Connects to WeConnect, polls selected domains at a configurable interval, and +appends records to a JSON file. Simultaneously runs a TCP push server so +clients receive every new snapshot the moment it is collected. + +### Quick start + +```bash +# Credentials via environment variables, all defaults +export WC_USER=me@example.com +export WC_PASS=secret +python collect.py + +# Credentials from a JSON file, 10-minute interval +python collect.py -c credentials/my_car.json -i 600 +``` + +### Credentials file (`-c FILE`) + +```json +{ + "username": "me@example.com", + "password": "secret", + "vin": "WVWZZZE1ZMP000000", + "domains": ["charging", "measurements", "readiness"] +} +``` + +All keys are optional; explicit CLI flags and `WC_USER` / `WC_PASS` +environment variables take precedence. + +### CLI reference + +| Flag | Default | Description | +|------|---------|-------------| +| `-c FILE` | — | JSON credentials file | +| `-u / -p` | `WC_USER` / `WC_PASS` | Username / password | +| `--vin` | first vehicle | Target VIN | +| `-d DOMAINS` | `charging,measurements,readiness` | Comma-separated domains or `all` | +| `-i SECONDS` | `300` | Collection interval | +| `-o FILE` | `logs/YYYY_MM_DD_HH_MM_SS_.json` | Output file | +| `--max-records N` | unlimited | Keep only last N records | +| `--host` | `0.0.0.0` | Push server bind address | +| `--port` | `9999` | Push server TCP port | +| `-v` | — | Verbose / debug logging | +| `--list-domains` | — | Print available domains and exit | + +**Available domains:** `charging`, `climatisation`, `measurements`, +`readiness`, `parking`, `access` + +### Log rotation + +When using the default `logs/` directory the collector automatically starts a +new file after midnight UTC, keeping all previous files intact. On startup the +full day's records are pre-loaded from disk so newly connected clients receive +the complete history. + +### Output format + +Each file is a JSON object: + +```json +{ + "meta": { "vin": "...", "created_at": "...", "interval_seconds": 300 }, + "records": [ + { + "ts": "2025-05-25T10:00:00+00:00", + "charging": { + "batteryStatus": { + "currentSOC": { "value": 80, "unit": "%" }, + "cruisingRangeElectric": { "value": 310, "unit": "km" } + } + } + } + ] +} +``` + +Physical values are always `{"value": …, "unit": …}` objects. + +## Push server protocol + +The server listens on `--host`:`--port` (default `0.0.0.0:9999`). + +* On connect: sends all records from the current day, sorted by timestamp. +* On each new poll: sends the new snapshot immediately. + +All messages are **newline-delimited JSON** (one record per line, `\n` +terminated). + +## GUI client (`gui_client.py`) + +```bash +python gui_client.py +python gui_client.py --host 192.168.1.10 --port 9999 +``` + +Dark theme matching nmea_client. Three tabs: + +### Connector tab + +Host / port fields, Connect / Disconnect button, and connection status. + +### Dashboard tab + +Live tree view of the latest snapshot — domain → status object → field, +showing value and unit. Expand state is preserved across updates. + +### Plot tab + +8 plots (4 × 2) in a scrollable grid. Each plot has its own control bar: + +* **Dropdown + `+`** — select a signal and add it as a trace (max 4 per plot). +* **Colored chips (`name ×`)** — click to remove a trace. +* Zoom / pan with mouse; crosshair cursor tracks mouse position. + +Trace selections are persisted to +`~/.config/we_monitor/plot_settings.json` and restored automatically on +the next launch once matching signals arrive. + +Signal labels use short human-readable names (e.g. *SOC*, *range*, +*outside temp*) defined in `_TRACE_LABELS` near the top of `gui_client.py`. + +## CLI test client (`client.py`) + +```bash +python client.py +python client.py --host 192.168.1.10 --port 9999 +``` + +Connects to the push server and prints every incoming snapshot as +pretty-printed JSON with a running record counter. \ No newline at end of file diff --git a/conversation_recap.md b/conversation_recap.md new file mode 100644 index 0000000..a784be5 --- /dev/null +++ b/conversation_recap.md @@ -0,0 +1,151 @@ +# Conversation Recap — we_monitor development + +## Overview + +Starting from a single `collect.py` script that polled a VW WeConnect vehicle +and wrote JSON to disk, the project was iteratively extended into a modular +Python package with a TCP push server and a PyQt5 / pyqtgraph GUI client. + +--- + +## Feature development (in order) + +### 1. Credentials file (`-c / --credentials`) +Added a JSON credentials file parameter alongside the existing `-u / -p` flags. +File supports `username`, `password`, `vin`, and `domains` keys. Domains may +be a JSON array or a comma-separated string. Explicit CLI flags and env vars +(`WC_USER` / `WC_PASS`) take precedence over the file. + +### 2. `--list-domains` +Added a CLI flag that prints all available domain names and exits. + +### 3. Physical value format +Refactored all numeric measurements to `{"value": x, "unit": "km"}` objects, +stripping unit suffixes from field names (e.g. `odometer_km` → `odometer`). + +### 4. Default log path + auto-create directory +Output defaults to `logs/YYYY_MM_DD_HH_MM_SS_.json`; the `logs/` +directory is created automatically if it does not exist. + +### 5. Midnight UTC log rotation +When using the default `logs/` directory the collector starts a new file after +midnight UTC. The old file is kept intact; the new file is auto-named with the +current date stamp. + +### 6. TCP push server +Added `--host` / `--port` flags (default `0.0.0.0:9999`). The server pushes +every new snapshot to all connected clients as newline-delimited JSON +immediately after each poll. The last snapshot is sent to newly connecting +clients so they have something to display right away. + +### 7. CLI test client (`client.py`) +Connects to the push server, pretty-prints every incoming JSON record, and +prints a running record counter after each reception. + +### 8. Refactor into modules +`collect.py` was split into focused server-side modules: + +| Module | Responsibility | +|--------|---------------| +| `server/network.py` | TCP accept loop, broadcast | +| `server/data_model.py` | Domain extractors, `collect_snapshot()` | +| `server/storage_helpers.py` | Load / save JSON store, `load_today_records()` | +| `server/we_connect.py` | WeConnect login and vehicle selection | +| `server/log_config.py` | `logging.basicConfig` setup | + +`collect.py` remains in the project root as the sole entry point. + +### 9. PyQt5 GUI client (`gui_client.py`) +Three-tab application: + +* **Connector** — host / port, Connect / Disconnect, status label. +* **Dashboard** — `QTreeWidget` showing domain → object → field/value/unit; + expand state preserved across live updates. +* **Plot** — time-series charts of all physical fields using pyqtgraph. + +### 10. Full-day history on connect +On client connect the server now sends all records from the current day instead +of only the most recent snapshot. Records are loaded from disk at startup into +`push_store_ref["preloaded"]` and merged with live in-memory records (dedup by +`ts`, sorted). + +### 11. Plot tab — user-selectable traces (pyqtgraph) +Replaced static matplotlib plots with four pyqtgraph `PlotWidget`s. Users +select up to 4 traces per plot from a dropdown; zoom/pan via mouse; +crosshair cursor with nearest-value readout. + +### 12. 8 plots with scrolling +Extended to 8 plots in a 4 × 2 scrollable grid (`QScrollArea`). Per-plot +control bars (add-combo + `+` button + colored chip buttons) sit directly +above each plot. + +### 13. Abbreviated trace names + label mapping +`_TRACE_LABELS` dict maps full dotted keys to short human-readable labels +(e.g. `charging.batteryStatus.currentSOC` → `"SOC"`). Combo boxes, chip +buttons, legends, and cursor readout all use the short label; the full key is +stored as item data. + +### 14. Persist plot settings +Trace selections saved to `~/.config/we_monitor/plot_settings.json` on every +add/remove. Restored silently on the next launch once matching signals arrive +from the server. + +### 15. Status bar record counter +The main window status bar shows `Last snapshot: | Records: `. +Counter resets on each new connection. + +### 16. Dark theme (nmea_client palette) +Color palette copied from `nmea_client/gui/plot_widget.py`: + +| Role | Hex | +|------|-----| +| Main background | `#1a1a2e` | +| Plot / input background | `#12121e` | +| Borders | `#2a2a4a` / `#3a3a5e` | +| Normal text | `#aaaacc` | +| Bright text | `#ddddee` | +| Connected | `#4CAF50` | +| Error | `#F44336` | + +Trace colors use the Material palette from nmea_client +(`#2196F3`, `#F44336`, `#4CAF50`, `#FF9800`, …). + +--- + +## Bugs fixed + +### Socket read timeout (gui_client spurious disconnect) +`socket.create_connection(timeout=5)` sets a permanent 5-second read timeout. +After the first message, if the collection interval > 5 s, `recv()` raised +`socket.timeout` and the GUI reported "Connection lost". +**Fix:** `self._sock.settimeout(None)` immediately after connect. + +### `load_today()` never called after first poll +The lazy-load condition `if not records` was never true once the first +WeConnect poll had written a record into memory, so disk history was never +loaded for late-connecting clients. +**Fix:** Eager loading — `load_today_records()` called at startup, result +stored in `push_store_ref["preloaded"]`; merged with live records on every +connect. + +### Erratic axis scaling on mouse hover +pyqtgraph's `InfiniteLine` (cursor crosshair) was included in auto-range +calculations. Moving the cursor triggered view rescaling on every mouse event. +**Fix:** `pw.addItem(vl, ignoreBounds=True)` in both the initial setup and +after each `_redraw()` call. + +--- + +## Key design decisions + +* **`store_ref` mutable dict** — shared between the main thread and the accept + loop thread under a single `threading.Lock`. Keys: `"current"` (live store) + and `"preloaded"` (today's disk records). +* **NDJSON protocol** — one JSON object per line over TCP; trivial to consume + with `nc`, `python client.py`, or the GUI. +* **pyqtgraph over matplotlib** — chosen for native Qt integration, live + update performance, and built-in zoom/pan without extra toolbars. +* **`_TRACE_LABELS` dict** — single place to maintain display names; falls + back to `key.split(".")[-1]` for unknown keys. +* **`QComboBox` item data** — short label displayed, full key stored via + `addItem(label, key)` / `currentData()`; no reverse-lookup dict needed. \ No newline at end of file