jens/we_monitor
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add README.md and conversation_recap.md

Browse Source 
README covers installation, collector CLI reference, push server protocol,
GUI client usage, and output format.  conversation_recap documents the full
development history: features, bugs fixed, and design decisions.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
jens
2026-05-25 22:42:00 +02:00
co-authored by Claude Sonnet 4.6
parent 9d7561ba2f
commit a01ae714c8
2 changed files with 311 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+160
View File
@@ -0,0 +1,160 @@
# we_monitor
Periodically polls a VW WeConnect vehicle via the `weconnect` library, stores
timestamped records as JSON, and pushes live data to connected TCP clients.
A PyQt5 GUI client visualises the data in real time.
## Repository layout
```
collect.py — collector daemon (entry point)
gui_client.py — PyQt5 GUI client
client.py — minimal CLI test client
server/
data_model.py — domain extractors, collect_snapshot()
network.py — TCP push server (accept loop + broadcast)
storage_helpers.py— load / save JSON store, midnight rotation helpers
we_connect.py — WeConnect login and vehicle selection
log_config.py — logging setup
requirements.txt
```
## Requirements
```
pip install weconnect PyQt5 pyqtgraph
```
## Collector (`collect.py`)
Connects to WeConnect, polls selected domains at a configurable interval, and
appends records to a JSON file. Simultaneously runs a TCP push server so
clients receive every new snapshot the moment it is collected.
### Quick start
```bash
# Credentials via environment variables, all defaults
export WC_USER=me@example.com
export WC_PASS=secret
python collect.py
# Credentials from a JSON file, 10-minute interval
python collect.py -c credentials/my_car.json -i 600
```
### Credentials file (`-c FILE`)
```json
{
"username": "me@example.com",
"password": "secret",
"vin": "WVWZZZE1ZMP000000",
"domains": ["charging", "measurements", "readiness"]
}
```
All keys are optional; explicit CLI flags and `WC_USER` / `WC_PASS`
environment variables take precedence.
### CLI reference
| Flag | Default | Description |
|------|---------|-------------|
| `-c FILE` | — | JSON credentials file |
| `-u / -p` | `WC_USER` / `WC_PASS` | Username / password |
| `--vin` | first vehicle | Target VIN |
| `-d DOMAINS` | `charging,measurements,readiness` | Comma-separated domains or `all` |
| `-i SECONDS` | `300` | Collection interval |
| `-o FILE` | `logs/YYYY_MM_DD_HH_MM_SS_<VIN>.json` | Output file |
| `--max-records N` | unlimited | Keep only last N records |
| `--host` | `0.0.0.0` | Push server bind address |
| `--port` | `9999` | Push server TCP port |
| `-v` | — | Verbose / debug logging |
| `--list-domains` | — | Print available domains and exit |
**Available domains:** `charging`, `climatisation`, `measurements`,
`readiness`, `parking`, `access`
### Log rotation
When using the default `logs/` directory the collector automatically starts a
new file after midnight UTC, keeping all previous files intact. On startup the
full day's records are pre-loaded from disk so newly connected clients receive
the complete history.
### Output format
Each file is a JSON object:
```json
{
"meta": { "vin": "...", "created_at": "...", "interval_seconds": 300 },
"records": [
{
"ts": "2025-05-25T10:00:00+00:00",
"charging": {
"batteryStatus": {
"currentSOC": { "value": 80, "unit": "%" },
"cruisingRangeElectric": { "value": 310, "unit": "km" }
}
}
}
]
}
```
Physical values are always `{"value": …, "unit": …}` objects.
## Push server protocol
The server listens on `--host`:`--port` (default `0.0.0.0:9999`).
* On connect: sends all records from the current day, sorted by timestamp.
* On each new poll: sends the new snapshot immediately.
All messages are **newline-delimited JSON** (one record per line, `\n`
terminated).
## GUI client (`gui_client.py`)
```bash
python gui_client.py
python gui_client.py --host 192.168.1.10 --port 9999
```
Dark theme matching nmea_client. Three tabs:
### Connector tab
Host / port fields, Connect / Disconnect button, and connection status.
### Dashboard tab
Live tree view of the latest snapshot — domain → status object → field,
showing value and unit. Expand state is preserved across updates.
### Plot tab
8 plots (4 × 2) in a scrollable grid. Each plot has its own control bar:
* **Dropdown + `+`** — select a signal and add it as a trace (max 4 per plot).
* **Colored chips (`name ×`)** — click to remove a trace.
* Zoom / pan with mouse; crosshair cursor tracks mouse position.
Trace selections are persisted to
`~/.config/we_monitor/plot_settings.json` and restored automatically on
the next launch once matching signals arrive.
Signal labels use short human-readable names (e.g. *SOC*, *range*,
*outside temp*) defined in `_TRACE_LABELS` near the top of `gui_client.py`.
## CLI test client (`client.py`)
```bash
python client.py
python client.py --host 192.168.1.10 --port 9999
```
Connects to the push server and prints every incoming snapshot as
pretty-printed JSON with a running record counter.
conversation_recap.md
+151
View File
@@ -0,0 +1,151 @@
# Conversation Recap — we_monitor development
## Overview
Starting from a single `collect.py` script that polled a VW WeConnect vehicle
and wrote JSON to disk, the project was iteratively extended into a modular
Python package with a TCP push server and a PyQt5 / pyqtgraph GUI client.
---
## Feature development (in order)
### 1. Credentials file (`-c / --credentials`)
Added a JSON credentials file parameter alongside the existing `-u / -p` flags.
File supports `username`, `password`, `vin`, and `domains` keys. Domains may
be a JSON array or a comma-separated string. Explicit CLI flags and env vars
(`WC_USER` / `WC_PASS`) take precedence over the file.
### 2. `--list-domains`
Added a CLI flag that prints all available domain names and exits.
### 3. Physical value format
Refactored all numeric measurements to `{"value": x, "unit": "km"}` objects,
stripping unit suffixes from field names (e.g. `odometer_km` → `odometer`).
### 4. Default log path + auto-create directory
Output defaults to `logs/YYYY_MM_DD_HH_MM_SS_<VIN>.json`; the `logs/`
directory is created automatically if it does not exist.
### 5. Midnight UTC log rotation
When using the default `logs/` directory the collector starts a new file after
midnight UTC. The old file is kept intact; the new file is auto-named with the
current date stamp.
### 6. TCP push server
Added `--host` / `--port` flags (default `0.0.0.0:9999`). The server pushes
every new snapshot to all connected clients as newline-delimited JSON
immediately after each poll. The last snapshot is sent to newly connecting
clients so they have something to display right away.
### 7. CLI test client (`client.py`)
Connects to the push server, pretty-prints every incoming JSON record, and
prints a running record counter after each reception.
### 8. Refactor into modules
`collect.py` was split into focused server-side modules:
| Module | Responsibility |
|--------|---------------|
| `server/network.py` | TCP accept loop, broadcast |
| `server/data_model.py` | Domain extractors, `collect_snapshot()` |
| `server/storage_helpers.py` | Load / save JSON store, `load_today_records()` |
| `server/we_connect.py` | WeConnect login and vehicle selection |
| `server/log_config.py` | `logging.basicConfig` setup |
`collect.py` remains in the project root as the sole entry point.
### 9. PyQt5 GUI client (`gui_client.py`)
Three-tab application:
* **Connector** — host / port, Connect / Disconnect, status label.
* **Dashboard** — `QTreeWidget` showing domain → object → field/value/unit;
expand state preserved across live updates.
* **Plot** — time-series charts of all physical fields using pyqtgraph.
### 10. Full-day history on connect
On client connect the server now sends all records from the current day instead
of only the most recent snapshot. Records are loaded from disk at startup into
`push_store_ref["preloaded"]` and merged with live in-memory records (dedup by
`ts`, sorted).
### 11. Plot tab — user-selectable traces (pyqtgraph)
Replaced static matplotlib plots with four pyqtgraph `PlotWidget`s. Users
select up to 4 traces per plot from a dropdown; zoom/pan via mouse;
crosshair cursor with nearest-value readout.
### 12. 8 plots with scrolling
Extended to 8 plots in a 4 × 2 scrollable grid (`QScrollArea`). Per-plot
control bars (add-combo + `+` button + colored chip buttons) sit directly
above each plot.
### 13. Abbreviated trace names + label mapping
`_TRACE_LABELS` dict maps full dotted keys to short human-readable labels
(e.g. `charging.batteryStatus.currentSOC` → `"SOC"`). Combo boxes, chip
buttons, legends, and cursor readout all use the short label; the full key is
stored as item data.
### 14. Persist plot settings
Trace selections saved to `~/.config/we_monitor/plot_settings.json` on every
add/remove. Restored silently on the next launch once matching signals arrive
from the server.
### 15. Status bar record counter
The main window status bar shows `Last snapshot: <ts> | Records: <N>`.
Counter resets on each new connection.
### 16. Dark theme (nmea_client palette)
Color palette copied from `nmea_client/gui/plot_widget.py`:
| Role | Hex |
|------|-----|
| Main background | `#1a1a2e` |
| Plot / input background | `#12121e` |
| Borders | `#2a2a4a` / `#3a3a5e` |
| Normal text | `#aaaacc` |
| Bright text | `#ddddee` |
| Connected | `#4CAF50` |
| Error | `#F44336` |
Trace colors use the Material palette from nmea_client
(`#2196F3`, `#F44336`, `#4CAF50`, `#FF9800`, …).
---
## Bugs fixed
### Socket read timeout (gui_client spurious disconnect)
`socket.create_connection(timeout=5)` sets a permanent 5-second read timeout.
After the first message, if the collection interval > 5 s, `recv()` raised
`socket.timeout` and the GUI reported "Connection lost".
**Fix:** `self._sock.settimeout(None)` immediately after connect.
### `load_today()` never called after first poll
The lazy-load condition `if not records` was never true once the first
WeConnect poll had written a record into memory, so disk history was never
loaded for late-connecting clients.
**Fix:** Eager loading — `load_today_records()` called at startup, result
stored in `push_store_ref["preloaded"]`; merged with live records on every
connect.
### Erratic axis scaling on mouse hover
pyqtgraph's `InfiniteLine` (cursor crosshair) was included in auto-range
calculations. Moving the cursor triggered view rescaling on every mouse event.
**Fix:** `pw.addItem(vl, ignoreBounds=True)` in both the initial setup and
after each `_redraw()` call.
---
## Key design decisions
* **`store_ref` mutable dict** — shared between the main thread and the accept
loop thread under a single `threading.Lock`. Keys: `"current"` (live store)
and `"preloaded"` (today's disk records).
* **NDJSON protocol** — one JSON object per line over TCP; trivial to consume
with `nc`, `python client.py`, or the GUI.
* **pyqtgraph over matplotlib** — chosen for native Qt integration, live
update performance, and built-in zoom/pan without extra toolbars.
* **`_TRACE_LABELS` dict** — single place to maintain display names; falls
back to `key.split(".")[-1]` for unknown keys.
* **`QComboBox` item data** — short label displayed, full key stored via
`addItem(label, key)` / `currentData()`; no reverse-lookup dict needed.