347 lines
12 KiB
Markdown
347 lines
12 KiB
Markdown
# we_monitor
|
||
|
||
Periodically polls a VW vehicle via the `carconnectivity` library, stores
|
||
timestamped records as JSON, and pushes live data to connected TCP clients.
|
||
A PyQt5 GUI client visualises the data in real time.
|
||
|
||
## Repository layout
|
||
|
||
```
|
||
server/
|
||
main.py — collector entry point (CLI + credential loading)
|
||
collect.py — run() main loop
|
||
data_model.py — domain extractors, collect_snapshot()
|
||
network.py — TCP push server (accept loop + broadcast)
|
||
storage_helpers.py — load / save JSON store, midnight rotation helpers
|
||
we_connect.py — CarConnectivity login and vehicle selection
|
||
log_config.py — logging setup
|
||
requirements.txt — server dependencies
|
||
install.sh — systemd user service installer (server only)
|
||
client/
|
||
gui_client.py — PyQt5 GUI client
|
||
client.py — minimal CLI test client
|
||
requirements.txt — client dependencies
|
||
install.sh — client installer (creates venv + run.sh launcher)
|
||
credentials/ — JSON config files (gitignored)
|
||
```
|
||
|
||
## Server
|
||
|
||
### Install (systemd user service)
|
||
|
||
```bash
|
||
cd server
|
||
chmod +x install.sh
|
||
./install.sh
|
||
# then edit ~/.config/we_monitor/credentials.json
|
||
systemctl --user start we_monitor
|
||
```
|
||
|
||
### Manual start
|
||
|
||
```bash
|
||
pip install -r server/requirements.txt
|
||
python -m server.main -c credentials/my_car.json
|
||
```
|
||
|
||
Username and password can also be supplied via environment variables
|
||
(override the credentials file):
|
||
|
||
```bash
|
||
export WC_USER=me@example.com
|
||
export WC_PASS=secret
|
||
python -m server.main -c credentials/my_car.json
|
||
```
|
||
|
||
### Credentials file
|
||
|
||
All connection and collection settings live in a single JSON file:
|
||
|
||
```json
|
||
{
|
||
"credentials": {
|
||
"username": "me@example.com",
|
||
"password": "secret"
|
||
},
|
||
"vin": "WVWZZZE1ZMP000000",
|
||
"domains": ["charging", "electric_drive", "connectivity"],
|
||
"interval_s": 300,
|
||
"host": "0.0.0.0",
|
||
"port": 9999,
|
||
"log_dir": "./logs",
|
||
"diff": false
|
||
}
|
||
```
|
||
|
||
| Key | Default | Description |
|
||
|-----|---------|-------------|
|
||
| `credentials.username` | — | MyVolkswagen email |
|
||
| `credentials.password` | — | MyVolkswagen password |
|
||
| `vin` | first vehicle | Target VIN |
|
||
| `domains` | `["charging","electric_drive","connectivity"]` | Domains to collect (list or `"all"`) |
|
||
| `interval_s` | `300` | Collection interval in seconds |
|
||
| `host` | `"0.0.0.0"` | Push server bind address |
|
||
| `port` | `9999` | Push server TCP port |
|
||
| `log_dir` | `"./logs"` | Directory for rotating JSON log files |
|
||
| `diff` | `false` | Enable diff-mode compression on the push server |
|
||
|
||
### CLI reference
|
||
|
||
CLI flags override the corresponding credentials-file value when given.
|
||
|
||
| Flag | Overrides | Description |
|
||
|------|-----------|-------------|
|
||
| `-c FILE` | — | Credentials / settings file |
|
||
| `-u / -p` | `credentials.username/password` | Username / password (also `WC_USER` / `WC_PASS`) |
|
||
| `-d DOMAINS` | `domains` | Domains to collect (comma-separated or `all`) |
|
||
| `-i SECONDS` | `interval_s` | Collection interval in seconds |
|
||
| `--host HOST` | `host` | Push server bind address |
|
||
| `--port PORT` | `port` | Push server TCP port |
|
||
| `--log-dir DIR` | `log_dir` | Directory for rotating log files |
|
||
| `--max-records N` | — | Keep only the last N records per file (default: unlimited) |
|
||
| `--diff` | `diff` | Enable diff-mode on the push server |
|
||
| `--dry` | — | Skip CarConnectivity login; run push server only (useful for GUI testing) |
|
||
| `-v` | — | Verbose / debug logging |
|
||
| `--list-domains` | — | Print available domains and exit |
|
||
|
||
**Available domains:** `charging`, `climatisation`, `electric_drive`,
|
||
`connectivity`, `vehicle`, `position`, `doors`
|
||
|
||
### Log rotation
|
||
|
||
A new file is started automatically after midnight UTC; previous files are kept
|
||
intact. On startup and on each new client connection the collector sends the
|
||
last 24 hours of records, so clients always receive a full rolling window
|
||
regardless of when midnight falls.
|
||
|
||
### Output format
|
||
|
||
Log files are named `YYYY_MM_DD_HH_MM_SS_<VIN>.json` and contain:
|
||
|
||
```json
|
||
{
|
||
"meta": { "vin": "...", "created_at": "...", "interval_seconds": 300 },
|
||
"records": [
|
||
{
|
||
"ts": "2025-05-25T10:00:00+00:00",
|
||
"charging": {
|
||
"state": "readyForCharging",
|
||
"power": { "value": 11.0, "unit": "kW" }
|
||
},
|
||
"electric_drive": {
|
||
"range": { "value": 310, "unit": "km" },
|
||
"battery": { "soc": { "value": 80, "unit": "%" } }
|
||
}
|
||
}
|
||
]
|
||
}
|
||
```
|
||
|
||
Physical values are always `{"value": …, "unit": …}` objects.
|
||
|
||
---
|
||
|
||
## Push server protocol
|
||
|
||
The server listens on the `host`:`port` from the credentials file
|
||
(default `0.0.0.0:9999`).
|
||
|
||
* **On connect:** sends all records from the last 24 hours, sorted by timestamp.
|
||
* **On each new poll:** sends the new snapshot immediately.
|
||
|
||
All messages are **newline-delimited JSON** (one record per line, `\n` terminated).
|
||
|
||
### Diff mode (`--diff`)
|
||
|
||
When diff mode is enabled the history burst and every live snapshot are
|
||
compressed using `jay_diff_full` before transmission. The wire format is
|
||
self-describing — diffs carry `update_add` / `delete` keys, full snapshots do
|
||
not — so no separate framing is needed.
|
||
|
||
The first message sent to a new client is always `jay_diff_full({}, record[0])`,
|
||
i.e. an `update_add` containing every field of the first record. Subsequent
|
||
messages contain only the fields that changed.
|
||
|
||
Enable `--diff` on the server **and** on the client (`client/client.py --diff` or the
|
||
**Diff mode** checkbox in the GUI Connector tab). Both sides must agree — a
|
||
mismatch produces garbled output.
|
||
|
||
---
|
||
|
||
## Client
|
||
|
||
### Install
|
||
|
||
```bash
|
||
cd client
|
||
chmod +x install.sh
|
||
./install.sh
|
||
```
|
||
|
||
This creates `client/.venv` and writes a `client/run.sh` launcher.
|
||
|
||
### Manual start
|
||
|
||
```bash
|
||
pip install -r client/requirements.txt
|
||
python client/gui_client.py
|
||
```
|
||
|
||
### GUI client (`client/gui_client.py`)
|
||
|
||
```bash
|
||
client/run.sh # after install.sh
|
||
```
|
||
|
||
Dark-theme PyQt5 application. Three tabs: **Connector**, **Dashboard**,
|
||
**Plot**. Connection settings, trace selections, and time windows are persisted
|
||
to `~/.config/we_monitor/plot_settings.json` and restored on the next launch.
|
||
|
||
#### Connector tab
|
||
|
||
| Control | Description |
|
||
|---------|-------------|
|
||
| **Host / Port** | Address of the push server |
|
||
| **Diff mode** checkbox | Must match the server's `--diff` setting |
|
||
| **Connect / Disconnect** | Opens or closes the TCP connection |
|
||
| **Status** | Shows `Connected`, `Disconnected`, or the error message |
|
||
|
||
Clicking **Connect** saves the current host, port, and diff-mode setting. All
|
||
plot data is cleared on each connect so the fresh server history is never mixed
|
||
with stale local data.
|
||
|
||
#### Dashboard tab
|
||
|
||
Live tree view of the most recent snapshot. The hierarchy mirrors the JSON
|
||
structure: domain → status object → field, with value and unit shown inline.
|
||
Expand / collapse state is preserved across updates.
|
||
|
||
#### Plot tab
|
||
|
||
Eight plots arranged in a 4 × 2 scrollable grid. Each plot is independent and
|
||
has its own control bar.
|
||
|
||
**Control bar (per plot)**
|
||
|
||
| Control | Description |
|
||
|---------|-------------|
|
||
| **Signal dropdown** | Lists every numeric `{value, unit}` field received so far |
|
||
| **`+` button** | Adds the selected signal as a new trace (max 4 per plot) |
|
||
| **Colored chips** (`name ×`) | One chip per active trace; click `×` to remove |
|
||
| **Time window spinner** | Show the last 1–24 hours of data (per plot) |
|
||
| **Date label** (right-aligned) | Shows the calendar date(s) of the visible window — single date or `YYYY-MM-DD – YYYY-MM-DD` when the view spans midnight |
|
||
|
||
**Plot area**
|
||
|
||
* **X axis** — time of day in `HH:MM:SS` format; no locale-dependent date prefix.
|
||
* **Y axis** — auto-zooms to fit the visible data on the first draw after connect
|
||
or after a trace is added / removed. Manual zoom is preserved between data
|
||
updates.
|
||
* **Crosshair** — dashed vertical line tracks the mouse cursor.
|
||
* **Floating tooltip** — appears next to the cursor and shows the nearest
|
||
sampled value for every active trace, with a colored bullet and unit.
|
||
* **Legend** — trace names and colors, shown inside the plot area.
|
||
|
||
**Mouse interaction**
|
||
|
||
| Action | Effect |
|
||
|--------|--------|
|
||
| Scroll wheel | Zoom X axis in / out, centred on cursor |
|
||
| Shift + scroll wheel | Zoom Y axis in / out, centred on cursor |
|
||
| Click + drag | Pan the view |
|
||
| Right-click | pyqtgraph context menu (export, auto-range, …) |
|
||
|
||
### CLI test client (`client/client.py`)
|
||
|
||
```bash
|
||
python client/client.py --host 192.168.1.10 --port 9999
|
||
python client/client.py --diff # reconstruct from diff stream
|
||
```
|
||
|
||
Connects to the push server and prints every incoming snapshot as
|
||
pretty-printed JSON with a running record counter.
|
||
|
||
| Flag | Description |
|
||
|------|-------------|
|
||
| `--host HOST` | Server host (default: `127.0.0.1`) |
|
||
| `--port PORT` | Server port (default: `9999`) |
|
||
| `--diff` | Reconstruct full snapshots from diff stream |
|
||
|
||
---
|
||
|
||
## Module reference
|
||
|
||
### `server/main.py`
|
||
|
||
Entry point for the collector daemon. Parses CLI / credentials-file arguments
|
||
and calls `run()`.
|
||
|
||
### `server/collect.py`
|
||
|
||
`run(args)` — establishes the CarConnectivity session, starts the push server,
|
||
and runs the main poll loop.
|
||
|
||
### `server/data_model.py`
|
||
|
||
`collect_snapshot(vehicle, domains) → dict`
|
||
Iterates over the requested domains on a CarConnectivity vehicle object and extracts
|
||
fields into a flat dict keyed by domain.
|
||
Physical values are normalised to `{"value": …, "unit": …}`. Returns a record
|
||
with a top-level `"ts"` ISO-8601 timestamp.
|
||
|
||
`ALL_DOMAINS`
|
||
Tuple of all domain names the collector knows about.
|
||
|
||
### `server/network.py`
|
||
|
||
`start_push_server(host, port) → (clients, lock, store_ref)`
|
||
Binds a TCP socket, starts the accept-loop thread (daemon), and returns the
|
||
shared state needed by `broadcast`.
|
||
|
||
`broadcast(snapshot, clients, lock)`
|
||
Serialises `snapshot` to a JSON line and sends it to every connected client.
|
||
Dead connections are removed silently.
|
||
|
||
`store_ref` dict keys:
|
||
| Key | Type | Description |
|
||
|-----|------|-------------|
|
||
| `"current"` | `dict` | The currently-open JSON store |
|
||
| `"preloaded"` | `list` | 24-h history records loaded at startup |
|
||
| `"diff_mode"` | `bool` | Whether to compress the history burst as diffs |
|
||
|
||
### `server/storage_helpers.py`
|
||
|
||
`load_store(path, vin, interval) → dict`
|
||
Loads an existing store file or creates a fresh one.
|
||
|
||
`save_store(path, store, max_records)`
|
||
Writes the store to disk, optionally trimming to the last `max_records` entries.
|
||
|
||
`auto_out_path(log_dir, vin) → Path`
|
||
Returns a timestamped file path for a new store.
|
||
|
||
`load_last_24h_records(log_dir, vin) → list`
|
||
Scans the log directory for files belonging to `vin`, loads any records from
|
||
the last 24 hours, and returns them sorted by timestamp.
|
||
|
||
### `server/we_connect.py`
|
||
|
||
`connect(username, password) → CarConnectivity`
|
||
Authenticates and returns a connected `CarConnectivity` instance.
|
||
|
||
`select_vehicle(wc, vin) → (vehicle, error)`
|
||
Returns the target vehicle (first available if `vin` is empty) or an error string.
|
||
|
||
### `client/gui_client.py` — class overview
|
||
|
||
| Class | Role |
|
||
|-------|------|
|
||
| `TcpReader` | Background thread; reads newline-delimited JSON from the socket, reconstructs full snapshots in diff mode via `jay_merge_full`, and emits each snapshot via the `message` Qt signal |
|
||
| `ConnectorTab` | Host / port / diff-mode form; emits `connect_requested` and `disconnect_requested` signals; persists settings |
|
||
| `DashboardTab` | Live `QTreeWidget` showing the latest snapshot; preserves expand state across updates |
|
||
| `PlotTab` | Eight independent time-series plots in a scrollable grid |
|
||
| `TimeAxisItem` | `pg.AxisItem` subclass; formats x-axis tick labels as `HH:MM:SS` |
|
||
| `PlotViewBox` | `pg.ViewBox` subclass; routes plain scroll to X zoom and Shift+scroll to Y zoom |
|
||
| `PlotTooltip` | Frameless child `QWidget` per plot; shows nearest sampled values for all active traces at the cursor position |
|
||
| `MainWindow` | `QMainWindow`; wires the three tabs together and owns the `TcpReader` |
|