# Conversation Recap — we_monitor development

## Overview

Starting from a single `collect.py` script that polled a VW WeConnect vehicle
and wrote JSON to disk, the project was iteratively extended into a modular
Python package with a TCP push server and a PyQt5 / pyqtgraph GUI client.

---

## Feature development (in order)

### 1. Credentials file (`-c / --credentials`)
Added a JSON credentials file parameter alongside the existing `-u / -p` flags.
File supports `username`, `password`, `vin`, and `domains` keys.  Domains may
be a JSON array or a comma-separated string.  Explicit CLI flags and env vars
(`WC_USER` / `WC_PASS`) take precedence over the file.

### 2. `--list-domains`
Added a CLI flag that prints all available domain names and exits.

### 3. Physical value format
Refactored all numeric measurements to `{"value": x, "unit": "km"}` objects,
stripping unit suffixes from field names (e.g. `odometer_km` → `odometer`).

### 4. Default log path + auto-create directory
Output defaults to `logs/YYYY_MM_DD_HH_MM_SS_<VIN>.json`; the `logs/`
directory is created automatically if it does not exist.

### 5. Midnight UTC log rotation
When using the default `logs/` directory the collector starts a new file after
midnight UTC.  The old file is kept intact; the new file is auto-named with the
current date stamp.

### 6. TCP push server
Added `--host` / `--port` flags (default `0.0.0.0:9999`).  The server pushes
every new snapshot to all connected clients as newline-delimited JSON
immediately after each poll.  The last snapshot is sent to newly connecting
clients so they have something to display right away.

### 7. CLI test client (`client.py`)
Connects to the push server, pretty-prints every incoming JSON record, and
prints a running record counter after each reception.

### 8. Refactor into modules
`collect.py` was split into focused server-side modules:

| Module | Responsibility |
|--------|---------------|
| `server/network.py` | TCP accept loop, broadcast |
| `server/data_model.py` | Domain extractors, `collect_snapshot()` |
| `server/storage_helpers.py` | Load / save JSON store, `load_today_records()` |
| `server/we_connect.py` | WeConnect login and vehicle selection |
| `server/log_config.py` | `logging.basicConfig` setup |

`collect.py` remains in the project root as the sole entry point.

### 9. PyQt5 GUI client (`gui_client.py`)
Three-tab application:

* **Connector** — host / port, Connect / Disconnect, status label.
* **Dashboard** — `QTreeWidget` showing domain → object → field/value/unit;
  expand state preserved across live updates.
* **Plot** — time-series charts of all physical fields using pyqtgraph.

### 10. Full-day history on connect
On client connect the server now sends all records from the current day instead
of only the most recent snapshot.  Records are loaded from disk at startup into
`push_store_ref["preloaded"]` and merged with live in-memory records (dedup by
`ts`, sorted).

### 11. Plot tab — user-selectable traces (pyqtgraph)
Replaced static matplotlib plots with four pyqtgraph `PlotWidget`s.  Users
select up to 4 traces per plot from a dropdown; zoom/pan via mouse;
crosshair cursor with nearest-value readout.

### 12. 8 plots with scrolling
Extended to 8 plots in a 4 × 2 scrollable grid (`QScrollArea`).  Per-plot
control bars (add-combo + `+` button + colored chip buttons) sit directly
above each plot.

### 13. Abbreviated trace names + label mapping
`_TRACE_LABELS` dict maps full dotted keys to short human-readable labels
(e.g. `charging.batteryStatus.currentSOC` → `"SOC"`).  Combo boxes, chip
buttons, legends, and cursor readout all use the short label; the full key is
stored as item data.

### 14. Persist plot settings
Trace selections saved to `~/.config/we_monitor/plot_settings.json` on every
add/remove.  Restored silently on the next launch once matching signals arrive
from the server.

### 15. Status bar record counter
The main window status bar shows `Last snapshot: <ts>  |  Records: <N>`.
Counter resets on each new connection.

### 16. Dark theme (nmea_client palette)
Color palette copied from `nmea_client/gui/plot_widget.py`:

| Role | Hex |
|------|-----|
| Main background | `#1a1a2e` |
| Plot / input background | `#12121e` |
| Borders | `#2a2a4a` / `#3a3a5e` |
| Normal text | `#aaaacc` |
| Bright text | `#ddddee` |
| Connected | `#4CAF50` |
| Error | `#F44336` |

Trace colors use the Material palette from nmea_client
(`#2196F3`, `#F44336`, `#4CAF50`, `#FF9800`, …).

---

## Bugs fixed

### Socket read timeout (gui_client spurious disconnect)
`socket.create_connection(timeout=5)` sets a permanent 5-second read timeout.
After the first message, if the collection interval > 5 s, `recv()` raised
`socket.timeout` and the GUI reported "Connection lost".
**Fix:** `self._sock.settimeout(None)` immediately after connect.

### `load_today()` never called after first poll
The lazy-load condition `if not records` was never true once the first
WeConnect poll had written a record into memory, so disk history was never
loaded for late-connecting clients.
**Fix:** Eager loading — `load_today_records()` called at startup, result
stored in `push_store_ref["preloaded"]`; merged with live records on every
connect.

### Erratic axis scaling on mouse hover
pyqtgraph's `InfiniteLine` (cursor crosshair) was included in auto-range
calculations.  Moving the cursor triggered view rescaling on every mouse event.
**Fix:** `pw.addItem(vl, ignoreBounds=True)` in both the initial setup and
after each `_redraw()` call.

---

## Key design decisions

* **`store_ref` mutable dict** — shared between the main thread and the accept
  loop thread under a single `threading.Lock`.  Keys: `"current"` (live store)
  and `"preloaded"` (today's disk records).
* **NDJSON protocol** — one JSON object per line over TCP; trivial to consume
  with `nc`, `python client.py`, or the GUI.
* **pyqtgraph over matplotlib** — chosen for native Qt integration, live
  update performance, and built-in zoom/pan without extra toolbars.
* **`_TRACE_LABELS` dict** — single place to maintain display names; falls
  back to `key.split(".")[-1]` for unknown keys.
* **`QComboBox` item data** — short label displayed, full key stored via
  `addItem(label, key)` / `currentData()`; no reverse-lookup dict needed.