collect: fix stale vehicle ref after token re-login; increase update verbosity

After a full re-login (doLogin), the library rebuilds its vehicle list with
new objects. The old vehicle reference in the collect loop was never updated
again, causing data to silently freeze. Re-select the vehicle from wc.vehicles
after every update() call to stay on the current object.

Also: update() now returns and logs its bool result; the record log line shows
which top-level keys changed each cycle (or "none" when the backend returned
identical data).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

.gitignore

@@ -3,3 +3,5 @@ credentials/
 vehicle_data.json
 logs/
 __pycache__/
+.venv/
+client/run.sh

README.md

@@ -7,45 +7,41 @@ A PyQt5 GUI client visualises the data in real time.
 ## Repository layout
 
 ```
-collect.py              — collector daemon (entry point)
-gui_client.py           — PyQt5 GUI client
-client.py               — minimal CLI test client
 server/
+  main.py               — collector entry point (CLI + credential loading)
+  collect.py            — run() main loop
   data_model.py         — domain extractors, collect_snapshot()
   network.py            — TCP push server (accept loop + broadcast)
   storage_helpers.py    — load / save JSON store, midnight rotation helpers
   we_connect.py         — CarConnectivity login and vehicle selection
   log_config.py         — logging setup
-utils/
-  jay_diff.py           — JSON diff / merge library
+  requirements.txt      — server dependencies
+  install.sh            — systemd user service installer (server only)
+client/
+  gui_client.py         — PyQt5 GUI client
+  client.py             — minimal CLI test client
+  requirements.txt      — client dependencies
+  install.sh            — client installer (creates venv + run.sh launcher)
 credentials/            — JSON config files (gitignored)
-requirements.txt
 ```
 
-## Requirements
+## Server
 
-```
-pip install carconnectivity carconnectivity-connector-volkswagen PyQt5 pyqtgraph
-```
-
-Or use the installer which also sets up a systemd user service:
+### Install (systemd user service)
 
 ```bash
+cd server
+chmod +x install.sh
 ./install.sh
+# then edit ~/.config/we_monitor/credentials.json
+systemctl --user start we_monitor
 ```
 
----
-
-## Collector (`collect.py`)
-
-Connects via CarConnectivity, polls selected domains at a configurable interval, and
-appends timestamped records to a rotating JSON log.  Simultaneously runs a TCP
-push server so clients receive every new snapshot the moment it is collected.
-
-### Quick start
+### Manual start
 
 ```bash
-python collect.py -c credentials/my_car.json
+pip install -r server/requirements.txt
+python -m server.main -c credentials/my_car.json
 ```
 
 Username and password can also be supplied via environment variables
@@ -54,7 +50,7 @@ Username and password can also be supplied via environment variables
 ```bash
 export WC_USER=me@example.com
 export WC_PASS=secret
-python collect.py -c credentials/my_car.json
+python -m server.main -c credentials/my_car.json
 ```
 
 ### Credentials file
@@ -166,27 +162,47 @@ The first message sent to a new client is always `jay_diff_full({}, record[0])`,
 i.e. an `update_add` containing every field of the first record.  Subsequent
 messages contain only the fields that changed.
 
-Enable `--diff` on the server **and** on the client (`client.py --diff` or the
+Enable `--diff` on the server **and** on the client (`client/client.py --diff` or the
 **Diff mode** checkbox in the GUI Connector tab).  Both sides must agree — a
 mismatch produces garbled output.
 
 ---
 
-## GUI client (`gui_client.py`)
+## Client
+
+### Install
 
 ```bash
-python gui_client.py
+cd client
+chmod +x install.sh
+./install.sh
+```
+
+This creates `client/.venv`, writes a `client/run.sh` launcher, and installs
+`~/.local/bin/we_monitor_gui` so the GUI is available system-wide.
+
+### Manual start
+
+```bash
+pip install -r client/requirements.txt
+python client/gui_client.py
+```
+
+### GUI client (`client/gui_client.py`)
+
+```bash
+we_monitor_gui       # after install.sh
 ```
 
 Dark-theme PyQt5 application.  Three tabs: **Connector**, **Dashboard**,
 **Plot**.  Connection settings, trace selections, and time windows are persisted
 to `~/.config/we_monitor/plot_settings.json` and restored on the next launch.
 
-### Connector tab
+#### Connector tab
 
 | Control | Description |
 |---------|-------------|
-| **Host / Port** | Address of the `collect.py` push server |
+| **Host / Port** | Address of the push server |
 | **Diff mode** checkbox | Must match the server's `--diff` setting |
 | **Connect / Disconnect** | Opens or closes the TCP connection |
 | **Status** | Shows `Connected`, `Disconnected`, or the error message |
@@ -195,18 +211,18 @@ Clicking **Connect** saves the current host, port, and diff-mode setting.  All
 plot data is cleared on each connect so the fresh server history is never mixed
 with stale local data.
 
-### Dashboard tab
+#### Dashboard tab
 
 Live tree view of the most recent snapshot.  The hierarchy mirrors the JSON
 structure: domain → status object → field, with value and unit shown inline.
 Expand / collapse state is preserved across updates.
 
-### Plot tab
+#### Plot tab
 
 Eight plots arranged in a 4 × 2 scrollable grid.  Each plot is independent and
 has its own control bar.
 
-#### Control bar (per plot)
+**Control bar (per plot)**
 
 | Control | Description |
 |---------|-------------|
@@ -216,7 +232,7 @@ has its own control bar.
 | **Time window spinner** | Show the last 1–24 hours of data (per plot) |
 | **Date label** (right-aligned) | Shows the calendar date(s) of the visible window — single date or `YYYY-MM-DD – YYYY-MM-DD` when the view spans midnight |
 
-#### Plot area
+**Plot area**
 
 * **X axis** — time of day in `HH:MM:SS` format; no locale-dependent date prefix.
 * **Y axis** — auto-zooms to fit the visible data on the first draw after connect
@@ -227,7 +243,7 @@ has its own control bar.
   sampled value for every active trace, with a colored bullet and unit.
 * **Legend** — trace names and colors, shown inside the plot area.
 
-#### Mouse interaction
+**Mouse interaction**
 
 | Action | Effect |
 |--------|--------|
@@ -236,13 +252,11 @@ has its own control bar.
 | Click + drag | Pan the view |
 | Right-click | pyqtgraph context menu (export, auto-range, …) |
 
----
-
-## CLI test client (`client.py`)
+### CLI test client (`client/client.py`)
 
 ```bash
-python client.py --host 192.168.1.10 --port 9999
-python client.py --diff   # reconstruct from diff stream
+python client/client.py --host 192.168.1.10 --port 9999
+python client/client.py --diff   # reconstruct from diff stream
 ```
 
 Connects to the push server and prints every incoming snapshot as
@@ -258,11 +272,15 @@ pretty-printed JSON with a running record counter.
 
 ## Module reference
 
-### `collect.py`
+### `server/main.py`
 
-Entry point for the collector daemon.  Parses CLI / credentials-file arguments,
-establishes the CarConnectivity session, starts the push server, and runs the main
-poll loop.
+Entry point for the collector daemon.  Parses CLI / credentials-file arguments
+and calls `run()`.
+
+### `server/collect.py`
+
+`run(args)` — establishes the CarConnectivity session, starts the push server,
+and runs the main poll loop.
 
 ### `server/data_model.py`
 
@@ -315,22 +333,7 @@ Authenticates and returns a connected `CarConnectivity` instance.
 `select_vehicle(wc, vin) → (vehicle, error)`  
 Returns the target vehicle (first available if `vin` is empty) or an error string.
 
-### `utils/jay_diff.py`
-
-Lightweight JSON diff / patch library.
-
-`jay_diff_full(old, new, combine_upd_add=True) → dict`  
-Computes a diff between two dicts.  With `combine_upd_add=True` (the default
-used everywhere in this project) the result uses `update_add` for all additions
-and updates, plus a separate `delete` key.  Returns `{}` when nothing changed.
-
-`jay_merge_full(old, diff) → dict`  
-Applies a `jay_diff_full` diff to `old`.  The diff structure is self-describing:
-if `diff` contains none of the known diff keys (`update`, `update_add`, `add`,
-`delete`) and `old` is empty, `diff` is returned as-is (full-snapshot
-passthrough for bootstrap).
-
-### `gui_client.py` — class overview
+### `client/gui_client.py` — class overview
 
 | Class | Role |
 |-------|------|

cariad-hybrid-auth-fix.patch

@@ -0,0 +1,283 @@
+diff --git a/volkswagencarnet/vw_connection.py b/volkswagencarnet/vw_connection.py
+index aac7fcc..944e8e1 100644
+--- a/volkswagencarnet/vw_connection.py
++++ b/volkswagencarnet/vw_connection.py
+@@ -4,10 +4,12 @@
+ from __future__ import annotations
+ 
+ import asyncio
++import base64
+ from datetime import UTC, datetime, timedelta
+ import hashlib
+ import logging
+ from random import randint, random
++import secrets
+ from urllib.parse import parse_qs, urljoin, urlparse
+ from typing import Dict, Optional
+ 
+@@ -138,26 +140,43 @@ class Connection:
+             )
+         return await req.json()
+ 
+-    async def get_authorization_page(self, authorization_endpoint: str) -> str:
+-        """Get authorization page (login page)."""
+-        # https://identity.vwgroup.io/oidc/v1/authorize?nonce={NONCE}&state={STATE}&response_type={TOKEN_TYPES}&scope={SCOPE}&redirect_uri={APP_URI}&client_id={CLIENT_ID}
+-        # https://identity.vwgroup.io/oidc/v1/authorize?client_id={CLIENT_ID}&scope={SCOPE}&response_type={TOKEN_TYPES}&redirect_uri={APP_URI}
++    @staticmethod
++    def _generate_pkce_pair() -> tuple[str, str]:
++        """Return (code_verifier, code_challenge) for PKCE S256."""
++        verifier = base64.urlsafe_b64encode(secrets.token_bytes(32)).rstrip(b"=").decode()
++        digest = hashlib.sha256(verifier.encode()).digest()
++        challenge = base64.urlsafe_b64encode(digest).rstrip(b"=").decode()
++        return verifier, challenge
++
++    async def get_authorization_page(
++        self, authorization_endpoint: str, extra_params: dict | None = None
++    ) -> tuple[str, str]:
++        """Get authorization page (login page).
++
++        Returns:
++            (page_html, page_url) — the URL is needed as the POST target when
++            the form has no explicit action attribute.
++        """
+         _LOGGER.debug('Requesting authorization page from "%s"', authorization_endpoint)
+         self._session_auth_headers.pop("Referer", None)
+         self._session_auth_headers.pop("Origin", None)
+         _LOGGER.debug('Request headers: "%s"', self._session_auth_headers)
+ 
++        params = {
++            "redirect_uri": APP_URI,
++            "response_type": CLIENT_TOKEN_TYPES,
++            "client_id": CLIENT_ID,
++            "scope": CLIENT_SCOPE,
++        }
++        if extra_params:
++            params.update(extra_params)
++
+         try:
+             req = await self._session.get(
+                 url=authorization_endpoint,
+                 headers=self._session_auth_headers,
+                 allow_redirects=False,
+-                params={
+-                    "redirect_uri": APP_URI,
+-                    "response_type": CLIENT_TOKEN_TYPES,
+-                    "client_id": CLIENT_ID,
+-                    "scope": CLIENT_SCOPE,
+-                },
++                params=params,
+             )
+ 
+             # Check if the response contains a redirect location
+@@ -177,15 +196,20 @@ class Connection:
+                 _LOGGER.info("Authorization error: %s", error_description)
+                 raise AuthenticationError(f"{error_msg}: {error_description}")
+ 
+-            # If redirected, fetch the new location
+-            req = await self._session.get(
+-                url=ref, headers=self._session_auth_headers, allow_redirects=False
+-            )
+-
+-            if req.status != 200:
+-                raise AuthenticationError("Failed to fetch authorization endpoint")
+-
+-            return await req.text()
++            # Follow redirects to the login page (may be more than one hop)
++            while True:
++                req = await self._session.get(
++                    url=ref, headers=self._session_auth_headers, allow_redirects=False
++                )
++                if req.status == 200:
++                    return await req.text(), ref
++                if req.status in (301, 302, 303, 307, 308):
++                    next_loc = req.headers.get("Location")
++                    if not next_loc:
++                        raise AuthenticationError("Missing Location header during login redirect chain")
++                    ref = urljoin(ref, next_loc)
++                    continue
++                raise AuthenticationError(f"Failed to fetch authorization endpoint: HTTP {req.status}")
+ 
+         except Exception as e:
+             _LOGGER.warning("Error during fetching authorization page: %s", str(e))
+@@ -200,6 +224,26 @@ class Connection:
+             return None
+         return state_input["value"]
+ 
++    def extract_login_form_data(self, page_content: str) -> dict:
++        """Extract all hidden fields and submit button values from the login form."""
++        soup = BeautifulSoup(page_content, "html.parser")
++        form = soup.select_one("form")
++        if not form:
++            return {}
++        form_data = {}
++        for inp in form.find_all("input"):
++            name = inp.get("name")
++            value = inp.get("value", "")
++            if name and inp.get("type") in ("hidden", None):
++                form_data[name] = value
++        # Include submit button value if it carries an action name (e.g. name="action" value="default")
++        for btn in form.find_all("button", type="submit"):
++            name = btn.get("name")
++            value = btn.get("value", "")
++            if name:
++                form_data[name] = value
++        return form_data
++
+     async def post_form(
+         self, session, url: str, headers: dict, form_data: dict, redirect: bool = True
+     ) -> str:
+@@ -227,7 +271,8 @@ class Connection:
+                 if error_code == "wrong-email-credentials":
+                     raise AuthenticationError("Wrong username or password")
+ 
+-            # Unknown 400 error
++            # Unknown 400 error — log the response body to aid debugging
++            _LOGGER.debug("Login 400 response body: %s", page_content[:2000])
+             raise AuthenticationError(
+                 "Login form validation failed with unknown 400 error"
+             )
+@@ -303,10 +348,22 @@ class Connection:
+         authorization_endpoint = openid_config["authorization_endpoint"]
+         auth_issuer = openid_config["issuer"]
+ 
+-        # Get authorization page
+-        authorization_page = await self.get_authorization_page(authorization_endpoint)
++        # PKCE + nonce (nonce required when id_token appears in response_type)
++        code_verifier, code_challenge = self._generate_pkce_pair()
++        nonce = base64.urlsafe_b64encode(secrets.token_bytes(16)).rstrip(b"=").decode()
++
++        # Get authorization page; returns (html, page_url) — use page_url as POST target
++        # because the form has no action attribute and implicitly submits to itself.
++        authorization_page, login_url = await self.get_authorization_page(
++            authorization_endpoint,
++            extra_params={
++                "code_challenge": code_challenge,
++                "code_challenge_method": "S256",
++                "nonce": nonce,
++            },
++        )
+ 
+-        # Extract form data
++        # Extract state token (still needed for the login URL)
+         state_token = self.extract_state_token(authorization_page)
+ 
+         if not state_token:
+@@ -316,13 +373,12 @@ class Connection:
+             )
+             raise AuthenticationError("Invalid login page - missing state token")
+ 
+-        # Do login
+-        login_form = {
+-            "username": self._session_auth_username,
+-            "password": self._session_auth_password,
+-            "state": state_token,
+-        }
+-        login_url = f"{auth_issuer}/u/login?state={state_token}"
++        # Build form from all hidden fields on the page so Auth0 gets every
++        # required field (e.g. csrf tokens, action), then overlay credentials.
++        login_form = self.extract_login_form_data(authorization_page)
++        login_form["username"] = self._session_auth_username
++        login_form["password"] = self._session_auth_password
++        login_form.setdefault("state", state_token)
+ 
+         redirect_location = await self.post_form(
+             self._session,
+@@ -337,36 +393,40 @@ class Connection:
+             self._session, auth_issuer, redirect_location
+         )
+ 
+-        jwt_auth_code = parse_qs(urlparse(redirect_response).query)["code"][0]
+-        return jwt_auth_code
++        # Parse both query string and fragment — hybrid flow puts tokens in the fragment
++        all_params = {
++            **parse_qs(urlparse(redirect_response).query),
++            **parse_qs(urlparse(redirect_response).fragment),
++        }
++        jwt_auth_code = all_params["code"][0]
++        callback_id_token = all_params.get("id_token", [None])[0]
++        callback_access_token = all_params.get("access_token", [None])[0]
++        return jwt_auth_code, code_verifier, callback_id_token, callback_access_token
+ 
+     async def _exchange_code_for_tokens(
+-        self, auth_code: str, token_endpoint: str
++        self, auth_code: str, token_endpoint: str, code_verifier: str | None = None
+     ) -> dict:
+-        """Exchange authorization code for access tokens.
+-
+-        Args:
+-            auth_code: Authorization code from login flow
+-            token_endpoint: Token endpoint URL
+-
+-        Returns:
+-            Dictionary containing tokens
+-
+-        Raises:
+-            AuthenticationError: If token exchange fails
+-        """
++        """Exchange authorization code for access tokens via the CARIAD BFF token endpoint."""
+         token_body = {
+             "client_id": CLIENT_ID,
+             "grant_type": "authorization_code",
+             "code": auth_code,
+             "redirect_uri": APP_URI,
+         }
++        if code_verifier:
++            token_body["code_verifier"] = code_verifier
++
++        token_headers = {
++            "Accept-Encoding": "gzip, deflate, br",
++            "Connection": "keep-alive",
++            "Content-Type": "application/x-www-form-urlencoded",
++            "User-Agent": USER_AGENT,
++            "x-android-package-name": ANDROID_PACKAGE_NAME,
++        }
+ 
+-        # Token endpoint
+         token_response = await self.post_form(
+-            self._session, token_endpoint, self._session_auth_headers, token_body
++            self._session, token_endpoint, token_headers, token_body
+         )
+-
+         return json_loads(token_response)
+ 
+     async def _login(self) -> bool:
+@@ -385,11 +445,21 @@ class Connection:
+             openid_config = await self.get_openid_config()
+             token_endpoint = openid_config["token_endpoint"]
+ 
+-            # Get authorization code
+-            auth_code = await self._get_authorization_code(openid_config)
+-
+-            # Exchange code for tokens
+-            tokens = await self._exchange_code_for_tokens(auth_code, token_endpoint)
++            # Get authorization code; hybrid response_type also delivers tokens directly
++            auth_code, code_verifier, cb_id_token, cb_access_token = await self._get_authorization_code(openid_config)
++
++            # Hybrid flow: tokens arrive in the callback URL — use them directly.
++            # The CARIAD BFF token endpoint requires a client_secret embedded in the
++            # official VW app, so we skip it when the callback already carries the tokens.
++            if cb_id_token and cb_access_token:
++                _LOGGER.debug("Using tokens from hybrid flow callback")
++                tokens = {
++                    "access_token": cb_access_token,
++                    "id_token": cb_id_token,
++                    "token_type": "Bearer",
++                }
++            else:
++                tokens = await self._exchange_code_for_tokens(auth_code, token_endpoint, code_verifier)
+ 
+             # Validate token structure
+             required_keys = ["access_token", "id_token", "token_type"]
+diff --git a/volkswagencarnet/vw_const.py b/volkswagencarnet/vw_const.py
+index 17666a1..711fcf9 100644
+--- a/volkswagencarnet/vw_const.py
++++ b/volkswagencarnet/vw_const.py
+@@ -7,7 +7,7 @@ COUNTRY = "DE"
+ # Data used in communication
+ CLIENT_ID = "a24fba63-34b3-4d43-b181-942111e6bda8@apps_vw-dilab_com"
+ CLIENT_SCOPE = "openid profile badge cars dealers vin"
+-CLIENT_TOKEN_TYPES = "code"
++CLIENT_TOKEN_TYPES = "code id_token token"
+ 
+ USER_AGENT = "Volkswagen/3.61.0-android/14"
+ APP_URI = "weconnect://authenticated"

client/gui_client.py

@@ -31,7 +31,7 @@ from pathlib import Path
 from PyQt5.QtCore import QObject, QTimer, Qt, pyqtSignal
 from PyQt5.QtGui import QColor, QFont
 from PyQt5.QtWidgets import (
-    QApplication, QCheckBox, QComboBox, QFormLayout, QGridLayout, QGroupBox,
+    QApplication, QComboBox, QFormLayout, QGridLayout, QGroupBox,
     QHBoxLayout, QLabel, QLineEdit, QMainWindow, QPushButton, QScrollArea,
     QSizePolicy, QSpinBox, QStatusBar, QTabWidget, QTreeWidget, QTreeWidgetItem,
     QVBoxLayout, QWidget,
@@ -95,16 +95,16 @@ _SETTINGS_PATH   = Path.home() / ".config" / "we_monitor" / "plot_settings.json"
 # cursor readout.  Falls back to the last path component for unknown keys.
 _TRACE_LABELS: dict[str, str] = {
     "charging.power":                                "charge power",
-    "charging.remain":                               "time to full",
-    "charging.settings.target_level":               "target SOC",
-    "climatisation.remain":                          "clim. time",
-    "climatisation.settings.target_temperature":     "target temp",
-    "climatisation.environment.temperature_outside": "outside temp",
-    "electric_drive.battery.soc":                    "SOC",
+    "charging.remain":                               "charge time remain",
+    "charging.settings.target_level":                "battery SOC set",
+    "climatisation.remain":                          "clima time remain",
+    "climatisation.settings.target_temperature":     "clima temperature set",
+    "climatisation.environment.temperature_outside": "outside temperature",
+    "electric_drive.battery.soc":                    "battery soc",
     "electric_drive.range":                          "range",
     "electric_drive.range_full":                     "range (full)",
-    "electric_drive.battery.temperature_max":        "batt. temp max",
-    "electric_drive.battery.temperature_min":        "batt. temp min",
+    "electric_drive.battery.temperature_max":        "battery temperature (max)",
+    "electric_drive.battery.temperature_min":        "battery temperature (min)",
     "electric_drive.odometer.odometer":              "odometer",
     "procedural.range_at_100":                       "range 100%",
 }
@@ -124,20 +124,18 @@ class TcpReader(QObject):
 
     def __init__(self):
         super().__init__()
-        self._sock      = None
-        self._running   = False
-        self._diff_mode = False
+        self._sock    = None
+        self._running = False
         self._state:  dict = {}
 
-    def connect_to(self, host: str, port: int, diff_mode: bool = False) -> str | None:
+    def connect_to(self, host: str, port: int) -> str | None:
         """Open connection; returns error string or None on success."""
         try:
             self._sock = socket.create_connection((host, port), timeout=5)
             self._sock.settimeout(None)
         except OSError as exc:
             return str(exc)
-        self._diff_mode = diff_mode
-        self._state     = {}
+        self._state   = {}
         self._running   = True
         threading.Thread(target=self._read_loop, daemon=True).start()
         return None
@@ -165,11 +163,8 @@ class TcpReader(QObject):
                     if line:
                         try:
                             data = json.loads(line)
-                            if self._diff_mode:
-                                self._state = jay_merge_full(self._state, data)
-                                self.message.emit(copy.deepcopy(self._state))
-                            else:
-                                self.message.emit(data)
+                            self._state = jay_merge_full(self._state, data)
+                            self.message.emit(copy.deepcopy(self._state))
                         except json.JSONDecodeError:
                             pass
         except OSError:
@@ -185,6 +180,23 @@ def _is_phys(v) -> bool:
     return isinstance(v, dict) and "value" in v and "unit" in v
 
 
+_CHANGED_BG = QColor("#1e3a00")
+
+
+def _flatten_values(data: dict, prefix: str = "") -> dict[str, object]:
+    """Flat dict of path → value for every physical leaf, using '/' separator."""
+    result = {}
+    for k, v in data.items():
+        if k == "ts":
+            continue
+        path = f"{prefix}/{k}" if prefix else k
+        if _is_phys(v):
+            result[path] = v.get("value")
+        elif isinstance(v, dict):
+            result.update(_flatten_values(v, path))
+    return result
+
+
 def _extract_phys(data: dict, prefix: str = "") -> dict[str, dict]:
     """Return a flat dict of path → {value, unit} for every physical leaf."""
     result = {}
@@ -217,11 +229,9 @@ class ConnectorTab(QWidget):
         self._port = QSpinBox()
         self._port.setRange(1, 65535)
         self._port.setValue(9999)
-        self._diff_cb = QCheckBox("Diff mode")
         self._load_conn_settings()
         form.addRow("Host:", self._host)
         form.addRow("Port:", self._port)
-        form.addRow(self._diff_cb)
 
         btn_row = QHBoxLayout()
         self._btn = QPushButton("Connect")
@@ -244,10 +254,6 @@ class ConnectorTab(QWidget):
         else:
             self.disconnect_requested.emit()
 
-    @property
-    def diff_mode(self) -> bool:
-        return self._diff_cb.isChecked()
-
     def _save_conn_settings(self):
         try:
             _SETTINGS_PATH.parent.mkdir(parents=True, exist_ok=True)
@@ -256,9 +262,8 @@ class ConnectorTab(QWidget):
                 data = json.loads(_SETTINGS_PATH.read_text())
             except (OSError, json.JSONDecodeError):
                 pass
-            data["host"]      = self._host.text()
-            data["port"]      = self._port.value()
-            data["diff_mode"] = self._diff_cb.isChecked()
+            data["host"] = self._host.text()
+            data["port"] = self._port.value()
             _SETTINGS_PATH.write_text(json.dumps(data, indent=2))
         except OSError:
             pass
@@ -270,8 +275,6 @@ class ConnectorTab(QWidget):
                 self._host.setText(data["host"])
             if "port" in data:
                 self._port.setValue(int(data["port"]))
-            if "diff_mode" in data:
-                self._diff_cb.setChecked(bool(data["diff_mode"]))
         except (OSError, json.JSONDecodeError, KeyError, ValueError):
             pass
 
@@ -307,19 +310,25 @@ class DashboardTab(QWidget):
         layout.addWidget(self._ts)
 
         self._tree = QTreeWidget()
-        self._tree.setColumnCount(3)
-        self._tree.setHeaderLabels(["Field", "Value", "Unit"])
+        self._tree.setColumnCount(4)
+        self._tree.setHeaderLabels(["Field", "Value", "Unit", "Human Readable State"])
         self._tree.setColumnWidth(0, 320)
-        self._tree.setColumnWidth(1, 180)
-        self._tree.setColumnWidth(2, 80)
+        self._tree.setColumnWidth(1, 120)
+        self._tree.setColumnWidth(2, 60)
+        self._tree.setColumnWidth(3, 200)
         self._tree.setAlternatingRowColors(True)
         self._tree.setRootIsDecorated(True)
         layout.addWidget(self._tree)
+        self._prev_flat: dict = {}
 
     def update(self, snapshot: dict):
         self._ts.setText(f"Last update:  {snapshot.get('ts', '—')}")
 
-        expanded = self._expanded_paths()
+        curr_flat = _flatten_values(snapshot)
+        changed = {p for p, v in curr_flat.items() if v != self._prev_flat.get(p)}
+        self._prev_flat = curr_flat
+
+        collapsed = self._collapsed_paths()
         self._tree.clear()
 
         bold = QFont()
@@ -333,63 +342,78 @@ class DashboardTab(QWidget):
             d_item.setFont(0, bold)
             d_item.setForeground(0, domain_color)
             for obj_name, fields in domain_data.items():
+                path = f"{domain}/{obj_name}"
                 if _is_phys(fields):
-                    item = QTreeWidgetItem([obj_name, str(fields["value"]), fields["unit"]])
+                    item = QTreeWidgetItem([obj_name, str(fields["value"]), fields["unit"], fields.get("str_state", "")])
                     item.setTextAlignment(1, Qt.AlignRight | Qt.AlignVCenter)
+                    if path in changed:
+                        for col in range(4):
+                            item.setBackground(col, _CHANGED_BG)
                     d_item.addChild(item)
                 elif isinstance(fields, dict):
                     obj_item = QTreeWidgetItem([obj_name])
                     obj_item.setFont(0, bold)
-                    self._add_fields(obj_item, fields, f"{domain}/{obj_name}")
+                    self._add_fields(obj_item, fields, path, changed)
                     d_item.addChild(obj_item)
+                else:
+                    item = QTreeWidgetItem([obj_name, str(fields), "", ""])
+                    item.setTextAlignment(1, Qt.AlignRight | Qt.AlignVCenter)
+                    if path in changed:
+                        for col in range(4):
+                            item.setBackground(col, _CHANGED_BG)
+                    d_item.addChild(item)
             self._tree.addTopLevelItem(d_item)
 
         self._tree.expandAll()
-        self._restore_expanded(expanded)
+        self._apply_collapsed(collapsed)
 
-    def _add_fields(self, parent: QTreeWidgetItem, data: dict, path: str):
+    def _add_fields(self, parent: QTreeWidgetItem, data: dict, path: str, changed: set):
         for k, v in data.items():
             child_path = f"{path}/{k}"
             if _is_phys(v):
-                item = QTreeWidgetItem([k, str(v["value"]), v["unit"]])
+                item = QTreeWidgetItem([k, str(v["value"]), v["unit"], v.get("str_state", "")])
                 item.setTextAlignment(1, Qt.AlignRight | Qt.AlignVCenter)
+                if child_path in changed:
+                    for col in range(4):
+                        item.setBackground(col, _CHANGED_BG)
                 parent.addChild(item)
             elif isinstance(v, dict):
                 node = QTreeWidgetItem([k])
-                self._add_fields(node, v, child_path)
+                self._add_fields(node, v, child_path, changed)
                 parent.addChild(node)
             else:
-                item = QTreeWidgetItem([k, str(v), ""])
+                item = QTreeWidgetItem([k, str(v), "", ""])
                 item.setTextAlignment(1, Qt.AlignRight | Qt.AlignVCenter)
+                if child_path in changed:
+                    for col in range(4):
+                        item.setBackground(col, _CHANGED_BG)
                 parent.addChild(item)
 
-    # ── preserve expand state across refreshes ──────────────────────────────
+    # ── preserve collapse state across refreshes ─────────────────────────────
 
-    def _expanded_paths(self) -> set[str]:
+    def _collapsed_paths(self) -> set[str]:
         paths = set()
-        root = self._tree.invisibleRootItem()
-        self._collect_expanded(root, "", paths)
+        self._collect_collapsed(self._tree.invisibleRootItem(), "", paths)
         return paths
 
-    def _collect_expanded(self, item, prefix, paths):
+    def _collect_collapsed(self, item, prefix, paths):
         for i in range(item.childCount()):
             child = item.child(i)
             path = f"{prefix}/{child.text(0)}"
-            if child.isExpanded():
+            if child.childCount() > 0 and not child.isExpanded():
                 paths.add(path)
-            self._collect_expanded(child, path, paths)
+            self._collect_collapsed(child, path, paths)
 
-    def _restore_expanded(self, paths: set[str]):
-        root = self._tree.invisibleRootItem()
-        self._apply_expanded(root, "", paths)
+    def _apply_collapsed(self, paths: set[str]):
+        self._set_collapsed(self._tree.invisibleRootItem(), "", paths)
 
-    def _apply_expanded(self, item, prefix, paths):
+    def _set_collapsed(self, item, prefix, paths):
         for i in range(item.childCount()):
             child = item.child(i)
             path = f"{prefix}/{child.text(0)}"
             if path in paths:
-                child.setExpanded(True)
-            self._apply_expanded(child, path, paths)
+                child.setExpanded(False)
+            self._set_collapsed(child, path, paths)
 
 
 # ── Plot tab ──────────────────────────────────────────────────────────────────
@@ -840,7 +864,7 @@ class MainWindow(QMainWindow):
 
     def _connect(self, host: str, port: int):
         self._plot.clear_data()
-        err = self._reader.connect_to(host, port, self._connector.diff_mode)
+        err = self._reader.connect_to(host, port)
         if err:
             self._connector.set_disconnected(err)
             self._statusbar.showMessage(f"Connection failed: {err}")

client/requirements.txt

@@ -1,6 +1,3 @@
-carconnectivity
-carconnectivity-connector-volkswagen
-jaydiff @ git+http://192.168.22.90:3001/jayfield/jaypy.git
 PyQt5
 pyqtgraph
-pytest
+jaydiff @ git+http://192.168.22.90:3001/jayfield/jaypy.git

install_client.sh

@@ -0,0 +1,82 @@
+#!/usr/bin/env bash
+# CLIENT installer — sets up the we_monitor GUI client.
+# Does NOT install the server (see install_server.sh).
+#
+# Fresh install:
+#   chmod +x install_client.sh
+#   ./install_client.sh
+#
+# Update (code already pulled via git):
+#   ./install_client.sh
+
+set -euo pipefail
+
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+REPO_DIR="$SCRIPT_DIR"
+
+VENV="$REPO_DIR/.venv"
+PYTHON="$VENV/bin/python"
+
+IS_UPDATE=false
+if [[ -x "$PYTHON" ]]; then
+    IS_UPDATE=true
+fi
+
+if $IS_UPDATE; then
+    echo "==> we_monitor GUI updater"
+else
+    echo "==> we_monitor GUI installer"
+fi
+echo "    Project : $REPO_DIR"
+echo "    User    : $USER"
+echo ""
+
+# ── 1. virtual environment ────────────────────────────────────────────────────
+if [[ ! -x "$PYTHON" ]]; then
+    echo "==> Creating virtual environment : $VENV"
+    python3 -m venv "$VENV"
+else
+    echo "==> Virtual environment exists   : $VENV"
+fi
+
+echo "==> Installing / updating dependencies"
+"$VENV/bin/pip" install --quiet --upgrade pip
+"$VENV/bin/pip" install --quiet --upgrade -r "$REPO_DIR/client/requirements.txt"
+echo "    Python  : $PYTHON"
+echo ""
+
+# ── 2. launcher script ────────────────────────────────────────────────────────
+LAUNCHER="$REPO_DIR/client/run.sh"
+cat > "$LAUNCHER" <<EOF
+#!/usr/bin/env bash
+exec "$PYTHON" "$REPO_DIR/client/gui_client.py" "\$@"
+EOF
+chmod +x "$LAUNCHER"
+echo "==> Launcher written             : $LAUNCHER"
+
+# ── 3. system command ─────────────────────────────────────────────────────────
+BIN_DIR="$HOME/.local/bin"
+CMD="$BIN_DIR/we_monitor_gui"
+mkdir -p "$BIN_DIR"
+cat > "$CMD" <<EOF
+#!/usr/bin/env bash
+exec "$PYTHON" "$REPO_DIR/client/gui_client.py" "\$@"
+EOF
+chmod +x "$CMD"
+echo "==> System command               : $CMD"
+if [[ ":$PATH:" != *":$BIN_DIR:"* ]]; then
+    echo "    Note: $BIN_DIR is not in your PATH."
+    echo "    Add to ~/.bashrc or ~/.profile:  export PATH=\"\$HOME/.local/bin:\$PATH\""
+fi
+
+echo ""
+echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
+if $IS_UPDATE; then
+    echo " Update complete."
+else
+    echo " Installation complete."
+    echo ""
+    echo " Start the GUI:"
+    echo "        we_monitor_gui"
+fi
+echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"

install_server.sh

@@ -1,5 +1,6 @@
 #!/usr/bin/env bash
-# Install or update we_monitor as a systemd user service.
+# SERVER installer — installs the we_monitor collector as a systemd user service.
+# Does NOT install the GUI client (see client/requirements.txt).
 #
 # Fresh install:
 #   chmod +x install.sh
@@ -14,14 +15,15 @@
 set -euo pipefail
 
 SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+REPO_DIR="$SCRIPT_DIR"
 SERVICE_NAME="we_monitor"
 CREDS_DIR="$HOME/.config/we_monitor"
 CREDS_FILE="$CREDS_DIR/credentials.json"
 SYSTEMD_DIR="$HOME/.config/systemd/user"
 SERVICE_FILE="$SYSTEMD_DIR/$SERVICE_NAME.service"
-LOG_DIR="$SCRIPT_DIR/logs"
+LOG_DIR="$REPO_DIR/logs"
 
-VENV="$SCRIPT_DIR/.venv"
+VENV="$REPO_DIR/.venv"
 PYTHON="$VENV/bin/python"
 
 # detect fresh install vs update
@@ -39,7 +41,7 @@ if $IS_UPDATE; then
 else
     echo "==> we_monitor installer"
 fi
-echo "    Project : $SCRIPT_DIR"
+echo "    Project : $REPO_DIR"
 echo "    User    : $USER"
 echo ""
 
@@ -53,7 +55,17 @@ fi
 
 echo "==> Installing / updating dependencies"
 "$VENV/bin/pip" install --quiet --upgrade pip
-"$VENV/bin/pip" install --quiet --upgrade -r "$SCRIPT_DIR/requirements.txt"
+"$VENV/bin/pip" install --quiet --upgrade -r "$REPO_DIR/server/requirements.txt"
+
+echo "==> Applying cariad-hybrid-auth-fix.patch to volkswagencarnet"
+SITE_PKG="$("$PYTHON" -c 'import sysconfig; print(sysconfig.get_path("purelib"))')"
+PATCH_FILE="$REPO_DIR/cariad-hybrid-auth-fix.patch"
+if patch -d "$SITE_PKG" -p1 -N --dry-run --silent < "$PATCH_FILE" 2>/dev/null; then
+    patch -d "$SITE_PKG" -p1 -N --silent < "$PATCH_FILE"
+    echo "    Patch applied"
+else
+    echo "    Patch already applied (skipping)"
+fi
 echo "    Python  : $PYTHON"
 echo ""
 
@@ -99,8 +111,8 @@ Wants=network-online.target
 
 [Service]
 Type=simple
-WorkingDirectory=$SCRIPT_DIR
-ExecStart=$PYTHON $SCRIPT_DIR/collect.py -c $CREDS_FILE
+WorkingDirectory=$REPO_DIR
+ExecStart=$PYTHON -m server.main -c $CREDS_FILE
 Restart=on-failure
 RestartSec=60
 StandardOutput=journal

server/collect.py

@@ -0,0 +1,127 @@
+import argparse
+import logging
+import sys
+import time
+from datetime import datetime, timezone
+from pathlib import Path
+
+from . import log_config, network, we_connect
+from jaydiff.diff import diff_full as jay_diff_full
+
+from .data_model import ALL_DOMAINS, apply_procedural, extract_all
+from .storage_helpers import auto_out_path, load_last_24h_records, load_store, save_store
+
+from volkswagencarnet.vw_exceptions import AuthenticationError
+
+log = logging.getLogger(__name__)
+
+
+def run(args: argparse.Namespace) -> None:
+    log_config.setup(args.verbose)
+
+    raw = [d.strip() for d in args.domains.split(",")]
+    domains = list(ALL_DOMAINS) if raw == ["all"] else raw
+    unknown = [d for d in domains if d not in ALL_DOMAINS]
+    if unknown:
+        log.error("Unknown domain(s): %s.  Available: %s", unknown, list(ALL_DOMAINS))
+        sys.exit(1)
+
+    if args.dry:
+        log.info("Dry-run mode — skipping volkswagencarnet login")
+        wc = None
+        vehicle = None
+        vin = args.vin or "UNKNOWN"
+    else:
+        log.info("Connecting via volkswagencarnet…")
+        wc = we_connect.connect(args.username, args.password)
+        vehicle, err = we_connect.select_vehicle(wc, args.vin)
+        if err:
+            log.error(err)
+            sys.exit(1)
+        vin = vehicle.vin
+
+    log.info("Vehicle: %s", vin)
+
+    logs_dir = Path(args.log_dir)
+    logs_dir.mkdir(parents=True, exist_ok=True)
+    out = auto_out_path(logs_dir, vin)
+
+    push_clients, push_lock, push_store_ref = network.start_push_server(args.host, args.port)
+    push_store_ref["preloaded"] = load_last_24h_records(logs_dir, vin)
+    for _r in push_store_ref["preloaded"]:
+        if _r is not None and "procedural" not in _r:
+            apply_procedural(_r)
+    if push_store_ref["preloaded"]:
+        log.info("Pre-loaded %d record(s) from the last 24 h", len(push_store_ref["preloaded"]))
+
+    current_day = datetime.now(timezone.utc).date()
+    store = load_store(out, vin, args.interval)
+    for _r in store["records"]:
+        if _r is not None and "procedural" not in _r:
+            apply_procedural(_r)
+    push_store_ref["current"] = store
+    last_broadcast: dict = {}
+
+    if args.dry:
+        log.info("Push server running on %s:%d — no live collection  (Ctrl-C to stop)", args.host, args.port)
+    else:
+        log.info(
+            "Collecting [%s] every %ds → %s  (Ctrl-C to stop)",
+            ", ".join(domains),
+            args.interval,
+            out,
+        )
+
+    try:
+        while True:
+            if not args.dry:
+                today = datetime.now(timezone.utc).date()
+                if today != current_day:
+                    out = auto_out_path(logs_dir, vin)
+                    store = load_store(out, vin, args.interval)
+                    push_store_ref["current"] = store
+                    current_day = today
+                    log.info("Midnight UTC rotation → %s", out)
+
+                try:
+                    ok = we_connect.update(wc)
+                    # Re-select vehicle after every update: if doLogin() was triggered
+                    # (rare token-expiry full re-login), the library rebuilds its vehicle
+                    # list with new objects, making our old reference permanently stale.
+                    vehicle, err = we_connect.select_vehicle(wc, args.vin)
+                    if err:
+                        log.warning("Vehicle re-selection after update failed: %s", err)
+                        continue
+                    log.debug(
+                        "update ok=%s  vehicle._states keys: %s",
+                        ok,
+                        list(vehicle.attrs.keys()),
+                    )
+                    snapshot = extract_all(vehicle)
+                    if snapshot is None:
+                        log.warning("extract_all returned None — skipping this cycle")
+                        continue
+                    snapshot = apply_procedural(snapshot)
+                    store["records"].append(snapshot)
+                    save_store(out, store, args.max_records)
+                    payload = jay_diff_full(last_broadcast, snapshot, combine_upd_add=True)
+                    last_broadcast = snapshot
+                    network.broadcast(payload, push_clients, push_lock)
+                    changed = [k for k in payload if k != "ts"]
+                    log.info(
+                        "Record #%d at %s  changed=%s",
+                        len(store["records"]),
+                        snapshot["ts"],
+                        changed or "none",
+                    )
+                except KeyboardInterrupt:
+                    raise
+                except AuthenticationError:
+                    log.warning("Authentication error — will retry at next interval")
+                except Exception:
+                    log.exception("Collection failed — will retry at next interval")
+
+            time.sleep(args.interval)
+
+    except KeyboardInterrupt:
+        log.info("Stopped by user.  %d records in %s", len(store["records"]), out)

server/data_model.py

@@ -1,69 +1,86 @@
 import logging
+import re
 from collections.abc import Callable
 from datetime import datetime, timezone
 
+from volkswagencarnet.vw_const import Paths
+from volkswagencarnet.vw_dashboard import Dashboard
+from volkswagencarnet.vw_utilities import find_path
+
 log = logging.getLogger(__name__)
 
 
 # ── value helpers ─────────────────────────────────────────────────────────────
 
-def _str(attr) -> str:
-    """Return the string value of a CarConnectivity attribute."""
-    return str(attr)
-
-
 def _phys(value, unit: str) -> dict:
     return {"value": value, "unit": unit}
 
 
-def _remaining_min(estimated_date_reached_attr) -> float | None:
-    """Compute remaining minutes from an estimated_date_reached DateAttribute."""
-    edr = estimated_date_reached_attr.value
-    if edr is None:
-        return None
-    return max(0.0, (edr - datetime.now(timezone.utc)).total_seconds() / 60)
-
-
 # ── domain extractors ─────────────────────────────────────────────────────────
 
 def extract_charging(vehicle) -> dict | None:
     result: dict = {}
 
     try:
-        ch = vehicle.charging
-        result["state"] = _str(ch.state)
-        result["type"] = _str(ch.type)
-        try:
-            result["power"] = _phys(ch.power.value, "kW")
-        except Exception:
-            pass
-        try:
-            rem = _remaining_min(ch.estimated_date_reached)
-            if rem is not None:
-                result["remain"] = _phys(round(rem), "min")
-        except Exception:
-            pass
-
+        result["state"] = vehicle.charging_state
     except Exception:
-        log.debug("vehicle.charging unavailable", exc_info=True)
+        log.debug("charging.state unavailable", exc_info=True)
 
     try:
-        cfg = vehicle.charging.settings
-        result["settings"] = {
-            "target_level": _phys(cfg.target_level.value, "%"),
-            "maximum_current": _phys(cfg.maximum_current.value, "A"),
-            "auto_unlock": _str(cfg.auto_unlock),
-        }
+        if vehicle.is_charger_type_supported:
+            result["type"] = vehicle.charger_type
     except Exception:
-        log.debug("vehicle.charging.settings unavailable", exc_info=True)
+        pass
 
     try:
-        conn = vehicle.charging.connector
-        result["plugStatus"] = {
-            "connection_state": _str(conn.connection_state),
-            "lock_state": _str(conn.lock_state),
-            "external_power": _str(conn.external_power),
-        }
+        power = vehicle.charging_power
+        if power is not None:
+            result["power"] = _phys(power, "kW")
+    except Exception:
+        log.debug("charging.power unavailable", exc_info=True)
+
+    try:
+        remain = vehicle.charging_time_left
+        if remain is not None:
+            result["remain"] = _phys(remain, "min")
+    except Exception:
+        log.debug("charging.remain unavailable", exc_info=True)
+
+    settings: dict = {}
+    try:
+        target = vehicle.battery_target_charge_level
+        if target is not None:
+            settings["target_level"] = _phys(target, "%")
+    except Exception:
+        pass
+    try:
+        max_a = vehicle.charge_max_ac_ampere
+        if max_a is not None:
+            settings["maximum_current"] = _phys(max_a, "A")
+    except Exception:
+        pass
+    try:
+        auto_unlock = vehicle.auto_release_ac_connector_state
+        if auto_unlock is not None:
+            settings["auto_unlock"] = auto_unlock
+    except Exception:
+        pass
+    if settings:
+        result["settings"] = settings
+
+    try:
+        plug: dict = {}
+        conn_state = find_path(vehicle.attrs, Paths.PLUG_CONN)
+        if conn_state is not None:
+            plug["connection_state"] = conn_state
+        lock_state = find_path(vehicle.attrs, Paths.PLUG_LOCK)
+        if lock_state is not None:
+            plug["lock_state"] = lock_state
+        ext_pwr = find_path(vehicle.attrs, Paths.PLUG_EXT_PWR)
+        if ext_pwr is not None:
+            plug["external_power"] = ext_pwr
+        if plug:
+            result["plugStatus"] = plug
     except Exception:
         log.debug("plugStatus unavailable", exc_info=True)
 
@@ -74,40 +91,39 @@ def extract_climatisation(vehicle) -> dict | None:
     result: dict = {}
 
     try:
-        cl = vehicle.climatization
-        result["state"] = _str(cl.state)
-        try:
-            rem = _remaining_min(cl.estimated_date_reached)
-            if rem is not None:
-                result["remain"] = _phys(round(rem), "min")
-        except Exception:
-            pass
+        state = vehicle.climatisation_state
+        if state is not None:
+            result["state"] = state
     except Exception:
-        log.debug("vehicle.climatization unavailable", exc_info=True)
+        log.debug("climatisation.state unavailable", exc_info=True)
 
     try:
-        cs = vehicle.climatization.settings
-        result["settings"] = {
-            "target_temperature": _phys(cs.target_temperature.value, "degC"),
-        }
+        remain = vehicle.electric_remaining_climatisation_time
+        if remain is not None:
+            result["remain"] = _phys(remain, "min")
     except Exception:
-        log.debug("climatization.settings unavailable", exc_info=True)
+        pass
 
     try:
-        wh = vehicle.window_heatings
-        result["window_heatings"] = {
-            name: _str(win.heating_state)
-            for name, win in wh.heatings.items()
-        }
+        target = vehicle.climatisation_target_temperature
+        if target is not None:
+            result["settings"] = {"target_temperature": _phys(target, "degC")}
     except Exception:
-        log.debug("vehicle.window_heatings unavailable", exc_info=True)
+        log.debug("climatisation.settings unavailable", exc_info=True)
 
     try:
-        result["environment"] = {
-            "temperature_outside": _phys(vehicle.outside_temperature.value, "degC"),
-        }
+        window_status = find_path(vehicle.attrs, Paths.CLIMATISATION_WINDOW_HEATING_STATUS)
+        if window_status:
+            heatings: dict = {}
+            for entry in window_status:
+                loc = entry.get("windowLocation")
+                state = entry.get("windowHeatingState")
+                if loc and state is not None:
+                    heatings[loc] = state
+            if heatings:
+                result["window_heatings"] = heatings
     except Exception:
-        log.debug("vehicle.outside_temperature unavailable", exc_info=True)
+        log.debug("window_heatings unavailable", exc_info=True)
 
     return result or None
 
@@ -116,23 +132,45 @@ def extract_electric_drive(vehicle) -> dict | None:
     result: dict = {}
 
     try:
-        ed = vehicle.get_electric_drive()
-        result["range"] = _phys(ed.range.value, "km")
-        result["range_full"] = _phys(round(float(ed.range_estimated_full.value), 1), "km")
-        try:
-            bat = ed.battery
-            result["battery"] = {
-                "soc": _phys(ed.level.value, "%"),
-                "temperature_max": _phys(round(float(bat.temperature_max.value) - 273.15, 1), "degC"),
-                "temperature_min": _phys(round(float(bat.temperature_min.value) - 273.15, 1), "degC"),
-            }
-        except Exception:
-            log.debug("electric_drive/battery unavailable", exc_info=True)
+        rng = vehicle.electric_range
+        if rng is not None:
+            result["range"] = _phys(rng, "km")
     except Exception:
-        log.debug("electric_drive unavailable", exc_info=True)
+        log.debug("electric_range unavailable", exc_info=True)
 
     try:
-        result["odometer"] = {"odometer": _phys(vehicle.odometer.value, "km")}
+        rng_full = vehicle.battery_cruising_range
+        if rng_full is not None:
+            result["range_full"] = _phys(round(float(rng_full), 1), "km")
+    except Exception:
+        log.debug("battery_cruising_range unavailable", exc_info=True)
+
+    battery: dict = {}
+    try:
+        soc = vehicle.battery_level
+        if soc is not None:
+            battery["soc"] = _phys(soc, "%")
+    except Exception:
+        log.debug("battery.soc unavailable", exc_info=True)
+    try:
+        tmax = vehicle.hv_battery_max_temperature
+        if tmax is not None:
+            battery["temperature_max"] = _phys(round(tmax, 1), "degC")
+    except Exception:
+        pass
+    try:
+        tmin = vehicle.hv_battery_min_temperature
+        if tmin is not None:
+            battery["temperature_min"] = _phys(round(tmin, 1), "degC")
+    except Exception:
+        pass
+    if battery:
+        result["battery"] = battery
+
+    try:
+        odo = vehicle.distance
+        if odo is not None:
+            result["odometer"] = {"odometer": _phys(odo, "km")}
     except Exception:
         log.debug("odometer unavailable", exc_info=True)
 
@@ -143,32 +181,43 @@ def extract_connectivity(vehicle) -> dict | None:
     result: dict = {}
 
     try:
-        state = str(vehicle.connection_state)
-        result["state"] = state
+        is_online = vehicle.connection_state_is_online
+        is_active = vehicle.connection_state_is_active
+        if is_online:
+            result["state"] = "online"
+        elif is_active:
+            result["state"] = "active"
+        else:
+            result["state"] = "offline"
     except Exception:
         log.debug("connectivity.state unavailable", exc_info=True)
 
     return result or None
 
+
 def extract_vehicle(vehicle) -> dict | None:
     result: dict = {}
 
     try:
-        state = str(vehicle.state)
-        result["state"] = state
+        overall = find_path(vehicle.attrs, Paths.ACCESS_OVERALL_STATUS)
+        if overall is not None:
+            result["state"] = overall
     except Exception:
         log.debug("vehicle.state unavailable", exc_info=True)
 
     return result or None
 
+
 def extract_position(vehicle) -> dict | None:
     result: dict = {}
 
     try:
         pos = vehicle.position
-        result["lat"] = pos.latitude.value
-        result["lon"] = pos.longitude.value
-
+        lat = pos.get("lat")
+        lng = pos.get("lng")
+        if lat is not None and lat != "?" and lng is not None and lng != "?":
+            result["lat"] = lat
+            result["lon"] = lng  # volkswagencarnet uses "lng" internally; we keep "lon" for compat
     except Exception:
         log.debug("position unavailable", exc_info=True)
 
@@ -179,27 +228,130 @@ def extract_doors(vehicle) -> dict | None:
     result: dict = {}
 
     try:
-        doors = vehicle.doors
-        result["doors"] = {
-            "overallState": _str(doors.lock_state),
-            "doors": {
-                name: {
-                    "lockState": _str(door.lock_state),
-                    "openState": _str(door.open_state),
-                }
-                for name, door in doors.doors.items()
-            },
-            "windows": {
-                name: {"openState": _str(win.open_state)}
-                for name, win in vehicle.windows.windows.items()
-            },
-        }
+        lock_state = find_path(vehicle.attrs, Paths.ACCESS_DOOR_LOCK)
+        if lock_state is not None:
+            result["lock_state"] = lock_state
+
+        doors_list = find_path(vehicle.attrs, Paths.ACCESS_DOORS) or []
+        door_entries: dict = {}
+        all_closed: bool | None = None
+        for door in doors_list:
+            name = door.get("name")
+            status = door.get("status") or []
+            if not name:
+                continue
+            entry: dict = {}
+            if "closed" in status or "open" in status:
+                open_state = "closed" if "closed" in status else "open"
+                entry["open_state"] = open_state
+                all_closed = open_state == "closed" if all_closed is None else (all_closed and open_state == "closed")
+            if "locked" in status or "unlocked" in status:
+                entry["lock_state"] = "locked" if "locked" in status else "unlocked"
+            if entry:
+                door_entries[name] = entry
+        if all_closed is not None:
+            result["open_state"] = "closed" if all_closed else "open"
+        if door_entries:
+            result["doors"] = door_entries
     except Exception:
         log.debug("vehicle.doors unavailable", exc_info=True)
 
     return result or None
 
 
+def extract_windows(vehicle) -> dict | None:
+    result: dict = {}
+
+    try:
+        windows_list = find_path(vehicle.attrs, Paths.ACCESS_WINDOWS) or []
+        window_entries: dict = {}
+        all_closed: bool | None = None
+        for win in windows_list:
+            name = win.get("name")
+            status = win.get("status") or []
+            if not name or "unsupported" in status:
+                continue
+            if "closed" in status or "open" in status:
+                open_state = "closed" if "closed" in status else "open"
+                window_entries[name] = {"open_state": open_state}
+                all_closed = open_state == "closed" if all_closed is None else (all_closed and open_state == "closed")
+        if all_closed is not None:
+            result["open_state"] = "closed" if all_closed else "open"
+        if window_entries:
+            result["windows"] = window_entries
+    except Exception:
+        log.debug("vehicle.windows unavailable", exc_info=True)
+
+    return result or None
+
+
+_UNIT_WHITELIST: frozenset[str] = frozenset({"min", "%", "d", "kW", "°C", "km", "V", "W", "A"})
+_UNIT_CONVERSIONS: dict[str, str] = {"d": "days", "°C": "degC"}
+
+
+def _normalise_unit(unit: str) -> str:
+    if unit not in _UNIT_WHITELIST:
+        return ""
+    return _UNIT_CONVERSIONS.get(unit, unit)
+
+
+def _unit_from_str_state(str_state: str, state) -> str:
+    """Parse the unit suffix from a str_state like '42 %' or '21.5 degC'."""
+    state_repr = str(state)
+    if str_state.startswith(state_repr):
+        suffix = str_state[len(state_repr):].strip()
+        if suffix:
+            return suffix
+    # fallback: last whitespace-separated token if it looks like a unit
+    parts = str_state.rsplit(None, 1)
+    if len(parts) == 2 and re.fullmatch(r"[^\d\s]\S*", parts[1]):
+        return parts[1]
+    return ""
+
+
+def extract_all(vehicle) -> dict | None:
+    """Read every supported instrument via the Dashboard and return a flat snapshot.
+
+    Structure: snapshot[component][attr] = {"value": state, "unit": unit, "str_state": str_state}
+    Unit is taken from instrument.unit when available, otherwise parsed from str_state.
+    """
+    snapshot: dict = {"ts": datetime.now(timezone.utc).isoformat()}
+    try:
+        dashboard = Dashboard(vehicle)
+    except Exception:
+        log.debug("extract_all: Dashboard setup failed", exc_info=True)
+        return None
+
+    for instrument in dashboard.instruments:
+        try:
+            state = instrument.state
+            if state is None:
+                continue
+
+            try:
+                str_state = str(instrument.str_state)
+            except Exception:
+                str_state = str(state)
+
+            if hasattr(instrument, "unit") and instrument.unit:
+                raw_unit = instrument.unit
+            else:
+                raw_unit = _unit_from_str_state(str_state, state)
+            unit = _normalise_unit(raw_unit)
+
+            component = instrument.component
+            attr = instrument.attr
+            snapshot.setdefault(component, {})[attr] = {
+                "value": state,
+                "unit": unit,
+                "str_state": str_state,
+            }
+        except Exception:
+            log.debug("extract_all: %s unavailable", instrument.attr, exc_info=True)
+
+    return snapshot or None
+
+
 ALL_DOMAINS: dict[str, Callable] = {
     "charging": extract_charging,
     "climatisation": extract_climatisation,
@@ -208,6 +360,7 @@ ALL_DOMAINS: dict[str, Callable] = {
     "vehicle": extract_vehicle,
     "position": extract_position,
     "doors": extract_doors,
+    "windows": extract_windows,
 }
 
 
@@ -240,18 +393,17 @@ def apply_procedural(record: dict) -> dict:
     """
     proc: dict = {}
 
-    range_at_soc = _get(record, "electric_drive", "range", "value")
-    soc          = _get(record, "electric_drive", "battery", "soc", "value")
+    range_at_soc = _get(record, "sensor", "electric_range", "value")
+    soc          = _get(record, "sensor", "battery_level", "value")
     if range_at_soc is not None and soc:
         proc["range_at_100"] = _phys(round(100 * float(range_at_soc) / float(soc), 1), "km")
 
     # ── add computed fields here ──────────────────────────────────────────
-    # Use _get(record, "domain", "statusObject", "field", "value") to
-    # safely read any nested value.  Always use {"value": ..., "unit": ...}
-    # format so the GUI picks up the field automatically.
+    # Use _get(record, "sensor"|"binary_sensor"|..., "attr_name", "value")
+    # Always use {"value": ..., "unit": ...} format so the GUI picks it up.
     #
     # Example — usable energy estimated from SOC and battery capacity:
-    # soc = _get(record, "electric_drive", "battery", "soc", "value")
+    # soc = _get(record, "sensor", "battery_level", "value")
     # if soc is not None:
     #     proc["energy_stored"] = {"value": round(soc / 100 * 77.0, 1), "unit": "kWh"}
     # ─────────────────────────────────────────────────────────────────────

server/main.py

@@ -9,16 +9,16 @@ the domain hierarchy: domain → status-object → field.
 Usage examples
 --------------
 # Credentials file with all settings:
-  python collect.py -c credentials/alex.json
+  python -m server.main -c credentials/alex.json
 
 # Override username/password via env vars:
   export WC_USER=me@example.com WC_PASS=secret
-  python collect.py -c credentials/alex.json
+  python -m server.main -c credentials/alex.json
 
 # Override domains on the CLI:
-  python collect.py -c credentials/alex.json -d charging,measurements
+  python -m server.main -c credentials/alex.json -d charging,measurements
 
-Available domains: charging, climatisation, electric_drive, connectivity, vehicle, position, doors
+Available domains: charging, climatisation, electric_drive, connectivity, vehicle, position, doors, windows
 
 Credentials file format (JSON)
 -------------------------------
@@ -45,119 +45,13 @@ Connect with:  nc localhost 9999  or any TCP client that reads lines.
 
 import argparse
 import json
-import logging
 import os
 import sys
-import time
-from datetime import datetime, timezone
 from pathlib import Path
 
-from server import log_config, network, we_connect
-from carconnectivity.errors import AuthenticationError, TemporaryAuthenticationError
-from server.data_model import ALL_DOMAINS, collect_snapshot, apply_procedural
-from server.storage_helpers import auto_out_path, load_last_24h_records, load_store, save_store
-from jaydiff.diff import diff_full as jay_diff_full
+from server.collect import run
+from server.data_model import ALL_DOMAINS
 
-log = logging.getLogger(__name__)
-
-
-# ── main loop ────────────────────────────────────────────────────────────────
-
-def run(args: argparse.Namespace) -> None:
-    log_config.setup(args.verbose)
-
-    raw = [d.strip() for d in args.domains.split(",")]
-    domains = list(ALL_DOMAINS) if raw == ["all"] else raw
-    unknown = [d for d in domains if d not in ALL_DOMAINS]
-    if unknown:
-        log.error("Unknown domain(s): %s.  Available: %s", unknown, list(ALL_DOMAINS))
-        sys.exit(1)
-
-    if args.dry:
-        log.info("Dry-run mode — skipping CarConnectivity login")
-        wc = None
-        vehicle = None
-        vin = args.vin or "UNKNOWN"
-    else:
-        log.info("Connecting via CarConnectivity…")
-        wc = we_connect.connect(args.username, args.password)
-        vehicle, err = we_connect.select_vehicle(wc, args.vin)
-        if err:
-            log.error(err)
-            sys.exit(1)
-        vin = vehicle.vin.value
-
-    log.info("Vehicle: %s", vin)
-
-    logs_dir = Path(args.log_dir)
-    logs_dir.mkdir(parents=True, exist_ok=True)
-    out = auto_out_path(logs_dir, vin)
-
-    push_clients, push_lock, push_store_ref = network.start_push_server(args.host, args.port)
-    push_store_ref["diff_mode"] = args.diff
-    push_store_ref["preloaded"] = load_last_24h_records(logs_dir, vin)
-    for _r in push_store_ref["preloaded"]:
-        if "procedural" not in _r:
-            apply_procedural(_r)
-    if push_store_ref["preloaded"]:
-        log.info("Pre-loaded %d record(s) from the last 24 h", len(push_store_ref["preloaded"]))
-
-    current_day = datetime.now(timezone.utc).date()
-    store = load_store(out, vin, args.interval)
-    for _r in store["records"]:
-        if "procedural" not in _r:
-            apply_procedural(_r)
-    push_store_ref["current"] = store
-    last_broadcast: dict = {}
-
-    if args.dry:
-        log.info("Push server running on %s:%d — no live collection  (Ctrl-C to stop)", args.host, args.port)
-    else:
-        log.info(
-            "Collecting [%s] every %ds → %s  (Ctrl-C to stop)",
-            ", ".join(domains),
-            args.interval,
-            out,
-        )
-
-    try:
-        while True:
-            if not args.dry:
-                today = datetime.now(timezone.utc).date()
-                if today != current_day:
-                    out = auto_out_path(logs_dir, vin)
-                    store = load_store(out, vin, args.interval)
-                    push_store_ref["current"] = store
-                    current_day = today
-                    log.info("Midnight UTC rotation → %s", out)
-
-                try:
-                    we_connect.update(wc)
-                    snapshot = collect_snapshot(vehicle, domains)
-                    apply_procedural(snapshot)
-                    store["records"].append(snapshot)
-                    save_store(out, store, args.max_records)
-                    if args.diff and last_broadcast:
-                        payload = jay_diff_full(last_broadcast, snapshot, combine_upd_add=True)
-                    else:
-                        payload = snapshot
-                    network.broadcast(payload, push_clients, push_lock)
-                    last_broadcast = snapshot
-                    log.info("Record #%d saved at %s", len(store["records"]), snapshot["ts"])
-                except KeyboardInterrupt:
-                    raise
-                except (AuthenticationError, TemporaryAuthenticationError):
-                    log.warning("Authentication error — will retry at next interval")
-                except Exception:
-                    log.exception("Collection failed — will retry at next interval")
-
-            time.sleep(args.interval)
-
-    except KeyboardInterrupt:
-        log.info("Stopped by user.  %d records in %s", len(store["records"]), out)
-
-
-# ── CLI ──────────────────────────────────────────────────────────────────────
 
 def main() -> None:
     p = argparse.ArgumentParser(
@@ -226,12 +120,6 @@ def main() -> None:
         help="Directory for rotating log files (overrides credentials file, default: ./logs)",
     )
     p.add_argument("-v", "--verbose", action="store_true", help="Enable debug logging")
-    p.add_argument(
-        "--diff",
-        action="store_true",
-        default=None,
-        help="Send diffs instead of full snapshots over the push server (overrides credentials file, default: off)",
-    )
     p.add_argument(
         "--dry",
         action="store_true",
@@ -259,7 +147,6 @@ def main() -> None:
         except json.JSONDecodeError as exc:
             p.error(f"Invalid JSON in credentials file: {exc}")
 
-    # username / password: env / CLI flag > credentials.username > credentials file root
     nested = creds_data.get("credentials", {})
     if args.username is None:
         args.username = nested.get("username") or creds_data.get("username")
@@ -283,9 +170,6 @@ def main() -> None:
         args.port = int(creds_data.get("port", 9999))
     if args.log_dir is None:
         args.log_dir = creds_data.get("log_dir", "./logs")
-    if args.diff is None:
-        args.diff = bool(creds_data.get("diff", False))
-
     if not args.dry and (not args.username or not args.password):
         p.error(
             "Username and password are required.  "

server/network.py

@@ -24,28 +24,30 @@ def _accept_loop(
                 preloaded = store_ref.get("preloaded", [])
 
                 if preloaded:
-                    seen_ts = {r.get("ts") for r in preloaded}
-                    extra   = [r for r in in_mem if r.get("ts") not in seen_ts]
-                    records = sorted(preloaded + extra, key=lambda r: r.get("ts", ""))
+                    seen_ts = {r.get("ts") for r in preloaded if r is not None}
+                    extra   = [r for r in in_mem if r is not None and r.get("ts") not in seen_ts]
+                    records = sorted(preloaded + extra, key=lambda r: r.get("ts", "") if r is not None else "")
                 else:
-                    records = in_mem
+                    records = [r for r in in_mem if r is not None]
 
                 try:
-                    if store_ref.get("diff_mode"):
-                        prev: dict = {}
-                        for record in records:
-                            diff = jay_diff_full(prev, record, combine_upd_add=True)
-                            prev = record
-                            conn.sendall((json.dumps(diff, default=str) + "\n").encode())
-                    else:
-                        for record in records:
-                            conn.sendall((json.dumps(record, default=str) + "\n").encode())
+                    prev: dict = {}
+                    for record in records:
+                        if record is None:
+                            continue
+                        diff = jay_diff_full(prev, record, combine_upd_add=True)
+                        prev = record
+                        conn.sendall((json.dumps(diff, default=str) + "\n").encode())
                     if records:
                         log.info("Sent %d record(s) to %s", len(records), addr)
                 except OSError:
                     log.debug("Failed to send history to %s", addr)
                     conn.close()
                     continue
+                except Exception:
+                    log.exception("Error sending history to %s — dropping client", addr)
+                    conn.close()
+                    continue
                 clients.append(conn)
         except OSError:
             break
@@ -73,7 +75,7 @@ def start_push_server(host: str, port: int) -> tuple:
     server_sock.listen()
     clients: list  = []
     lock           = threading.Lock()
-    store_ref: dict = {"current": None, "preloaded": [], "diff_mode": False}
+    store_ref: dict = {"current": None, "preloaded": []}
     t = threading.Thread(
         target=_accept_loop,
         args=(server_sock, clients, lock, store_ref),

server/requirements.txt

@@ -0,0 +1,2 @@
+volkswagencarnet @ git+https://github.com/robinostlund/volkswagencarnet.git@c30dc37
+jaydiff @ git+http://192.168.22.90:3001/jayfield/jaypy.git

server/storage_helpers.py

@@ -53,11 +53,11 @@ def load_last_24h_records(logs_dir: Path, vin: str) -> list:
             try:
                 data = json.loads(path.read_text())
                 if isinstance(data, dict) and "records" in data:
-                    kept = [r for r in data["records"] if r.get("ts", "") >= cutoff_str]
+                    kept = [r for r in data["records"] if r is not None and r.get("ts", "") >= cutoff_str]
                     records.extend(kept)
                     log.debug("Loaded %d/%d records from %s", len(kept), len(data["records"]), path.name)
             except (json.JSONDecodeError, OSError):
                 log.warning("Could not load %s", path)
 
-    records.sort(key=lambda r: r.get("ts", ""))
+    records.sort(key=lambda r: r.get("ts", "") if r is not None else "")
     return records

server/we_connect.py

@@ -1,52 +1,70 @@
+import asyncio
 import logging
-import sys
 
-try:
-    from carconnectivity.carconnectivity import CarConnectivity
-except ImportError:
-    print(
-        "carconnectivity is not installed.  Run:  "
-        "pip install carconnectivity carconnectivity-connector-volkswagen",
-        file=sys.stderr,
-    )
-    sys.exit(1)
+import aiohttp
+from volkswagencarnet.vw_connection import Connection
+from volkswagencarnet.vw_exceptions import AuthenticationError  # noqa: F401 — re-exported for collect.py
 
 log = logging.getLogger(__name__)
 
 
-def connect(username: str, password: str) -> CarConnectivity:
-    config = {
-        "carConnectivity": {
-            "connectors": [
-                {
-                    "type": "volkswagen",
-                    "config": {
-                        "username": username,
-                        "password": password,
-                    },
-                }
-            ]
-        }
-    }
-    cc = CarConnectivity(config=config)
-    cc.startup()
-    cc.fetch_all()
-    return cc
+class _Session:
+    """Holds the asyncio event loop, aiohttp session, and Connection together."""
+
+    def __init__(
+        self,
+        loop: asyncio.AbstractEventLoop,
+        session: aiohttp.ClientSession,
+        connection: Connection,
+    ) -> None:
+        self._loop = loop
+        self._http_session = session
+        self.connection = connection
+
+    def run(self, coro):
+        return self._loop.run_until_complete(coro)
+
+    @property
+    def vehicles(self):
+        return self.connection.vehicles
 
 
-def select_vehicle(cc: CarConnectivity, vin: str | None):
+def connect(username: str, password: str) -> _Session:
+    loop = asyncio.new_event_loop()
+    asyncio.set_event_loop(loop)
+
+    async def _do():
+        session = aiohttp.ClientSession(headers={"Connection": "keep-alive"})
+        conn = Connection(session, username, password)
+        ok = await conn.doLogin()
+        if not ok:
+            await session.close()
+            raise AuthenticationError("Login failed")
+        return session, conn
+
+    session, conn = loop.run_until_complete(_do())
+    return _Session(loop, session, conn)
+
+
+def select_vehicle(wc: _Session, vin: str | None):
     """Return (vehicle, error_message). error_message is None on success."""
-    vehicles = list(cc.get_garage().list_vehicles())
+    vehicles = wc.vehicles
     if not vehicles:
-        return None, "No vehicles found in this CarConnectivity account"
+        return None, "No vehicles found in this account"
     if vin:
-        vehicle = next((v for v in vehicles if v.vin.value == vin), None)
+        vehicle = next((v for v in vehicles if v.vin == vin), None)
         if not vehicle:
-            available = [v.vin.value for v in vehicles]
-            return None, f"VIN {vin} not found.  Available: {available}"
+            available = [v.vin for v in vehicles]
+            return None, f"VIN {vin} not found. Available: {available}"
         return vehicle, None
     return vehicles[0], None
 
 
-def update(cc: CarConnectivity) -> None:
-    cc.fetch_all()
+def update(wc: _Session) -> bool:
+    """Run one update cycle. Returns True if the library reported success."""
+    ok = wc.run(wc.connection.update())
+    if not ok:
+        log.warning("connection.update() returned False — data may not have refreshed")
+    else:
+        log.debug("connection.update() succeeded")
+    return bool(ok)