we_monitor

Periodically polls a VW vehicle via the carconnectivity library, stores timestamped records as JSON, and pushes live data to connected TCP clients. A PyQt5 GUI client visualises the data in real time.

Repository layout

collect.py              — collector daemon (entry point)
gui_client.py           — PyQt5 GUI client
client.py               — minimal CLI test client
server/
  data_model.py         — domain extractors, collect_snapshot()
  network.py            — TCP push server (accept loop + broadcast)
  storage_helpers.py    — load / save JSON store, midnight rotation helpers
  we_connect.py         — CarConnectivity login and vehicle selection
  log_config.py         — logging setup
utils/
  jay_diff.py           — JSON diff / merge library
credentials/            — JSON config files (gitignored)
requirements.txt

Requirements

pip install carconnectivity carconnectivity-connector-volkswagen PyQt5 pyqtgraph

Or use the installer which also sets up a systemd user service:

./install.sh

---

Collector (collect.py)

Connects via CarConnectivity, polls selected domains at a configurable interval, and appends timestamped records to a rotating JSON log. Simultaneously runs a TCP push server so clients receive every new snapshot the moment it is collected.

Quick start

python collect.py -c credentials/my_car.json

Username and password can also be supplied via environment variables (override the credentials file):

export WC_USER=me@example.com
export WC_PASS=secret
python collect.py -c credentials/my_car.json

Credentials file

All connection and collection settings live in a single JSON file:

{
  "credentials": {
    "username": "me@example.com",
    "password": "secret"
  },
  "vin":        "WVWZZZE1ZMP000000",
  "domains":    ["charging", "electric_drive", "connectivity"],
  "interval_s": 300,
  "host":       "0.0.0.0",
  "port":       9999,
  "log_dir":    "./logs",
  "diff":       false
}

Key	Default	Description
credentials.username	—	MyVolkswagen email
credentials.password	—	MyVolkswagen password
vin	first vehicle	Target VIN
domains	["charging","electric_drive","connectivity"]	Domains to collect (list or "all")
interval_s	300	Collection interval in seconds
host	"0.0.0.0"	Push server bind address
port	9999	Push server TCP port
log_dir	"./logs"	Directory for rotating JSON log files
diff	false	Enable diff-mode compression on the push server

CLI reference

CLI flags override the corresponding credentials-file value when given.

Flag	Overrides	Description
-c FILE	—	Credentials / settings file
-u / -p	credentials.username/password	Username / password (also WC_USER / WC_PASS)
-d DOMAINS	domains	Domains to collect (comma-separated or all)
-i SECONDS	interval_s	Collection interval in seconds
--host HOST	host	Push server bind address
--port PORT	port	Push server TCP port
--log-dir DIR	log_dir	Directory for rotating log files
--max-records N	—	Keep only the last N records per file (default: unlimited)
--diff	diff	Enable diff-mode on the push server
--dry	—	Skip CarConnectivity login; run push server only (useful for GUI testing)
-v	—	Verbose / debug logging
--list-domains	—	Print available domains and exit

Available domains: charging, climatisation, electric_drive, connectivity, vehicle, position, doors

Log rotation

A new file is started automatically after midnight UTC; previous files are kept intact. On startup and on each new client connection the collector sends the last 24 hours of records, so clients always receive a full rolling window regardless of when midnight falls.

Output format

Log files are named YYYY_MM_DD_HH_MM_SS_<VIN>.json and contain:

{
  "meta": { "vin": "...", "created_at": "...", "interval_seconds": 300 },
  "records": [
    {
      "ts": "2025-05-25T10:00:00+00:00",
      "charging": {
        "state": "readyForCharging",
        "power": { "value": 11.0, "unit": "kW" }
      },
      "electric_drive": {
        "range":   { "value": 310, "unit": "km" },
        "battery": { "soc": { "value": 80, "unit": "%" } }
      }
    }
  ]
}

Physical values are always {"value": …, "unit": …} objects.

---

Push server protocol

The server listens on the host:port from the credentials file (default 0.0.0.0:9999).

• On connect: sends all records from the last 24 hours, sorted by timestamp.
• On each new poll: sends the new snapshot immediately.

All messages are newline-delimited JSON (one record per line, \n terminated).

Diff mode (--diff)

When diff mode is enabled the history burst and every live snapshot are compressed using jay_diff_full before transmission. The wire format is self-describing — diffs carry update_add / delete keys, full snapshots do not — so no separate framing is needed.

The first message sent to a new client is always jay_diff_full({}, record[0]), i.e. an update_add containing every field of the first record. Subsequent messages contain only the fields that changed.

Enable --diff on the server and on the client (client.py --diff or the Diff mode checkbox in the GUI Connector tab). Both sides must agree — a mismatch produces garbled output.

---

GUI client (gui_client.py)

python gui_client.py

Dark-theme PyQt5 application. Three tabs: Connector, Dashboard, Plot. Connection settings, trace selections, and time windows are persisted to ~/.config/we_monitor/plot_settings.json and restored on the next launch.

Connector tab

Control	Description
Host / Port	Address of the collect.py push server
Diff mode checkbox	Must match the server's --diff setting
Connect / Disconnect	Opens or closes the TCP connection
Status	Shows Connected, Disconnected, or the error message

Clicking Connect saves the current host, port, and diff-mode setting. All plot data is cleared on each connect so the fresh server history is never mixed with stale local data.

Dashboard tab

Live tree view of the most recent snapshot. The hierarchy mirrors the JSON structure: domain → status object → field, with value and unit shown inline. Expand / collapse state is preserved across updates.

Plot tab

Eight plots arranged in a 4 × 2 scrollable grid. Each plot is independent and has its own control bar.

Control bar (per plot)

Control	Description
Signal dropdown	Lists every numeric {value, unit} field received so far
+ button	Adds the selected signal as a new trace (max 4 per plot)
Colored chips (name ×)	One chip per active trace; click × to remove
Time window spinner	Show the last 1–24 hours of data (per plot)
Date label (right-aligned)	Shows the calendar date(s) of the visible window — single date or YYYY-MM-DD – YYYY-MM-DD when the view spans midnight

Plot area

• X axis — time of day in HH:MM:SS format; no locale-dependent date prefix.
• Y axis — auto-zooms to fit the visible data on the first draw after connect or after a trace is added / removed. Manual zoom is preserved between data updates.
• Crosshair — dashed vertical line tracks the mouse cursor.
• Floating tooltip — appears next to the cursor and shows the nearest sampled value for every active trace, with a colored bullet and unit.
• Legend — trace names and colors, shown inside the plot area.

Mouse interaction

Action	Effect
Scroll wheel	Zoom X axis in / out, centred on cursor
Shift + scroll wheel	Zoom Y axis in / out, centred on cursor
Click + drag	Pan the view
Right-click	pyqtgraph context menu (export, auto-range, …)

---

CLI test client (client.py)

python client.py --host 192.168.1.10 --port 9999
python client.py --diff   # reconstruct from diff stream

Connects to the push server and prints every incoming snapshot as pretty-printed JSON with a running record counter.

Flag	Description
--host HOST	Server host (default: 127.0.0.1)
--port PORT	Server port (default: 9999)
--diff	Reconstruct full snapshots from diff stream

---

Module reference

collect.py

Entry point for the collector daemon. Parses CLI / credentials-file arguments, establishes the CarConnectivity session, starts the push server, and runs the main poll loop.

server/data_model.py

collect_snapshot(vehicle, domains) → dict
Iterates over the requested domains on a CarConnectivity vehicle object and extracts fields into a flat dict keyed by domain. Physical values are normalised to {"value": …, "unit": …}. Returns a record with a top-level "ts" ISO-8601 timestamp.

ALL_DOMAINS
Tuple of all domain names the collector knows about.

server/network.py

start_push_server(host, port) → (clients, lock, store_ref)
Binds a TCP socket, starts the accept-loop thread (daemon), and returns the shared state needed by broadcast.

broadcast(snapshot, clients, lock)
Serialises snapshot to a JSON line and sends it to every connected client. Dead connections are removed silently.

store_ref dict keys:

Key	Type	Description
"current"	dict	The currently-open JSON store
"preloaded"	list	24-h history records loaded at startup
"diff_mode"	bool	Whether to compress the history burst as diffs

server/storage_helpers.py

load_store(path, vin, interval) → dict
Loads an existing store file or creates a fresh one.

save_store(path, store, max_records)
Writes the store to disk, optionally trimming to the last max_records entries.

auto_out_path(log_dir, vin) → Path
Returns a timestamped file path for a new store.

load_last_24h_records(log_dir, vin) → list
Scans the log directory for files belonging to vin, loads any records from the last 24 hours, and returns them sorted by timestamp.

server/we_connect.py

connect(username, password) → CarConnectivity
Authenticates and returns a connected CarConnectivity instance.

select_vehicle(wc, vin) → (vehicle, error)
Returns the target vehicle (first available if vin is empty) or an error string.

utils/jay_diff.py

Lightweight JSON diff / patch library.

jay_diff_full(old, new, combine_upd_add=True) → dict
Computes a diff between two dicts. With combine_upd_add=True (the default used everywhere in this project) the result uses update_add for all additions and updates, plus a separate delete key. Returns {} when nothing changed.

jay_merge_full(old, diff) → dict
Applies a jay_diff_full diff to old. The diff structure is self-describing: if diff contains none of the known diff keys (update, update_add, add, delete) and old is empty, diff is returned as-is (full-snapshot passthrough for bootstrap).

gui_client.py — class overview

Class	Role
TcpReader	Background thread; reads newline-delimited JSON from the socket, reconstructs full snapshots in diff mode via jay_merge_full, and emits each snapshot via the message Qt signal
ConnectorTab	Host / port / diff-mode form; emits connect_requested and disconnect_requested signals; persists settings
DashboardTab	Live QTreeWidget showing the latest snapshot; preserves expand state across updates
PlotTab	Eight independent time-series plots in a scrollable grid
TimeAxisItem	pg.AxisItem subclass; formats x-axis tick labels as HH:MM:SS
PlotViewBox	pg.ViewBox subclass; routes plain scroll to X zoom and Shift+scroll to Y zoom
PlotTooltip	Frameless child QWidget per plot; shows nearest sampled values for all active traces at the cursor position
MainWindow	QMainWindow; wires the three tabs together and owns the TcpReader