diff --git a/jaydiff/README.jaydiff.md b/jaydiff/README.jaydiff.md index 2976b5c..fc3e531 100644 --- a/jaydiff/README.jaydiff.md +++ b/jaydiff/README.jaydiff.md @@ -1,10 +1,8 @@ -# jay_diff — JSON diff / patch library +# jaypy/jaydiff — JSON diff / patch library -`utils/jay_diff.py` is a lightweight, pure-Python library for computing and -applying structural diffs between two JSON-compatible dicts. It is used by -`collect.py` and `gui_client.py` to compress the push-server stream: instead -of retransmitting full snapshots on every poll, the server sends only the -fields that changed since the previous snapshot. +`jaydiff` is a lightweight, pure-Python library for computing and +applying structural diffs between two JSON-compatible dicts. It is used to +compress the json data packets with redundant data, by providing a delta instead of the full packet --- @@ -24,7 +22,7 @@ state to a **new** state using one or more of these keys: everywhere in this project). The separate `update` / `add` form is available via `combine_upd_add=False` for completeness. -An **empty dict `{}`** returned by `jay_diff_full` means the two states are +An **empty dict `{}`** returned by `diff_full` means the two states are identical. ### Delete-delta encoding @@ -52,7 +50,7 @@ completely. ### Diff functions ```python -jay_diff_full(old, new, combine_upd_add=True) -> dict +diff_full(old, new, combine_upd_add=True) -> dict ``` The main entry point. Computes a complete diff from `old` to `new`. @@ -62,53 +60,53 @@ The main entry point. Computes a complete diff from `old` to `new`. Returns `{}` when `old == new`. ```python -jay_diff_add(old, new) -> dict +diff_add(old, new) -> dict ``` Returns keys present in `new` but absent from `old` (additions only). ```python -jay_diff_del(old, new) -> dict +diff_del(old, new) -> dict ``` Returns keys present in `old` but absent from `new`, or sub-keys that were removed from a shared dict. Uses `None` as sentinel for entirely-absent dict keys. ```python -jay_diff_upd(old, new) -> dict +diff_upd(old, new) -> dict ``` Returns keys present in both `old` and `new` whose values differ (updates only; new keys are ignored). ```python -jay_diff_upd_add(old, new) -> dict +diff_upd_add(old, new) -> dict ``` -Like `jay_diff_upd` but also includes keys new in `new` — equivalent to -`jay_diff_full(..., combine_upd_add=True)` minus the delete section. +Like `diff_upd` but also includes keys new in `new` — equivalent to +`diff_full(..., combine_upd_add=True)` minus the delete section. --- ### Merge functions ```python -jay_merge_full(old, diff) -> dict +merge_full(old, diff) -> dict ``` -The main entry point. Applies a diff produced by `jay_diff_full` to `old` +The main entry point. Applies a diff produced by `diff_full` to `old` and returns the reconstructed new state. **Bootstrap passthrough:** if `old` is `{}` and `diff` contains none of the recognised diff keys, `diff` is returned as-is. This is how the first message from the push server is handled: the server sends -`jay_diff_full({}, record[0])`, which looks like a plain snapshot, and the +`diff_full({}, record[0])`, which looks like a plain snapshot, and the client stores it directly. Applies operations in order: `update` → `update_add` → `add` → `delete`. ```python -jay_merge_update(old, new) -> dict # apply an update/update_add delta -jay_merge_add(old, new) -> dict # apply an add delta -jay_merge_delete(old, new) -> dict # apply a delete delta +merge_update(old, new) -> dict # apply an update/update_add delta +merge_add(old, new) -> dict # apply an add delta +merge_delete(old, new) -> dict # apply a delete delta ``` -Low-level helpers; prefer `jay_merge_full` unless you need fine-grained +Low-level helpers; prefer `merge_full` unless you need fine-grained control. --- @@ -116,27 +114,27 @@ control. ## Round-trip guarantee ``` -jay_merge_full(old, jay_diff_full(old, new)) == new +merge_full(old, diff_full(old, new)) == new ``` This holds for all inputs where values are JSON-compatible scalars (strings, numbers, booleans, `None`, lists) or nested dicts thereof. It is verified -by `tests/test_jay_diff_real_data.py` against 674 consecutive pairs of real +by `tests/test_diff_real_data.py` against 674 consecutive pairs of real vehicle snapshots. --- ## Wire format -`jay_diff_full` / `jay_merge_full` are used by the push server when +`diff_full` / `merge_full` are used by the push server when `--diff` is enabled. The diff dict is serialised as a newline-delimited JSON line. -- The **first** message to a new client is `jay_diff_full({}, record[0])`, +- The **first** message to a new client is `diff_full({}, record[0])`, which always contains `update_add` with every field of the first record. - Subsequent messages contain only the changed fields. - The format is self-describing: a message with none of the diff keys is - treated as a full snapshot by `jay_merge_full` (bootstrap passthrough). + treated as a full snapshot by `merge_full` (bootstrap passthrough). `null` in JSON round-trips to Python `None`, so the delete sentinel survives serialisation transparently. @@ -146,10 +144,10 @@ serialisation transparently. ## Tests ``` -tests/test_jay_diff.py — 104 unit tests (all diff/merge functions, +tests/test_diff_.py — 104 unit tests (all diff/merge functions, corner cases: type changes, empty dicts, deep nesting, round-trips) -tests/test_jay_diff_real_data.py — parametrised over 675 real VW snapshots +tests/test_diff_real_data.py — parametrised over 675 real VW snapshots in tests/data/ (674 consecutive pairs): round-trip, identical diff = {}, and empty-diff-is-noop checks @@ -165,23 +163,23 @@ pytest tests/ ## Bug history -### `jay_merge_delete`: crash on aliased dicts (fixed) +### `merge_delete`: crash on aliased dicts (fixed) -`jay_diff_rev` (used internally by the old `jay_diff_del`) stored references +`diff_rev` (used internally by the old `diff_del`) stored references to sub-dicts of `_old` directly in the delete delta without copying them. -When `jay_merge_full(old, jay_diff_full(old, new))` was called, some +When `merge_full(old, diff_full(old, new))` was called, some sub-dicts in the delete delta were the same Python objects as sub-dicts in `old`. Iterating `_new.items()` while deleting from `_old` (the same object) raised `RuntimeError: dictionary changed size during iteration`. **Fix:** `list(_new.items())` snapshots the items before the loop. Also, -`jay_diff_del` was rewritten as a standalone recursive function that never +`diff_del` was rewritten as a standalone recursive function that never stores aliases to its inputs. -### `jay_merge_delete`: empty dict wrongly pruned (fixed) +### `merge_delete`: empty dict wrongly pruned (fixed) When the new state had an empty dict `{}` at a location where the old state -had a non-empty dict, `jay_merge_delete` would delete all sub-keys and then +had a non-empty dict, `merge_delete` would delete all sub-keys and then prune the now-empty parent key entirely. The result differed from `new`, which retained the parent key as `{}`. @@ -189,7 +187,7 @@ Root cause: the old delta format stored the full old sub-dict for both "key absent from new" and "key present but empty in new" — the two cases were indistinguishable at merge time. -**Fix:** `jay_diff_del` now stores `None` for dict keys entirely absent from -`new`, and a non-empty sub-dict for partial deletions. `jay_merge_delete` +**Fix:** `diff_del` now stores `None` for dict keys entirely absent from +`new`, and a non-empty sub-dict for partial deletions. `merge_delete` no longer prunes: a `None` (or scalar) value in the delta is the explicit signal to remove the parent key. \ No newline at end of file