jens/jaypy
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Update internal README to reflect jaydiff refactor

Browse Source 
Rename function references from jay_ prefix to the internal names used
in the module (diff_add, diff_full, merge_full, etc.) and update the
package description to remove the original project-specific context.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
jens
2026-05-27 13:47:07 +02:00
co-authored by Claude Sonnet 4.6
parent d54148fdf6
commit 344a6e4973
1 changed files with 34 additions and 36 deletions
Download Patch File Download Diff File Expand all files Collapse all files
jaydiff/README.jaydiff.md
+34 -36
View File
@@ -1,10 +1,8 @@
# jay_diff — JSON diff / patch library # jaypy/jaydiff — JSON diff / patch library
`utils/jay_diff.py` is a lightweight, pure-Python library for computing and `jaydiff` is a lightweight, pure-Python library for computing and
applying structural diffs between two JSON-compatible dicts. It is used by applying structural diffs between two JSON-compatible dicts. It is used to
`collect.py` and `gui_client.py` to compress the push-server stream: instead compress the json data packets with redundant data, by providing a delta instead of the full packet
of retransmitting full snapshots on every poll, the server sends only the
fields that changed since the previous snapshot.
--- ---
@@ -24,7 +22,7 @@ state to a **new** state using one or more of these keys:
everywhere in this project). The separate `update` / `add` form is everywhere in this project). The separate `update` / `add` form is
available via `combine_upd_add=False` for completeness. available via `combine_upd_add=False` for completeness.
An **empty dict `{}`** returned by `jay_diff_full` means the two states are An **empty dict `{}`** returned by `diff_full` means the two states are
identical. identical.
### Delete-delta encoding ### Delete-delta encoding
@@ -52,7 +50,7 @@ completely.
### Diff functions ### Diff functions
```python ```python
jay_diff_full(old, new, combine_upd_add=True) -> dict diff_full(old, new, combine_upd_add=True) -> dict
``` ```
The main entry point. Computes a complete diff from `old` to `new`. The main entry point. Computes a complete diff from `old` to `new`.
@@ -62,53 +60,53 @@ The main entry point. Computes a complete diff from `old` to `new`.
Returns `{}` when `old == new`. Returns `{}` when `old == new`.
```python ```python
jay_diff_add(old, new) -> dict diff_add(old, new) -> dict
``` ```
Returns keys present in `new` but absent from `old` (additions only). Returns keys present in `new` but absent from `old` (additions only).
```python ```python
jay_diff_del(old, new) -> dict diff_del(old, new) -> dict
``` ```
Returns keys present in `old` but absent from `new`, or sub-keys that were Returns keys present in `old` but absent from `new`, or sub-keys that were
removed from a shared dict. Uses `None` as sentinel for entirely-absent removed from a shared dict. Uses `None` as sentinel for entirely-absent
dict keys. dict keys.
```python ```python
jay_diff_upd(old, new) -> dict diff_upd(old, new) -> dict
``` ```
Returns keys present in both `old` and `new` whose values differ (updates Returns keys present in both `old` and `new` whose values differ (updates
only; new keys are ignored). only; new keys are ignored).
```python ```python
jay_diff_upd_add(old, new) -> dict diff_upd_add(old, new) -> dict
``` ```
Like `jay_diff_upd` but also includes keys new in `new` — equivalent to Like `diff_upd` but also includes keys new in `new` — equivalent to
`jay_diff_full(..., combine_upd_add=True)` minus the delete section. `diff_full(..., combine_upd_add=True)` minus the delete section.
--- ---
### Merge functions ### Merge functions
```python ```python
jay_merge_full(old, diff) -> dict merge_full(old, diff) -> dict
``` ```
The main entry point. Applies a diff produced by `jay_diff_full` to `old` The main entry point. Applies a diff produced by `diff_full` to `old`
and returns the reconstructed new state. and returns the reconstructed new state.
**Bootstrap passthrough:** if `old` is `{}` and `diff` contains none of the **Bootstrap passthrough:** if `old` is `{}` and `diff` contains none of the
recognised diff keys, `diff` is returned as-is. This is how the first recognised diff keys, `diff` is returned as-is. This is how the first
message from the push server is handled: the server sends message from the push server is handled: the server sends
`jay_diff_full({}, record[0])`, which looks like a plain snapshot, and the `diff_full({}, record[0])`, which looks like a plain snapshot, and the
client stores it directly. client stores it directly.
Applies operations in order: `update` → `update_add` → `add` → `delete`. Applies operations in order: `update` → `update_add` → `add` → `delete`.
```python ```python
jay_merge_update(old, new) -> dict # apply an update/update_add delta merge_update(old, new) -> dict # apply an update/update_add delta
jay_merge_add(old, new) -> dict # apply an add delta merge_add(old, new) -> dict # apply an add delta
jay_merge_delete(old, new) -> dict # apply a delete delta merge_delete(old, new) -> dict # apply a delete delta
``` ```
Low-level helpers; prefer `jay_merge_full` unless you need fine-grained Low-level helpers; prefer `merge_full` unless you need fine-grained
control. control.
--- ---
@@ -116,27 +114,27 @@ control.
## Round-trip guarantee ## Round-trip guarantee
``` ```
jay_merge_full(old, jay_diff_full(old, new)) == new merge_full(old, diff_full(old, new)) == new
``` ```
This holds for all inputs where values are JSON-compatible scalars (strings, This holds for all inputs where values are JSON-compatible scalars (strings,
numbers, booleans, `None`, lists) or nested dicts thereof. It is verified numbers, booleans, `None`, lists) or nested dicts thereof. It is verified
by `tests/test_jay_diff_real_data.py` against 674 consecutive pairs of real by `tests/test_diff_real_data.py` against 674 consecutive pairs of real
vehicle snapshots. vehicle snapshots.
--- ---
## Wire format ## Wire format
`jay_diff_full` / `jay_merge_full` are used by the push server when `diff_full` / `merge_full` are used by the push server when
`--diff` is enabled. The diff dict is serialised as a newline-delimited `--diff` is enabled. The diff dict is serialised as a newline-delimited
JSON line. JSON line.
- The **first** message to a new client is `jay_diff_full({}, record[0])`, - The **first** message to a new client is `diff_full({}, record[0])`,
which always contains `update_add` with every field of the first record. which always contains `update_add` with every field of the first record.
- Subsequent messages contain only the changed fields. - Subsequent messages contain only the changed fields.
- The format is self-describing: a message with none of the diff keys is - The format is self-describing: a message with none of the diff keys is
treated as a full snapshot by `jay_merge_full` (bootstrap passthrough). treated as a full snapshot by `merge_full` (bootstrap passthrough).
`null` in JSON round-trips to Python `None`, so the delete sentinel survives `null` in JSON round-trips to Python `None`, so the delete sentinel survives
serialisation transparently. serialisation transparently.
@@ -146,10 +144,10 @@ serialisation transparently.
## Tests ## Tests
``` ```
tests/test_jay_diff.py — 104 unit tests (all diff/merge functions, tests/test_diff_.py — 104 unit tests (all diff/merge functions,
corner cases: type changes, empty dicts, corner cases: type changes, empty dicts,
deep nesting, round-trips) deep nesting, round-trips)
tests/test_jay_diff_real_data.py — parametrised over 675 real VW snapshots tests/test_diff_real_data.py — parametrised over 675 real VW snapshots
in tests/data/ (674 consecutive pairs): in tests/data/ (674 consecutive pairs):
round-trip, identical diff = {}, and round-trip, identical diff = {}, and
empty-diff-is-noop checks empty-diff-is-noop checks
@@ -165,23 +163,23 @@ pytest tests/
## Bug history ## Bug history
### `jay_merge_delete`: crash on aliased dicts (fixed) ### `merge_delete`: crash on aliased dicts (fixed)
`jay_diff_rev` (used internally by the old `jay_diff_del`) stored references `diff_rev` (used internally by the old `diff_del`) stored references
to sub-dicts of `_old` directly in the delete delta without copying them. to sub-dicts of `_old` directly in the delete delta without copying them.
When `jay_merge_full(old, jay_diff_full(old, new))` was called, some When `merge_full(old, diff_full(old, new))` was called, some
sub-dicts in the delete delta were the same Python objects as sub-dicts in sub-dicts in the delete delta were the same Python objects as sub-dicts in
`old`. Iterating `_new.items()` while deleting from `_old` (the same `old`. Iterating `_new.items()` while deleting from `_old` (the same
object) raised `RuntimeError: dictionary changed size during iteration`. object) raised `RuntimeError: dictionary changed size during iteration`.
**Fix:** `list(_new.items())` snapshots the items before the loop. Also, **Fix:** `list(_new.items())` snapshots the items before the loop. Also,
`jay_diff_del` was rewritten as a standalone recursive function that never `diff_del` was rewritten as a standalone recursive function that never
stores aliases to its inputs. stores aliases to its inputs.
### `jay_merge_delete`: empty dict wrongly pruned (fixed) ### `merge_delete`: empty dict wrongly pruned (fixed)
When the new state had an empty dict `{}` at a location where the old state When the new state had an empty dict `{}` at a location where the old state
had a non-empty dict, `jay_merge_delete` would delete all sub-keys and then had a non-empty dict, `merge_delete` would delete all sub-keys and then
prune the now-empty parent key entirely. The result differed from `new`, prune the now-empty parent key entirely. The result differed from `new`,
which retained the parent key as `{}`. which retained the parent key as `{}`.
@@ -189,7 +187,7 @@ Root cause: the old delta format stored the full old sub-dict for both "key
absent from new" and "key present but empty in new" — the two cases were absent from new" and "key present but empty in new" — the two cases were
indistinguishable at merge time. indistinguishable at merge time.
**Fix:** `jay_diff_del` now stores `None` for dict keys entirely absent from **Fix:** `diff_del` now stores `None` for dict keys entirely absent from
`new`, and a non-empty sub-dict for partial deletions. `jay_merge_delete` `new`, and a non-empty sub-dict for partial deletions. `merge_delete`
no longer prunes: a `None` (or scalar) value in the delta is the explicit no longer prunes: a `None` (or scalar) value in the delta is the explicit
signal to remove the parent key. signal to remove the parent key.