diff --git a/README.md b/README.md index e69de29..6105608 100644 --- a/README.md +++ b/README.md @@ -0,0 +1,67 @@ +# Tic Tac Toe + +A Tic Tac Toe implementation where two reinforcement-learning agents +(temporal-difference / value-based learning) train against each other, and a +human can play against a trained agent from the command line. + +## How it works + +- `state.py` — board representation (3x3 numpy array of `'X'`, `'O'`, `'-'`) + and helpers to detect potential moves and winning lines. +- `a_player.py` — `APlayer` base class shared by all player types (move, + win-check, game lifecycle hooks). +- `machine_player.py` — `MachinePlayer`, a TD-learning agent. It keeps a + value table (state string -> expected reward) in a dict, picks the move + leading to the best-valued next state, occasionally explores a random move + (`p_explore`), and updates values towards the outcome of each game using a + learning rate `alpha`. +- `human_player.py` — `HumanPlayer`, lets a human enter moves via the + console (German prompts). +- `player_provider.py` — `PlayerProvider` pairs two players for a game and + randomly decides who starts. +- `helper.py` — small numeric helpers (softmax/sampling), not currently used + by the main game loop. +- `tic_tac_toe.py` — entry point: runs training and/or play loops. +- `values_x.json` / `values_o.json` — persisted value tables for the X and O + machine players, loaded/saved across runs so training progress carries + over. + +## Requirements + +- Python 3.12+ +- numpy (see `.venv` for an existing virtual environment, or install with + `pip install numpy`) + +## Usage + +Run from the project root: + +```bash +python tic_tac_toe.py [options] +``` + +Options: + +| Flag | Default | Description | +| ----------- | ------- | --------------------------------------------------------------- | +| `--verbose` | `False` | Print detailed per-move debug output | +| `--train` | `False` | Train the machine players before playing | +| `--num` | `5` | Number of training iterations / batches | +| `--human` | `False` | Let a human play as X against the machine player as O | +| `--load` | `True` | Load existing value tables from `values_x.json`/`values_o.json` | + +Examples: + +```bash +# Train the two machine players against each other, saving progress +python tic_tac_toe.py --train --num 50 + +# Play as a human (X) against the trained machine player (O) +python tic_tac_toe.py --human + +# Watch two machine players play a few verbose games +python tic_tac_toe.py --verbose +``` + +During training, `values_x.json` and `values_o.json` are updated after each +batch of games, so training can be interrupted and resumed.