[bugfix] - support large CPU-side datasets; fix numpy/CuPy interop
train.py: add _to_gpu() helper; convert mini-batches from CPU numpy to GPU CuPy on the fly so large datasets can stay in RAM. Use numpy permutation for numpy batches during shuffle. Limit status err_rms to first 5000 samples to avoid GPU OOM on status checks. model.py: replace np.copy() with np.array() — cupy.copy() does not convert numpy arrays to CuPy; cupy.array() does. Fixes entity.forward() crash after training when batch was passed as numpy. test_faces_sub_image.py: load_patches normalizes on CPU and returns numpy, avoiding multi-copy GPU OOM for large datasets (500 images ≈ 1.4 GB). show_reconstructions converts numpy patches to CuPy before inference. Add --n_images CLI arg. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
+1
-1
@@ -22,7 +22,7 @@ class Model(ABC):
|
||||
|
||||
def train(self, batch: Mat):
|
||||
entities = self.objects(Entity)
|
||||
_batch = np.copy(batch)
|
||||
_batch = np.array(batch) # np.array() converts numpy→CuPy; np.copy() does not
|
||||
for entity in entities:
|
||||
if entity.enable_training:
|
||||
train(entity, _batch, Status())
|
||||
|
||||
Reference in New Issue
Block a user