fix: heater task permanently died after a mid-brew unplug, never recovered

Root cause (from the brewpi Pi's log): HendiCtrl.disconnect() only
caught HendiException (a protocol-level NAK), but a genuine unplug
fails deep inside pyserial itself - flushInput()/readline() raise
termios.error/OSError on the now-dead fd. That escaped disconnect()
uncaught (skipping self.ser.close() too), then kept propagating up
through every caller that invoked disconnect() from its own except
handler (HeaterHendi.process()/activate(), AHeater.open()'s
context-manager exit), all the way out of HeaterTask.on_process()'s
coroutine entirely. Nothing restarts a dead ATask, so the heater's
process()/duty-cycle loop was gone for the rest of the process's
life - reconnecting afterward changed `connected` back to True but
nothing was left running to ever call process() again.

Fixes:
- HendiCtrl.disconnect() now catches broadly and always closes the
  port, matching Pololu1376.disconnect()'s already-correct pattern.
- HeaterTask/StirrerTask.on_process() gain an outer retry loop as a
  safety net: even an unanticipated exception now logs, marks
  disconnected, and re-enters rather than permanently killing the
  task.

Verified by reproducing the exact failure (comm error raising a raw
OSError, not HendiException) against a fake device - the task now
survives and resumes process()ing once reconnected.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YaPLuRPpyjWcwhMvCvpHCL

components/actor/hendiCtrl.py

@@ -213,9 +213,23 @@ class HendiCtrl:
 		if self.ser.is_open:
 			try:
 				self.remoteEnable(False)
-			except HendiException:
+			except Exception:
+				# Not just HendiException (a protocol-level NAK) - a genuine
+				# unplug fails deep inside pyserial itself (termios.error/
+				# OSError from flushInput()/readline() on a dead fd), and
+				# disconnect() must never let that escape: every caller here
+				# (HeaterHendi.process()/activate()'s own except handlers,
+				# AHeater.open()'s context-manager exit) calls disconnect()
+				# specifically to recover from a comm failure, so if this
+				# raises too it propagates all the way out of HeaterTask's
+				# on_process() and permanently kills that task's loop - the
+				# device then never gets process()'d again even after a
+				# later successful reconnect.
 				pass
+			try:
 				self.ser.close()
+			except Exception:
+				pass
 		self.sw_id = None
 		self.sw_ver = None
 

tasks/heater.py

@@ -122,6 +122,21 @@ class HeaterTask(ATask):
 		# connected.
 		await self.send({'Connected': self.device.connected})
 
+		# Outer retry loop - belt-and-suspenders around the tick loop's own
+		# per-call guards below. A comm failure a device-level guard didn't
+		# anticipate (e.g. a raw pyserial/termios exception escaping from
+		# somewhere other than device.process(), the one call already
+		# wrapped below) must never be allowed to fall out of this
+		# coroutine entirely: nothing restarts a dead ATask, so a single
+		# uncaught exception here would permanently stop this device from
+		# ever being process()'d again - even after a later, otherwise-
+		# successful reconnect. See the incident this was added for: a
+		# mid-brew unplug raised termios.error from deep inside
+		# HendiCtrl.disconnect() itself (fixed there too), which escaped
+		# all the way out of this loop and killed it for the rest of the
+		# process's life.
+		while True:
+			try:
 				with self.device.open():
 					while True:
 						# Closed-loop: TC has full control. Open-loop: direct manual power only.
@@ -152,6 +167,13 @@ class HeaterTask(ATask):
 							print(f"HeaterTask: comm error, marking disconnected: {e}")
 							self.device.disconnect()
 						await asyncio.sleep(self.interval)
+			except Exception as e:
+				print(f"HeaterTask: unexpected error, recovering: {e}")
+				try:
+					self.device.disconnect()
+				except Exception:
+					pass
+				await asyncio.sleep(self.interval)
 
 
 

tasks/stirrer.py

@@ -66,6 +66,12 @@ class StirrerTask(ATask):
 		# observer.
 		await self.send({'Connected': self.device.connected})
 
+		# Outer retry loop - see HeaterTask.on_process()'s own comment for
+		# why: nothing restarts a dead ATask, so an exception escaping this
+		# coroutine entirely would permanently stop this device from ever
+		# being process()'d again, even after a later successful reconnect.
+		while True:
+			try:
 				with self.device.open():
 					while True:
 						try:
@@ -74,3 +80,10 @@ class StirrerTask(ATask):
 							print(f"StirrerTask: comm error, marking disconnected: {e}")
 							self.device.disconnect()
 						await asyncio.sleep(self.interval)
+			except Exception as e:
+				print(f"StirrerTask: unexpected error, recovering: {e}")
+				try:
+					self.device.disconnect()
+				except Exception:
+					pass
+				await asyncio.sleep(self.interval)