Discovered while testing the smart-plug power-switch feature: unplugging
the printer left the dashboard stuck showing it as online/"ready"
indefinitely. Live-tested against a real printer to isolate two
independent, compounding causes:
1. The MQTT socket had no TCP keepalive. A connection killed without a
clean TCP close (unplugged, not a graceful shutdown) looks alive to the
OS for as long as its default dead-connection timeout - 15+ minutes on
Linux - since a send on a half-open connection is buffered by the
kernel and doesn't fail immediately. Fixed with SO_KEEPALIVE (short
idle/interval/count) plus TCP_USER_TIMEOUT, since keepalive probes alone
only fire on an idle connection - verified live that a printer
disappearing while a send was still in flight (the common case, since
the poll loop sends every few seconds) instead falls back to the far
slower normal TCP retransmission timer, which keepalive settings don't
affect at all.
2. Even after the socket was correctly detected as dead, the status poll
loop could hang indefinitely inside publish() waiting for a reconnect
attempt already running on the MQTT reader thread (the Issue #105
reconnect-lock serialization), and therefore never reached the
is_connected() check that flips kobra_state to "offline". _reconnect()
now takes wait_if_in_progress/persist flags so the poll loop's call
returns immediately with at most one attempt instead of blocking
through someone else's multi-minute backoff loop - persistent retrying
stays the reader thread's job.
A disconnected printer is now detected and reflected on the dashboard
within about 15 seconds. Live-verified across repeated disconnect/
reconnect cycles that no sockets, threads, or file descriptors are left
behind (checked via /proc/<pid>/fd and /proc/<pid>/task) - the transient
FIN-WAIT-2 entries seen while the printer's TLS service is still booting
belong to the kernel's own connection teardown, not to processes held by
the bridge, and clear on their own.
The bridge could get permanently stuck after a printer went offline and
came back, even with the printer confirmed reachable via ping/nc - only
a full container restart recovered it. Two compounding bugs:
1. Two independent code paths could trigger _reconnect() concurrently
with no coordination: the reader thread (on a failed keepalive ping)
and publish()/publish_web() (on a failed sendall(), which happens
constantly once the socket is dead, since the poll loop calls
query_info() every poll_interval). Both would race into their own
_do_connect(), each opening a competing TLS handshake against a
printer that likely only accepts one mTLS session at a time - so
neither converges, and the failure repeats every ~3s instead of
backing off. Added a lock so a second _reconnect() call waits for
the first to finish instead of starting a competing handshake.
2. publish() swallows send/reconnect failures internally and returns
None instead of raising - so _poll_loop's `if info: ...` branch was
silently skipped on failure, but the surrounding except-block (which
would have triggered the existing, correct offline/reconnect
transition) was never reached, since no exception was ever thrown.
The poll loop had no way to tell "printer sent nothing this tick"
apart from "the MQTT session is dead". Added client.is_connected()
and check it explicitly when query_info() returns falsy, routing a
dead session into the same clean offline branch already used for a
TCP-unreachable printer.
Verified: bridge continued printing normally throughout (live print
in progress on the real printer during this fix), full test suite
green (111 tests), new tests cover the concurrent-reconnect lock and
the poll-loop offline transition on a swallowed send failure.
