Found during a targeted code review, not from a user report:
- _drain() could get permanently stuck: valid JSON that isn't an object
(e.g. a bare number or list, which json.loads() happily accepts) crashed
_dispatch()'s dict-oriented logic, and the exception escaped before
self._buf was advanced past the bad packet - so the same malformed bytes
sat at the front of the buffer and re-crashed every subsequent _drain()
call, forcing a reconnect each time. Now the buffer always advances (via
try/finally) and _dispatch() rejects non-dict payloads with a warning log
instead of crashing.
- _pending_msgid/_pending_report were mutated without synchronization
while the reader thread reads/resolves them in _dispatch() - a
check-then-set race on the shared report_key slot meant two concurrent
publish() calls for the same msg_type could have one silently miss its
reply. Added a dedicated lock around all registration/cleanup.
- The report-topic resolution path never verified a reply's msgid matched
the waiter it was about to resolve, so a late reply for an
already-timed-out request could be delivered to an unrelated, newer
caller waiting on the same report_key. Entries now carry their own msgid
for this comparison; replies without a msgid still resolve normally
(most printer push-reports don't carry one).
- upload_gcode() silently sent a request with an empty session token when
the upload URL was missing "?s=", instead of raising a clear error at
the actual point of failure. It also leaked the upload socket's file
descriptor on any send/recv failure other than a timeout, since there
was no try/finally around its lifetime.
New tests in tests/test_client_robustness.py cover all of the above.
Discovered while testing the smart-plug power-switch feature: unplugging
the printer left the dashboard stuck showing it as online/"ready"
indefinitely. Live-tested against a real printer to isolate two
independent, compounding causes:
1. The MQTT socket had no TCP keepalive. A connection killed without a
clean TCP close (unplugged, not a graceful shutdown) looks alive to the
OS for as long as its default dead-connection timeout - 15+ minutes on
Linux - since a send on a half-open connection is buffered by the
kernel and doesn't fail immediately. Fixed with SO_KEEPALIVE (short
idle/interval/count) plus TCP_USER_TIMEOUT, since keepalive probes alone
only fire on an idle connection - verified live that a printer
disappearing while a send was still in flight (the common case, since
the poll loop sends every few seconds) instead falls back to the far
slower normal TCP retransmission timer, which keepalive settings don't
affect at all.
2. Even after the socket was correctly detected as dead, the status poll
loop could hang indefinitely inside publish() waiting for a reconnect
attempt already running on the MQTT reader thread (the Issue #105
reconnect-lock serialization), and therefore never reached the
is_connected() check that flips kobra_state to "offline". _reconnect()
now takes wait_if_in_progress/persist flags so the poll loop's call
returns immediately with at most one attempt instead of blocking
through someone else's multi-minute backoff loop - persistent retrying
stays the reader thread's job.
A disconnected printer is now detected and reflected on the dashboard
within about 15 seconds. Live-verified across repeated disconnect/
reconnect cycles that no sockets, threads, or file descriptors are left
behind (checked via /proc/<pid>/fd and /proc/<pid>/task) - the transient
FIN-WAIT-2 entries seen while the printer's TLS service is still booting
belong to the kernel's own connection teardown, not to processes held by
the bridge, and clear on their own.
Logs every incoming MQTT message on INFO regardless of topic, including
dedup'd duplicates and topics with no registered callback - useful for
capturing printer behavior the bridge doesn't normally surface without
needing to know the topic name in advance (the existing wildcard
subscribe already receives everything, this just makes it visible).
The bridge could get permanently stuck after a printer went offline and
came back, even with the printer confirmed reachable via ping/nc - only
a full container restart recovered it. Two compounding bugs:
1. Two independent code paths could trigger _reconnect() concurrently
with no coordination: the reader thread (on a failed keepalive ping)
and publish()/publish_web() (on a failed sendall(), which happens
constantly once the socket is dead, since the poll loop calls
query_info() every poll_interval). Both would race into their own
_do_connect(), each opening a competing TLS handshake against a
printer that likely only accepts one mTLS session at a time - so
neither converges, and the failure repeats every ~3s instead of
backing off. Added a lock so a second _reconnect() call waits for
the first to finish instead of starting a competing handshake.
2. publish() swallows send/reconnect failures internally and returns
None instead of raising - so _poll_loop's `if info: ...` branch was
silently skipped on failure, but the surrounding except-block (which
would have triggered the existing, correct offline/reconnect
transition) was never reached, since no exception was ever thrown.
The poll loop had no way to tell "printer sent nothing this tick"
apart from "the MQTT session is dead". Added client.is_connected()
and check it explicitly when query_info() returns falsy, routing a
dead session into the same clean offline branch already used for a
TCP-unreachable printer.
Verified: bridge continued printing normally throughout (live print
in progress on the real printer during this fix), full test suite
green (111 tests), new tests cover the concurrent-reconnect lock and
the poll-loop offline transition on a swallowed send failure.
