When the "start ota" command was issued from the web panel, a race
condition could occur between the main loop thread and the HTTP server
thread:
- HTTP thread: prepareForOTAStart() -> stopRedirectServer() ->
httpd_stop(_redirect_server) [blocking]
- Main thread: loop() -> ensureWebServer() -> stopRedirectServer() ->
httpd_stop(_redirect_server) [double free!]
Because stopRedirectServer() sets _redirect_server = nullptr only after
httpd_stop() returns, both threads could pass the nullptr check
simultaneously, resulting in a double free and heap corruption:
CORRUPT HEAP: Bad head at 0x3fcb1f24. Expected 0xabba1234 got 0x3fca5f34
assert failed: multi_heap_free multi_heap_poisoning.c:259 (head != NULL)
The call in ensureWebServer() was added as a safety net in commit
4b83142b, but is redundant: prepareForOTAStart() already calls
stopRedirectServer() synchronously before startOTAUpdate() occupies
port 80. Removing it eliminates the race condition.
The bug only manifested when OTA was triggered from the web panel
(HTTP thread), not from radio (main thread), because in the latter
case both calls happen on the same thread and cannot race.
The previous while loop computed a ternary `chunk_len` on every
iteration to handle the final partial chunk as a special case, and
called vTaskDelay(1) after every chunk including the last one,
adding an unnecessary yield immediately before the terminating null
chunk.
Restructure the loop using the identity:
last_size = ((len - 1) % kWebPageChunkSize) + 1
This gives `last_size` in [1, kWebPageChunkSize] for any non-zero `len`,
so the final partial chunk is always non-empty and can be sent after
the loop without a special case. The for loop then iterates only over
full kWebPageChunkSize chunks, making every iteration identical and
branch-free, and vTaskDelay(1) is called only after full chunks.
Also add a `len == 0` guard to short-circuit immediately when there is
nothing to send, and remove the best-effort terminator send on the
error path since a failed send makes a follow-up send equally likely
to fail.
The HTML pages (login, app, stats) are string literals compiled into
PROGMEM. Their sizes are known at compile time, but the previous
sendProgmemChunked() called strlen() on every request to determine
the length, requiring a full scan of potentially tens of kilobytes of
PROGMEM data before the first byte was sent.
Replace sendProgmemChunked() with a sendWhole() function that accepts
an explicit length parameter, and introduce a sendProgmem() macro that
passes sizeof(mem) - 1 at the call site. For array literals this
resolves to a compile-time constant, eliminating the runtime strlen
scan entirely.
The rename from sendProgmemChunked() to sendWhole() also better reflects
what the function does — it sends the entire buffer in chunks — rather
than where the data comes from.
The login, app, and stats HTML pages are static assets compiled into
PROGMEM. All dynamic content is fetched separately via JavaScript API
calls after the page loads — the HTML itself never changes between
requests within the same firmware version.
The "Cache-Control: no-store" header forced the browser to re-download
the full HTML on every visit, including a complete chunked transfer
from the ESP32. Given that the ESP32 is slow and serves responses in
small chunks, this was an unnecessary repeated cost that added latency
before the page became interactive.
Removing no-store allows the browser to cache the HTML pages locally.
Subsequent requests are served from the browser cache instantly, with no
transfer from the ESP32 at all, leaving the server free to handle the
API requests that actually carry dynamic data.
The HTTP redirect server has max_open_sockets = 2. A client that
connects and then disappears silently (network drop, browser crash,
mobile radio loss) leaves its socket occupying a slot indefinitely —
the server has no way to detect the loss until it tries to write to
the socket again.
Enable lru_purge_enable on the redirect server so that when both
slots are occupied by such stale connections, the least-recently-used
one is evicted automatically to make room for a new incoming
connection, rather than refusing it outright.
Add a "Connection: close" header to the 302 response as a complementary
measure. For well-behaved clients this triggers an immediate TCP
teardown after the redirect is received, shrinking the window during
which a connection can turn into a zombie. Together the two changes
provide defence in depth: "Connection: close" prevents stale connections
from forming in the first place; lru_purge_enable cleans them up when
they do.
The ESP32 processes HTTP requests sequentially on a single task.
Under load — generating chunked HTML pages, running commands, or
serving sequential stats API calls — it can be slow to send data
or loop back to accept the next request. The previous 2-second
recv/send timeouts were too tight for this: the httpd layer would
drop connections mid-transfer, causing the browser to hang or
show incomplete pages.
Raise recv_wait_timeout and send_wait_timeout from 2 s to 10 s
on both the HTTPS and HTTP-redirect servers, giving the ESP32
enough headroom to finish generating and sending responses without
the transport layer tearing down the connection prematurely.
Set backlog_conn to 0 on both servers. With the previous backlog
of 2, incoming connections completed the TCP handshake and queued
in the kernel while the ESP32 was busy. The browser saw the
connection as open but received no HTTP response, causing it to
freeze indefinitely. With backlog 0 (lwIP minimum), connections
that cannot be immediately accepted are refused outright, giving
the browser a fast, recoverable error instead of a silent hang.
Enable lru_purge_enable on the HTTPS server. Browsers hold
keep-alive connections open for reuse. With max_open_sockets = 2,
both slots can be occupied by idle keep-alive connections from the
same session, blocking a new connection attempt entirely. LRU
purge automatically closes the least-recently-used idle keep-alive
connection to make room, ensuring the single client can always
reconnect without a server restart.
When sending large PROGMEM content in chunks over HTTPS, the lwIP TCP/IP
task (tiT) could monopolize CPU 0 for an extended period without ever
yielding, starving the IDLE0 task and triggering the task watchdog timer.
Add vTaskDelay(1) at the end of each iteration in sendProgmemChunked()
to yield to the scheduler between chunks, allowing the IDLE task to reset
the watchdog and preventing spurious reboots during web panel page loads.
The hardcoded kWebPageChunkSize of 768 bytes caused the task watchdog
to trigger when serving the web panel over HTTPS. Each call to
httpd_resp_send_chunk() results in a separate TLS record encryption
via mbedTLS, which on ESP32-S3 uses DMA-backed AES-GCM (esp_aes_process_dma).
The gdma_disconnect() call inside that path enters a critical section,
blocking the IDLE0 task. With 768-byte chunks, a large page response
requires many such DMA operations in tight succession, starving the IDLE
task long enough to trip the watchdog.
Replacing the hardcoded value with MBEDTLS_SSL_OUT_CONTENT_LEN aligns the
chunk size to the TLS output record buffer, minimising the number of TLS
records (and thus DMA encryption operations) needed to send a full page,
and keeping the httpd task within the watchdog timeout.
Fixes: task_wdt abort in sendProgmemChunked() -> httpd_ssl_send() ->
esp_aes_process_dma() -> gdma_disconnect() on ESP32-S3.
Set task_priority = tskIDLE_PRIORITY + 2 and core_id = 0 for both the
HTTPS server and the HTTP-to-HTTPS redirect server. This keeps web
serving off core 1, which handles radio and application logic, reducing
interference with time-sensitive operations.
_have_time_sync is reset to false whenever WiFi disconnects, even
though the ESP32 RTC continues to hold accurate time after a
successful SNTP sync. This caused hasTimeSync() to return false
during transient WiFi outages, unnecessarily tearing down MQTT
broker connections and suppressing packet publishing.
Introduce _last_time_sync to record the wall-clock time of the
most recent confirmed sync. Move hasTimeSync() out of the header
into NetworkService.cpp and extend its logic: in addition to the
existing _have_time_sync flag, return true if the system clock is
still sane (>= kMinSaneEpoch) and no more than kMaxOutOfSync (24h)
has elapsed since the last confirmed sync.
This makes the MQTT uplink resilient to brief WiFi dropouts without
requiring any changes to callers of hasTimeSync().
Also bump kMinSaneEpoch from 2025-01-01 to 2026-01-01.
When the MQTT WebSocket handshake fails before a connection is fully
established (e.g. "Sec-WebSocket-Accept not found"), the IDF v4
transport teardown path writes only 3 of the 4 bytes of the heap block
tail canary (expected 0xbaad5678, actual 0xbaad5600). The subsequent
esp_mqtt_client_destroy() call frees that block, causing multi_heap_free
to detect the broken canary and abort:
CORRUPT HEAP: Bad tail at 0x3fcb42a4. Expected 0xbaad5678 got 0xbaad5600
assert failed: multi_heap_free multi_heap_poisoning.c:259 (head != NULL)
The fix is to check heap integrity with heap_caps_check_integrity_all(false)
between esp_mqtt_client_stop() and esp_mqtt_client_destroy(). If corruption
is detected, scan internal SRAM (0x3FC00000–0x3FD00000) for the truncated
canary pattern and restore it to the correct value before
destroy() runs.
The scan is a no-op when the heap is clean and is compiled out entirely
on IDF v5+, where the underlying bug does not exist.
This fixes crash-on-reconnect observed with ESP32-S3 + IDF v4 + WSS transport.
Valentin V. Bartenev