fix(web): stop HTTPS panel heap fragmentation, add self-heal and heap counters

Both Clyde North observers stopped answering over WiFi after days of
uptime while LoRa and MQTT kept working. `memory` showed heap_min ~1KB
and a largest internal block of 19KB: the HTTPS listener was alive but
no mbedTLS handshake (~40KB contiguous) could be allocated.

The trigger was the Aug 9 lru_purge change: with a 2-socket pool, every
browser page-load burst evicted a live session and forced a fresh TLS
handshake, and the repeated 40KB alloc/free cycles fragmented internal
RAM. Before that change the pool simply jammed, so nothing churned.

- Send `Connection: close` and trigger a session close on one-shot
  responses (pages, favicon, redirect, login, 401s) so page-load bursts
  release their sockets immediately. The authenticated /api/* polling
  connection keeps keep-alive so it does not pay a handshake per poll.
- Gate WebPanelServer::start() on internal heap headroom (56KB free /
  32KB largest), retrying every 15s instead of every loop tick.
- Self-heal: when the panel is idle and the largest internal block drops
  below 24KB, stop and re-create the server to return its pools.
- Count the MQTT teardown path that deliberately abandons a client on a
  heap-integrity failure.
- Expose the above as `heals:`/`deferred:` in `get web.status` and
  `leaked:` in `get mqtt.status`.
- Fix the stats page Channel/Gateway cells showing `--`: the /api/stats
  summary JSON never carried channel, gateway health, or the watchdog
  count (only the `get wifi.status` string did).
- Add eastmesh-tools/web-heap-check.sh to read heap/service health over
  the API and optionally stress the panel with browser-style bursts.
- Release notes 2026.8.3 for both observer tracks.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Jared Dohrman
2026-08-23 10:53:53 +00:00
co-authored by Claude Fable 5
parent 4445fb9f4d
commit 866498299e
12 changed files with 343 additions and 30 deletions
+11 -1
View File
@@ -2644,6 +2644,9 @@ bool MyMesh::formatWebStatsSummaryJson(char* reply, size_t reply_size) {
int wifi_rssi = 0;
int wifi_quality = 0;
int wifi_code = 0;
int wifi_channel = 0;
const char* wifi_gateway = "--";
unsigned wifi_watchdog_count = 0;
#if defined(ESP32)
if (network.getWifiSSID()[0] == 0) {
@@ -2664,6 +2667,9 @@ bool MyMesh::formatWebStatsSummaryJson(char* reply, size_t reply_size) {
escapeJsonString(ip.c_str(), wifi_ip, sizeof(wifi_ip));
wifi_rssi = WiFi.RSSI();
wifi_code = static_cast<int>(WiFi.status());
wifi_channel = WiFi.channel();
wifi_gateway = network.isGatewayReachable() ? "ok" : "lost";
wifi_watchdog_count = network.getWatchdogReconnectCount();
if (wifi_rssi <= -100) {
wifi_quality = 0;
strncpy(wifi_signal, "poor", sizeof(wifi_signal) - 1);
@@ -2778,7 +2784,8 @@ bool MyMesh::formatWebStatsSummaryJson(char* reply, size_t reply_size) {
"\"packets\":{\"recv\":%u,\"sent\":%u,\"flood_tx\":%u,\"direct_tx\":%u,\"flood_rx\":%u,\"direct_rx\":%u,"
"\"recv_errors\":%u,\"direct_dups\":%u,\"flood_dups\":%u,\"neighbors\":%u},"
"\"memory\":{\"heap_free\":%u,\"heap_min\":%u,\"heap_max\":%u,\"psram_free\":%u,\"psram_min\":%u,\"psram_max\":%u},"
"\"wifi\":{\"ssid\":\"%s\",\"status\":\"%s\",\"connected\":%s,\"state\":\"%s\",\"code\":%d,\"ip\":\"%s\",\"rssi\":%d,\"quality\":%d,\"signal\":\"%s\",\"powersave\":\"%s\"},"
"\"wifi\":{\"ssid\":\"%s\",\"status\":\"%s\",\"connected\":%s,\"state\":\"%s\",\"code\":%d,\"ip\":\"%s\",\"rssi\":%d,\"quality\":%d,\"signal\":\"%s\",\"powersave\":\"%s\","
"\"channel\":%d,\"gateway\":\"%s\",\"watchdog_count\":%u},"
"\"services\":{\"mqtt_connected\":%s,\"mqtt_state\":\"%s\",\"web_enabled\":%s,\"web_panel_up\":%s,\"web_auth\":\"%s\","
"\"archive_available\":%s}",
(_stats_history.isEnabled() && _stats_history.isRecentHistoryAvailable()) ? "true" : "false",
@@ -2841,6 +2848,9 @@ bool MyMesh::formatWebStatsSummaryJson(char* reply, size_t reply_size) {
wifi_quality,
wifi_signal,
wifi_powersave,
wifi_channel,
wifi_gateway,
wifi_watchdog_count,
mqtt_connected ? "true" : "false",
mqtt_state,
web.isWebEnabled() ? "true" : "false",