Lands @Vybo's storage-stack rework from #179 — runtime SD wedge/remount recovery,
chat-store health diagnostics, and a segmented, append-friendly message store that
replaces the whole-ring file rewrite (crash-atomic migration, per-segment
corruption quarantine) — plus a fix for the segment-table-cap latch that made SD
I/O fail after a few hours of uptime: at the cap, segBuildJob emitted a create-append
segCommitJob could not table, freezing the durability watermark into a permanent
write-spin that leaked orphan segment files until the card filled.
Full credit to Dan Vybiral (@Vybo) for the design and implementation. The cap-latch
fix is the only change on top of his branch. Landed directly (his fork branch could
not carry the fix), so #179 is closed rather than merged.
Co-Authored-By: Dan Vybiral <dan.vybiral@greencode.cz>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Root cause of the reported "SD writes fail after a few hours of uptime": when the
segment table is full (s_seg_count == k_ui_seg_max), segBuildJob still emits a
CREATE-append, but segCommitJob only tables a create while s_seg_count < the cap.
The new segment file is written but never recorded, the durability watermark
(s_seg_flushed_seq) never advances, so the same append re-arms every flush cycle
(permanent write-spin). As the ring evicts the frozen records the target first_seq
shifts, so a new orphan seg_*.bin is written each cycle until the card fills and
every SD write fails; records evicted before they are credited are lost.
It surfaces only after hours because s_seg_count creeps from its ~20 steady state
(5000/256) up to the 24 cap slowly, via delete-driven fragmentation and
step-1-append starving step-2 retirement.
Fix: at the cap, defer the create-append instead of emitting an untableable one.
segBuildJob skips step 1 (before the gather/watermark advance, so the watermark is
never moved past un-written records) and falls through to step 2, which retires the
oldest segment first. At the cap the oldest 256 records have aged out of the ring
(24*256 > 5000), so that segment is fully evicted -> compact_dirty -> unlinked,
freeing a slot with no live-record loss; the deferred records are the newest (above
the watermark, evicted last) and land next cycle. saveMsgsToStorage's early
"armed == 0 -> durable" return is guarded with segMoreWorkPending so a deferred
append is reported as a failure, not a false success. Below the cap: no-op.
Builds green on T-Deck + Heltec V4 TFT. Needs a multi-hour on-device soak to
confirm against the original repro.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
beta_49 hard-pinned s_ui_fscale=100 on the P4 to stop >100% chrome overflow, but
the "UI size" dropdown stays shown (CAP_LARGE_SCREEN), so changing it did nothing
(reported: "changing text size doesn't work on the P4"). Honour the saved pref like
the Tanmatsu does. Large/Huge can still make some P4 chrome that uses unscaled dims
tight; that's a per-screen SC() follow-up, not a reason to disable scaling.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
After a reinsert the chat store re-landed but the map stayed blank: the
tile layer was never told the storage changed. It keeps decoded tiles in
s_map_tiles across renders and only renders on pan / zoom / tab-open, so
a card that came back produced no reload; a boot with no card never
adopted a later-inserted one as the tile cache at all.
mapNoteStorageChanged() now runs from every mount-state transition
(wedge-recovery remount, 30 s reinsert watch, file-manager insert poll,
post-format, and both card-lost paths), right next to the existing
flushHistorySoon(). It adopts the card as the tile backend when this
boot resolved none, drops the decoded tiles (a swapped card holds
different content), clears the fetch dedup ring so downloads that failed
during the outage are retried instead of being remembered as in-flight
forever, and re-renders immediately when the map is the visible tab.
Builds verified: all five SD-relevant touch envs.
Signed-off-by: Dan Vybiral <dan.vybiral@greencode.cz>
The home chip now carries the clock time of the last successful save
(HH:MM, 12/24h-aware, with a '-<N>D' suffix once it is a day or more
old) instead of the segment count and byte total, and the About page's
'last save' line switches from 'OK 12s ago' / 'FAIL xN (12s ago)' to the
same timestamp form. A clock time answers 'is my history safe right
now?' directly, and the day suffix stops a store that quietly stopped
saving yesterday from reading as fresh. Segment count, byte total and
the failure stage + errno stay on the About panel.
Backed by new wall-clock stamps taken alongside the existing millis
ones in uiMsgsWriteResult; they read 0 (rendered '--:--') when the
system clock wasn't set at save time, since no meaningful timestamp
exists then.
Builds verified: all five SD-relevant touch envs.
Signed-off-by: Dan Vybiral <dan.vybiral@greencode.cz>
A second adversarial review pass (11 confirmed findings, 2 critical) hit
the crash and failure paths the on-device happy path never touches. Six
root causes, fixed at the root rather than per-symptom:
1. Migration was not crash-atomic (CRITICAL). Chunks landed one rename
at a time, so a power cut mid-migration left a valid oldest-first
PREFIX that the loader preferred over the still-intact old file --
permanently showing only the oldest few hundred messages. There is
now a commit marker (/msgs/store.ok) written only after full
verification: its absence means the segment set is provisional, and
the loader wipes those segments and re-reads the old file. The 60 s
migration retry on a sick card therefore no longer risks history.
2. Migration assumed a linear ring (CRITICAL) but its retry paths run
mid-session, after appendMessage may have wrapped it -- producing
segments in slot order (first_seq > last_seq, scrambled history that
per-segment read-back could not detect) before deleting the old
file. It now walks the ring chronologically, skips tombstones (which
it used to persist as ghost records), and verification additionally
requires strictly increasing seqs within and across segments.
3. A partially-failed append was retried by appending the same batch
again, duplicating records (or misaligning every later record if the
failure split one). Segments now carry a rewrite_open flag meaning
"file content untrusted": the scheduler REPAIRS such a segment with
a full rewrite (header + records, tmp+rename) before any append can
target it, and the repair absorbs the unflushed tail.
4. Post-resync appends targeted deleted files (headerless segments) and
could push a chunk's range past the 256-record gather cap, silently
dropping records the watermark already called durable. Fixed by the
same repair-first rule: retable marks every chunk rewrite_open.
5. Resync deleted every on-disk segment up front, so a same-card wedge
recovery left the whole history RAM-only for minutes. It no longer
deletes anything: same-key files are replaced by each repair's
rename, and leftovers are swept only once the re-land is complete.
6. A stale worker descriptor could be committed twice after
persistHistoryNow drained past a busy worker (double-counted totals,
duplicate table entries), and a leaked redirty flag could make the
drain re-compact one segment ~28 times and then report failure.
uiHistWaitWorkerIdle now always drops the descriptor and the flag,
marking the abandoned job's segment for repair instead. The drain
also refuses to run at all while a worker is still writing -- with
FF_FS_LOCK=0, removing or renaming a path under a stalled open
handle can cross-link clusters and corrupt the volume.
Also: the loader applies a strictly-monotonic seq filter, so duplicate
or out-of-order records from any interrupted write are dropped at load
and the affected segment is rewritten; append commits credit their own
segment by key instead of blindly crediting the last table entry.
Builds verified: all five SD-relevant touch envs.
Signed-off-by: Dan Vybiral <dan.vybiral@greencode.cz>
Adds a compact store-status chip sharing the TX/RX legend row above the
home chart, parked on the right half of the chart width (fixed width +
right-aligned + CLIP, so it can never grow into the traffic counts at
any UI scale). Small font (g_font_12), not clickable — the legend still
owns this row's taps.
Shows a floppy glyph plus, in order of precedence: 'FAIL xN' in red
while saves are failing, 'migrating' in amber while an old-format store
hasn't converted yet, else the segment count and total bytes ('9s 512K'
/ '21s 1.1M') in the normal subdued colour. Refreshed independently of
the TX/RX chart so the big-screen scaled layout (which drops the chart)
still shows it, and re-flowed with the legend on Expansion-Kit boards.
The full diagnosis (backend, failure stage + errno) stays on the About
page's Chat store panel.
Builds verified: all five SD-relevant touch envs.
Signed-off-by: Dan Vybiral <dan.vybiral@greencode.cz>
An adversarial review pass over the refactor surfaced these; the first
is a silent-permanent-loss bug:
- CRITICAL: the finished-job check ran AFTER the failure branch had
already reset s_hist_flush_ok, so a FAILED worker job read as ok and
got committed — advancing the durability watermark over records that
never reached disk. The completion check now runs strictly first, and
a failed job just drops its descriptor (data stays pending).
- A message deleted while its APPEND job was in flight resurrected on
reboot: segMarkSeqDirty skipped it (seq above the watermark = 'never
reaches disk') but the in-flight job then put it on disk. After a
committed append the ring's tombstones inside the newly-flushed range
now get their compaction marks.
- segCommitJob cleared compact_dirty unconditionally, erasing marks for
deletes that landed WHILE the compact job was writing (its snapshot
predates them). A redirty flag keeps the segment dirty for a
follow-up pass.
- The sync drain shared the worker's job descriptor, snapshot buffer
and req flag: a worker stalled past the 9 s idle-wait cap could read
a torn buffer, and the raised req let the worker steal and
double-execute a job the drain was running inline. segBuildJob is now
pure (fills a caller buffer + local SegJob); only the async path arms
the worker statics, and the drain owns a separate buffer.
- An in-session FAT32 format erased all segments while the table (and
watermark) still claimed them — old records permanently lost and the
next append targeted a missing file. The format path now runs the
same resync as a card swap (retable + re-land off-thread).
- uiSegAppendRecords with create=true used FILE_APPEND, so a stale
same-key file (crash residue) got a second header mid-file; create
now opens truncating.
Builds verified: all five SD-relevant touch envs.
Signed-off-by: Dan Vybiral <dan.vybiral@greencode.cz>
- About 'Chat store' shows the segment count + writer-maintained byte
total (zero file I/O at 1 Hz — the old size probe blocked on the FatFs
volume lock behind a sick write) and flags a pending migration.
- uiDataEnsureDirs() runs at boot (before the loader scans) and on all
three SD remount paths, so a fresh replacement card gets the segment
dir immediately.
- sdEnsureMeshcomodFolders (post-FAT32-format) now also recreates
/meshcomod and /meshcomod/msgs — an in-session format used to break
every chat/telemetry write with ENOENT until a reboot.
- The boot SPIFFS->SD adoption creates /meshcomod/msgs so flat-named
SPIFFS segments ('/msgs/seg_*.bin') copy onto the FAT card instead of
being silently skipped (missing parent dir).
- sdRestoreApply ('Copy internal data to SD') persists chat history
BEFORE the copy — it used to reboot without persisting, silently
dropping everything since the last lazy flush.
- Dead s_ui_data_resolved removed; stale writer comments updated.
Builds verified: all five SD-relevant touch envs.
Per .notes/segmented-store-plan.md phase 5.
Signed-off-by: Dan Vybiral <dan.vybiral@greencode.cz>
The flush pipeline now moves one bounded JOB at a time instead of the
whole ring: an append batch into the active segment (a few hundred
bytes per message burst) or a one-segment compaction rewritten from the
RAM ring. The loop task builds and COMMITS jobs (single table writer);
the core-1 hist_flush worker only executes the armed job and reports —
same busy/req protocol, same failure ladder (retry backoff, warning
toast, hist_sync_after sync-fallback), all ported verbatim.
- Durability watermark (s_seg_flushed_seq): ring records above it are
the pending-append backlog. Records deleted before ever flushing
advance it with no write. Job snapshot buffer is one segment (~80 KB
PSRAM) replacing the whole-ring 1.3 MB snapshot.
- markMsgsDirty clamps drop from 30 s/10 s (whole-ring-rewrite pacing)
to 2 s SD / 5 s SPIFFS — the hard-cut loss window shrinks ~15x.
- Deletes (message/thread/purge/web) mark owning segments compact-dirty;
eviction unlinks a segment only when ALL its records aged out (no
per-message rewrite churn; a partially-evicted boundary segment is
trimmed by the next boot's loader).
- saveMsgsToStorage is now the synchronous drain (shutdown/reboot/
delete flows/sync-fallback): it retries a failed migration, honors
resync, and never uses FILE_APPEND (a worker stalled past the 9 s
idle wait could hold an append handle on the same file — the active
segment is rewritten via the .tm2 namespace instead, rename-over
wins with each candidate internally consistent).
- flushHistorySoon (remount paths) now sets s_seg_resync: the segment
table is rebuilt from the ring, stale/foreign on-disk segments are
removed, and the full history re-lands one segment at a time
off-thread — replacing the old 'full-ring write fixes everything'
property without the >30 s UI freeze.
- Retired: uiWriteMsgsFile, uiHistWorkerFlush, the ring-sized snapshot.
Builds verified: all five SD-relevant touch envs.
Per .notes/segmented-store-plan.md phase 4.
Signed-off-by: Dan Vybiral <dan.vybiral@greencode.cz>
loadHistoryFromStorage now prefers segments over the split v6 file over
the legacy combined file. The segment loader reads oldest-first into the
ring, quarantines corrupt segments INDIVIDUALLY (one bad record used to
discard the entire history), truncates crash-ragged tails at the record
boundary, keeps the newest <= cap records when the disk set outgrows
this boot's ring (three-reversal rotation restores the linear-ring
invariant), reconciles the segment table (fully-dropped segments deleted,
the boundary segment marked compact-dirty), restores real seqs and seeds
the generator, and sweeps orphaned segment tmps.
When an old format supplied the ring, migrateRingToSegments() converts
it: segments written from the ring, every one read back and verified
(count + seq bounds) before the old msgs file is deleted; any failure
rolls the whole segment set back so the intact old file stays
authoritative next boot, and s_seg_store_ready stays false so the flush
scheduler (phase 4) retries the migration instead of writing segments
that would shadow it.
The companion message counter rides the threads-file header now (the
segments don't store it); the mesh re-asserts it per message anyway.
Builds all five envs (loader wired; writers wire in phase 4).
Per .notes/segmented-store-plan.md phase 3.
Signed-off-by: Dan Vybiral <dan.vybiral@greencode.cz>
The file-layer building blocks for the segmented chat store, no callers
yet: UiSegHeader/UiSegMsg on-disk format (v6 record + appended seq,
self-describing rec_size), segment naming (/msgs/seg_<first_seq>.bin,
real subdir on FAT backends, legal flat name on SPIFFS), uiDataEnsureDirs,
chunked internal-RAM record writer, active-segment append (header-first
on create, retry-once after ensuring dirs, ragged-tail crash semantics),
one-segment compact (tmp+rename with the .tmp/.tm2 dual-namespace
discipline, n==0 removes the file), retired-segment delete, validated
open + size-agnostic record reader (File::size()-free — the Tanmatsu FFat
metadata layer lies), and segment discovery (dir listing on FAT, flat
prefix scan on SPIFFS, orphaned-tmp sweep, sorted ascending).
All ops report through uiMsgsWriteResult/uiMsgsWriteFail so the About
diagnostics, failure toasts, sync-fallback and SD-wedge arbitration work
unchanged; new stage codes a/c/d/s join chatSaveFailText.
Builds all five envs (expected unused-function warnings until the
scheduler wires these in).
Per .notes/segmented-store-plan.md phase 2.
Signed-off-by: Dan Vybiral <dan.vybiral@greencode.cz>
Every ring record gets a monotonic uint32 seq at append time, and the
loader backfills seqs in chronological order for records loaded from the
pre-segment formats (which carry none on disk), seeding the generator
past them. seq is the segmented store's record key: segment membership
is a [first_seq, last_seq] range, stable across per-segment compaction.
Deliberately a dedicated counter (_ui_seq_next), NOT _msgcount: the
companion protocol can overwrite _msgcount (msgRead), so it cannot be a
unique key.
Builds verified: all five SD-relevant touch envs.
Signed-off-by: Dan Vybiral <dan.vybiral@greencode.cz>
Extend the channel top-row timestamp layout to every bubble-style
thread so DMs and room messages match channels.
Co-authored-by: Cursor <cursoragent@cursor.com>
EIO persisted on the core-1 flush task, which narrows the failure to
shared-SPI interleave rather than core-0 starvation alone: the one
always-working configuration is the LOOP TASK writing (reboot persist)
-- when it writes, nothing else interleaves radio traffic between the
SD transactions on the shared bus. (The pager port notes flagged
exactly this: 'keep an eye on SD-write + RX overlap during history
flush'.)
The flush now self-heals: after N consecutive failed background writes
it retries synchronously on the loop task (one UI hitch per flush while
degraded), and a single success flips it back to the async task. N is a
new setting (Settings -> General -> 'Chat save fallback', dropdown
Off/1/2/3/5/8, default 2; Off = background retries only). Stored as
TouchCfg v40 trailing field hist_sync_after with getter/setter.
Builds verified: all five SD-relevant touch envs.
Signed-off-by: Dan Vybiral <dan.vybiral@greencode.cz>
Make the shorter channel-bubble layout optional via a persisted
"Timestamp on top" switch (default ON), so classic footer layout remains available.
Co-authored-by: Cursor <cursoragent@cursor.com>
s_lock_wall_dsc is a static lv_img_dsc_t (stable address). LVGL's image
cache is keyed by that src pointer, not by dsc->data, so re-decoding a
custom lockscreen wallpaper into a fresh PSRAM buffer without first
invalidating the cache entry can leave LVGL drawing from a stale,
already-freed pointer on the next lockscreenShow(). The freed PSRAM is
commonly reclaimed soon after by the map's 128 KB tile buffers, so the
stale draw shows up as colorful RGB565 noise instead of the wallpaper --
exactly the symptom in #127. Only reboot 'fixed' it because the stale
LVGL cache entry otherwise persists indefinitely.
Same bug class already fixed for map tiles in freeMapTileSlot() (see
its comment); this applies the same lv_img_cache_invalidate_src() call
to the two places that free s_lock_wall: lockscreenShow() (re-decode)
and lockscreenHide() (teardown).
Fixes#127
Two failure signatures from on-device testing, both now explained and
closed:
- EIO ('card I/O error') on the periodic autosave while the identical
full-ring write succeeded from the reboot path: the flush rode the
core-0 tile_fetch worker -- the Wi-Fi core -- where the driver's
prio-23 tasks starve a prio-1 task for hundreds of ms. sd_diskio's
busy-waits measure WALL time, so timeouts expired while the task
simply wasn't scheduled, and healthy writes surfaced as EIO
(intermittently: the write sometimes won the timing, matching the
'succeeded after a few auto tries' observation). The flush now runs
on its own small task pinned to core 1 at the loop task's priority --
equal-priority time-slicing keeps the UI serviced during a write, and
chat persistence has no Wi-Fi affinity to lose.
- EBADF ('bad file number') after a shrink: FatFs FR_INVALID_OBJECT --
a handle whose volume was remounted underneath it. The file-manager
removal poll could probe + SD.end() while the flush held its open
temp file. All unmount paths now hold off under an in-flight flush
(the FM poll joins the sdHealthTick guard).
Builds verified: all five SD-relevant touch envs.
Signed-off-by: Dan Vybiral <dan.vybiral@greencode.cz>
Replaces the '@o e23' shorthand with a human-readable diagnosis
(chatSaveFailText): stage (open/header/write/rename) plus a plain-text
errno ('too many open files', 'card full', 'card I/O error', ...,
strerror fallback), e.g. 'open failed: too many open files (e23)'.
Shown as a second line on the About Chat-store section, on the repeated
autosave-failure alert, and on the reboot-time last-chance-save alert.
Diagnostic vocabulary stays English on purpose -- it is what ends up in
bug reports verbatim.
Builds verified: all five SD-relevant touch envs.
Signed-off-by: Dan Vybiral <dan.vybiral@greencode.cz>
Field evidence reframed the chat-save failures: a soft reboot wrote the
FULL 515 KB ring to the same card without trouble, while the periodic
worker autosave could not even create the file. That rules out a
size/bad-region card problem and points at contention: the SD VFS was
mounted with max_files=3 (set when the card only served the file
manager), but the card now serves telemetry log rewrites (2 handles),
battery log rewrites (2), the history flush, tile reads/writes, WAV
chimes and the About readout concurrently -- the flush's open("w") loses
that race with ENFILE while the card itself is fine, and at reboot
(everything quiesced) the same write sails through. All four mount
sites now use max_files=6 (~+1.7 KB while mounted).
To make the next such failure self-explaining on a device with no
readable serial, the About Chat-store line now includes the failure
stage and errno ('FAIL x3 @o e23'): o/h/b/r = open/header/body/rename;
errno 23/24 = VFS file table full, 28 = card full, 5 = card I/O error.
The header-write failure path also joins the failure accounting (it
returned early without counting before) and cleans up its temp file.
Builds verified: all five SD-relevant touch envs.
Signed-off-by: Dan Vybiral <dan.vybiral@greencode.cz>
Deleting messages tombstones them in place (thread[0]='\0'; the ring
cannot memmove — slot indexes are live UI handles), but the writer
emitted tombstones as full blank records: delete 1000+ messages and the
file never shrank, and every flush kept rewriting the high-water size.
The writer now skips tombstones (the file compacts to live records on
the next flush), and the loader accepts a legitimately empty header-only
file instead of quarantining it as corrupt.
Builds verified: all five SD-relevant touch envs.
Signed-off-by: Dan Vybiral <dan.vybiral@greencode.cz>
Field trace with the new Chat-store diagnostics (FAILING x1, then an
escalating UI freeze on every ~30 s retry) exposed three stall sources
around a card that mounts fine but times out on large writes:
- The About page's Chat-store readout opened the msgs file at 1 Hz on
the UI task; that open blocks on the FatFs volume lock behind the
worker's slow failing write, so just WATCHING the diagnostics froze
the UI. The file size is now cached, refreshed at most every 30 s and
never while a flush is in flight.
- The persist-after-remount ran the full ring write synchronously on
the UI task -- >30 s freeze on a write-sick card. Replaced with
flushHistorySoon(): arms an immediate flush without the min-delay
clamp, actual write stays on the core-0 worker.
- Retries against a hopelessly failing store ran every 30 s forever,
each costing seconds of bus stalls. After 5 consecutive failures the
retry backs off to 5 min; the RAM ring keeps everything meanwhile,
the on-screen warning already fired, and one success resets the
counter.
Builds verified: all five SD-relevant touch envs.
Signed-off-by: Dan Vybiral <dan.vybiral@greencode.cz>
Digging into 'messages still lost after a soft reboot': the loader is
sound (well-formed files load fully; a quarantine discards the whole
load, never a suffix) and the chat detail applies no time filter — so a
conversation missing only its recent messages means the ui_msgs file
itself is stale. Two ways that happens silently today:
- Persistent ring-write failures. uiWriteMsgsFile failing (full card,
FAT damage from hard cuts/card yanks, wedge) was 100% invisible: the
small threads file keeps succeeding, so the thread list shows fresh
last-message times while messages quietly stop persisting — exactly
the reported UX. Now every ring-write outcome is recorded
(uiMsgsWriteResult), repeated failures raise an on-screen warning, a
failure on the SD store stamps the SD health probe, and the About page
gets a 'Chat store' section showing the resolved backend (also exposes
the boot-with-unmountable-card trap where history silently goes to
internal flash), the msgs file size, and the last save result.
- The reboot-time persist had a hole and a corruption hazard. A worker
still busy past uiHistWaitWorkerIdle's 9 s cap left _msgs_dirty
cleared, so rebootDevice skipped the final sync save entirely; and
when the sync save DID overlap a stalled worker write, both truncated
the SAME temp file — interleaved garbage, and whichever rename landed
last installed a corrupt file the next boot quarantines (total
history loss). A still-busy worker now counts as unwritten data, the
synchronous writer uses its own temp (last-rename-wins, each candidate
internally consistent), a failed last-chance save shows a warning
before the reset, and boot reclaims orphaned temps.
Builds verified: all five SD-relevant touch envs.
Signed-off-by: Dan Vybiral <dan.vybiral@greencode.cz>
Message-loss analysis from testing: messages live in a RAM ring and the
SD file is written lazily (>=30 s coalesce for the SD-sized ring), with a
synchronous persist only on SOFT reboot/shutdown. A hard power cut
(power switch, reset button, reflash) therefore loses everything since
the last SUCCESSFUL flush -- and while the card is out, flushes fail for
the whole outage, so the loss window grows to the entire card-out span
plus up to 30 s after remount. This also explains the confusing UX where
the thread list still shows the correct last-message time while the
conversation lacks the recent messages: thread metadata
(ui_threads_v1.bin, ~1.5 s coalesce) and the message ring
(ui_msgs_v1.bin, >=30 s) are separate files with different freshness.
All three remount paths (wedge recovery, reinsert watch, FM insert poll)
now call persistHistoryNow() right after a successful mount, so the
entire RAM ring -- including every message received while the card was
out -- lands on the card within the same second it returns. They also
SD.mkdir(/meshcomod) first: a fresh replacement card has no data root,
and without it every history flush failed silently forever.
Builds verified: all five SD-relevant touch envs.
Signed-off-by: Dan Vybiral <dan.vybiral@greencode.cz>
On-device test: after a failed remount ('SD card lost', SD.end() run) a
REINSERTED card looked recovered -- battery stats quietly fall back to
SPIFFS -- but chat history kept writing into the unregistered SD VFS and
silently failing: new messages were never persisted again until reboot.
Nothing outside the file manager ever re-attempted a mount, and
sdHealthTick early-returned while unmounted.
- sdHealthTick now runs a reinsert watch while unmounted: one light
single-attempt SD.begin (no ladder; an empty slot fails fast) every
30 s while awake. A healthy warm card mounts on the first 4 MHz
attempt -> 'SD card remounted', and the armed history flush retries
then land (messages received while the card was out are still in the
RAM ring and flush too). The FM poll / SD-row tap keep the full ladder
for stubborn cards; the watch skips while the FM is open or the pager
card-detect says the slot is empty.
- Hot paths no longer walk the mount ladder at all: map tile lookups
(loadTileJpeg / tileExistsAt) check s_sd_mounted only -- with the card
out, the post-backoff re-ladder read as recurring multi-second UI
freezes on the map. The notify-task WAV open also drops its
fmSdTryMount(): laddering SD.end()/SD.begin() from a second task races
the loop task's SD use. Mounting is owned by boot adoption, the
reinsert watch, the FM paths, and the SD-tiles toggle.
Builds verified: all five SD-relevant touch envs.
Signed-off-by: Dan Vybiral <dan.vybiral@greencode.cz>
Adds a self-contained "Airtime" app that measures how busy the LoRa
channel actually is, so lost-message reports can be checked against real
data instead of assumed congestion.
It separates the two things that get conflated:
- Is the spectrum busy? channel utilization % = (RX airtime + TX
airtime) / wall-clock, sampled at 1 Hz, plotted over time, with a
session average and a congestion-threshold reference line.
- Are packets actually being lost at the radio? CRC errors
(collisions), packets heard but read too late (getRxEvents -
packetsRecv - recvErrors), and RX-queue drops.
A plain verdict combines both axes so a field tester can report
a concrete conclusion. Reads existing counters via the_mesh (Dispatcher
airtime/packet accumulators) and radio_driver (RadioLib wrapper stats);
no MeshCore core changes.
The 30 s background liveness probe now skips while touchSleep has the
device parked (screen off, loop throttled): nobody is watching for the
toast, and the periodic SEND_STATUS would keep the card from settling
into its own power-down. A card yanked while asleep is caught by the
first background probe after wake. The failure-driven probe stays active
during sleep on purpose -- scheduled telemetry keeps logging to SD while
parked, and a wedge there must still recover.
Builds verified: all five SD-relevant touch envs.
Signed-off-by: Dan Vybiral <dan.vybiral@greencode.cz>
On-device testing of the previous commit showed three gaps:
- No 'SD card lost' when the card is pulled while nothing writes:
detection was purely failure-driven, and the battery path never stamps
(battLogFs's SD.exists fails -> silently falls back to SPIFFS, which is
exactly the 'battery stats fail to load' symptom). sdHealthTick now
also runs a 30 s background probe (one sub-ms SEND_STATUS on a healthy
card), so idle removal is noticed and toasted; battLogFs stamps when
cardType() claims present but real I/O fails.
- Map froze the whole UI when the card was pulled with the map open:
every tile lookup ran up to five SD ops, each riding a full sd_diskio
timeout + re-init cycle, serializing the render pass into a
multi-minute freeze. Tile paths (loadTileJpeg / tileExistsAt / the
SD-backed tile cache) now short-circuit while a dead card is suspected,
and sdReadFailedCardDead() distinguishes a missing tile (healthy card:
cheap, silent) from a dead card (stamp + bail) on a failed read.
- Even after an unmount, hot paths calling fmSdTryMount() directly would
re-walk the multi-second mount ladder per call. fmSdTryMount() now
fast-fails inside the post-failure backoff window; the explicit user
retry (tap the SD row in the file manager) clears the backoff first.
Also renames sdNoteWriteFailure -> sdNoteIoFailure (it now covers reads).
Builds verified: LilyGo_TDeck_companion_radio_touch,
heltec_v4_tft_companion_radio_usb_tcp_touch,
heltec_v4_r8_tft_companion_radio_usb_tcp_touch,
tlora_pager_sx1262_companion_radio_touch,
ThinkNode_M9_companion_radio_touch.
Signed-off-by: Dan Vybiral <dan.vybiral@greencode.cz>
A runtime SD failure is a one-way door in arduino-esp32: one timed-out
command latches STA_NOINIT in sd_diskio (and a data-phase error leaves
every later op failing FR_DISK_ERR without even setting NOINIT, so FatFs
never re-inits the card). Meanwhile SD.cardType() returns the type CACHED
at mount -- only an explicit SD.end() ever makes it CARD_NONE again -- so
every 'cardType() != CARD_NONE' gate keeps passing, s_sd_mounted stays
true, fmSdTryMount() no-ops, and the file-manager removal poll (which
compared against CARD_NONE) could literally never fire. Net effect: after
one bad moment on the bus (heavy clustered writes -- scheduled telemetry
log rewrites, battery log, core-0 history flush / tile writes -- on the
shared rail) the card silently plays dead until a reboot power-cycles it.
Reported as: SD behaves as not present until the next reboot.
Fix, three parts:
- sdNoteWriteFailure(): any writer on any task (loop, core-0 worker)
stamps a volatile flag when an SD write open fails while the card is
supposedly mounted (telemetry log, battery log, wardrive CSV, chat
history/threads store, tile cache).
- sdHealthTick() in UITask:➿ verifies the stamp with real I/O --
sdProbeAlive() opens the root dir, whose opendir runs f_opendir ->
disk_status = a real SEND_STATUS transaction (and one re-init handshake
on a NOINIT'd card, so a recoverable card self-heals right there). If
the card stopped answering, drop the mount and re-run the full mount
ladder: exactly what the next reboot would have done, minus the reboot.
Success => 'SD card remounted' toast and every feature resumes
transparently (same VFS mountpoint, cached fs pointers stay valid).
Failure => honest 'SD card lost' with cardType() now reading CARD_NONE,
so features degrade gracefully and the existing remount paths (FM poll,
row tap, feature use) keep retrying with the usual backoff. Held off
while the core-0 worker owns the card (hist flush / FAT scan / firmware
download) and while a format is pending; probes rate-limited to 1/5 s.
- The FM removal poll now probes with real I/O instead of the dead
'SD.cardType() == CARD_NONE' check, so physically yanking the card is
finally detected too.
Builds verified: LilyGo_TDeck_companion_radio_touch,
heltec_v4_tft_companion_radio_usb_tcp_touch,
heltec_v4_r8_tft_companion_radio_usb_tcp_touch,
tlora_pager_sx1262_companion_radio_touch,
ThinkNode_M9_companion_radio_touch.
Signed-off-by: Dan Vybiral <dan.vybiral@greencode.cz>
OTA_BIN_NAME only branched T-Deck vs everything-else -> wadamesh-heltec-v4-tft, so
V4-R8, ThinkNode M9, RAK Tap V2, both T-LoRa Pagers and Attaky would self-update by
downloading the V4-TFT bin -- flashing the wrong firmware onto the wrong board.
All are dual-slot OTA-capable (CAP_OTA=1 + app0/app1 partitions + FIRMWARE_OTA_ENV),
so map each to its own release bin. Pager splits on USE_LR1121. Tanmatsu stays out
(AppFS/launcher, CAP_OTA=0, never reaches this file).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The update-channel UI (the "Get test builds (beta)" toggle + the Install-update
button) is gated on a non-empty FIRMWARE_OTA_ENV. That is set per-env in
platformio.ini for the 8 S3 boards but was never set for the P4 IDF build, so the
whole update UI was hidden — P4 testers had no way to opt into beta and stayed
stuck reading the stable channel (the beta_49 "no update, 48 up to date" report).
- tdisplay_p4 CMakeLists: define FIRMWARE_OTA_ENV so the UI is shown.
- UITask.cpp: add a P4 branch to OTA_BIN_NAME so the download URL points at
wadamesh-tdisplay-p4[-lcd].bin, not the V4-TFT bin. AMOLED vs LCD split on
HAS_TDP4_LCD.
The C6 WiFi shim already routes HTTPClient/Update.h and the P4 partition table has
dual ota_0/ota_1 slots, so the OTA path itself was wired; this only unhides and
correctly targets it. Builds green on P4 (AMOLED) + V4-TFT. On-device toggle/OTA
test still pending (P4 dropped off USB, needs a power cycle).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Backspace tap now peeks the lock screen (reveal without unlocking),
mirroring the T-Deck's trackball-click behavior; holding Backspace still
fully unlocks.
- New native 480x222 lock wallpaper asset so the icon/wordmark aren't
crushed by the shared 320x240 art's cover-fill on this wide/short panel.
- Lock text forced to pure white and repositioned for this layout: clock
top-left over the icon, unread count + "Screen locked" stacked
bottom-right directly above the unlock hint.
- Fixed lock text flipping from white to black a moment after every reveal:
the lock overlay is CLICKABLE (to absorb taps), so the keyboard/encoder
nav rebuild was collecting it as a focus target -- navFocusCb's focus
highlight then reverse-videoed it, flipping every label's text to dark.
Tagged the overlay NAV_SKIP so it's never a nav target and never inverted.
- serviceLockscreen() now runs in the pager's loop branch too, so the lock
clock/unread badge keep refreshing after the first reveal instead of
freezing there.
All changes gated to TLORA_PAGER; no other board's behavior changes.
Signed-off-by: Tesso M Costa <tesso.martins@gmail.com>
P4-focused beta. Non-P4 boards only pick up minor shared exit-trap code, nothing user-facing.
- Snake + full-screen apps (Monitor/Spectrum/Discover/Web mirror/Advert) exit on status-bar tap
- Screen timeout / standby / burn-in dimming wired for the AMOLED (setBrightness path)
- UI scale pinned to 100% on the P4 (text overflow above 100%)
- No hard-lock on screen-off (P4 has no lock screen yet)
- GPS: L76K detect at 9600 baud, skip boot-detect race, 4096 RX buffer
- Control center dropdown taller on tall screens so the GPS line clears the toggle buttons
- P4 LCD SKU: HI8561 backlight PWM on GPIO51 (fixes the black screen)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- New Discover app: active NODE_DISCOVER sweep -> signal-ranked nearby list, tap-to-add-contact, GPS wardriving log (SD CSV) + signal-coloured coverage overlay on the map. Board-agnostic.
- T-Display P4 TFT-LCD (HI8561) variant support (WADA_P4_LCD build; AMOLED bin untouched).
- Wire the Attaky Core board (#158, @attakygit) into the release matrix + web flasher.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* attaky_mesh_series: add board variant with FT6636 touch and landscape UI
* attaky_mesh_series: P3 battery (MAX17048) + GPS + shared-I2C mutex
- AttakySharedI2C: one Wire.begin() + a mutex serializing the shared board
I2C bus (SDA=2/SCL=1) across the FT6636 touch poll (core 0), the MAX17048
fuel gauge (core 1), and the future AW9523 keyboard. Arduino's TwoWire
buffers are not concurrency-safe.
- board.begin() owns the authoritative bus init; the touch driver now takes
the lock around every transaction instead of its own Wire.begin().
- getBattMilliVolts(): real MAX17048 VCELL read (78.125 uV/LSB -> mV = raw*5/64),
2 s cache + lock, replacing the 4000 mV stub. Upstream maps mV -> % itself.
Hardware-verified ~4058 mV.
- GPS: rename the dead GPS_BAUD flag to GPS_BAUD_RATE (the core reads that; 9600
only worked via the #else default). Pins verified on hardware: ESP RX = IO17
receives ATGM336H NMEA; MeshCore's module-centric PIN_GPS_TX=17 maps correctly.
* attaky_mesh_series: Control_Keyboard_(Bottom) dual-AW9523 keyboard
Add the Control_Keyboard_(Bottom)_1.0 module as an ADDITIVE hardware
keyboard on the touch board. Two AW9523 expanders (0x5A left / 0x5B
right) each scan a 5x5 matrix over the shared board I2C bus, serialized
through the AttakySharedI2C mutex alongside the FT6636 touch poll and the
MAX17048 fuel gauge.
The driver emits final bytes (printable ASCII, backspace, enter, space)
with momentary Shift resolved to uppercase in-driver. UITask injects them
into the focused textarea exactly like the web-mirror keystroke path, so
CAP_KEYBOARD stays 0 and the on-screen keyboard is not suppressed (touch
typing does not regress). Focus-gated: the matrices are only scanned
while an editable field is bound, so a key has zero side effect when
nothing is focused.
Hardware-verified, both halves: q/w/e/r/t (0x5A) + y (0x5B), Shift
uppercase, backspace, space, enter; on-screen keyboard intact.
* attaky_mesh_series: use reliable 115200 upload speed
* attaky_mesh_series: use core default GPS baud
* attaky_mesh_series: restore GPS_BAUD setting
* attaky_mesh_series: rename PlatformIO environment
* pager: keep encoder focus inside the chat message list, remap composer nav
The rotary encoder's plain turn stepped LVGL's focus group directly, which
only ever contains the currently-materialized bubbles (the message list is
virtualized; loading more history is scroll-driven, not focus-driven). So
turning past the first/last *loaded* bubble escaped the list onto the
header/composer even with plenty of un-loaded history in that direction.
A turn at that edge now scrolls the list instead when more messages exist
(reusing the existing free-scroll helper), which triggers virtualization to
load the next batch; focus only leaves the list at the true oldest/newest
message. Fast turning folds into a pending target so it doesn't escape
mid-load, and the render's focus re-aim now runs synchronously so the
transient auto-focus LVGL assigns a deleted row's neighbor never paints.
Also removes the two touch-only jump-to-oldest/latest arrows on this board
(no touchscreen to tap them) and remaps the chat's key bindings now that
plain turns stay in the list:
- Orange/Alt tap: jump to the newest message + focus the composer (was
Backspace's job).
- Backspace tap: jump to the first unread message below the "NEW ----"
divider, or the newest message if nothing's unread.
- Composer row edges: Send+NEXT reaches the channel-settings gear (falls
back to the newest bubble in a DM, which has no gear); the quick-reply
chip+PREV goes straight to the newest bubble.
TLORA_PAGER_SHORTCUTS.md documents the new chat-screen behavior.
Signed-off-by: Tesso M Costa <tesso.martins@gmail.com>
* fix scroll
---------
Signed-off-by: Tesso M Costa <tesso.martins@gmail.com>
- Contact-loss fix (microSD devices): atomic temp+swap saveContacts, unified robust
boot SD-mount ladder, and a storage-truth line in the Store-on-SD setting.
- Heltec V4-R8: route MISO for the shared FSPI bus so the microSD mounts; wake-from-sleep
freeze fixed (panel sleep goes through LovyanGFX's own bus, not the HSPI shim).
- Do Not Disturb (originally PR #156 by Tesso Codemonkey / @codemonkeybr): scheduled
sound-silencing window in Settings > Sound + a status-bar bell.
- Control-center redesign: round icon-only chips that spread to fill each row (T-Deck 2x4,
V4 portrait 2 rows), frosted-glass panel, real lock glyph + dynamic bell/bell-slash DND
chip + keyboard icon+word chip (new cc_icons_16 FontAwesome font), per-chip confirmation
toasts, press-grow clip fix, and a V4-portrait status-bar BLE/clock overlap fix.
- LilyGo T-Pager: map tile cache falls back to the microSD under the launcher (fixes the
"storage error / reflash the tiles partition" dead-end).
Co-Authored-By: Tesso Codemonkey <798026+codemonkeybr@users.noreply.github.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Heltec V4-R8: the CHSC6x touch ran on a 2nd I2C controller (Wire1) bound to the
SAME pads (17/18) as the board Wire (RTC + sensors), driven from two cores — a
classic SDA-stuck-low wedge = the tester's random hard-freezes-needing-reset.
Touch now shares the one board Wire (V4CAP_WIRE selector), so arduino-esp32's
per-instance TwoWire mutex serialises all access, the proven T-Deck pattern.
- Wi-Fi settings: the join/details/hidden sheet floats on the base layer above the
settings page; navMaybeRebuild only recognised the settings sheet + chat + top
layer as focus roots, so on no-touch/keypad-nav boards (Tanmatsu) its SSID/
password fields + Join button were never collected into the focus group and
couldn't be selected. Add s_wifi_sheet as a nav focus root (priority over the
settings sheet beneath it).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
