wada.mesh: read-only contacts/rx_log/stats/self via thin UITask bridges
(no mesh types cross into the host TU). wada.net.http_get: async on the
existing net worker, http-only, 32 KB cap, one in flight per app, callback
delivered on the UI thread under the guarded pcall. RF Monitor and Airtime
rewritten as Lua catalog apps (live on the VPS with snake); their native
versions deliberately stay compiled in for beta_59 until the ports are
field-proven. lib/lua joined both IDF component builds (P4 verified on
RISC-V). Catalog source tracked at deploy/apps/.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Follow-up to 3bf56cd, driven by the (correct) challenge that "SD writes upset
the panel electrically" could not be the whole story when tile writes never
flash and other P4 firmwares use their cards freely. The completed on-device
matrix now separates two distinct mechanisms:
M1 An SD write EXECUTING on core 1 (where display.begin() allocated the DSI
frame-restart ISR, and where the Arduino loop task runs) masks that
core's interrupts inside the SDMMC/FATFS critical sections long enough
to drop a display frame. Proven: 80 contact saves to SD from core 1
flash every time; the identical 80 saves hopped to core 0 never flash.
NOT electrical -- the challenge was right.
M2 An SD write CONCURRENT with live UI rendering still flashes even from
core 0 (manual add/delete flows pump LVGL while the hopped write runs;
the write-open trace shows those runs performed only correct core-0
writes and flashed anyway). The same manual flow with the card mounted
but ZERO SD writes (internal LittleFS store) is clean.
Shipped design therefore stays: hot store on internal LittleFS (immune to both
mechanisms, verified across every flow), SD for tiles/bulk. Tiles' long-standing
immunity is consistent: brief core-0 writes vs the store's long ones.
New in this commit, kept for good:
- p4StorageCall (DataStore.cpp): a core-0 storage task; every hot write entry
(saveContacts/saveChannels/savePrefs/blobs, history segs, threads file) hops
onto it, blocking the caller -- semantics unchanged, execution core moved.
With the internal store this is still wanted hygiene: no storage write ever
runs on the render core again.
- WADA_SD_STORE_TEST build gate: puts the store back on the SD card for future
M2 investigation (known to still flash on manual flows; do not ship).
- TDP4_POKE_TRACE now also logs every write-open with its issuing core ([SDW]),
which is what pinned M2.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Root cause, established by on-device elimination over one long session: SD-card
WRITE bursts electrically disturb the T-Display P4's AMOLED -- the whole-screen
blue flash of #167. The display stack itself measured clean at every layer while
flashes were visible: framebuffer content (flush tracer: every band black), DSI
bridge underrun interrupt (never fired), DSI host protocol errors incl. the host
payload-buffer underflow (never latched), DCS/brightness traffic and XL9535
expander writes (silent), disp on/off (never). The flash survived every digital
derate -- 4->1-bit SD bus, 20->10 MHz SD clock, 1000->750 Mbps DSI lane rate --
on every card tried, at any brightness (including near-zero, i.e. the panel is
being upset at the hardware level, not showing content), yet vanished the moment
the card was removed or left unwritten. The card's rail is switched off the same
3V3 domain as the panel (XL9535 SD_EN), so its program-current spikes reach the
panel and no firmware setting can filter them. Streaming tile writes never
flashed; rapid open-write-close store bursts always did. Meck-P4 not flashing
proves nothing about the board: it never writes the SD.
Fix: the hot store (contacts/channels/identity/blobs/prefs-file/chat history)
lives on the INTERNAL 'storage' partition; the SD keeps bulk, gentle writers
(map tiles, backups). The internal partition is mounted as LittleFS, not FAT:
the P4's FAT layer has documented broken metadata and the v3 migration proved
its writes are no better (a 102 KB contacts3 "copied" onto it read back as 0 B;
on LittleFS the same copy reads back byte-exact and loads all 689 contacts).
esp_littlefs looks partitions up by label with SUBTYPE_ANY, so the ex-FAT
partition converts in place on first boot (formatOnFail) -- no partition-table
change, OTA-safe.
One-time v4 migration (lessons v1-v3 encoded): substance-gated (only from a
card whose contacts3 beats the internal one -- a fresh test card once donated a
blank store), whitelist-only copy (v2 filled the partition with junk before
contacts3 landed), both historical card layouts probed (root vs /meshcomod),
marker written only after the copy, card never written (it remains the
rollback).
Also in this change, kept as opt-in diagnostics (dormant without their env
gates): WADA_P4_POKE_TRACE (DCS/expander/disp-on-off tracing + a DSI host
error-register poller), WADA_SD_KHZ / WADA_SD_1BIT (SD primary-rung derates),
WADA_DSI_MBPS (lane-rate override; default back at the vendor 1000 -- the 750
experiment changed nothing and is documented in the driver).
Verified on device: no flash on the contact-churn trigger with the store
internal (the previously 100%-reproducing trigger), 689/689 contacts after
migration, tiles still on SD.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Root-cause work, not another guess. The esp_lcd DSI driver's own ISR says it:
// when an underrun happens, the LCD display may already becomes blue
ESP_DRAM_LOGE(TAG, "can't fetch data from external memory fast enough, ...")
A DSI bridge FIFO underrun displays as exactly one blue frame -- the reported
whole-screen split-second blue flash -- and the driver logs an error line every
time it happens. Every serial capture until now grepped for tile/storage/trace
patterns that this line does not match, so it was never looked for.
The esp-iot-solution MIPI-DSI guide prescribes, for this exact error: L2 cache
256 KB, 128 B cache lines, and perf optimization (plus SPIRAM 200M + XIP from
PSRAM, already set). Applied all three for this board only:
- build.sh: sdkconfig fragment order general;wadamesh;tdisplay_p4 so the BOARD
fragment wins over the shared ones (it was last-loser before; the shared
wadamesh fragment's -Os, needed for the Tanmatsu AppFS ceiling, was
unoverridable). No net config change from the reorder itself: the board
fragment previously duplicated no shared keys.
- sdkconfigs/tdisplay_p4: COMPILER_OPTIMIZATION_PERF, CACHE_L2_CACHE_256KB,
CACHE_L2_CACHE_LINE_128B. App 2.99 -> 3.35 MB of the 4 MB slot.
- main/CMakeLists.txt: -O2's deeper analysis trips new warnings in vendored
components (esp-modbus etc.); same keep-visible-never-fatal policy as the
existing strips.
Verification harness (scratchpad, not shipped): the companion protocol driven
over the USB console (contact add/remove = the 100% reported trigger, pure
local DB ops, no LoRa TX) while the same session captures every console line.
Since the driver logs each underrun unconditionally, one watched run now gives
a decisive verdict whichever way it goes.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
None of the three fixed the flash, and each carried a cost:
- prefs-on-SD (1bc0c7a) also moved main.cpp onto a different set of headers than
it had been compiling against. SdNvsPrefs.h's own comment warns that exact
mismatch previously caused a free()-of-garbage bootloop, and a map tile
regression showed up on these builds. Not worth the risk for no benefit.
- LVGL bands 24 -> 64 (1e6e121): a full-screen repaint went from ~26 transfers
to ~10 with no change on device, so burst count is not the mechanism.
- XIP-off (0695ca6): no change on device, so PSRAM bandwidth starving the DSI is
not the mechanism either.
Back to the configuration where map tiles were confirmed working. Kept the two
changes that stand on their own: the framebuffer is blanked before the panel is
switched on (it was streaming uninitialised PSRAM to the screen), and the
WADA_P4_FLUSH_TRACE build gate.
Standing evidence for whoever picks this up: the tracer shows every large band
LVGL pushes is 0x0000 black, including through the boot window where the flash
occurs, so the colour is never in the framebuffer and the UI layer is excluded.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Ran the flush tracer instead of theorising further. Result is decisive: every
large uniform band LVGL pushes to the panel is rgb565=0x0000 (black), across the
whole boot window where the flash occurs. LVGL never paints the flash colour, so
the panel is displaying something that was never written to the framebuffer.
That excludes the entire UI layer, and with it the redraw/repaint theories.
Adds WADA_P4_FLUSH_TRACE=1 as a build gate so the tracer can be run on demand
without shipping it (it slows the flush path, so never confirm a fix with it on).
With the UI excluded the remaining candidate is the DSI starving. The panel
streams ~84 MB/s out of PSRAM continuously, and XIP put every instruction fetch
and rodata read on that same bus. The triggers are all bursts of code and data
churn (rebuilding the contact/chat list); panning the map reuses a warm tile
cache and never flashes. Turning XIP off leaves the PSRAM bus essentially to the
display. Experiment, not a claimed fix.
Also supersedes the burst-count theory from 1e6e121: widening LVGL bands from 24
to 64 lines (26 transfers -> 10 for a full repaint) changed nothing on device.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The T-Display P4 never called SdNvsPrefs::useFile, so although contacts and chat
history correctly moved to the SD card, every settings/thread-state write still
landed in internal-flash NVS. On this board that is not just wear: the DSI panel
streams its framebuffer continuously out of PSRAM, and an internal-flash write
visibly disturbs it -- the whole-screen flash reported as #167. The trigger list
matches exactly: a message arriving, opening a chat with unread messages, and
deleting a contact (which writes repeatedly, hence flickering for the whole
operation). Map panning writes tiles to SD continuously and never flickers,
which is what pinned it down.
Only when a card is mounted. With no card there is nowhere better to put prefs,
and moving them from NVS to the internal FAT partition would trade one flash
write for another while needlessly relocating existing users' settings. Old NVS
values still load and migrate on their next save, so this is transparent.
Fixing it also required an include-path fix. The vendored core component
publishes core/src publicly and still carries OLD copies of the headers whose
.cpp files it deliberately excludes because our src/ ships them. main.cpp's
angle includes therefore resolved to the stale core headers while the linker
bound the new objects, so anything added since the core tag looked like it did
not exist (SdNvsPrefs::useFile, touchPrefsReload). Putting the repo's src/ ahead
of the core dir in main's INCLUDE_DIRS makes main.cpp and the app layer agree.
Verified on device: "[PREFS] backend = FILE /meshcomod".
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Three independent 100%-reproducible triggers finally pinned this down, and they
all turn out to be the same thing: a write to flash.
- every received message -> chat-history write
- opening a channel that has unread messages -> mark-read write
("if you open them back up and there aren't unread messages, there's no flash")
- deleting a contact -> the chunked bulk delete writes repeatedly, and
it flickers CONTINUOUSLY for the whole operation
Mechanism. This panel is a MIPI-DSI DPI/video one: its single ~1.4 MB framebuffer
can only live in PSRAM and the DSI DMA streams it to the glass continuously. PSRAM
sits behind the same cache as flash, and a flash write has to DISABLE that cache
while it runs. For the duration of the write the DMA cannot fetch the framebuffer,
the panel underruns, and a frame comes out as a whole-screen colour flash. Nothing
to do with what is being drawn, which is why it appeared on every screen, on every
board revision, and from the very start of the port.
Fix: CONFIG_SPIRAM_XIP_FROM_PSRAM=y. Code and constants are relocated into PSRAM
at startup so the cache no longer has to be dropped for a flash write. The SoC
supports it (CONFIG_SOC_SPIRAM_XIP_SUPPORTED=y) and IDF ships the same pairing as
a P4 reference config (esp_mm/test_apps/mm/sdkconfig.ci.xip_psram_esp32p4). It
pulls in SPIRAM_FETCH_INSTRUCTIONS / SPIRAM_RODATA / SPIRAM_FLASH_LOAD_TO_PSRAM.
This also explains why 7aaff80 (moving the flush scratch out of PSRAM) did not
help: that reduced PSRAM BANDWIDTH contention, but the cache is disabled outright
during a flash write, so bandwidth was never the constraint. That commit is still
worth keeping on its own merits, but it was not the cause.
Also restores CONFIG_ESP_HOSTED_RESET_SLAVE_USING_CALLBACK and its SDIO twin to
this fragment. They existed only in the GENERATED sdkconfig, so regenerating it
for the XIP change silently dropped them and esp-hosted fell back to resetting the
slave over GPIO 54 — a line this board does not have (C6_EN is on the XL9535, so
it can only be driven by callback). Inert today because TDP4_C6_READY is 0, but
the config is now reproducible from its fragments instead of depending on an
untracked file nobody could regenerate.
Boots clean with XIP enabled: display up, GPS probe reading, only rst:0x1
(POWERON). Needs field confirmation against the contact-delete trigger, which is
the reliable one.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Root cause of "GPS on the P4 stays on searching forever", reported repeatedly and
chased through pins, baud, wake polarity, antennas and a comparison against the
Meck-P4 port. None of those were wrong. The loop simply never called
sensors.loop().
Every S3 board calls it once per iteration (src/main.cpp: STALL_SCOPE("sensors",
sensors.loop())). That file is excluded from this build and the P4 has its own
main loop, which is where the call went missing during the port: sensors.begin()
ran at boot and nothing ever pumped the manager again.
sensors.loop() is what drains the GPS UART into MicroNMEA. Without it the parser
never receives a byte, so isValid() is false forever and the UI shows "searching"
regardless of signal. It also starves the telemetry sensors and the GPS->RTC time
sync that hang off the same manager.
The evidence lines up exactly:
- a T-Deck with the same L76K fixes almost instantly, the P4 never does;
- 15 minutes under clear sky changed nothing, because no amount of signal helps
a parser that is never run;
- and a raw UART probe showed the MODULE reaching a valid 3D fix
($GNGGA,...,1,05,2.2 at HDOP 2.2, plus $GPTXT ANTENNA OK) that the firmware
had no idea about. That reading came from the probe, not the provider, which
is precisely why it looked like the GPS "worked" while the device disagreed.
Builds clean and boots clean on the P4.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Follow-up to 6430866. Two changes.
Print one example of each distinct NMEA sentence type, not just the first line.
A single GxGSV is not enough to judge satellite visibility: the first boot showed
"$GLGSV,1,1,00" and I read that as zero satellites, but GLGSV is GLONASS ONLY and
says nothing about GPS, Galileo or BeiDou. The full set showed 10 GPS + 5 GLONASS
in view with SNR 33/35/23, plus "$GPTXT,...,ANTENNA OK" from the module's own
antenna check, i.e. a healthy receiver mid-acquisition rather than a fault.
Shorten the drain from 3s to 1.2s. It runs on EVERY boot and it consumes bytes
the NMEA parser would otherwise see, so it only needs to be long enough to catch
one of each sentence type (the L76K repeats the full set about once a second).
1.2s still captured all nine types on the verification run.
Confirmed working on hardware: a later boot reported
$GNGGA,073620.000,5050.80018,N,00452.95932,E,1,05,2.2,36.0,M
$GNGSA,A,3,08,22,32,...,2.2
i.e. a valid 3D fix on 5 satellites at HDOP 2.2, down from no fix / HDOP 25.5 a
few minutes earlier. The GPS path was never broken; the old diagnostic just could
not tell anyone that, because it reported a flag ENV_SKIP_GPS_DETECT forces true.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The line here claimed "[GPS] L76K DETECTED (NMEA on GPIO22)" but never looked at
the UART. It reported whether the "gps" SETTING had been registered, and on this
board ENV_SKIP_GPS_DETECT forces that true unconditionally (we set it because the
P4's heavy boot misses the core's 1 s detect window, and a false negative hides
the GPS toggle entirely). So it printed DETECTED on every boot whether or not a
single byte had ever arrived, and it survived several rounds of "GPS still isn't
working" while proving nothing.
Replaced with a real 3 s drain of Serial1 that reports byte count, complete NMEA
sentence count, the configured baud and RX pin, and the first sentence seen. That
splits the two cases that were being conflated: bytes arriving means pins, rails,
wake and baud are all correct and any stuck "acquiring" is a FIX problem (antenna
or sky view); zero bytes means the link itself is dead.
First run on hardware answered the open question immediately:
[GPS] UART probe: 1985 bytes, 48 lines in 3s @ 9600 baud, module TX -> GPIO22
[GPS] first sentence: $GLGSV,1,1,00,0*79
1985 bytes of well-formed NMEA proves the link is good, and GLGSV ",00," is zero
satellites in view, i.e. an antenna/sky-view issue and not firmware.
For the record, the config was already identical to Meck-P4's working port
(components/meshcore/variant.h: PIN_GPS_TX 22 / PIN_GPS_RX 23, wake on XL9535
IO11), and their L76K driver confirms both our other values: begin() defaults to
9600 baud, and sleep(false) drives the wake pin HIGH.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The update-channel UI (the "Get test builds (beta)" toggle + the Install-update
button) is gated on a non-empty FIRMWARE_OTA_ENV. That is set per-env in
platformio.ini for the 8 S3 boards but was never set for the P4 IDF build, so the
whole update UI was hidden — P4 testers had no way to opt into beta and stayed
stuck reading the stable channel (the beta_49 "no update, 48 up to date" report).
- tdisplay_p4 CMakeLists: define FIRMWARE_OTA_ENV so the UI is shown.
- UITask.cpp: add a P4 branch to OTA_BIN_NAME so the download URL points at
wadamesh-tdisplay-p4[-lcd].bin, not the V4-TFT bin. AMOLED vs LCD split on
HAS_TDP4_LCD.
The C6 WiFi shim already routes HTTPClient/Update.h and the P4 partition table has
dual ota_0/ota_1 slots, so the OTA path itself was wired; this only unhides and
correctly targets it. Builds green on P4 (AMOLED) + V4-TFT. On-device toggle/OTA
test still pending (P4 dropped off USB, needs a power cycle).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
P4-focused beta. Non-P4 boards only pick up minor shared exit-trap code, nothing user-facing.
- Snake + full-screen apps (Monitor/Spectrum/Discover/Web mirror/Advert) exit on status-bar tap
- Screen timeout / standby / burn-in dimming wired for the AMOLED (setBrightness path)
- UI scale pinned to 100% on the P4 (text overflow above 100%)
- No hard-lock on screen-off (P4 has no lock screen yet)
- GPS: L76K detect at 9600 baud, skip boot-detect race, 4096 RX buffer
- Control center dropdown taller on tall screens so the GPS line clears the toggle buttons
- P4 LCD SKU: HI8561 backlight PWM on GPIO51 (fixes the black screen)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- New Discover app: active NODE_DISCOVER sweep -> signal-ranked nearby list, tap-to-add-contact, GPS wardriving log (SD CSV) + signal-coloured coverage overlay on the map. Board-agnostic.
- T-Display P4 TFT-LCD (HI8561) variant support (WADA_P4_LCD build; AMOLED bin untouched).
- Wire the Attaky Core board (#158, @attakygit) into the release matrix + web flasher.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
