Handbook

Outlook Web read strategies (OWA)

Read-only harvest paths for forge-lcdl Outlook tasks and forge-cockpit-web mail migration. All paths use an attached Playwright session (prefer Edge CDP with an already signed-in M365 profile).

Updated

Strategy selection

Goal Governed task source_kind / notes
Gate before sync outlook_probe_mail_health page_health
Folder inventory outlook_list_folders folders
Quick visible sample outlook_list_messages_viewport mail_viewport
Large folder (10k+ rows) outlook_list_messages_scroll mail_scroll + max_scroll_rounds
Bulk inbox (chunked + resume) outlook_list_inbox_chunk chunk_size + checkpoint / next_cursor; query.list_strategy auto | rest | dom
Inbox list (Graph REST) outlook_list_inbox_rest Cookie session GET; folder_total_count; see OUTLOOK-LIST-STRATEGIES.md
Date window outlook_list_inbox_chunk query.date_from, query.date_to (ISO YYYY-MM-DD) → OWA search URL
Thread bodies + per-message IDs outlook_read_conversation_thread conversation_thread + row_index
Deep link parse/build outlook_normalize_item_url pure URL (no session)
Layout probe outlook_apply_read_layout v1 detects only; set OWA manually
Classic list read playwright_read_items inbox, recent, search, folder, message_detail

View modes (OWA UI)

Layout When to use LCDL support
Default inbox + reading pane Thread expansion, body capture conversation_thread, playwright_read_items
Compact / comfortable density Stable list selectors outlook_apply_read_layout (detect); set in OWA UI
Reading pane off More rows per viewport before scroll Set in OWA UI; then mail_scroll
Search Filtered backfill playwright_read_items + search + query.q
Per-folder Migration batches outlook_list_folders then mail_scroll with query.container

Read state (observe vs mutate)

Task Observes unread Mutates OWA read
outlook_probe_mail_health optional unread_count (REST) No
outlook_list_inbox_rest extra.is_unread No
outlook_list_inbox_chunk / scroll / viewport extra.is_unread (DOM heuristics) No
outlook_read_conversation_thread before click Yes — selects row (marks read)

preserve_unread (default on conversation read): after harvest, allowlisted evaluate restore_row_unread for the same row_index when the row was unread. Disable with task input preserve_unread: false or FORGE_LCDL_OUTLOOK_PRESERVE_UNREAD=0. Failures emit ConnectorWarning preserve_unread_failed; harvest still succeeds.

Wire field: UnifiedItem.extra.is_unread and optional state: unread.

Read-only allowlist

Session methods: navigate, snapshot, evaluate_readonly, scroll_readonly, click_readonly.

click_readonly targets only:

  • list_row — [role="listbox"] [role="option"] (select conversation)
  • reading_pane_article — reading-pane [role="article"] (focus message in thread)

No send, delete, archive, move, or compose.

CDP setup (Cockpit / dev)

  1. Quit all Edge windows.
  2. COCKPIT_CDP_URL=http://127.0.0.1:9222 — start via forge-cockpit-web scripts/start-edge-with-debug-for-cockpit.sh.
  3. Sign in to https://outlook.office.com/mail/ once.
  4. Run guided tests or sync; LCDL opens a new tab when possible.

Tab visibility (no OS focus steal)

OWA list harvest and search-day navigation (received:M/D/YYYY in the mail search box) require the tab to report visible. Background tabs (document.hidden === true) often leave the search query typed but return zero rows.

Default (Cockpit): FORGE_LCDL_CDP_BRING_TO_FRONT=0 and COCKPIT_EDGE_FOCUS_EDGE=0 — LCDL enables CDP focus emulation (Emulation.setFocusEmulationEnabled) via apply_cdp_focus_emulation(page) on attach. The operator's Edge window does not need to come to the foreground.

Variable Default Use
FORGE_LCDL_CDP_FOCUS_EMULATION auto-on when bring-to-front off Set 0 only to disable the workaround
FORGE_LCDL_CDP_BRING_TO_FRONT / COCKPIT_EDGE_FOCUS_EDGE 0 Set 1 to allow Page.bringToFront (steals OS focus)

Full policy: EDGE-CDP-ATTACH.md.

Timing and caps

Task Typical live Default cap
outlook_probe_mail_health 2–8 s n/a
outlook_list_messages_viewport 2–5 s limit ≤ 200
outlook_list_messages_scroll 30 s – several min max_scroll_rounds 80; env FORGE_LCDL_OUTLOOK_SCROLL_MAX for tests
outlook_read_conversation_thread 5–20 s per thread 14 articles (max_articles)

Pause ~1200 ms between scroll rounds to avoid OWA throttling.

Health probe: viewport row estimate

outlook_probe_mail_health may return list_row_count_estimate (DOM count of visible list rows). That number is not total mailbox size and not a consumer download quota — only “rows visible in the viewport right now.” Host apps should label it accordingly in progress UI.

Consumer integration (orchestration)

LCDL does not import consumer repos. Host apps compose governed tasks and map wire output locally.

See Messaging consumer boundary for the atomic catalog and forbidden coupling.

Example consumer: forge-cockpit-web maps UnifiedItem batches via lcdl_outlook_bridge → upsert_unified_message (owa_item_message_key, source_folder, source_url).

Recommended loop for full mailbox migration:

  1. outlook_list_inbox_chunk with list_strategy: auto (or dedicated outlook_list_inbox_rest) and chunk_size 100–500; pass next_cursor as checkpoint until chunk.has_more is false. Prefer REST when cookies allow; DOM harvest_list_rows is the fallback (see OUTLOOK-LIST-STRATEGIES.md).
  2. Optional query.date_from / query.date_to to backfill by calendar range (run one range per job).
  3. outlook_read_conversation_thread when list rows lack full body_text (second pass: open conversation / reading pane per row URL).
  4. Cockpit persists checkpoint under var/data/outlook_chunk_checkpoints/account_<id>.json between Connect runs.

Cockpit two-layer mode (COCKPIT_OUTLOOK_INGESTION_LAYERS=list_then_detail, default for guided 10k harvest): phase Scan = step 1 only (metadata + preview); phase Open = step 3 per newly imported row (harvest_phase list_metadata → message_detail).

References