Handbook

Outlook OWA two-pass sync — plan

Default today (Cockpit): single-pass scroll+save — scroll, harvest exchange_item_id, export new mail in one loop. See OUTLOOK-EML-EXPORT-DUAL-ARCHITECTURE.md, forge-cockpit-web outlook-owa-eml-dual-architecture.md, and…

Updated

Legacy on this page: discover → download two-pass when COCKPIT_OUTLOOK_OWA_LEGACY_TWO_PASS=1. Cockpit still sets COCKPIT_OUTLOOK_OWA_TWO_PASS=1 for DOM list ingest (download_then_analyze, list_strategy=dom).

Pass 1 — Discover (fast, no message open)

Goal: Build an ordered list of download targets (up to N) without opening the reading pane.

Step Mechanism Mutates read state?
Scroll inbox outlook_list_inbox_dom_step / harvest_list_rows No (scroll inner container only; no row focus/click/PageDown)
Row metadata DOM: from_email, from, full subject, UTC received_at, exchange_item_id, conv_id, list_row.index No
Dedupe exchange_item_id + message_identity_key vs Cockpit local store; conv_id for OWA row locate; legacy content ui_hash read-only No
Optional RFC822 outlook_fetch_message_rfc822 — context menu View → message details (headers only; row still in list) No reading pane
Scroll metrics Cockpit tracks unique conv_id count vs viewport touches (rows_scanned); burst scroll when scroll_moved=false No
Anchor scroll_top at first new row No

Must not: left-click list rows, open conversation, or depend on reading-pane DOM during Pass 1 (RFC822 uses context menu only).

Pass 1 (Cockpit / KA posture — OWA only)

Same stability idea as SharePoint library harvest in KA: governed evaluate scripts + scroll, no opportunistic Graph calls when Graph is unavailable on Edge.

  1. Visible progress — active_step=discover, counters: catalogued targets, unique rows, viewport touches, already local.
  2. Report every scroll round — discover is fast; UI should show movement within 1–2s.
  3. list_strategy=dom — harvest_list_rows only; do not rely on outlook_list_inbox_rest / Graph MIME in Cockpit (COCKPIT_OUTLOOK_OWA_ONLY=1).
  4. Probe copy — Mailbox health check is separate; message must say “then Pass 1 scroll+DOM”.

Pass 2 — Download (slow)

Goal: Write raw .eml per target. Cockpit renames to {Message-ID}.eml under files/…/messages/…/ (not source.eml; not the subject name in Edge Downloads).

Priority Path Opens reading pane?
1 OWA list context menu — right-click row → Download / .eml No (Cockpit default)
2 OWA reading pane — left-click row → toolbar / overflow Yes (opt-in only; avoid)

Graph MIME remains in LCDL for other hosts that have a working Graph session on Edge; Cockpit does not use it when COCKPIT_OUTLOOK_OWA_ONLY=1.

OWA export implementation (current)

  • Default (safe): context_menu only — right-click row, Download as EML (hover Download when OWA shows a submenu), Escape to dismiss flyouts. Does not open the reading pane.
  • Graph MIME when graph_id is present and REST works; on failure, falls back to Download as EML (unless FORGE_LCDL_OUTLOOK_OWA_ONLY=1).
  • No reading-pane fallback unless FORGE_LCDL_OUTLOOK_EML_ALLOW_READING_PANE_FALLBACK=1 (and mode context_menu_then_reading_pane). Avoid on operator desktops — it left-clicks rows and hits More actions / unrelated menus.
  • Force legacy: FORGE_LCDL_OUTLOOK_EML_DOWNLOAD_MODE=reading_pane

Cockpit progress text should say “OWA context menu” when on this path, not “open message”.

Pass 2 scroll navigation (outlook_owa_nav.py)

After Pass 1, each discovered row carries _discover_container_scroll (inner list scroll when the row was seen).

Strategy Start scroll Download order Locate scroll
top_then_reverse_up (default) Top of batch (range_min) Last → first (reverse Pass 1) Prefer up; scroll down only when target is below current position

Pass 2 restores scroll to the top of the range, downloads the deepest row first, then works upward. seek_scroll_direction() compares current container scroll to each row's scroll_hint. If re-find fails but Pass 1 left extra.list_row.index, Cockpit uses that row metadata (no left-click).

Isolated E2E (100 discover + download + teardown)

From forge-cockpit-web (Edge CDP + signed-in OWA tab):

./scripts/run-lcdl-outlook-two-pass-e2e-100.sh

Uses a temp cockpit-e2e.sqlite and files/ tree (production var/data/cockpit.sqlite untouched). Removes the workspace and asserts no legacy source.eml files remain. Pytest gate: COCKPIT_OUTLOOK_E2E_100=1.

Pass 2 improvements (planned)

  1. Re-find scroll labeled Pass 2 — Scrolling to locate a target by identity key is not Pass 1; UI should say “Pass 2 — locating row (no click)”.
  2. Metrics — download.json / warnings record via: owa_list_context_menu only under OWA-only mode.

Environment

Variable Default Purpose
COCKPIT_OUTLOOK_OWA_TWO_PASS 1 Enable discover → download
COCKPIT_OUTLOOK_LIST_STRATEGY auto REST when session allows
FORGE_LCDL_OUTLOOK_EML_DOWNLOAD_MODE context_menu_then_reading_pane OWA export UX
COCKPIT_CDP_AUTO_RECOVER 1 Restart stale Edge CDP
FORGE_LCDL_CDP_FOCUS_EMULATION auto-on CDP focus emulation so OWA lists render without OS focus steal
FORGE_LCDL_CDP_BRING_TO_FRONT / COCKPIT_EDGE_FOCUS_EDGE 0 1 = raise Edge window (Page.bringToFront)

Verification

  1. Prepare Edge (CDP 9222), signed-in OWA inbox.
  2. Sync with goal 5–10 (small first).
  3. Pass 1: Discover stage; rows scanned and “already local” increase; reading pane does not change for each target during discover.
  4. Pass 2: Either fast Graph saves or context-menu export without full message body in reading pane.
  5. download.json / warnings show via as expected.

References