Handbook

Org Explorer harvest — dual architecture (LCDL)

Consumer mirror: forge-cockpit-web org-explorer-dual-architecture.md.

Updated

LCDL owns Playwright read atoms (Delve iframe isolated-world evaluate, read-only clicks); Cockpit owns SQLCipher, crawl state, depth policy, on-disk person folders, org chart UI, and contact linking; Edge owns operator session on an existing OWA Org Explorer tab (loki.delve.office.com/orgexplorer/).

No EML / Downloads / unified_message: Org harvest is people + edges JSON only.

Parallel surface: Outlook People — host-page /people/ atoms (outlook_people_*) write the same consumer store with harvest_source=outlook_people. See ORG-DIRECTORY-CONSUMER-BOUNDARY.md.

Stable state boundary

LCDL does not dedupe people by display_name, write files/org/, maintain org_ingest_queue, or stamp profile_check. The wire may list the same human under different graph_id values when OWA surfaces them in different contexts (focus card, manager strip, team column, works-with).

Cockpit consumer (disk_canonical.py, ingest_queue.py, profile_resolve.py, profile_checker.py) owns:

  • canonical folder selection (canonical_graph_id)
  • queueing incomplete / id-only detections
  • offline compaction and checker stamps

LCDL task list and wire shape are unchanged — only Cockpit ingest semantics after OrgHarvestWire.from_task_result.

Governed tasks (v1)

task_id Purpose
outlook_probe_org_explorer_health Iframe loaded, search box ready
outlook_org_explorer_home_snapshot Focused person + visible hierarchy wire JSON
outlook_org_explorer_open_profile_card View profile panel (email, LinkedIn if present)
outlook_org_explorer_prepare_focus Profile-enrich: mail-first search, optional chart click, View profile
outlook_org_explorer_host_dr_panel Host flyout structure when iframe focus fails
outlook_org_explorer_focus_search Search picker → confirmed focus (display_name, optional graph_id, optional mail)
outlook_org_explorer_click_person Card click when it changes focused person
outlook_org_explorer_focus_url Navigate/focus via URL when supported

Input: { "session_ref": "…" } where the session page is a CdpOwaPageShim on an Org Explorer host tab.

Evaluate scripts: src/forge_lcdl/playwright/scripts/outlook_org_explorer_readonly.py (and related org explorer modules).

End-to-end (LCDL slice)

session_ref resolved → PlaywrightPageSession (Org Explorer host tab)
  → evaluate_readonly(org_explorer_health)     # probe
  → evaluate_readonly(home_snapshot)           # GraphQL walk + DOM fallback
  → wire JSON: people[], edges[], focused_graph_id, tenant_id, upn
  → optional: click_person / focus_search / open_profile_card

Consumer calls via cockpit_server.lcdl_outlook_bridge.run_task — LCDL does not import Cockpit.

Responsibility matrix

Concern forge-lcdl Cockpit consumer Edge
Governed read tasks tasks/outlook_read_v1.py (org explorer + people runners) lcdl_outlook_bridge.run_task —
Delve iframe selectors / evaluate JS outlook_org_explorer_readonly.py Must not add page.locator in explorer_harvest.py Org Explorer UI
CDP tab pick pick_org_explorer_cdp_page_target in edge_cdp_attach.py _attach_org_explorer_tab Existing tab only
Tab visibility (background crawl) apply_cdp_focus_emulation via connect_owa_page_via_selective_cdp FORGE_LCDL_CDP_BRING_TO_FRONT=0 (default) Delve iframe expand/show-more needs visible tab
Isolated world iframe read Page.createIsolatedWorld + Runtime.evaluate — Signed-in Delve session
Photo bytes on wire Delve blob: fetch in snapshot ingest_wire_photos → photo_120.jpg Avatars rendered
Wire shape people[], edges[], focused_graph_id OrgHarvestWire.from_task_result —
SQLCipher, crawl caps — explorer_harvest.py, ingest.py —
Depth policy / guided phases — depth_policy.py, guided_crawl_phases —
Graph MSAL sync — org/sync_service.py Device sign-in
Contact linker — contact_link.py —
Read-only guarantee READONLY_POLICY.md, task read_only: true _ORG_CDP_LOCK Operator obeys policy

What LCDL must not do

Forbidden in LCDL Belongs in consumer
org_people.sqlite, org_graph.sqlite Cockpit SQLCipher
files/org/ paths, harvest_manifest.json Cockpit org/ingest.py
COCKPIT_ORG_SCAN_MAX_PEOPLE cap explorer_harvest.py
Crawl state, NDJSON live events Cockpit crawl_state.py, API
MSAL / Graph device sync Cockpit org/sync_service.py
contact.org_graph_id linker Cockpit contact_link.py

Data sources (priority)

  1. Delve GraphQL (useOrgExplorerQueryRef_OrganizationQuery) — recursive JSON walk
  2. personaphoto?aadObjectId= URLs — stable graph_id
  3. DOM innerText fallback — names/titles only (no graph_id)

Wire shape (summary)

{
  "ok": true,
  "focused_graph_id": "…",
  "people": [{ "graph_id": "…", "display_name": "…", "job_title": "…", "photo_bytes_b64": "…" }],
  "edges": [{ "src_graph_id": "…", "rel": "reports_to", "dst_graph_id": "…" }]
}

Edge rel values: reports_to, peer_of, works_with.

Profile-enrich harness vs guided crawl

Guided ingest (explorer_harvest.py) focuses people via chart click / URL when the crawl policy allows — dropdown search is not the primary path. Profile-enrich harness (COCKPIT_ORG_SEARCH_ENTER_ONLY=1, Cockpit scripts/org-profile-enrich/) types mail or display name and submits a single CDP Enter with no suggest-row poll; prepare_person_focus(..., open_view_profile=False) leaves View Profile to a later phase. LCDL exposes both behaviors via org_search_enter_only() and open_view_profile on prepare_person_focus; consumers choose per workflow.