Playwright PDCA implementation map

This document records repository discovery for the Playwright-assisted, read-only extension. It is the handoff artifact for subsequent PDCA steps; no runtime code ships in step 00.

Repo confirmation

Product scope (v1, preserved)

In scope: attach to an already-authorized playwright browser/session; navigate approved domains; read visible metadata/text and DOM snapshots through a facade; read-only file extraction when session policy permits; normalize into a source-neutral item model.

Out of scope (v1): writes/submits/approvals; cookie/storage/token/header export; Microsoft Graph, SharePoint REST, Workday REST, or other private app APIs; bypassing MFA or harvesting credentials.

Alignment with existing MCP policy: McpPolicy.readmostly_playwright() already denies cookie/storage mutation tools and treats click/type/fill as approval_required — playwright v1 should stay at or below that posture.

Existing package layout (relevant areas)

src/forge_lcdl/
├── __init__.py              # Public exports (TaskRunner, run_task, operators, generic, …)
├── runner.py                # run_task / TaskRunner dispatch
├── result.py                # Ok / Err Result alias
├── errors.py                # LlmFailure hierarchy
├── contracts/               # 39 task_ids × v1 (contract.md + contract.json)
├── contracts_api.py         # ContractSpec v2, UnknownContractError
├── tasks/
│   ├── catalog_v1.py        # TASK_REGISTRY_V1 (generic + RAG + games LLM tasks)
│   ├── registry.py          # register_task / resolve_task / lazy packs
│   └── packs/install.py     # generic | rag | games | playwright packs
├── mcp_client/              # Optional MCP client (pip install forge-lcdl[mcp])
│   ├── hub.py               # Sync McpHub
│   ├── policy.py            # Deny-by-default; readmostly_playwright preset
│   └── adapters/playwright.py  # PlaywrightAdapter, fetch_snapshot, …
├── generic/                 # OpenAI-compat transport helpers
├── execution/               # LcdlClient, ExecutionEngine
├── context/                 # Context packs (byte-budget repo scan)
├── verification/            # Schema/shape checks, pytest runner
└── testing/phased_gates.py  # Subprocess gate helpers

tests/                       # ~84 Python test modules (`tests/**/test_*.py`)
├── test_runner.py, test_catalog_tasks.py
├── mcp_client/              # Fake hub adapter tests (offline)
├── integration/             # granite + mcp_live (skipped by default)
└── tasks/, games/, game_engine/, docs/

docs/
├── reference/runtime-boundary.md   # Core vs forge-lcdl-runtime vs consumers
├── agents/MCP-CLIENT.md            # Playwright MCP client patterns
└── playwright/                      # This map (new)

Task / contract inventory

Contracts live at src/forge_lcdl/contracts/<task_id>/v1/{contract.md,contract.json}. 39 contract folders exist; four have schema/input artifacts only (no register_task runner): page_probe, interaction_probe, page_mechanics, validation_failure — consumed as JSON shapes by Playwright LLM tasks (e.g. pw_page_kind_route, pw_quiz_mechanics_discover).

Sidecar generation and audits: scripts/generate_contract_sidecars.py, scripts/audit_contract_docs.py.

Playwright note: Enterprise read modules are not required to be LLM run_task entries on day one. Prefer a dedicated forge_lcdl.playwright package with optional governed tasks added later if normalization needs LLM assistance. Do not overload existing pw_* Playwright LLM tasks — those ingest page mechanics for QA/education, not M365/Workday mailboxes.

Patterns to reuse (do not refactor)

Proposed module layout (subsequent steps)

Target new tree under src/forge_lcdl/playwright/ (names tentative):

Public surface (later): export minimal types + attach_session from forge_lcdl.playwright; do not add to top-level forge_lcdl.__all__ until API stabilizes.

Unified item model (common shape)

Single dataclass/TypedDict in playwright/types.py:

• schema_version (e.g. "1")
• item_id, source_system (outlook | teams | workday | sharepoint | onedrive | file)
• item_type, source_kind
• title, body_preview, body_text (optional per policy)
• actor, participants
• created_at, updated_at (ISO-8601 strings)
• url, thread_id, container, state
• attachments (metadata list)
• provenance, raw_ref (non-secret locator, e.g. DOM path or stable page ref — never tokens)

JSON Schema sidecar optional in a later step under contracts/playwright_* if governed tasks wrap reads.

Likely files to touch (by PDCA theme)

Avoid in early steps: broad edits to catalog_v1.py, runner.py, transport.py, or mcp_client/hub.py.

Test strategy

Default (CI / offline)

• All unit tests use FakePlaywrightSession or static HTML/JSON fixtures under tests/playwright/fixtures/.
• Mirror tests/mcp_client/test_playwright_adapter.py: capture (method, args) call log; no network.
• Default pytest invocation (existing): pytest → addopts = ["-m", "not granite"] in pyproject.toml.

Markers (existing + planned)

Suggested env gates (later): FORGE_LCDL_PLAYWRIGHT_LIVE, FORGE_LCDL_PLAYWRIGHT_SESSION_ID — document in docs/playwright/TESTING.md.

Discovery command (step 00)

cd /home/lzvyahin/Code/forge-lcdl
python -m pytest --collect-only -q

Expected: hundreds of tests collected; granite and mcp_live cases remain skip-conditioned. Re-run after adding tests/playwright/.

Operator verify (2026-05-31, Act): shell execution was unavailable in the Act agent session; run the command locally before step 02 and record the trailing N tests collected line here when known.

Risks and mitigations

Gaps deferred to later PDCA steps

1. Playwright runtime contract — exact attach API, session handle, and tool surface (stub Protocol only next).
2. pyproject.toml markers — add playwright / playwright_live when first integration tests land.
3. Optional dependencies — PDF/Office parsing libraries not yet declared.
4. Governed LLM tasks — whether reads get run_task wrappers or stay pure Python APIs.
5. Handbook publishing — docs/playwright/TESTING.md and deeper nav when content stabilizes; home pointer added in step 01 Act.
6. forge-lcdl-runtime bridge — consumer-side attachment wiring; core documents hook only.
7. Live pytest verification — operator should run pytest --collect-only -q locally before step 02 (Act session could not execute shell).

Architecture docs (step 01)

Suggested PDCA step sequence (high level)

1. 01 — Architecture docs (complete)
2. 02 — UnifiedItem + errors + fake session protocol + unit tests
3. 03 — Read-only policy module + domain presets
4. 04–09 — One source module per step (outlook → teams → workday → sharepoint → onedrive → files)
5. 10 — Optional MCP bridge adapter (feature-flagged)
6. 11 — Integration tests (skipped live), docs (README, testing guide)
7. 12+ — Optional governed tasks / catalog entries if product requires

References

• .cursor/rules/lcdl-playwright-pdca.mdc — implementation guardrails
• docs/reference/runtime-boundary.md — core vs runtime
• docs/agents/MCP-CLIENT.md — Playwright MCP client (related, not identical to playwright)
• src/forge_lcdl/mcp_client/policy.py — readmostly_playwright() tool allow/deny lists
• CONTRIBUTING.md — responsible use; no credential theft

---

Step 00 complete (Do + Act): discovery recorded; Check nits applied to inventory counts and contract-vs-runner note; implementation code intentionally not added.

Step 01 complete (Do + Act): architecture/policy/extension docs; Check PASS; Act renumbered PDCA sequence, updated operator handoff notes, added docs/index.md discoverability pointer.

Step 04 — connector protocols and fakes (Act 2026-05-31)

Status: Act applied Check fixes; operator must run pytest to close verification.

Verify:

cd /home/lzvyahin/Code/forge-lcdl && python3 -m pytest tests/playwright/ -q --tb=short

Next PDCA step: per-source readers under playwright/sources/ (outlook inbox reader beyond stub, teams, workday, …).

Step 08 — SharePoint reader (Act 2026-05-31)

Status: Do + Check PASS; Act applied no code changes (no pytest failures or blocking Check findings). Operator must run pytest to close verification.

Source kinds (v1): news, list, library — session navigate → snapshot → HTML parse → UnifiedItem.

Verify:

cd /home/lzvyahin/Code/forge-lcdl
python -m pytest tests/playwright/test_sharepoint_reader.py -v --tb=short
python -m pytest tests/playwright/ -q
python -m compileall -q src tests

Known limitations (deferred):

• Fixture-oriented data-* selectors; real modern SharePoint UI mapping is a runtime adapter concern.
• SharePointReader() defaults to permissive_https() for offline tests; production callers should pass sharepoint_source_policy() without permissive override.
• body_text not populated by v1 parser (preview only).
• No task-registry / contract wiring in this step.

Next PDCA step: 09 — OneDrive reader.

Step 21 — SharePoint modern bridge + offline tests (Act 2026-05-31)

Status: Implemented in core; offline pytest is the default verification path.

Verify:

cd /home/lzvyahin/Code/forge-lcdl
.venv/bin/python -m pytest tests/playwright/test_sharepoint_urls.py \
  tests/playwright/test_sharepoint_modern.py \
  tests/playwright/test_evaluating_fake_session.py \
  tests/playwright/test_sharepoint_scroll.py \
  tests/playwright/test_sharepoint_reader_modern.py \
  tests/playwright/test_playwright_sharepoint_tasks.py -q --tb=short
.venv/bin/python -m pytest tests/test_contract_docs_completeness.py -q

Out of scope: forge-knowledge-assistant register_playwright_page wiring; live tenant CI.

Step 09 — OneDrive reader (Act 2026-05-31)

Status: Do + Check PASS; Act applied no code changes (no pytest failures or blocking Check findings). Operator must run pytest to close verification.

Source kinds (v1): recent, shared, activity — session navigate → snapshot → HTML parse → UnifiedItem (SharePoint-hosted OneDrive layout URLs).

Verify:

cd /home/lzvyahin/Code/forge-lcdl
python -m pytest tests/playwright/test_onedrive_reader.py -q --tb=short
python -m pytest tests/playwright/ -q
python -m compileall -q src tests

Known limitations (deferred):

• Fixture-oriented data-* / row class selectors; real OneDrive DOM may need runtime adapter tuning.
• Navigation targets SharePoint _layouts/15/onedrive.aspx views; onedrive.live.com consumer UI variants not mapped.
• No governed task contract / TASK_REGISTRY_V1 wiring in this step.
• File content extraction (10_files_reader) is separate from list/activity reads.
• Live playwright / MCP bridge tests remain out of scope (skipped unless env-gated later).

Optional Check hardening (not applied in Act): site_url filter navigation test, invalid source_kind → invalid_query test, parse_onedrive_snapshot_html body data-kind override test — add only if a future failure demands them.

Next PDCA step: 10 — files reader (PPTX, XLSX, DOCX, PDF) and/or MCP bridge per PDCA pack sequence.

Step 11 — DOCX reader (Act 2026-05-31)

Status: Do + Check PASS; Act added warning-path tests and this handoff note.

Optional dependency: pip install 'forge-lcdl[files]' → python-docx>=1.1.0 (extra name files, not playwright-files).

Verify:

cd /home/lzvyahin/Code/forge-lcdl
pip install -e '.[files,dev]'
python -m pytest tests/playwright/test_docx_reader.py -q --tb=short
python -m compileall -q src tests

Known limitations (deferred): PPTX/PDF readers; sources/files.py facade; playwright session download wiring; TASK_REGISTRY_V1 contracts; footnotes/headers/footers/images. (XLSX reader: step 13.)

Next PDCA step: 12 — PPTX reader (completed; see below).

Step 12 — PPTX reader (Act 2026-05-31)

Status: Do + Check PASS; Act narrowed notes exception handling, added pptx_notes_unavailable warning test, and this handoff note.

Verify:

cd /home/lzvyahin/Code/forge-lcdl
pip install -e '.[files,dev]'
python -m pytest tests/playwright/test_pptx_reader.py -v --tb=short
python -m pytest tests/playwright/ -q
python -m compileall -q src/forge_lcdl/playwright/files/pptx_reader.py

Without [files]: 3 passed, 9 skipped (import probe, invalid metadata, missing-dep monkeypatch; extraction tests skip).

Known limitations (deferred): sources/files.py unified facade; playwright session download wiring; stdlib-only PPTX fixture bytes; charts/images/SmartArt/masters; TASK_REGISTRY_V1 file contracts.

Next PDCA step: 13 — XLSX reader (completed; see below).

Step 13 — XLSX reader (Act 2026-05-31)

Status: Do + Check PASS after Act (row-limit assertion aligned; hidden-sheet test; map note).

Verify:

cd /home/lzvyahin/Code/forge-lcdl
pip install -e '.[files,dev]'
python -m pytest tests/playwright/test_xlsx_reader.py -v --tb=short
python -m pytest tests/playwright/ -q
python -m compileall -q src tests

Without [files]: 3 passed, 8 skipped (import probe, invalid metadata, missing-dep monkeypatch; extraction tests skip).

API notes: max_table_rows / table_chunks[].row_count count body rows only; xlsx_sheets[].row_count is the bounded grid row count (includes header row).

Known limitations (deferred): sources/files.py unified facade; playwright session download wiring; Excel <table>/charts/macros; TASK_REGISTRY_V1 file contracts; live playwright tests.

Next PDCA step: 14 — PDF reader (completed; see below).

Step 14 — PDF reader (Act 2026-05-31)

Status: Do + Check PASS; Act aligned encrypted pdf_pages with successful extractions (empty list), hardened stdlib PDF fixture streams, and this handoff note.

Verify:

cd /home/lzvyahin/Code/forge-lcdl
pip install -e '.[files,dev]'
python -m pytest tests/playwright/test_pdf_reader.py -v --tb=short
python -m compileall -q src tests

Without [files]: 3 passed, 7 skipped (import probe, invalid metadata, missing-dep monkeypatch; extraction tests skip).

Known limitations (deferred): sources/files.py unified facade; playwright session download wiring; OCR / scanned PDFs; password-protected PDFs (blank-password attempt only); TASK_REGISTRY_V1 file contracts; live playwright tests.

Next PDCA step: 15 — unified feed processor (completed; see below).

Step 15 — Unified feed processor (Act 2026-05-31)

Status: Do + Check PASS; Act added source_kinds filter test, public id/dedupe exports, and dedupe_items ordering note.

Verify:

cd /home/lzvyahin/Code/forge-lcdl
python -m pytest tests/playwright/test_processor.py -q --tb=short
python -m pytest tests/playwright/ -q
python -m compileall -q src tests

API notes: Standalone dedupe_items returns key-sorted survivors (not time-sorted); use sort_items or process_unified_feed for newest-first feeds. ProcessedUnifiedFeed has no wire/JSON helper yet.

Known limitations (deferred): merging multiple ReadResult batches with cursor pagination; date-bucket grouping; TASK_REGISTRY_V1 contracts for processed feeds; ProcessedUnifiedFeed wire export; playwright_live integration tests.

Next PDCA step: per PDCA pack (e.g. sources/files.py facade, session download wiring, or governed task wrapping).

---

Step 20 — Final hardening and handoff (2026-05-31)

Status: Act complete (2026-05-31). Public API test uses only forge_lcdl.playwright exports (no direct policy import). Operator must run verification commands below to confirm green pytest locally.

Implemented surface (v1)

Public import: import forge_lcdl.playwright — stable __all__; not re-exported from top-level forge_lcdl until product stabilizes.

Optional deps: pip install -e '.[files,dev]' for Office/PDF parsers (pyproject.toml extra files).

Pytest markers (configured)

Verification commands

cd /home/lzvyahin/Code/forge-lcdl
pip install -e '.[dev,files]'   # optional; file tests skip without [files]
python -m compileall -q src tests
python -m pytest tests/playwright/ -q --tb=short
python -m pytest -q --tb=line    # full suite minus granite
python -m pytest tests/test_contract_docs_completeness.py -q

With live session (operator only):

FORGE_LCDL_PLAYWRIGHT_LIVE=1 FORGE_LCDL_PLAYWRIGHT_SESSION_ID=… \
  python -m pytest -m playwright_live -q

Tests added in step 20

Read-only guarantees (enforced in code)

• PlaywrightPolicy / filter_tool_name deny write and credential-export operations.
• GuardedPlaywrightSession wraps backends with navigation URL checks.
• sanitize_for_wire / redact_raw_ref strip secret-like keys and patterns.
• Connectors use navigate → snapshot only; no Graph/REST clients in v1.
• Governed tasks call get_playwright_session_provider(); no LLM transport.

Known limitations (deferred)

Next extension points

See EXTENSION_POINTS.md: implement PlaywrightSession in runtime, register sessions on InMemoryPlaywrightSessionProvider (or host provider), tune *source_policy() presets per tenant, add mcp_bridge behind feature flag.

Step 20 complete: exports tightened, public API test (public-surface-only assertions), handoff recorded. Sign-off after local pytest block passes.