Handbook
Web extractor explore (DomLayoutSpec)
Deterministic website MCQ extraction explore path — analogue of PDF-EXTRACTOR-PARITY.md for Playwright DOM layouts.
Updated
DomLayoutSpec catalog
Frozen entries in src/forge_lcdl/ingest/dom_layout_catalog.yaml:
| layout_id | page_kind | Use case |
|---|---|---|
oep_graded_radio_snapshot |
graded_single_page | OEP graded snapshot |
mop_blog_h2_abcd |
static_blog_list | MOP blog fix00 |
mas_data_attr_blocks |
data_attr_blocks | MAS-like fix01 |
static_quiz_item_visible |
static_mcq_page | Static mechanics fix02 |
watu_single_radio |
watu_single_question | Watu fix03 (Tier B) |
watu_result_graded |
watu_result_page | Watu result fix04 (Tier B) |
Runtime: forge_lcdl.ingest.pm_cert_dom_layout — get_dom_layout_spec, compile_dom_extractor_python, extract_with_layout.
LCDL tasks (playwright pack)
| Task | LLM | Role |
|---|---|---|
pw_layout_spec_from_profile |
No | Profile row → catalog spec |
pw_extractor_compile |
No | Spec → extract_questions(page) module |
pw_extractor_parity_evaluate |
No | Playwright exec + pm_cert_parity (Tier A) or stem/ci match (Tier B) |
pw_layout_spec_refine |
No | Eval → catalog-safe spec patches |
Refine hierarchy: forge_lcdl.tasks.pw_explore_instructions (catalog > profile > parity feedback > history).
lmeta flow
Shared library: forge_lcdl.ingest.explore_runner (embedded flows under src/forge_lcdl/flows/).
Workbench pack: forge-composer-workbench/project-management-certification/web-extractor-explore-pack/
- Flow:
flows/pw-extractor-explore.lmeta(also shipped in forge-lcdl) - Runner:
pipeline/pw_lmeta_explore_runner.py(thin wrapper →run_extractor_explore)
Certificator Bank Health: sources with runner.type: lcdl_explore and a catalog layout_id use the same library in-process (POST /v1/bank-health/jobs/extractor-explore-dry-run, source-ingest dry-run/production). Register via layout catalog picker (GET /v1/bank-health/layout-catalogs).
No pw_extractor_synthesize_* inside the explore flow.
Parity vs incremental MC
| Path | Orchestration | Produce |
|---|---|---|
| Explore (this doc) | lmeta compile/refine | Deterministic DomLayoutSpec compile |
| Incremental MC | Python (incremental_extractor_mc.py) |
LLM exemplar synthesis |
Both can share trusted oracle extractors on the same fixture URLs.
Cursor handoff (GATE RED)
pw_explore_instructions.build_cursor_handoff() — L5 layout_spec.yaml + explore_report.json before prose hints.