Extractors

An extractor is a named, reusable way to collect a specific kind of data. Guida 0.9 turns extractors into a platform: descriptors live in your workspace, Guida loads them into a searchable catalog, and operators can validate, certify, compare, and inspect them without turning implementation details into an informal checklist.

Extractors run in Guida Complete and Guida Node. Guida Control provides a remote operator view over the same Remote Ops contracts; it does not keep a second extractor catalog.

Descriptor and implementation

The descriptor is the public contract. It identifies the implementation, source, workflow, queue, default input, target domains, output contract, and evidence policy. The implementation remains ordinary workspace code.

{
  "$schema": "../../../extractors.schema.json",
  "id": "careers.example.jobs",
  "profile": "web.discovery.browser",
  "label": "Example engineering jobs",
  "domain": "careers",
  "workflowName": "careers-example",
  "queueName": "careers_example",
  "implementation": {
    "module": "lib/careers-example-worker.js",
    "export": "discoverExampleJobs",
    "language": "javascript"
  },
  "source": {
    "kind": "web",
    "url": "https://example.com/careers",
    "label": "Example careers"
  },
  "targetDomains": ["example.com"],
  "input": {
    "defaultUrl": "https://example.com/careers",
    "defaultCompanyName": "Example",
    "defaultSlug": "example"
  },
  "evidence": {
    "requiresPageLoaded": true,
    "requiresResultEvidence": true,
    "allowQualifiedEmpty": true
  }
}

A small extractors.json root can include descriptor files, select profiles, and divide a large catalog into deterministic shards. Profiles provide shared defaults; each descriptor remains independently addressable and retains its source path.

Contracts before execution

Catalog validation catches malformed descriptors, duplicate IDs, missing implementations, unresolved profiles, invalid domain policy, and incompatible input or output contracts. Contract compatibility checks make the boundary between an extractor and its consumers explicit.

The scripting API also exposes synchronous, read-only catalog inspection through g.extractors.list(), g.extractors.get(), and g.extractors.validate(). Remote operators use the corresponding workspace.extractors.* and workspace.extractors.contracts.* methods.

Certification with evidence

Certification is bounded operational verification, not a permanent badge.

• Quick mode checks a deliberately small target set for fast feedback.
• Coverage mode expands across the selected catalog within explicit bounds.
• A successful empty result needs evidence that the page loaded and that an empty result is allowed by the descriptor.
• Selector evidence must be observed, not merely declared.
• Traversal records completeness and truncation, so a partial scan is not presented as full coverage.

Plan first with workspace.extractors.certification.plan. A plan reports what is runnable or blocked and the limits that will apply. Starting, cancelling, or exporting certification evidence is a gated operation. Guida records run and result history in operations.db; authored policy remains in workspace files.

Operational record

Guida keeps the latest verification outcome and the underlying history separate. Operators can inspect:

• recent certification runs and per-extractor results;
• statistics and their history;
• anomaly findings under the selected policy;
• contract changes and compatibility results;
• exported evidence for review or release qualification.

This lets you distinguish a current failure from a long-running pattern without using vague labels. History retention is controlled by the workspace’s extractor history policy.

Catalog scale

The catalog and operational surfaces have been validated with 5,000 extractors using deterministic, single-host tests. That is a catalog-scale claim, not a promise of 5,000 simultaneous browser sessions or an Enterprise fleet limit.

From extractor to pipeline

An extractor produces contracted data. A semantic pipeline connects that output to validation gates, mappers, and destinations. Generated pipelines use explicit descriptor bindings such as $extractor.id or $extractor.source.url; Guida does not infer meaning from filenames or naming conventions.

Remote operation

The Remote Ops API exposes bounded catalog, contract, certification, history, statistics, and anomaly methods. Read operations can support a monitoring-only client. Certification execution and evidence export retain Guida’s local capability, policy, and confirmation checks.