Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

Beyond Pixels: Exploring DOM Downsampling for LLM-Based Web Agents

License

Notifications You must be signed in to change notification settings

webfuse-com/D2Snap

Open more actions menu

Repository files navigation

D2Snap

Example of downsampling on an image (top) and a DOM (bottom) instance

D2Snap is a first-of-its-kind DOM downsampling algorithm, designed for use with LLM-based web agents.

Integrate

D2Snap.d2Snap(
  dom: DOM,
  k: number, l: number, m: number,
  options?: Options
): Promise<string>

D2Snap.adaptiveD2Snap(
  dom: DOM,
  maxTokens: number = 4096,
  maxIterations: number = 5,
  options?: Options
): Promise<string>
type DOM = Document | Element | string;
type Options = {
  assignUniqueIDs?: boolean;          // false (this option is not available with string input)
  debug?: boolean;                    // false
  keepUnknownElements?: boolean;      // false
  skipMarkdownTranslation?: boolean;  // false
};

Browser

<script src="https://cdn.jsdelivr.net/gh/webfuse-com/D2Snap@main/dist/D2Snap.browser.js"></script>

Module

npm install webfuse-com/D2Snap

Install jsdom to use the library with Node.js:

npm install jsdom
import * as D2Snap from "@webfuse-com/d2snap";

Example

<section tabindex="3" class="container" required="true">
  # Our Pizza
  <div class="shadow-lg">
    ## Margherita
    A simple classic mozzarela tomatoes and basil
    <button type="button">Add</button>
    ## Capricciosa
    A rich taste A true favourite
    <button type="button">Add</button>
  </div>
</section>

↓ D2Snap ↓

<section class="container" required="true">
  # Our Pizza
  <div class="shadow-lg">
    ## Margherita
    A simple classic
    <button>Add</button>
    ## Capricciosa
    A rich taste
    <button>Add</button>
  </div>
</section>

↓ D2Snap ↓

<section>
  # Our Pizza
  ## Margherita
  A simple classic
  <button>Add</button>
  ## Capricciosa
  A rich taste
  <button>Add</button>
</section>

Experiment

Setup

npm install
npm install jsdom

Build

npm run build

Test

npm run test

Evaluate

Provide LLM API provider key(s) to .env (compare example).

npm run eval:<snapshot>

<snapshot> ∈ { gui, dom, bu, D2Snap }

npm run eval:D2Snap -- --verbose --split 10,20 --provider openai --model gpt-4o

Re-create Snapshots

npm run snapshots:create

Beyond Pixels: Exploring DOM Downsampling for LLM-Based Web Agents
Thassilo M. SchiepanskiNicholas Piël
Surfly BV

About

Beyond Pixels: Exploring DOM Downsampling for LLM-Based Web Agents

Resources

License

Stars

Watchers

Forks

Contributors

Morty Proxy This is a proxified and sanitized view of the page, visit original site.