LLM Parse Website

LLM Parse Website is a base agent on Somnia Agents. A contract names a site, a prompt and the shape of the answer; the agent finds the pages (or scrapes the URL it was given), renders them in a real browser, converts the HTML to Markdown and has an LLM extract the fields, which come back ABI-encoded with a receipt of every step.

Use it for data that is on a web page but not behind a JSON API: a price on a retail site, a headline, a result on a results page, a change on a page you watch. For a JSON API, JSON API Request is cheaper; for a prompt with no web page behind it, use LLM Inference.

How a request runs

1. The contract sends a base URL, a prompt and a schema: a string, a number, a boolean, a choice from a list, or a struct of those.
2. If resolveUrl is set, a search service finds the relevant pages on the domain; otherwise the agent fetches the URL as given.
3. The pages are fetched in a browser, so JavaScript-rendered content is included, and converted to Markdown.
4. An LLM extracts the fields the schema names from that context.
5. The typed result and a receipt, one step per stage (request, search, scrape, sanitise, extract), are returned to the contract's callback.

Reference

The methods (ExtractString, ExtractUint and the rest), their inputs, the receipt format and the price per subcommittee member are in the agent's reference page, LLM Parse Website. Calling any agent from Solidity is in Invoking from Solidity.

Every registered agent, with its methods and price, is in the Agents directory in the Somnia Explorer.