LLM Web Scraping with Anthropic API

This workflow automates web data retrieval using Anthropic’s Claude-3-haiku model and an HTTP tool agent for processing queries like fetching website content. Key nodes include Manual Trigger (initiates workflow), Input Validation (sets user_query), Validate Input (ensures query length ≥5 characters), HTTP Tool Agent (processes query with Anthropic), HTTP Request Tool (fetches URLs like example.com), Anthropic Chat Model (handles AI processing), Format Success Response (returns response, intermediate_steps, tool_calls_count), and Format Validation Error (handles invalid inputs). It leverages Anthropic’s API for language processing and HTTP requests for web interactions.\n\nSetup Requirements and Configuration: Install n8n by downloading from n8n.io for self-hosting or sign up at cloud.n8n.io for cloud setup. Follow installation prompts or create a cloud account. Obtain an Anthropic API key from console.anthropic.com: sign up, navigate to ‘API Keys,’ and generate a key. In n8n, import the JSON workflow. Configure the Anthropic Chat Model node by adding your API key under ‘Credentials’ > ‘Anthropic API’ in n8n’s interface. Replace the Manual Trigger with a Webhook Trigger for automation: set path to ‘http-tool-agent,’ configure for POST requests, and optionally enable header authentication with a secure key. Ensure sufficient server resources for API calls and HTTP requests.\n\nTesting and Deployment Steps: Test by sending a POST request to the Webhook URL using Postman. Example JSON: {'user_query': 'Fetch content from https://example.com'} with an authentication header if enabled. Alternatively, use Manual Trigger in n8n’s editor. The ‘Format Success Response’ node returns a 200 status with response (e.g., website content), intermediate_steps, and tool_calls_count, or ‘Format Validation Error’ (400 status) for queries shorter than 5 characters. Monitor logs for errors like ‘Invalid API key’ or ‘HTTP request failed,’ and verify credentials or URL accessibility if issues arise. Activate the workflow in n8n for deployment. Validate output to ensure response contains expected web content and tool_calls_count reflects HTTP request executions.