LLM From Upload to Answer: RAG Pipeline for Context-Aware Document Queries

This workflow implements a Retrieval-Augmented Generation (RAG) pipeline for document processing and conversational queries, using a secure webhook for document uploads and chat interactions. It replaces manual document analysis or basic search tools by uploading documents, splitting them into chunks, generating embeddings with OpenAI, storing them in a Qdrant vector database, and enabling context-aware chat responses via an AI agent. Key nodes include Webhook for request intake, IF for request routing, Document Loader and Text Splitter for processing, OpenAI Embeddings and Qdrant for vector storage, and RAG Agent for contextual responses. It benefits knowledge workers or support teams in small to mid-size businesses (10-100 employees) handling 50+ document-based queries daily, reducing response time from 10-15 minutes to seconds.\n\nThe ROI saves 8-12 hours weekly for teams processing 100+ queries, improving knowledge retrieval accuracy. Use cases include customer support answering queries based on manuals, legal teams searching contracts, or researchers analyzing reports. Requirements: OpenAI API key (~$0.01/1K tokens for embeddings, ~$0.02/1K tokens for chat), Qdrant instance (free community edition or cloud ~$30/month), n8n instance (free or cloud.n8n.io, ~$20/month), DEVHUB_API_KEY for webhook authentication. Scalability supports thousands of documents; limited by Qdrant storage (~1M vectors free tier) and OpenAI API rate limits (~1,000 requests/minute). Environment variable: DEVHUB_API_KEY.\n\nInstall n8n from n8n.io or cloud.n8n.io. Set up Qdrant (local or qdrant.cloud) and obtain API key. Get OpenAI API key from platform.openai.com. Configure n8n credentials: HTTP Header Auth (X-API-Key), OpenAI API (for embeddings and chat), Qdrant API. Set nodes: Webhook (POST, path: 'devhub-rag-pipeline', header auth), Document Loader (binary input), Text Splitter (chunk size: 1000), Qdrant (collection: devhubconnect_documents), RAG Agent (GPT-4o-mini). Expose webhook via ngrok.\n\nTest with POST requests (e.g., multipart/form-data for document uploads or {message: 'What is in the document?'} for chat) using Postman; verify document storage or chat responses with sources. Common errors: Invalid API key (401—check DEVHUB_API_KEY, OpenAI, Qdrant credentials), missing document (400—ensure file in multipart request), rate limits (429—add retry logic). Deploy by activating workflow and sharing webhook URL. Maintenance: Monitor logs, rotate API keys quarterly, check Qdrant storage limits. Optimize by adjusting chunk size, topK retrieval (5-10), or caching frequent queries.", "businessValue": "Saves 8-12 hours/week automating 100+ document-based queries for support or research", "setupTime": "30-45 minutes", "difficulty": "Advanced", "requirements": ["OpenAI API key", "Qdrant instance with API key", "DEVHUB_API_KEY", "n8n installation, webhook and AI knowledge"], "useCase": "Automating document analysis and contextual query responses for support, legal, or research teams"