Kafka Message Publisher

This workflow automates publishing messages to Kafka topics via a webhook. Key nodes include Webhook (receives POST requests), Input Validation (sanitizes inputs), Check Input Valid (verifies operation), Prepare Message Data (formats messages), Kafka Publish (sends messages), Check Publish Success (confirms delivery), Process Publishing Result (summarizes output), Success Response (returns JSON), and error responses (Invalid Input, Publish Error, Kafka Operations). It uses the Kafka service. Setup Requirements Install n8n from n8n.io or sign up at cloud.n8n.io. Import the JSON into n8n’s workflow editor. Set up a Kafka cluster locally (e.g., using Docker) or use a cloud provider (e.g., Confluent Cloud). Obtain Kafka credentials (bootstrap servers, SASL mechanism, username, password) from your cluster or provider’s dashboard. In n8n, configure Kafka credentials under kafka-credentials with bootstrap servers (e.g., broker:9092), SASL mechanism (e.g., PLAIN), username, and password. Configure the Webhook node by copying its URL (/kafka-publish). Set allowedOrigins (e.g., 'https://your-domain.com') for security. The workflow expects a JSON payload with 'operation' (e.g., publish_message), 'topic', 'message', 'key', 'partition', and 'compression'. Handle errors like invalid input (400: Invalid input) or connection failure (502: Failed to publish). Secure credentials and webhook URL. Testing and Deployment Import the JSON into n8n and activate the workflow. Ensure Kafka is running and credentials are configured. Test the Webhook with a POST request (e.g., curl -X POST [webhook-url] -d '{"operation": "publish_message", "topic": "test-topic", "message": "Hello Kafka"}' -H 'Content-Type: application/json'). Verify Kafka Publish sends the message, Check Publish Success confirms delivery and Success Response returns JSON with publish details (e.g., offset, partition). Test error cases: invalid topic/message (400: Invalid input), wrong credentials (502: Failed to connect), or unavailable broker (502: Kafka operations failed). Check n8n and Kafka logs for errors. Integrate the webhook URL into your application, ensure Kafka accessibility, activate the workflow, and deploy for automated messaging. Monitor Kafka broker connectivity and message delivery.