If you’re using OpenClaw for more than just simple, single-shot tasks, you’ve probably hit a wall when trying to manage multiple, ongoing AI workflows. The default OpenClaw setup, while robust for individual agent runs, doesn’t inherently parallelize complex, multi-stage operations very well. This often leads to a bottleneck where a single long-running task hogs resources, preventing other, potentially more urgent, agents from executing. Imagine you’re running a market sentiment analysis agent that processes daily news, an automated customer support agent that handles incoming queries, and a content generation agent that drafts blog posts – all simultaneously. Without a proper sub-agent architecture, these tasks will often queue up, leading to delays and missed opportunities.
Looking to get a VPS for your project? Vultr offers reliable VPS hosting starting at $5/month with global data centers. Many OpenClaw users self-host on Vultr for consistent uptime and affordable pricing.
\n\n
Understanding the Problem with Single-Threaded Agents
\n\n
By default, when you initiate an OpenClaw agent, say with openclaw run my_main_agent.py, that script typically encompasses the entire logic for a specific task. If that task involves multiple steps – fetching data, processing it with one LLM, then refining the output with another – it’s all handled sequentially within that single agent’s execution context. This is perfectly fine for many use cases. However, when you introduce concurrency requirements, like needing to process ten independent data streams or respond to five simultaneous user queries, the single agent model quickly becomes a bottleneck. Your main agent script ends up being a monolithic entity trying to manage disparate concerns, often leading to complex state management and error handling within one file, making it harder to debug and scale.
\n\n
The core issue is resource contention and sequential execution. If your my_main_agent.py is busy fetching a large dataset or waiting for a slow LLM response, any other logical “task” within that same script has to wait. Even if you try to manage multiple internal “workflows” with async/await patterns in Python, you’re still constrained by the overall execution context of that single agent process. For true parallel processing, especially across different LLM calls that might have varying latencies or rate limits, you need a more distributed approach.
\n\n
Implementing a Sub-Agent Architecture
\n\n
The solution lies in breaking down your complex workflows into smaller, independent OpenClaw agents – let’s call them “sub-agents” – each responsible for a specific, atomic part of the overall process. The main agent then acts as an orchestrator, dispatching tasks to these sub-agents and collecting their results. This mimics a microservices architecture, but within the OpenClaw ecosystem.
\n\n
Here’s how you structure it:
\n
- \n
- Orchestrator Agent: This is your primary OpenClaw agent. Its role is to define the overall workflow, determine which sub-agents need to run, and manage their execution.
- Sub-Agents: These are individual OpenClaw agent scripts (e.g.,
data_fetch_agent.py,sentiment_analysis_agent.py,summary_generation_agent.py). Each sub-agent should encapsulate a single, well-defined task. They are designed to be run independently and return a specific output.
\n
\n
\n\n
To make this work, you’ll leverage OpenClaw’s ability to run agents programmatically and, crucially, to pass data between them. The orchestrator won’t just execute sub-agents; it will often need to feed them inputs and consume their outputs.
\n\n
Orchestrator Configuration and Logic
\n\n
Let’s say your main agent is orchestrator_agent.py. Inside this agent, you’ll define the sequence of operations. Instead of directly calling LLMs for every step, you’ll invoke other OpenClaw agents. OpenClaw provides a programmatic API for this, but a more robust way for persistent, decoupled execution is to use file-based communication or a lightweight message queue if your setup permits. For simplicity and reliability on a VPS, we’ll stick to file-based communication.
\n\n
Your orchestrator_agent.py might look something like this:
\n
\nimport json\nimport subprocess\nimport os\nimport time\n\nclass OrchestratorAgent:\n def __init__(self):\n self.output_dir = "/var/openclaw/outputs"\n os.makedirs(self.output_dir, exist_ok=True)\n\n def run_sub_agent(self, agent_name, input_data):\n input_file = os.path.join(self.output_dir, f"{agent_name}_input.json")\n output_file = os.path.join(self.output_dir, f"{agent_name}_output.json")\n\n with open(input_file, "w") as f:\n json.dump(input_data, f)\n\n # Run the sub-agent in the background or wait for it\n # For true parallelism, you'd daemonize this or use a process pool\n # For simplicity here, we'll run it synchronously and wait.\n # To run truly async and check for completion, you'd manage PIDs.\n print(f"Running sub-agent: {agent_name} with input: {input_file}")\n command = [\n "openclaw", "run",\n f"agents/{agent_name}.py",\n "--input-file", input_file,\n "--output-file", output_file\n ]\n \n try:\n # Use Popen to run asynchronously if you want to dispatch multiple\n # and then collect. For this example, we'll wait.\n process = subprocess.run(command, capture_output=True, text=True, check=True)\n print(f"Sub-agent {agent_name} finished. STDOUT: {process.stdout} STDERR: {process.stderr}")\n\n if os.path.exists(output_file):\n with open(output_file, "r") as f:\n return json.load(f)\n else:\n print(f"Warning: {output_file} not found after {agent_name} execution.")\n return {"error": "output file not found"}\n except subprocess.CalledProcessError as e:\n print(f"Error running sub-agent {agent_name}: {e}")\n print(f"Sub-agent STDOUT: {e.stdout}")\n print(f"Sub-agent STDERR: {e.stderr}")\n return {"error": str(e), "stdout": e.stdout, "stderr": e.stderr}\n finally:\n # Clean up input file if no longer needed\n if os.path.exists(input_file):\n os.remove(input_file)\n # You might want to keep output files for debugging or auditing\n\n def run(self):\n # Example workflow\n data_to_process = {"text": "The stock market saw a significant rally today, boosting investor confidence."}\n\n # Step 1: Fetch data (might be a simple pass-through or actual fetching logic)\n fetched_data = self.run_sub_agent("data_fetcher", data_to_process)\n if fetched_data and not fetched_data.get("error"):\n print(f"Fetched data: {fetched_data.get('content', 'N/A')}")\n else:\n print("Data fetch failed, exiting.")\n return\n\n # Step 2: Analyze sentiment\n sentiment_result = self.run_sub_agent("sentiment_analyzer", {"text": fetched_data.get("content")})\n if sentiment_result and not sentiment_result.get("error"):\n print(f"Sentiment: {sentiment_result.get('sentiment', 'N/A')}")\n else:\n print("Sentiment analysis failed, exiting.")\n return\n\n # Step 3: Generate a summary based on sentiment\n summary_input = {\n "text": fetched_data.get("content"),\n "sentiment": sentiment_result.get("sentiment")\n }\n summary_result = self.run_sub_agent("summary_generator", summary_input)\n if summary_result and not summary_result.get("error"):\n print(f"Summary: {summary_result.get('summary', 'N/A')}")\n else:\n print("Summary generation failed, exiting.")\n return\n\n print("Workflow completed successfully!")\n\n# To run this orchestrator:\nif __name__ == "__main__":\n agent = OrchestratorAgent()\n agent.run()\n\n\n
Sub-Agent Structure
\n\n
Each sub-agent needs to be designed to read its input from a specified file (--input-file) and write its output to another specified file (--output-file). This standardized interface is critical for the orchestrator to interact with them effectively. Here’s an example of a sentiment_analyzer.py sub-agent:
\n\n
\nimport json\nimport argparse\nimport os\nfrom openclaw import OpenClaw\n\nclass SentimentAnalyzerAgent:
Want to see what OpenClaw can really do? Check out this wild project building AI agents with physical bodies →
Related: OpenClaw Setup: From Zero to Running in 30 Minutes (Part 2)
Related: Building an OpenClaw-Powered Affiliate Site: Architecture and Automation Stack
Related: OpenClaw Setup: From Zero to Running in 30 Minutes (Part 2)
Related: Building an OpenClaw-Powered Affiliate Site: Architecture and Automation Stack
Related: OpenClaw Setup: From Zero to Running in 30 Minutes (Part 2)
Related: Building an OpenClaw-Powered Affiliate Site: Architecture and Automation Stack
Related: OpenClaw Setup: From Zero to Running in 30 Minutes (Part 2)
Related: Building an OpenClaw-Powered Affiliate Site: Architecture and Automation Stack
Related: OpenClaw Setup: From Zero to Running in 30 Minutes (Part 2)
Related: Building an OpenClaw-Powered Affiliate Site: Architecture and Automation Stack