MASON - A Multi Agenttic LLM Framework
A multi agent LLM framework for no-code software generation using CrewAI, benchmarked on MBPP and HumanEval.
Project Overview
MASON (Multi Agent System for Open No-Code Development) is a modular LLM orchestration framework designed to explore the capabilities of multi-agent architectures in no-code software development tasks. Built atop CrewAI, MASON coordinates specialized agents—such as prompt parsers, requirement analysts, code generators, and validators—to collaboratively generate, validate, and output executable Python code.
This system benchmarks its effectiveness against standard Python generation datasets like MBPP and HumanEval. The architecture facilitates scalable agent interaction and transparency through structured config files and visualized results. The project evaluates whether a distributed agent setup outperforms single-LLM solutions in quality, modularity, and reliability.
Key Features
- •Multi-agent orchestration with CrewAI
- •Support for MBPP and HumanEval benchmarks
- •Role-specific agents (Input Parser, Analyst, Generator, Validator, Output)
- •Customizable agent and task definitions via YAML files
- •Modular and extensible architecture for LLM workflows
- •Visualizations for performance metrics and comparisons
- •Structured output logging with execution stats
Technical Challenges
- →Managing latency and coordination overhead in multi-agent systems
- →State consistency across multiple lightweight LLMs
- →Sequential local execution due to lack of GPU parallelism
- →Agent communication complexity for multi-step code synthesis
- →Balancing modularity with performance in LLM workflows