MM-WebAgent: A Hierarchical Multimodal Agent for Automated Webpage Generation

Microsoft Research Asia proposes the MM-WebAgent framework. To address the issues of inconsistent style and lack of global coherence when integrating AIGC tools into webpage generation processes, it coordinates various AIGC tools through hierarchical planning and iterative self-reflection mechanisms, enabling the automated generation of multimodal webpages with consistent style and global coherence.

The rapid development of AIGC technology is reshaping the landscape of the creative industry. In the field of web design, it provides flexibility for UI/UX design, but directly integrating AIGC tools faces core challenges: inconsistent style and lack of global coherence. Isolated generation of page elements easily leads to issues such as uncoordinated color schemes, chaotic layout, and ambiguous visual hierarchy, which affect user experience and limit the application value of AIGC.

MM-WebAgent adopts a three-layer architecture: the global planning layer determines the overall structure and layout strategy of the webpage; the content generation layer coordinates AIGC tools to generate visual elements; the integration and optimization layer integrates elements and performs iterative optimization. The core mechanisms include hierarchical planning (local decisions serve global goals) and iterative self-reflection (multi-dimensional evaluation and problem correction), forming a "planning-generation-reflection-optimization" closed loop.

The research team built a multimodal webpage generation benchmark test set (covering different types of webpage tasks) and designed a three-layer evaluation protocol: code quality (normativity, compatibility), visual quality (aesthetics, style consistency), and multimodal integration (element coordination). Experimental results show that MM-WebAgent significantly outperforms traditional methods and baselines in all dimensions, especially with obvious advantages in multimodal integration and style consistency.

MM-WebAgent marks a new stage in web design automation, enhancing the practicality of AIGC tools and providing a technical path for creative automation. For designers, it can assist in generating prototypes to release creativity; for non-professional users, it lowers the threshold for webpage creation. In the future, with the progress of multimodal large models, it is expected to promote the popularization of human-machine collaboration models in more creative fields.