Researchers at Carnegie Mellon University in the United States have introduced a new artificial intelligence model called LegoGPT, which can generate physically stable LEGO constructions based on text descriptions. This model allows users to create real-world buildable LEGO designs from simple textual prompts.
LegoGPT is a modified version of Meta’s large language model, LLaMA-3.2-1B-Instruct. Instead of predicting the next word, the model predicts the “next brick,” using an additional mathematical module to ensure physical stability. This module takes into account structural forces and gravity to ensure the constructions are sound.
The researchers trained LegoGPT using a dataset called StableText2Lego, which includes over 47,000 stable LEGO structures along with their corresponding text descriptions. This dataset contains 3D models of various LEGO objects and their detailed annotations.
One of LegoGPT’s key features is its “physics-aware rollback” system, which verifies the stability of generated designs and eliminates unstable components by redesigning them. This approach significantly improves the model’s ability to produce physically stable designs.
Currently, LegoGPT can generate designs within a 20x20x20 grid using eight basic LEGO brick types. The designs can be assembled by both humans and robots. The research team plans to extend the model in the future to support larger and more complex structures.
The code and dataset for LegoGPT are open-source and available on GitHub, allowing the public to freely access and use the model. Users can create their own LEGO designs using simple text descriptions and build them in real life.
