date
type
status
slug
summary
tags
category
icon
password
URL
by Simon Meng, mp.weixin.qq.com • See original
Large language models are growing larger, seemingly poised to take over everything. However, it seems that no one has yet considered "space" as a compatible "modality" for LLMs 🤔—here, space is not narrowly defined as a 3D model or point cloud but includes spatial relationships that reflect human perception habits. For instance, at the end of a narrow, dimly lit corridor, people expect a bright and tranquil courtyard, with a cozy room nearby; such relationships are crucial for providing comfort in virtual environments.
As a group of tech enthusiasts architects , we have developed a GNN (Graph Neural Network) model that can learn spatial topological relationships and architectural experiences. This model reconnects LLMs with game engines and 3D assets, enabling the abstract spatial descriptions output by LLMs to be rapidly transformed into fully traversable, immersive experiences that align with human perceptual habits in 3D (game level) prototypes. Furthermore, it supports quick modifications and adjustments through dialogue!
🤩 Ultimately, we aim to create a cross-modal spatial reasoning model that is ready for the AI-native game + XR + metaverse era on the horizon!
🤗 This is just one facet of our spatial computing research; our ongoing work also includes full-scene 3D content migration based on text input and LLM dialogue, akin to generalized neural radiance fields (NeRF). We are well aware that these are difficult yet right pursuits. Fortunately, we are lucky enough to have completed a seed round financing led by Qiji Chuangtan, allowing our dreams to continue.
- 作者:Simon Shengyu Meng
- 链接:https://simonsy.net/article/LLM-SPACE-en
- 声明:本文采用 CC BY-NC-SA 4.0 许可协议，转载请注明出处。
相关文章
Microcosmic Universe
Project Title: One and Three Objects, and An Attempt at Exhausting the Object
One and three objects, and an attempt at exhausting the object
DreamGaussian: The Stable Diffusion Moment of AIGC 3D Generation
How I Used AI to Create a Promotional Video for Xiaomi's Daniel Arsham Limited Edition Smartphone
3D scene editing has entered the era of AI text interaction
