Spaces:
Running
Exploring the Possibility of 3D Generative AI with MLX: Point Clouds, Gaussian Splatting, and Beyond
I am a first-year PhD student interested in starting research on 3D Generative AI. I understand that NVIDIA currently dominates the AI field, but my available hardware is a 2023 Mac Pro with an Apple Silicon M2 Ultra (top configuration). I am wondering whether it is feasible to use MLX or PyTorch with MPS for research in areas such as 3D Gen-AI, point clouds, and Gaussian Splatting.
Any advice or guidance would be greatly appreciated!
Unfortunately the Gen 3D ecosystem tends to have a lot of CUDA dependency. A solution you could try is Spaces Dev Mode so you can access nvidia hardware when needed, and adapt from there to MLX where possible 😊
Thank you all for the advice—I’ll definitely consider using a cloud service like Spaces Dev Mode.
In the meantime, I noticed that @rbourgeat has a migration of TripoSG that runs successfully on macOS with Apple Silicon, although the diso package isn’t supported. I’ll start from there to dig deeper into this field.
Hey Henry,
Yes, you have pinpointed a very common problem. Unfortunately, a lot of essential 3D packages are developed only for CUDA.
A perfect example is spconv, a widely used library for point clouds. A recent project like Tencent's Hunyuan3D Part relies on it, which basically makes it unusable on a Mac.
It's a bit frustrating, but to get started, I would recommend focusing on models built almost exclusively with the core tools provided by frameworks like PyTorch or MLX.
That way, you can be pretty sure the code will run on your Mac (via the MPS backend), without getting blocked right away by a CUDA dependency.
Good luck with your PhD ;)