What are point clouds in Unity AR?

Point clouds refer to a set of points representing the spatial features of the real-world environment detected by the AR system. The AR platform generates these points based on data collected from various sensors, such as the device’s camera and depth sensors.

Applications

Point clouds are commonly used in AR applications for the following purposes:

Environment understanding

Point clouds help the AR system understand the structure and layout of the real-world environment. They provide information about surfaces, like walls, floors, tables, and other objects, allowing AR apps to interact with and place virtual content accurately in the physical world.

Surface detection

Point clouds play a crucial role in surface detection, which involves identifying flat surfaces in the real world where virtual objects can be anchored and interacted with.

Object recognition

Point clouds can recognize and track specific 3D objects or patterns in the environment. This is useful for implementing AR object tracking and interactions.

Spatial mapping

By combining data from multiple point cloud frames, AR systems can create a spatial map of the environment. This map allows AR apps to understand the 3D geometry of the surroundings.

Mesh generation

In some cases, point clouds can be used as input to generate a 3D mesh representation of the real-world environment. Meshes are surfaces composed of triangles that provide a more detailed and realistic representation of the surroundings.

Implementing point clouds

Open Unity hub and create an AR project.

AR default point cloud

The AR default point cloud represents the spatial features of the real-world environment as detected by the AR system. The particle system in the AR default point cloud is a visual effect that simulates and animates a large number of small particles, such as sparks, smoke, raindrops, or other visual elements. To create an AR default point cloud. Go to create > XR > AR default point cloud.

AR point cloud manager

The AR point cloud manager, will manage the the point cloud features for your Unity project. In your XR origin, add the AR point cloud manager component. Drag the default point cloud to the point cloud prefab.

Save the scene. Connect your Android or iOS device. Go to File > Build and Run.

Demonstration

Conclusion

You can access and utilize point cloud data and create more accurate and immersive AR experiences that blend virtual content with the physical world.