Access Official Technical PDF

Share Summary:LinkedInEmail

mmWave radar point cloud segmentation offers a way to classify individual radar detections when monitoring traffic with pedestrians, cars and surrounding clutter. The challenge is that radar point clouds are much sparser than camera images or LiDAR data, leaving fewer spatial details for a classifier to use. A Gaussian mixture model (GMM), combined with features describing each point relative to its tracked object, provides one unsupervised approach to this problem.

The method is relevant beyond traffic research wherever radar sensing must support object-level interpretation. For automation engineers and system integrators, it illustrates how radar signal processing outputs can be turned into point-wise classifications without relying on manually labeled training data for model fitting.

Why radar point clouds are difficult to segment

After processing a frequency-modulated continuous-wave (FMCW) radar signal, detections can be represented as a point cloud. Each point describes a reflection from the scene and can include measurements such as range, azimuth and elevation angles, Doppler velocity, signal-to-noise ratio and noise level. The result resembles a LiDAR point cloud, but typically contains fewer points per object.

That sparsity makes it difficult to separate objects using shape alone. Image segmentation can draw on dense pixel patterns, while LiDAR systems generally provide denser spatial samples. Conventional commercial radar can have limited range and angular resolution, resulting in a relatively sparse representation. Higher-resolution mmWave radar can provide a richer point cloud, particularly at shorter distances, but does not eliminate the underlying classification challenge.

Radar nevertheless has characteristics that suit traffic monitoring: it can operate in poor lighting and adverse weather, and its Doppler measurements provide information about motion. These properties make it useful for applications that need to distinguish transportation modes and estimate their speeds.

How mmWave radar point cloud segmentation uses GMM

A multivariate GMM represents observed data as a weighted combination of Gaussian distributions. Each distribution corresponds to a cluster or class, described by its mean and covariance; the weights represent the model’s prior probability for each class. For a new point, the model estimates how likely it is to belong to each distribution and assigns it to the class with the highest posterior probability.

Because the point classes are not specified during model fitting, the GMM is trained in an unsupervised manner. Its parameters are estimated iteratively using expectation-maximization (EM). During the expectation step, the model calculates each class’s responsibility for explaining each data point. During the maximization step, it updates the class weights, means and covariance matrices. The process repeats until the parameters or likelihood converge.

The choice of input features is important. Instead of describing a radar detection only by its absolute coordinates, the approach expresses it relative to the centroid of the tracked object. For each point, the feature vector contains:

  • Relative position: Differences in x, y and z between the point’s position and the tracked object’s centroid, describing the point’s location on the object.
  • Relative Doppler: The point’s Doppler velocity adjusted for the tracked object’s velocity projected in the point’s direction.
  • Radar cross section: A reflection-strength feature calculated from range, signal-to-noise ratio and the CFAR window noise measurement.

Together, these features describe where a reflection lies on an object, how its measured motion relates to the object’s motion, and the strength of the return. The underlying idea is that points from different object types can exhibit different feature distributions. A GMM can then use those differences to classify individual points.

What the traffic monitoring test showed

The evaluation used a high-resolution mmWave radar to collect data for pedestrians and cars, along with unavoidable environmental clutter. The reported detection area extended to 15 meters in range and 18 meters in cross-range. The training data included 8,035 frames collected over about 13 minutes, while the test data included 1,222 frames collected over about two minutes. The classifier was evaluated on individual frames rather than by accumulating multiple frames.

For testing, the data was labeled using the position of each tracked object relative to the radar’s line of sight: points associated with centroids on one side were labeled as pedestrians, points on the other side as cars, and points without an associated centroid as clutter. This setup provided ground truth for evaluation; it was separate from the unsupervised GMM fitting. Because the model’s cluster identifiers do not inherently correspond to the human-assigned labels, the predicted classes were associated with the ground-truth labels before scoring.

The reported intersection-over-union (IoU) was 0.80 for pedestrians, 0.56 for cars and 0.66 for clutter. Precision and recall showed a similar difference between pedestrian and car results: pedestrian precision was 0.85 and recall was 0.93, while car precision was 0.88 and recall was 0.61. In practical terms, the car class was detected with comparatively lower recall in this test. The authors attributed the weaker car classification performance in part to having less car data than pedestrian data, which biased the fitted model toward pedestrians.

Practical considerations and next steps

The results suggest that a relatively simple unsupervised model can provide useful point-wise segmentation for a limited set of classes. They should not be read as evidence that the same performance will hold across different sites, radar configurations or a larger range of transportation modes. The test covered pedestrians, cars and clutter, and the data collection used a controlled arrangement to establish ground-truth labels.

Class balance is one practical consideration: the reported results indicate that a shortage of examples for one class can affect classification performance. Another is the distinction between segmentation and object-level understanding. Classifying each reflection point does not, by itself, solve the broader tasks of grouping points into objects, tracking them over time or recognizing additional vehicle types.

Further work described for this approach includes collecting data at a traffic intersection and adding modes such as motorcycles, bicycles, trucks and buses. The proposed direction is to combine classification with clustering and tracking, including using GMM as a preliminary classifier to support DBSCAN-based grouping.

Conclusion

For industrial and traffic-monitoring teams assessing radar perception, the key takeaway is that mmWave radar point cloud segmentation depends on both sensor resolution and feature design. Relative position, motion and radar cross section give a GMM more useful information than isolated coordinates alone. The reported test demonstrates a promising starting point for separating pedestrians, cars and clutter, while also highlighting the importance of representative class data and further validation before extending the method to more complex scenes.

Access Official Technical PDF

About The Author