Access Official Technical PDF

Share Summary:LinkedInEmail

Industrial data analytics often involves datasets with many measured variables, from process signals and sensor readings to equipment conditions and quality indicators. While these features may contain useful information, some can also be redundant or irrelevant to a classification task. Correlation-based feature selection offers one way to reduce this complexity by grouping related attributes and retaining a smaller subset for model training.

A method explored across four datasets—including the SECOM semiconductor manufacturing dataset—combines covariance-based feature representation with Affinity Propagation (AP) clustering. Its results illustrate both the potential value of feature reduction and a practical caution for automation teams: a smaller dataset does not guarantee better model accuracy in every application.

Why reduce features in industrial datasets?

In a high-dimensional dataset, each feature represents an attribute that a model can use to distinguish among classes. As the number of features grows, analysis and model development can become more demanding. Some attributes may contribute little to a classification decision, while others may carry overlapping information because they are strongly related.

Feature selection aims to keep attributes relevant to the target while removing unnecessary or redundant ones. This differs from simply collecting fewer measurements: the selection process works with the available data to identify a more manageable subset. For system integrators and plant teams, this can be useful when evaluating classification models built from large collections of process signals. It does not, by itself, establish that a selected signal is causal or operationally important.

How correlation-based feature selection with clustering works

The described approach has three main stages: preprocess the data, represent relationships among features, and cluster similar features before selecting representatives.

1. Prepare the data

Preprocessing addresses issues that can interfere with analysis, including missing values and categorical data. The method uses K-nearest-neighbor imputation to estimate missing values from nearby records, and converts categorical values into numeric form for processing. These choices matter because the quality and representation of input data can influence both feature grouping and the classifier’s results.

2. Represent feature relationships with covariance

Covariance describes how two variables vary together. The method calculates a covariance matrix, where each feature is represented by its relationships with the other features. These feature vectors provide the input for grouping attributes with similar patterns.

Covariance should not be read as proof that one process variable causes a change in another. It is a measure of association, and its usefulness depends on the data and the intended classification task. In particular, teams should consider whether the variables’ scales and distributions make the resulting comparisons appropriate.

3. Group similar features and select a subset

Affinity Propagation clustering uses pairwise similarities to form groups and identify representative examples, called exemplars. Unlike approaches that require a predefined number of clusters, AP determines exemplars through an iterative exchange of messages between data points.

After clustering, the method randomly selects 30% of the features in each group and combines them into a reduced dataset. The selected attributes are then used to train classification algorithms. The aim is to retain useful information while decreasing the number of inputs; random selection also means the exact subset may vary, so repeatability and validation are important considerations in practical use.

What the dataset comparisons show

The evaluation covers four datasets: smartphone activity recognition (HAPT), gas sensors for home activity monitoring, weight-lifting exercise measurements, and SECOM, which contains semiconductor manufacturing process data. The study compares model accuracy on the original datasets with accuracy after feature selection, using several algorithms, including decision trees, random forests, AdaBoost, logistic regression, bagging, and support vector machines.

Results varied by both dataset and classifier. For the reported ID3 decision-tree comparison, the SECOM feature count fell from 590 to 173, while accuracy rose from 87.26% to 88.85%. On HAPT, the count decreased from 561 to 176 and accuracy changed from 91.63% to 92.34%. These examples show that a reduced feature set can retain comparable accuracy, and in some cases improve it.

However, the outcome was not consistently positive. For the weight-lifting dataset, the reported bagging accuracy fell from 72.24% on the original data to 54.62% after selection. Random forest accuracy for that dataset also declined, from 74.61% to 67.11%. The authors likewise report that the method did not perform equally well across all algorithms and datasets. Feature reduction should therefore be treated as an approach to test, not a guaranteed route to better performance.

Practical considerations for automation teams

When applying this kind of workflow to industrial classification, validate the reduced feature set against the full dataset using the same evaluation process. Check performance for each relevant class, rather than relying only on an overall accuracy figure, particularly when class frequencies are uneven. Compare more than one classifier where appropriate, since the reported results demonstrate that performance depends on the model as well as the data.

  • Document preprocessing decisions, including how missing and categorical values are handled.
  • Record which features are selected and whether results change across repeated runs of the random selection step.
  • Assess reduced and full feature sets using consistent training and testing conditions.
  • Review the operational meaning of retained variables with process and controls specialists before drawing conclusions from a model.

The study also identifies further comparison with other correlation-based feature selection methods and evaluation on lower-dimensional datasets as areas for future work.

Conclusion

Correlation-based feature selection with Affinity Propagation provides a structured way to group related attributes and build a smaller input set for multiclass classification. Results on the SECOM and other datasets show that substantial feature reduction can sometimes preserve or improve reported accuracy, but declines on other dataset–classifier combinations make validation essential. For industrial teams, the practical lesson is to measure the trade-off for the specific process data and classification objective before adopting a reduced feature set.

Access Official Technical PDF

About The Author