A Fairness-Controlled Framework for Comparing One-Hot and Compact Distance-Based Output Encoding in Multi-Class Deep Neural Networks

Abstract

This thesis examines output-layer design as a practical mechanism for improving the accuracy efficiency trade-off of deep neural networks in multi-class classification. Conventional one hot encoding represents each class with a dedicated output unit, causing output-layer dimensionality, parameter count, and computation in the network head to increase linearly with the number of classes. Although compact encodings, error-correcting output codes, label embeddings, and metric-learning approaches have been explored, existing studies often vary architecture, loss function, activation, training budget, or decoding strategy alongside the encoding. This makes it difficult to attribute observed gains to the output representation itself. In addition, many evaluations emphasize accuracy while giving limited attention to deployment-relevant measures such as training time, inference latency, energy consumption, model size, and memory footprint. To address these limitations, this thesis compares the conventional one-hot baseline with three engineered compact, distance-based binary output encodings: Closest, Longest, and Reverse. For a classification problem with K classes, each compact scheme assigns every class a binary code of length 𝑏 = ⌈𝑙𝑜𝑔2𝐾⌉, thereby reducing the number of output units relative to one-hot encoding. The comparison is conducted within a fairness-controlled experimental framework in which all models share the same feed-forward DNN backbone, data splits, hyperparameters, training budget, initialization controls, and repeated random-seed configurations. Experiments are performed across seven datasets from five domains: wearable-sensor activity recognition, remote-sensing land-cover classification, handwritten digit recognition, electric motor condition recognition, and wine-quality assessment.