MicroQonv: Reshaping Convolution Tensors for Efficient Microscaling in Training and Inference
Making neural networks 7 times smaller without losing accuracy
Researchers developed MicroQonv, a technique that shrinks the memory footprint of neural networks during training and use by up to 7.5 times while keeping accuracy nearly identical. The method reorders how convolutional layers—the building blocks that power image recognition—compress their data, cutting unnecessary data movement in half for weights and up to nine times for intermediate calculations.
Smaller models run faster and cheaper on phones, edge devices, and data centers. This technique enables object detection systems like YOLO to cut their memory demands by 3.5 times while maintaining real-world accuracy, making it practical to deploy powerful AI on devices with limited power and storage. It also unlocks four-bit compression for continual learning on edge devices—scenarios where models must adapt to new tasks without forgetting old ones—boosting accuracy by up to 11%.