Error Correction Code Transformer

Error correction code is a major part of the communication physical layer, ensuring the reliable transfer of data over noisy channels. Recently, neural decoders were shown to outperform classical decoding techniques. However, the existing neural approaches present strong overfitting due to the exponential training complexity,

or a restrictive inductive bias due to reliance on Belief Propagation. Recently, Transformers have become methods of choice in many applications thanks to their ability to represent complex interactions between elements. 


Our proposed model is more efficient than other standard Transformers due to the sparse design of existing block codes, making its training and deployment extremely affordable. Even with a limited number of computational blocks, it outperforms popular (neural) decoders on a broad range of code families. 
We believe this new architecture will allow the development new standards in this field.

In our solution, we propose to extend for the first time the Transformer architecture to the soft
decoding of linear codes at arbitrary block lengths. We encode each channel’s output dimension to high dimension for better representation of the bits information to be processed separately. 
The element-wise processing allows the analysis of the channel output reliability, while the algebraic code and the interaction between the bits are inserted into the model via an adapted masked self-attention module. The proposed approach demonstrates the extreme power and flexibility of Transformers and outperforms existing state-of-the-art neural decoders by large margins at a fraction of their time complexity.
We study the impact of the proposed embedding and the masking procedure and we analyze and
compare the complexity of our method to other methods. 
 Figure 1 below: BER for various values of SNR for (a) Polar(64,32) and (b) BCH(63,51) codes. Comparison
with the best model of Bennatan et al. [2] on BCH(127,64) code (c).

1. Data storage – Storage devices such as hard drives, solid-state drives, and memory cards. ECC ensures that data stored on these devices is accurate and free from errors.
2. Computer memory: ECC is used in computer memory systems such as RAM to detect and correct errors that can occur due to electrical noise or other factors.
3. Communication systems-  Satellite links, mobile networks, and wireless networks to improve the reliability of data transmission over noisy or unreliable channels.
4. Digital signal processing –  Remove errors from signals that have been corrupted by noise or interference, such as audio and video processing, where errors can have a significant impact on the quality of the output.

Provisional patent application 

Sign up for
our events

    Life Science