A lone inventor has developed a data compression algorithm that defies the theoretical “Shannon Limit“. The press hasn’t covered this recent news, even though it has dramatic implications. This is probably because the technique is so very arcane. The inventor is none other than the great-great-great granddaughter of the inventor of the tabulated punch card, Herman Hollerith.
The algorithm reduces most of the data while converting the remaining information into as many ones as possible. This not only shrinks storage requirements and costs, but in the case of flash memory, it also has an important impact on total power. Flash is erased by setting all bits to ones, and bits are written by either leaving them alone (one) or by changing them (zero). The fewer zeros in the code, the less energy required to change the bits. Energy is also saved during an erase, since fewer bits need to be brought back to the erased state.
To explain the algorithm in its simplest terms, a byte of data is evaluated. If it has more zero bits than one bits the byte is inverted and an index bit is set to reflect this fact. Next, the four bits on either side of the byte are evaluated and if one has more zeros than ones it is inverted and another index bit is set. This process continues until Continue reading