A binary code \(C\) is a nonempty subset of \(\mathbb{Z}_2^n\). The elements of \(C\) are the code words, and \(n\) is the length of the code (i.e., the length of the code words).
Let \(u,v \in C\). The Hamming distance \(d(u,v)\) is the number of positions in which \(u\) and \(v\) differ.
A nearest neighbor to \(v \in C\) is a code word \(u \in C\) such that minimizes \(d(u,v)\).
The code rate of a binary code is \(\frac{\log_2 M}{n}\), where \(M\) is the number of code words and \(n\) is the length of the code.
Theorem. (Triangle inequality) Let \(u,v,w\) be code words in a binary code \(C\). Then \[ d(u,v) \leq d(u,w) + d(w,v) \]
Proof. (Induction on the length of \(C\).)
Let \(C\) be a binary code. The minimum distance \(d(C)\) of \(C\) is the minimum of \(d(u,v)\) over all pairs \(u,v\in C\), \(u\neq v\).
Theorem. (Minimum distance) Let \(C\) be a binary code.
If \(d(C) \geq s+1\), then \(C\) can detect \(s\) errors.
If \(d(C) \geq 2t+1\), then \(C\) can correct \(t\) errors.
Proof. 1. \(\checkmark\). 2. Triangle inequality.
Application: Hamming \([7,4]\) code.
A binary code with length \(n\), \(M\) code words, and minimum distance \(d\) is called an \((n,M,d)\) code.
Exercise. Complete the table.
| Code | \(n\) | \(M\) | \(d\) | code rate | errors detected | errors corrected |
|---|---|---|---|---|---|---|
| \(3\)-fold repetition | \(3\) | |||||
| \(8\)th-bit parity check | \(8\) | |||||
| Hamming \([7,4]\) | ||||||
| \(32\times 32\) Hadamard |
\(M =\) number of code words.
\(n =\) length of the code words.
\(d =\) distance of the code (min distance between words).
Tradeoffs:
Definition. An \(n\)-dimensional Hamming sphere of radius \(r\) with center \(c\) is the set of all vectors \(v\) in \(\mathbb{Z}_2^n\) such that \(d(v,c) \leq r\).
Exercise:
00.000.0000?Lemma. A Hamming sphere of radius \(r\) in dimension \(n\) has \[ \binom{n}{0} + \binom{n}{1} + \binom{n}{2} + \cdots + \binom{n}{r} \] elements.
Theorem. A binary \((n,M,d)\) code with \(d \geq 2t+1\) satisfies \[ M \leq \frac{2^n}{\sum_{j=0}^t \binom{n}{j}}. \]
Definition. A binary code with \(d = 2t+1\) that achieves the above bound is called perfect.
That is, a perfect code is an \((n,M,d)\) code that corrects \(t\) errors such that \[M = \frac{2^n}{\sum_{j=0}^t \binom{n}{j}}.\]
Is the Hamming \((7,4)\) code perfect?
A binary linear code is a binary code whose code words are formed by taking all the possible linear combinations of rows of some generating matrix.
## [,1] [,2] [,3] [,4] [,5] [,6]
## [1,] 1 1 0 2 3 4
## [2,] 0 1 2 1 1 4
## [3,] 3 0 0 1 3 1## [,1] [,2] [,3] [,4] [,5] [,6]
## [1,] 1 2 0 1 3 1
## [2,] 1 3 1 1 0 3
## [3,] 0 4 2 4 0 1## [,1] [,2] [,3] [,4] [,5] [,6]
## [1,] 1 1 0 2 3 4
## [2,] 0 1 2 1 1 4
## [3,] 3 0 0 1 3 1## [,1] [,2] [,3] [,4] [,5] [,6]
## [1,] 10 3 4 7 14 15## [1] 10 3 4 7 14 15## [,1] [,2] [,3] [,4] [,5] [,6]
## [1,] 1 1 0 2 3 4
## [2,] 0 1 2 1 1 4
## [3,] 3 0 0 1 3 1## [,1]
## [1,] 22
## [2,] 23
## [3,] 11## Matrix Multiplication
##
## Description:
##
## Multiplies two matrices, if they are conformable. If one argument
## is a vector, it will be promoted to either a row or column matrix
## to make the two arguments conformable. If both are vectors of the
## same length, it will return the inner product (as a matrix).
##
## Usage:
##
## x %*% y
##
## Arguments:
##
## x, y: numeric or complex matrices or vectors.
##
## Details:
##
## When a vector is promoted to a matrix, its names are not promoted
## to row or column names, unlike 'as.matrix'.
##
## Promotion of a vector to a 1-row or 1-column matrix happens when
## one of the two choices allows 'x' and 'y' to get conformable
## dimensions.
##
## This operator is S4 generic but not S3 generic. S4 methods need
## to be written for a function of two arguments named 'x' and 'y'.
##
## Value:
##
## A double or complex matrix product. Use 'drop' to remove
## dimensions which have only one level.
##
## Note:
##
## The propagation of NaN/Inf values, precision, and performance of
## matrix products can be controlled by 'options("matprod")'.
##
## References:
##
## Becker, R. A., Chambers, J. M. and Wilks, A. R. (1988) _The New S
## Language_. Wadsworth & Brooks/Cole.
##
## See Also:
##
## For matrix _cross_products, 'crossprod()' and 'tcrossprod()' are
## typically preferable. 'matrix', 'Arithmetic', 'diag'.
##
## Examples:
##
## x <- 1:4
## (z <- x %*% x) # scalar ("inner") product (1 x 1 matrix)
## drop(z) # as scalar
##
## y <- diag(x)
## z <- matrix(1:12, ncol = 3, nrow = 4)
## y %*% z
## y %*% x
## x %*% z## [,1] [,2] [,3] [,4] [,5] [,6]
## [1,] 1 1 0 2 3 4
## [2,] 0 1 2 1 1 4
## [3,] 3 0 0 1 3 1## [,1] [,2] [,3]
## [1,] 1 0 3
## [2,] 1 1 0
## [3,] 0 2 0
## [4,] 2 1 1
## [5,] 3 1 3
## [6,] 4 4 1## [,1] [,2] [,3]
## [1,] 31 22 18
## [2,] 22 23 8
## [3,] 18 8 20## [,1] [,2] [,3] [,4] [,5] [,6]
## [1,] 10 1 0 5 12 7
## [2,] 1 2 2 3 4 8
## [3,] 0 2 4 2 2 8
## [4,] 5 3 2 6 10 13
## [5,] 12 4 2 10 19 19
## [6,] 7 8 8 13 19 33For the assignment, you might want to be able to convert an integer
n to a binary vector v (e.g., if you want to
represent all possible binary vectors of a certain length.)
Notice that intToBits gives you 32 bits, no matter
what.
## [1] 01 00 01 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
## [26] 00 00 00 00 00 00 00Make those numeric instead of binary, so matrix multiplication will work:
## [1] 1 0 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0You probably don’t want all those 0’s, so you can just subscript the ones you want.
## [1] 1 0 1 1The result is the binary vector corresponding to the binary representation of 13, with the digits reversed. If you want them in the correct order, you can reverse the subscripts:
## [1] 1 1 0 1