  In this paper, we study codes with sparse generator matrices. More
specifically, low-density generator matrix (LDGM) codes with a certain
constraint on the weight of the columns in the generator matrix are considered.
In this paper, it is first shown that when a BMS channel W and a constant s>0
are given, there exists a polarization kernel such that the corresponding polar
code is capacity-achieving and the column weights of the generator matrix (GM)
are bounded from above by $N^s$. Then, a general construction based on a
concatenation of polar codes and a rate-$1$ code, and a new column-splitting
algorithm that guarantees a much sparser GM, is given. More specifically, for
any BMS channel and any $\epsilon > 2\epsilon^*$, where $\epsilon^* \approx
0.085$, an existence of a sequence of capacity-achieving codes with all the GM
column weights upper bounded by $(\log N)^{1+\epsilon}$ is shown. Furthermore,
two coding schemes for BEC and BMS channels, based on a second column-splitting
algorithm, are devised with low-complexity decoding that uses
successive-cancellation. The second splitting algorithm allows for the use of a
low-complexity decoder by preserving the reliability of the bit-channels
observed by the source bits, and by increasing the code block length. The
concatenation-based construction can also be applied to the random linear code
ensemble to yield capacity-achieving codes with all the GM column weights being
$O(\log N)$ and with (large-degree) polynomial decoding complexity.
