tokenize(normalized) {
        const lattice = new TokenLattice(normalized, this.bos_token_id, this.eos_token_id);
        this.populateNodes(lattice);
        return lattice.tokens();
    }
Encodes an array of tokens into an array of subtokens using the unigram model.

@param {string} normalized The normalized string.
@returns {string[]} An array of subtokens obtained by encoding the input tokens using the unigram model.