encode(tokens) {
        throw Error("encode should be implemented in subclass.")
    }
Encodes a list of tokens into a list of token IDs.
@param {string[]} tokens The tokens to encode.
@returns {string[]} The encoded tokens.
@throws Will throw an error if not implemented in a subclass.