H2O’s Grid Search for GBM is exposed for the Spark pipelines. This tutorial demonstrates how it is used in a simple Spark pipeline on the Ham or Spam dataset.
This Spark Transformer tokenizes the messages and splits sentences into words.
val tokenizer = new RegexTokenizer().
setInputCol("text").
setOutputCol("words").
setMinTokenLength(3).
setGaps(false).
Remove words that do not bring much value for the model.
val stopWordsRemover = new StopWordsRemover().
setInputCol(tokenizer.getOutputCol).
setOutputCol("filtered").
setStopWords(Array("the", "a", "", "in", "on", "at", "as", "not", "for")).
Crete hashes for the observed words.
val hashingTF = new HashingTF().
setNumFeatures(1 << 10).
setInputCol(stopWordsRemover.getOutputCol).
Create an IDF model. This creates a numerical representation of how much information a given word provides in the whole message.
val idf = new IDF().
setMinDocFreq(4).
setInputCol(hashingTF.getOutputCol).
First, we need to define the hyper parameters. Hyper parameters are stored in the map where key is the name of the parameter and value is an array of possible values.
We also need to specify the algorithm on which we want to run Grid Search together with its arguments. For this, we can use setAlgo method.
val grid = new H2OGridSearch().
setPredictionCol("label").
setHyperParameters(hyperParams).
val colPruner = new ColumnPruner().
val pipeline = new Pipeline().
© Copyright 2016-2019 H2O.ai. Last updated on Apr 03, 2019.
Built with Sphinx using a theme provided by Read the Docs.
