<NME> mlogit_Benchmark--Heating-checkpoint.ipynb
<BEF> {
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Mlogit Benchmark 2: Kenneth Train's Heating Data\n",
    "The purpose of this notebook is to:\n",
    "<ol>\n",
    "    <li> Demonstrate the use of the pyLogit to estimate conditional logit models.</li>\n",
    "    <li> Benchmark the results reported pyLogit against those reported by the mlogit package.</li>\n",
    "</ol>\n",
    "\n",
    "The models estimated in this notebook will be the same models detailed in \"Kenneth Train’s exercises using the mlogit package for R.\" In particular, the following models will be estimated:\n",
    "<ol>\n",
    "    <li> The model with installation cost and operating cost, without intercepts (p.2). \n",
    "        <pre> mlogit(depvar~ic+oc|0, H) </pre>\n",
    "    </li>\n",
    "    <li> The model that imposes the constraint that r = 0.12 (such that wtp = 8.33) (p. 4).\n",
    "        <pre> H$lcc=H$ic+H$oc/0.12\n",
    " mlcc <- mlogit(depvar~lcc|0, H)\n",
    "        </pre>\n",
    "    </li>\n",
    "    <li> The model with installation cost, operating cost, and all intercepts except that of the \"hp\" alternative (p.5).\n",
    "    <pre> mc <- mlogit(depvar~ic+oc, H, reflevel = 'hp')\n",
    "    </pre>\n",
    "    </li>\n",
    "    <li> The model with installation cost divided by income, operating cost, and all intercepts except that of the \"hp\" alternative (p. 7).\n",
    "    <pre> mi <- mlogit(depvar~oc+I(ic/income), H, reflevel = 'hp')\n",
    "    </pre>\n",
    "    </li>\n",
    "    <li> The model with intallation costs, operating costs, alternative specific coefficients for income, and all intercepts except that of the \"hp\" alternative (p.7).\n",
    "    <pre> mi2 <- mlogit(depvar~oc+ic|income, H, reflevel=\"hp\")\n",
    "    </pre>\n",
    "    </li>\n",
    "</ol>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. Import Needed libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from collections import OrderedDict    # For recording the model specification \n",
    "\n",
    "import pandas as pd                    # For file input/output\n",
    "import numpy as np                     # For vectorized math operations\n",
    "\n",
    "import pylogit as pl                   # For MNL model estimation and\n",
    "                                       # conversion from wide to long format\n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. Load and look at the required datasets"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Load the Heating data, noting that the data is in wide data format\n",
    "wide_heating_df = pd.read_csv(\"heating_data_r.csv\")\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "      <th>4</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>idcase</th>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>4</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>depvar</th>\n",
       "      <td>gc</td>\n",
       "      <td>gc</td>\n",
       "      <td>gc</td>\n",
       "      <td>er</td>\n",
       "      <td>er</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ic.gc</th>\n",
       "      <td>866</td>\n",
       "      <td>727.93</td>\n",
       "      <td>599.48</td>\n",
       "      <td>835.17</td>\n",
       "      <td>755.59</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ic.gr</th>\n",
       "      <td>962.64</td>\n",
       "      <td>758.89</td>\n",
       "      <td>783.05</td>\n",
       "      <td>793.06</td>\n",
       "      <td>846.29</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ic.ec</th>\n",
       "      <td>859.9</td>\n",
       "      <td>796.82</td>\n",
       "      <td>719.86</td>\n",
       "      <td>761.25</td>\n",
       "      <td>858.86</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ic.er</th>\n",
       "      <td>995.76</td>\n",
       "      <td>894.69</td>\n",
       "      <td>900.11</td>\n",
       "      <td>831.04</td>\n",
       "      <td>985.64</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ic.hp</th>\n",
       "      <td>1135.5</td>\n",
       "      <td>968.9</td>\n",
       "      <td>1048.3</td>\n",
       "      <td>1048.7</td>\n",
       "      <td>883.05</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>oc.gc</th>\n",
       "      <td>199.69</td>\n",
       "      <td>168.66</td>\n",
       "      <td>165.58</td>\n",
       "      <td>180.88</td>\n",
       "      <td>174.91</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>oc.gr</th>\n",
       "      <td>151.72</td>\n",
       "      <td>168.66</td>\n",
       "      <td>137.8</td>\n",
       "      <td>147.14</td>\n",
       "      <td>138.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>oc.ec</th>\n",
       "      <td>553.34</td>\n",
       "      <td>520.24</td>\n",
       "      <td>439.06</td>\n",
       "      <td>483</td>\n",
       "      <td>404.41</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>oc.er</th>\n",
       "      <td>505.6</td>\n",
       "      <td>486.49</td>\n",
       "      <td>404.74</td>\n",
       "      <td>425.22</td>\n",
       "      <td>389.52</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>oc.hp</th>\n",
       "      <td>237.88</td>\n",
       "      <td>199.19</td>\n",
       "      <td>171.47</td>\n",
       "      <td>222.95</td>\n",
       "      <td>178.49</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>income</th>\n",
       "      <td>7</td>\n",
       "      <td>5</td>\n",
       "      <td>4</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>agehed</th>\n",
       "      <td>25</td>\n",
       "      <td>60</td>\n",
       "      <td>65</td>\n",
       "      <td>50</td>\n",
       "      <td>25</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>rooms</th>\n",
       "      <td>6</td>\n",
       "      <td>5</td>\n",
       "      <td>2</td>\n",
       "      <td>4</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>region</th>\n",
       "      <td>ncostl</td>\n",
       "      <td>scostl</td>\n",
       "      <td>ncostl</td>\n",
       "      <td>scostl</td>\n",
       "      <td>valley</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "             0       1       2       3       4\n",
       "idcase       1       2       3       4       5\n",
       "depvar      gc      gc      gc      er      er\n",
       "ic.gc      866  727.93  599.48  835.17  755.59\n",
       "ic.gr   962.64  758.89  783.05  793.06  846.29\n",
       "ic.ec    859.9  796.82  719.86  761.25  858.86\n",
       "ic.er   995.76  894.69  900.11  831.04  985.64\n",
       "ic.hp   1135.5   968.9  1048.3  1048.7  883.05\n",
       "oc.gc   199.69  168.66  165.58  180.88  174.91\n",
       "oc.gr   151.72  168.66   137.8  147.14   138.9\n",
       "oc.ec   553.34  520.24  439.06     483  404.41\n",
       "oc.er    505.6  486.49  404.74  425.22  389.52\n",
       "oc.hp   237.88  199.19  171.47  222.95  178.49\n",
       "income       7       5       4       2       2\n",
       "agehed      25      60      65      50      25\n",
       "rooms        6       5       2       4       6\n",
       "region  ncostl  scostl  ncostl  scostl  valley"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Look at the raw Heating data\n",
    "wide_heating_df.head().T"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. Convert the wide format dataframes to long format"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3a. Perform needed data cleaning"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Noting that the column denoting the choice (depvar) contains string objects, we need to convert the choice column into an integer based column."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Convert the choice column for the Train data into integers\n",
    "# Note that we will use a 1 to denote 'choice1' and a 2 to \n",
    "# represent 'choice2'\n",
    "wide_heating_df[\"choice\"] = wide_heating_df[\"depvar\"].map(dict(zip(['gc', 'gr',\n",
    "                                                                    'ec', 'er',\n",
    "                                                                    'hp'],\n",
    "                                                                   range(1,6))))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For the Heating data, all of the alternatives are available in all choice situations. Note that, in general, this is not the case for choice data. As such we need to have columns that denote the availability of each alternative for each individual.\n",
    "\n",
    "These columns will all be filled with ones for each row in the wide format dataframes because all of the alternatives are always available for each individual."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Create the needed availability columns for the Heating data\n",
    "for i in range(1, 6):\n",
    "    wide_heating_df[\"availability_{}\".format(i)] = 1\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3b. Convert the Heating dataset to long format"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Index([u'idcase', u'depvar', u'ic.gc', u'ic.gr', u'ic.ec', u'ic.er', u'ic.hp',\n",
       "       u'oc.gc', u'oc.gr', u'oc.ec', u'oc.er', u'oc.hp', u'income', u'agehed',\n",
       "       u'rooms', u'region', u'choice', u'availability_1', u'availability_2',\n",
       "       u'availability_3', u'availability_4', u'availability_5'],\n",
       "      dtype='object')"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Look at the columns that we need to account for when converting from\n",
    "# the wide data format to the long data format.\n",
    "wide_heating_df.columns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "##########\n",
    "# Define lists of the variables pertaining to each variable type\n",
    "# that we need to account for in the data format transformation\n",
    "##########\n",
    "# Determine the name for the alternative ids in the long format \n",
    "# data frame\n",
    "heating_alt_id = \"alt_id\"\n",
    "# Determine the column that denotes the id of what we're treating\n",
    "# as individual observations, i.e. the choice situations.\n",
    "heating_obs_id_col = \"idcase\"\n",
    "# Determine what column denotes the choice that was made\n",
    "heating_choice_column = \"choice\"\n",
    "\n",
    "\n",
    "# Create the list of observation specific variables\n",
    "heating_ind_variables = [\"depvar\", \"income\", \"agehed\", \"rooms\", \"region\"]\n",
    "\n",
    "# Specify the variables that vary across individuals and some or all alternatives\n",
    "# Note that each \"main\" key should be the desired name of the column in the long\n",
    "# data format. The inner keys shoud be the alternative ids that that have some\n",
    "# value for the \"main\" key variable.\n",
    "heating_alt_varying_variables = {\"installation_costs\": {1: \"ic.gc\",\n",
    "                                                        2: \"ic.gr\",\n",
    "                                                        3: \"ic.ec\",\n",
    "                                                        4: \"ic.er\",\n",
    "                                                        5: \"ic.hp\"},\n",
    "                                 \"operating_costs\": {1: \"oc.gc\",\n",
    "                                                     2: \"oc.gr\",\n",
    "                                                     3: \"oc.ec\",\n",
    "                                                     4: \"oc.er\",\n",
    "                                                     5: \"oc.hp\"},\n",
    "                                  }\n",
    "\n",
    "# Specify the availability variables\n",
    "heating_availability_variables = OrderedDict()\n",
    "for alt_id, var in zip(range(1, 6),\n",
    "                       [\"availability_{}\".format(i) for i in range(1, 6)]):\n",
    "    heating_availability_variables[alt_id] = var\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>idcase</th>\n",
       "      <th>alt_id</th>\n",
       "      <th>choice</th>\n",
       "      <th>depvar</th>\n",
       "      <th>income</th>\n",
       "      <th>agehed</th>\n",
       "      <th>rooms</th>\n",
       "      <th>region</th>\n",
       "      <th>installation_costs</th>\n",
       "      <th>operating_costs</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>gc</td>\n",
       "      <td>7</td>\n",
       "      <td>25</td>\n",
       "      <td>6</td>\n",
       "      <td>ncostl</td>\n",
       "      <td>866.00</td>\n",
       "      <td>199.69</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>gc</td>\n",
       "      <td>7</td>\n",
       "      <td>25</td>\n",
       "      <td>6</td>\n",
       "      <td>ncostl</td>\n",
       "      <td>962.64</td>\n",
       "      <td>151.72</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>gc</td>\n",
       "      <td>7</td>\n",
       "      <td>25</td>\n",
       "      <td>6</td>\n",
       "      <td>ncostl</td>\n",
       "      <td>859.90</td>\n",
       "      <td>553.34</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>gc</td>\n",
       "      <td>7</td>\n",
       "      <td>25</td>\n",
       "      <td>6</td>\n",
       "      <td>ncostl</td>\n",
       "      <td>995.76</td>\n",
       "      <td>505.60</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "      <td>gc</td>\n",
       "      <td>7</td>\n",
       "      <td>25</td>\n",
       "      <td>6</td>\n",
       "      <td>ncostl</td>\n",
       "      <td>1135.50</td>\n",
       "      <td>237.88</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   idcase  alt_id  choice depvar  income  agehed  rooms  region  \\\n",
       "0       1       1       1     gc       7      25      6  ncostl   \n",
       "1       1       2       0     gc       7      25      6  ncostl   \n",
       "2       1       3       0     gc       7      25      6  ncostl   \n",
       "3       1       4       0     gc       7      25      6  ncostl   \n",
       "4       1       5       0     gc       7      25      6  ncostl   \n",
       "\n",
       "   installation_costs  operating_costs  \n",
       "0              866.00           199.69  \n",
       "1              962.64           151.72  \n",
       "2              859.90           553.34  \n",
       "3              995.76           505.60  \n",
       "4             1135.50           237.88  "
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "##########\n",
    "# Actually perform the conversion to long format\n",
    "##########\n",
    "long_heating_df = pl.convert_wide_to_long(wide_data=wide_heating_df,\n",
    "                                          ind_vars=heating_ind_variables,\n",
    "                                          alt_specific_vars=heating_alt_varying_variables,\n",
    "                                          availability_vars=heating_availability_variables,\n",
    "                                          obs_id_col=heating_obs_id_col,\n",
    "                                          choice_col=heating_choice_column,\n",
    "                                          new_alt_id_name=heating_alt_id)\n",
    "\n",
    "# Look at the long format Heating data\n",
    "long_heating_df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4. Create desired variables"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Create the life-cycle cost variable needed for model 2 where\n",
    "# we assume the discount rate, r, is 0.12.\n",
    "long_heating_df[\"life_cycle_cost\"] = (long_heating_df[\"installation_costs\"] + \n",
    "                                      long_heating_df[\"operating_costs\"] / 0.12)\n",
    "\n",
    "# Create the installation cost divided by income variable\n",
    "long_heating_df[\"installation_cost_burden\"] = (long_heating_df[\"installation_costs\"] /\n",
    "                                               long_heating_df[\"income\"])\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For numeric stability reasons, it is advised that one scale one's variables so that the estimated coefficients are similar in absolute magnitude, and if possible so that the estimated coefficients are close to 1 in absolute value (in other words, not terribly tiny or extremely large). This is done for the fishing data below"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5. Specify and estimate the desired models needed for benchmarking"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 5a. The model with installation cost and operating cost, without intercepts"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Create the model specification\n",
    "model_1_spec = OrderedDict()\n",
    "model_1_names = OrderedDict()\n",
    "\n",
    "# Note that for the specification dictionary, the\n",
    "# keys should be the column names from the long format\n",
    "# dataframe and the values should be a list with a combination\n",
    "# of alternative id's and/or lists of alternative id's. There \n",
    "# should be one element for each beta that will be estimated \n",
    "# in relation to the given column. Lists of alternative id's\n",
    "# mean that all of the alternatives in the list will get a\n",
    "# single beta for them, for the given variable.\n",
    "# The names dictionary should contain one name for each\n",
    "# element (that is each alternative id or list of alternative \n",
    "# ids) in the specification dictionary value for the same \n",
    "# variable\n",
    "\n",
    "model_1_spec[\"installation_costs\"] = [range(1, 6)]\n",
    "model_1_names[\"installation_costs\"] = [\"installation_costs\"]\n",
    "\n",
    "model_1_spec[\"operating_costs\"] = [range(1, 6)]\n",
    "model_1_names[\"operating_costs\"] = [\"operating_costs\"]\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Log-likelihood at zero: -1,448.4941\n",
      "Initial Log-likelihood: -1,448.4941\n",
      "Estimation Time: 0.37 seconds.\n",
      "Final log-likelihood: -1,095.2371\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/timothyb0912/anaconda/lib/python2.7/site-packages/pylogit/conditional_logit.py:482: OptimizeWarning: Unknown solver options: gtol\n",
      "  \"maxiter\": maxiter})\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<table class=\"simpletable\">\n",
       "<caption>Multinomial Logit Model Regression Results</caption>\n",
       "<tr>\n",
       "  <th>Dep. Variable:</th>         <td>choice</td>          <th>  No. Observations:  </th>     <td>900</td>   \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Model:</th>         <td>Multinomial Logit Model</td> <th>  Df Residuals:      </th>     <td>898</td>   \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Method:</th>                  <td>MLE</td>           <th>  Df Model:          </th>      <td>2</td>    \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Date:</th>             <td>Mon, 14 Mar 2016</td>     <th>  Pseudo R-squ.:     </th>    <td>0.244</td>  \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Time:</th>                 <td>18:51:13</td>         <th>  Pseudo R-bar-squ.: </th>    <td>0.242</td>  \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>converged:</th>              <td>True</td>           <th>  Log-Likelihood:    </th> <td>-1,095.237</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th> </th>                         <td> </td>            <th>  LL-Null:           </th> <td>-1,448.494</td>\n",
       "</tr>\n",
       "</table>\n",
       "<table class=\"simpletable\">\n",
       "<tr>\n",
       "           <td></td>             <th>coef</th>     <th>std err</th>      <th>z</th>      <th>P>|z|</th> <th>[95.0% Conf. Int.]</th> \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>installation_costs</th> <td>   -0.0062</td> <td>    0.000</td> <td>  -17.665</td> <td> 0.000</td> <td>   -0.007    -0.006</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>operating_costs</th>    <td>   -0.0046</td> <td>    0.000</td> <td>  -14.217</td> <td> 0.000</td> <td>   -0.005    -0.004</td>\n",
       "</tr>\n",
       "</table>"
      ],
      "text/plain": [
       "<class 'statsmodels.iolib.summary.Summary'>\n",
       "\"\"\"\n",
       "                     Multinomial Logit Model Regression Results                    \n",
       "===================================================================================\n",
       "Dep. Variable:                      choice   No. Observations:                  900\n",
       "Model:             Multinomial Logit Model   Df Residuals:                      898\n",
       "Method:                                MLE   Df Model:                            2\n",
       "Date:                     Mon, 14 Mar 2016   Pseudo R-squ.:                   0.244\n",
       "Time:                             18:51:13   Pseudo R-bar-squ.:               0.242\n",
       "converged:                            True   Log-Likelihood:             -1,095.237\n",
       "                                             LL-Null:                    -1,448.494\n",
       "======================================================================================\n",
       "                         coef    std err          z      P>|z|      [95.0% Conf. Int.]\n",
       "--------------------------------------------------------------------------------------\n",
       "installation_costs    -0.0062      0.000    -17.665      0.000        -0.007    -0.006\n",
       "operating_costs       -0.0046      0.000    -14.217      0.000        -0.005    -0.004\n",
       "======================================================================================\n",
       "\"\"\""
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Create an instance of the MNL model class\n",
    "model_1 = pl.create_choice_model(data=long_heating_df,\n",
    "                                 alt_id_col=heating_alt_id,\n",
    "                                 obs_id_col=heating_obs_id_col,\n",
    "                                 choice_col=heating_choice_column,\n",
    "                                 specification=model_1_spec,\n",
    "                                 model_type=\"MNL\",\n",
    "                                 names=model_1_names)\n",
    "\n",
    "# Estimate the given model, starting from a point of all zeros\n",
    "# as the initial values.\n",
    "model_1.fit_mle(np.zeros(2), method='newton-cg')\n",
    "\n",
    "# Look at the estimation summaries\n",
    "model_1.get_statsmodels_summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "Number of Parameters                                          2\n",
      "Number of Observations                                      900\n",
      "Null Log-Likelihood                                   -1448.494\n",
      "Fitted Log-Likelihood                                 -1095.237\n",
      "Rho-Squared                                           0.2438788\n",
      "Rho-Bar-Squared                                        0.242498\n",
      "Estimation Message        Optimization terminated successfully.\n",
      "dtype: object\n",
      "==============================\n",
      "                    parameters   std_err    t_stats      p_values  \\\n",
      "installation_costs   -0.006232  0.000353 -17.665275  7.763377e-70   \n",
      "operating_costs      -0.004580  0.000322 -14.216562  7.231945e-46   \n",
      "\n",
      "                    robust_std_err  robust_t_stats  robust_p_values  \n",
      "installation_costs        0.002489       -2.503814     1.228625e-02  \n",
      "operating_costs           0.000535       -8.560322     1.125527e-17  \n"
     ]
    }
   ],
   "source": [
    "# Look at the 'standard summary' since it includes robust p-values\n",
    "model_1.print_summaries()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Compare with mlogit"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The call from mlogit was as follows:\n",
    "<pre>\n",
    "Call:\n",
    "mlogit(formula = depvar ~ ic + oc | 0, data = H, method = \"nr\", \n",
    "    print.level = 0)\n",
    "\n",
    "Frequencies of alternatives:\n",
    "      ec       er       gc       gr       hp \n",
    "0.071111 0.093333 0.636667 0.143333 0.055556 \n",
    "\n",
    "nr method\n",
    "4 iterations, 0h:0m:0s \n",
    "g'(-H)^-1g = 1.56E-07 \n",
    "gradient close to zero \n",
    "\n",
    "Coefficients :\n",
    "      Estimate  Std. Error t-value  Pr(>|t|)    \n",
    "ic -0.00623187  0.00035277 -17.665 < 2.2e-16 \\*\\*\\*\n",
    "oc -0.00458008  0.00032216 -14.217 < 2.2e-16 \\*\\*\\*\n",
    "\\---\n",
    "Signif. codes:  0 ‘\\*\\*\\*’ 0.001 ‘\\*\\*’ 0.01 ‘\\*’ 0.05 ‘.’ 0.1 ‘ ’ 1\n",
    "\n",
    "Log-Likelihood: -1095.2\n",
    "</pre>\n",
    "As can be seen, the estimates, standard errors, t-values, and log-likelihood agree. The p-values differ but this is because mlogit calculates its p-values based on a t-distribution whereas pyLogit uses an asymptotic normal distribution."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 5b. The model that imposes the constraint that the discount rate, r = 0.12, still without intercepts."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Log-likelihood at zero: -1,448.4941\n",
      "Initial Log-likelihood: -1,448.4941\n",
      "Estimation Time: 0.26 seconds.\n",
      "Final log-likelihood: -1,248.7019\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<table class=\"simpletable\">\n",
       "<caption>Multinomial Logit Model Regression Results</caption>\n",
       "<tr>\n",
       "  <th>Dep. Variable:</th>         <td>choice</td>          <th>  No. Observations:  </th>     <td>900</td>   \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Model:</th>         <td>Multinomial Logit Model</td> <th>  Df Residuals:      </th>     <td>899</td>   \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Method:</th>                  <td>MLE</td>           <th>  Df Model:          </th>      <td>1</td>    \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Date:</th>             <td>Mon, 14 Mar 2016</td>     <th>  Pseudo R-squ.:     </th>    <td>0.138</td>  \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Time:</th>                 <td>18:51:13</td>         <th>  Pseudo R-bar-squ.: </th>    <td>0.137</td>  \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>converged:</th>              <td>True</td>           <th>  Log-Likelihood:    </th> <td>-1,248.702</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th> </th>                         <td> </td>            <th>  LL-Null:           </th> <td>-1,448.494</td>\n",
       "</tr>\n",
       "</table>\n",
       "<table class=\"simpletable\">\n",
       "<tr>\n",
       "           <td></td>             <th>coef</th>     <th>std err</th>      <th>z</th>      <th>P>|z|</th> <th>[95.0% Conf. Int.]</th> \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>installation_costs</th> <td>   -0.0007</td> <td> 4.28e-05</td> <td>  -16.741</td> <td> 0.000</td> <td>   -0.001    -0.001</td>\n",
       "</tr>\n",
       "</table>"
      ],
      "text/plain": [
       "<class 'statsmodels.iolib.summary.Summary'>\n",
       "\"\"\"\n",
       "                     Multinomial Logit Model Regression Results                    \n",
       "===================================================================================\n",
       "Dep. Variable:                      choice   No. Observations:                  900\n",
       "Model:             Multinomial Logit Model   Df Residuals:                      899\n",
       "Method:                                MLE   Df Model:                            1\n",
       "Date:                     Mon, 14 Mar 2016   Pseudo R-squ.:                   0.138\n",
       "Time:                             18:51:13   Pseudo R-bar-squ.:               0.137\n",
       "converged:                            True   Log-Likelihood:             -1,248.702\n",
       "                                             LL-Null:                    -1,448.494\n",
       "======================================================================================\n",
       "                         coef    std err          z      P>|z|      [95.0% Conf. Int.]\n",
       "--------------------------------------------------------------------------------------\n",
       "installation_costs    -0.0007   4.28e-05    -16.741      0.000        -0.001    -0.001\n",
       "======================================================================================\n",
       "\"\"\""
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Create the model specification\n",
    "model_2_spec = OrderedDict()\n",
    "model_2_names = OrderedDict()\n",
    "\n",
    "model_2_spec[\"life_cycle_cost\"] = [range(1, 6)]\n",
    "model_2_names[\"life_cycle_cost\"] = [\"installation_costs\"]\n",
    "\n",
    "# Create an instance of the MNL model class\n",
    "model_2 = pl.create_choice_model(data=long_heating_df,\n",
    "                                 alt_id_col=heating_alt_id,\n",
    "                                 obs_id_col=heating_obs_id_col,\n",
    "                                 choice_col=heating_choice_column,\n",
    "                                 specification=model_2_spec,\n",
    "                                 model_type=\"MNL\",\n",
    "                                 names=model_2_names)\n",
    "\n",
    "# Estimate the given model, starting from a point of all zeros\n",
    "# as the initial values.\n",
    "model_2.fit_mle(np.zeros(1), method='newton-cg')\n",
    "\n",
    "# Look at the estimation summaries\n",
    "model_2.get_statsmodels_summary()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Compare with mlogit\n",
    "Look at the corresponding results from mlogit:\n",
    "<pre>\n",
    "Call:\n",
    "mlogit(formula = depvar ~ lcc | 0, data = H, method = \"nr\", print.level = 0)\n",
    "\n",
    "Frequencies of alternatives:\n",
    "      ec       er       gc       gr       hp \n",
    "0.071111 0.093333 0.636667 0.143333 0.055556 \n",
    "\n",
    "nr method\n",
    "5 iterations, 0h:0m:0s \n",
    "g'(-H)^-1g = 9.32E-05 \n",
    "successive function values within tolerance limits \n",
    "\n",
    "Coefficients :\n",
    "       Estimate  Std. Error t-value  Pr(>|t|)    \n",
    "lcc -7.1585e-04  4.2761e-05 -16.741 < 2.2e-16 ***\n",
    "---\n",
    "Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1\n",
    "\n",
    "Log-Likelihood: -1248.7\n",
    "</pre>\n",
    "\n",
    "As before, all computed values agree except for the p-values, which we already know to be different due to the distribution being used to compute the p-values (t-distribution vs normal distribution)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 5c. The model with installation cost, operating cost, and all intercepts except that of the \"hp\" alternative"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Log-likelihood at zero: -1,448.4941\n",
      "Initial Log-likelihood: -1,448.4941\n",
      "Estimation Time: 0.02 seconds.\n",
      "Final log-likelihood: -1,008.2287\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/timothyb0912/anaconda/lib/python2.7/site-packages/scipy/optimize/_minimize.py:382: RuntimeWarning: Method BFGS does not use Hessian information (hess).\n",
      "  RuntimeWarning)\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<table class=\"simpletable\">\n",
       "<caption>Multinomial Logit Model Regression Results</caption>\n",
       "<tr>\n",
       "  <th>Dep. Variable:</th>         <td>choice</td>          <th>  No. Observations:  </th>     <td>900</td>   \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Model:</th>         <td>Multinomial Logit Model</td> <th>  Df Residuals:      </th>     <td>894</td>   \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Method:</th>                  <td>MLE</td>           <th>  Df Model:          </th>      <td>6</td>    \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Date:</th>             <td>Mon, 14 Mar 2016</td>     <th>  Pseudo R-squ.:     </th>    <td>0.304</td>  \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Time:</th>                 <td>18:51:14</td>         <th>  Pseudo R-bar-squ.: </th>    <td>0.300</td>  \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>converged:</th>              <td>False</td>          <th>  Log-Likelihood:    </th> <td>-1,008.229</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th> </th>                         <td> </td>            <th>  LL-Null:           </th> <td>-1,448.494</td>\n",
       "</tr>\n",
       "</table>\n",
       "<table class=\"simpletable\">\n",
       "<tr>\n",
       "           <td></td>             <th>coef</th>     <th>std err</th>      <th>z</th>      <th>P>|z|</th> <th>[95.0% Conf. Int.]</th> \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>ASC: gc</th>            <td>    1.7110</td> <td>    0.227</td> <td>    7.546</td> <td> 0.000</td> <td>    1.267     2.155</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>ASC: gr</th>            <td>    0.3083</td> <td>    0.207</td> <td>    1.492</td> <td> 0.136</td> <td>   -0.097     0.713</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>ASC: ec</th>            <td>    1.6588</td> <td>    0.448</td> <td>    3.699</td> <td> 0.000</td> <td>    0.780     2.538</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>ASC: er</th>            <td>    1.8534</td> <td>    0.362</td> <td>    5.121</td> <td> 0.000</td> <td>    1.144     2.563</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>installation_costs</th> <td>   -0.0015</td> <td>    0.001</td> <td>   -2.469</td> <td> 0.014</td> <td>   -0.003    -0.000</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>operating_costs</th>    <td>   -0.0070</td> <td>    0.002</td> <td>   -4.502</td> <td> 0.000</td> <td>   -0.010    -0.004</td>\n",
       "</tr>\n",
       "</table>"
      ],
      "text/plain": [
       "<class 'statsmodels.iolib.summary.Summary'>\n",
       "\"\"\"\n",
       "                     Multinomial Logit Model Regression Results                    \n",
       "===================================================================================\n",
       "Dep. Variable:                      choice   No. Observations:                  900\n",
       "Model:             Multinomial Logit Model   Df Residuals:                      894\n",
       "Method:                                MLE   Df Model:                            6\n",
       "Date:                     Mon, 14 Mar 2016   Pseudo R-squ.:                   0.304\n",
       "Time:                             18:51:14   Pseudo R-bar-squ.:               0.300\n",
       "converged:                           False   Log-Likelihood:             -1,008.229\n",
       "                                             LL-Null:                    -1,448.494\n",
       "======================================================================================\n",
       "                         coef    std err          z      P>|z|      [95.0% Conf. Int.]\n",
       "--------------------------------------------------------------------------------------\n",
       "ASC: gc                1.7110      0.227      7.546      0.000         1.267     2.155\n",
       "ASC: gr                0.3083      0.207      1.492      0.136        -0.097     0.713\n",
       "ASC: ec                1.6588      0.448      3.699      0.000         0.780     2.538\n",
       "ASC: er                1.8534      0.362      5.121      0.000         1.144     2.563\n",
       "installation_costs    -0.0015      0.001     -2.469      0.014        -0.003    -0.000\n",
       "operating_costs       -0.0070      0.002     -4.502      0.000        -0.010    -0.004\n",
       "======================================================================================\n",
       "\"\"\""
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Create the model specification\n",
    "model_3_spec = OrderedDict()\n",
    "model_3_names = OrderedDict()\n",
    "\n",
    "model_3_spec[\"intercept\"] = range(1, 5)\n",
    "model_3_names[\"intercept\"] = [\"ASC: {}\".format(x) \n",
    "                              for x in [\"gc\", \"gr\", \"ec\", \"er\"]]\n",
    "\n",
    "model_3_spec[\"installation_costs\"] = [range(1, 6)]\n",
    "model_3_names[\"installation_costs\"] = [\"installation_costs\"]\n",
    "\n",
    "model_3_spec[\"operating_costs\"] = [range(1, 6)]\n",
    "model_3_names[\"operating_costs\"] = [\"operating_costs\"]\n",
    "\n",
    "# Create an instance of the MNL model class\n",
    "model_3 = pl.create_choice_model(data=long_heating_df,\n",
    "                                 alt_id_col=heating_alt_id,\n",
    "                                 obs_id_col=heating_obs_id_col,\n",
    "                                 choice_col=heating_choice_column,\n",
    "                                 specification=model_3_spec,\n",
    "                                 model_type=\"MNL\",\n",
    "                                 names=model_3_names)\n",
    "\n",
    "# Estimate the given model, starting from a point of all zeros\n",
    "# as the initial values.\n",
    "model_3.fit_mle(np.zeros(6))\n",
    "\n",
    "# Look at the estimation summaries\n",
    "model_3.get_statsmodels_summary()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Compare with mlogit\n",
    "Look at the corresponding results from mlogit:\n",
    "<pre>\n",
    "Call:\n",
    "mlogit(formula = depvar ~ ic + oc, data = H, reflevel = \"hp\", \n",
    "    method = \"nr\", print.level = 0)\n",
    "\n",
    "Frequencies of alternatives:\n",
    "      hp       ec       er       gc       gr \n",
    "0.055556 0.071111 0.093333 0.636667 0.143333 \n",
    "\n",
    "nr method\n",
    "6 iterations, 0h:0m:0s \n",
    "g'(-H)^-1g = 9.58E-06 \n",
    "successive function values within tolerance limits \n",
    "\n",
    "Coefficients :\n",
    "                  Estimate  Std. Error t-value  Pr(>|t|)    \n",
    "ec:(intercept)  1.65884594  0.44841936  3.6993 0.0002162 ***\n",
    "er:(intercept)  1.85343697  0.36195509  5.1206 3.045e-07 ***\n",
    "gc:(intercept)  1.71097930  0.22674214  7.5459 4.485e-14 ***\n",
    "gr:(intercept)  0.30826328  0.20659222  1.4921 0.1356640    \n",
    "ic             -0.00153315  0.00062086 -2.4694 0.0135333 *  \n",
    "oc             -0.00699637  0.00155408 -4.5019 6.734e-06 ***\n",
    "---\n",
    "Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1\n",
    "\n",
    "Log-Likelihood: -1008.2\n",
    "McFadden R^2:  0.013691 \n",
    "Likelihood ratio test : chisq = 27.99 (p.value = 8.3572e-07)\n",
    "</pre>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Again, all calculated values except for the p-values and McFadden's $R^2$ seem to agree.\n",
    "\n",
    "As noted in the mlogit benchmark \\# 1 notebook, the mlogit values for McFadden's $R^2$ seem to be incorrect. Based off of the formula: $$\\begin{aligned} \\textrm{McFadden's }R^2 &= 1 - \\frac{\\mathscr{L}_M}{\\mathscr{L}_0} \\\\\n",
    "\\textrm{where } \\mathscr{L}_M &= \\textrm{the fitted log-likelihood} \\\\\n",
    "\\mathscr{L}_0 &= \\textrm{the null log-likelihood}\\end{aligned}$$\n",
    "from \"Coefficients of Determination for Multiple Logistic Regression Analysis\" by Scott Menard (2000), The American Statistician, 54:1, 17-24, the calculated value of McFadden's R^2 should be 0.303947 as reported by pyLogit.\n",
    "\n",
    "Note that the initial log-likelihood and McFadden $R^2$ are recomputed below for verification of its correctnes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "pyLogit's null log-likelihood is correct: True\n",
      "McFadden's R^2 is 0.30395\n"
     ]
    }
   ],
   "source": [
    "# Note that every observation in the Heating dataset\n",
    "# has 5 available alternatives, therefore the null \n",
    "# probability is 0.20\n",
    "null_prob = 0.20\n",
    "\n",
    "# Calculate how many observations are in the Heating \n",
    "# dataset\n",
    "num_heating_obs = wide_heating_df.shape[0]\n",
    "\n",
    "# Calculate the Fishing dataset's null log-likelihood \n",
    "null_heating_log_likelihood = (num_heating_obs * \n",
    "                               np.log(null_prob))\n",
    "\n",
    "# Determine whether pyLogit's null log-likelihood is correct\n",
    "correct_null_ll = np.allclose(null_heating_log_likelihood,\n",
    "                              model_3.null_log_likelihood)\n",
    "print \"pyLogit's null log-likelihood is correct:\", correct_null_ll\n",
    "\n",
    "# Calculate McFadden's R^2\n",
    "mcfaddens_r2 = 1 - (model_3.log_likelihood / model_3.null_log_likelihood)\n",
    "print \"McFadden's R^2 is {:.5f}\".format(mcfaddens_r2)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 5d. The model with installation cost divided by income, operating cost, and all intercepts except that for \"hp\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Log-likelihood at zero: -1,448.4941\n",
      "Initial Log-likelihood: -1,448.4941\n",
      "Estimation Time: 0.04 seconds.\n",
      "Final log-likelihood: -1,010.1975\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<table class=\"simpletable\">\n",
       "<caption>Multinomial Logit Model Regression Results</caption>\n",
       "<tr>\n",
       "  <th>Dep. Variable:</th>         <td>choice</td>          <th>  No. Observations:  </th>     <td>900</td>   \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Model:</th>         <td>Multinomial Logit Model</td> <th>  Df Residuals:      </th>     <td>894</td>   \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Method:</th>                  <td>MLE</td>           <th>  Df Model:          </th>      <td>6</td>    \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Date:</th>             <td>Mon, 14 Mar 2016</td>     <th>  Pseudo R-squ.:     </th>    <td>0.303</td>  \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Time:</th>                 <td>18:51:14</td>         <th>  Pseudo R-bar-squ.: </th>    <td>0.298</td>  \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>converged:</th>              <td>False</td>          <th>  Log-Likelihood:    </th> <td>-1,010.198</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th> </th>                         <td> </td>            <th>  LL-Null:           </th> <td>-1,448.494</td>\n",
       "</tr>\n",
       "</table>\n",
       "<table class=\"simpletable\">\n",
       "<tr>\n",
       "              <td></td>                <th>coef</th>     <th>std err</th>      <th>z</th>      <th>P>|z|</th> <th>[95.0% Conf. Int.]</th> \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>ASC: gc</th>                  <td>    1.9264</td> <td>    0.203</td> <td>    9.471</td> <td> 0.000</td> <td>    1.528     2.325</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>ASC: gr</th>                  <td>    0.4048</td> <td>    0.201</td> <td>    2.012</td> <td> 0.044</td> <td>    0.010     0.799</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>ASC: ec</th>                  <td>    1.8701</td> <td>    0.436</td> <td>    4.285</td> <td> 0.000</td> <td>    1.015     2.725</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>ASC: er</th>                  <td>    1.9341</td> <td>    0.360</td> <td>    5.372</td> <td> 0.000</td> <td>    1.228     2.640</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>installation_cost_burden</th> <td>   -0.0028</td> <td>    0.002</td> <td>   -1.460</td> <td> 0.144</td> <td>   -0.006     0.001</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>operating_costs</th>          <td>   -0.0071</td> <td>    0.002</td> <td>   -4.580</td> <td> 0.000</td> <td>   -0.010    -0.004</td>\n",
       "</tr>\n",
       "</table>"
      ],
      "text/plain": [
       "<class 'statsmodels.iolib.summary.Summary'>\n",
       "\"\"\"\n",
       "                     Multinomial Logit Model Regression Results                    \n",
       "===================================================================================\n",
       "Dep. Variable:                      choice   No. Observations:                  900\n",
       "Model:             Multinomial Logit Model   Df Residuals:                      894\n",
       "Method:                                MLE   Df Model:                            6\n",
       "Date:                     Mon, 14 Mar 2016   Pseudo R-squ.:                   0.303\n",
       "Time:                             18:51:14   Pseudo R-bar-squ.:               0.298\n",
       "converged:                           False   Log-Likelihood:             -1,010.198\n",
       "                                             LL-Null:                    -1,448.494\n",
       "============================================================================================\n",
       "                               coef    std err          z      P>|z|      [95.0% Conf. Int.]\n",
       "--------------------------------------------------------------------------------------------\n",
       "ASC: gc                      1.9264      0.203      9.471      0.000         1.528     2.325\n",
       "ASC: gr                      0.4048      0.201      2.012      0.044         0.010     0.799\n",
       "ASC: ec                      1.8701      0.436      4.285      0.000         1.015     2.725\n",
       "ASC: er                      1.9341      0.360      5.372      0.000         1.228     2.640\n",
       "installation_cost_burden    -0.0028      0.002     -1.460      0.144        -0.006     0.001\n",
       "operating_costs             -0.0071      0.002     -4.580      0.000        -0.010    -0.004\n",
       "============================================================================================\n",
       "\"\"\""
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Create the model specification\n",
    "model_4_spec = OrderedDict()\n",
    "model_4_names = OrderedDict()\n",
    "\n",
    "model_4_spec[\"intercept\"] = range(1, 5)\n",
    "model_4_names[\"intercept\"] = [\"ASC: {}\".format(x) \n",
    "                              for x in [\"gc\", \"gr\", \"ec\", \"er\"]]\n",
    "\n",
    "model_4_spec[\"installation_cost_burden\"] = [range(1, 6)]\n",
    "model_4_names[\"installation_cost_burden\"] = [\"installation_cost_burden\"]\n",
    "\n",
    "model_4_spec[\"operating_costs\"] = [range(1, 6)]\n",
    "model_4_names[\"operating_costs\"] = [\"operating_costs\"]\n",
    "\n",
    "# Create an instance of the MNL model class\n",
    "model_4 = pl.create_choice_model(data=long_heating_df,\n",
    "                                 alt_id_col=heating_alt_id,\n",
    "                                 obs_id_col=heating_obs_id_col,\n",
    "                                 choice_col=heating_choice_column,\n",
    "                                 specification=model_4_spec,\n",
    "                                 model_type=\"MNL\",\n",
    "                                 names=model_4_names)\n",
    "\n",
    "# Estimate the given model, starting from a point of all zeros\n",
    "# as the initial values.\n",
    "model_4.fit_mle(np.zeros(6))\n",
    "\n",
    "# Look at the estimation summaries\n",
    "model_4.get_statsmodels_summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Compare with mlogit\n",
    "Look at the results from mlogit:\n",
    "<pre>\n",
    "Call:\n",
    "mlogit(formula = depvar ~ oc + I(ic/income), data = H, reflevel = \"hp\", \n",
    "    method = \"nr\", print.level = 0)\n",
    "\n",
    "Frequencies of alternatives:\n",
    "      hp       ec       er       gc       gr \n",
    "0.055556 0.071111 0.093333 0.636667 0.143333 \n",
    "\n",
    "nr method\n",
    "6 iterations, 0h:0m:0s \n",
    "g'(-H)^-1g = 1.03E-05 \n",
    "successive function values within tolerance limits \n",
    "\n",
    "Coefficients :\n",
    "                 Estimate Std. Error t-value  Pr(>|t|)    \n",
    "ec:(intercept)  1.8700773  0.4364248  4.2850 1.827e-05 ***\n",
    "er:(intercept)  1.9340707  0.3599991  5.3724 7.768e-08 ***\n",
    "gc:(intercept)  1.9264254  0.2034031  9.4710 < 2.2e-16 ***\n",
    "gr:(intercept)  0.4047710  0.2011694  2.0121   0.04421 *  \n",
    "oc             -0.0071066  0.0015518 -4.5797 4.657e-06 ***\n",
    "I(ic/income)   -0.0027658  0.0018944 -1.4600   0.14430    \n",
    "---\n",
    "Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1\n",
    "\n",
    "Log-Likelihood: -1010.2\n",
    "McFadden R^2:  0.011765 \n",
    "Likelihood ratio test : chisq = 24.052 (p.value = 5.9854e-06)\n",
    "</pre>\n",
    "\n",
    "Again, all calculated values except for the p-values and McFadden's $R^2$ seem to agree."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 5e. The model with intallation costs, operating costs, alternative specific income, and all intercepts for \"hp.\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Log-likelihood at zero: -1,448.4941\n",
      "Initial Log-likelihood: -1,448.4941\n",
      "Estimation Time: 0.03 seconds.\n",
      "Final log-likelihood: -1,005.8885\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<table class=\"simpletable\">\n",
       "<caption>Multinomial Logit Model Regression Results</caption>\n",
       "<tr>\n",
       "  <th>Dep. Variable:</th>         <td>choice</td>          <th>  No. Observations:  </th>     <td>900</td>   \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Model:</th>         <td>Multinomial Logit Model</td> <th>  Df Residuals:      </th>     <td>890</td>   \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Method:</th>                  <td>MLE</td>           <th>  Df Model:          </th>     <td>10</td>    \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Date:</th>             <td>Mon, 14 Mar 2016</td>     <th>  Pseudo R-squ.:     </th>    <td>0.306</td>  \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Time:</th>                 <td>18:51:14</td>         <th>  Pseudo R-bar-squ.: </th>    <td>0.299</td>  \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>converged:</th>              <td>False</td>          <th>  Log-Likelihood:    </th> <td>-1,005.889</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th> </th>                         <td> </td>            <th>  LL-Null:           </th> <td>-1,448.494</td>\n",
       "</tr>\n",
       "</table>\n",
       "<table class=\"simpletable\">\n",
       "<tr>\n",
       "           <td></td>             <th>coef</th>     <th>std err</th>      <th>z</th>      <th>P>|z|</th> <th>[95.0% Conf. Int.]</th> \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>ASC: gc</th>            <td>    2.0552</td> <td>    0.486</td> <td>    4.225</td> <td> 0.000</td> <td>    1.102     3.008</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>ASC: gr</th>            <td>    1.1416</td> <td>    0.518</td> <td>    2.203</td> <td> 0.028</td> <td>    0.126     2.157</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>ASC: ec</th>            <td>    1.9545</td> <td>    0.704</td> <td>    2.778</td> <td> 0.005</td> <td>    0.576     3.333</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>ASC: er</th>            <td>    2.3056</td> <td>    0.624</td> <td>    3.695</td> <td> 0.000</td> <td>    1.083     3.528</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>installation_costs</th> <td>   -0.0015</td> <td>    0.001</td> <td>   -2.466</td> <td> 0.014</td> <td>   -0.003    -0.000</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>operating_costs</th>    <td>   -0.0070</td> <td>    0.002</td> <td>   -4.479</td> <td> 0.000</td> <td>   -0.010    -0.004</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>income_gc</th>          <td>   -0.0718</td> <td>    0.089</td> <td>   -0.809</td> <td> 0.419</td> <td>   -0.246     0.102</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>income_gr</th>          <td>   -0.1798</td> <td>    0.100</td> <td>   -1.796</td> <td> 0.073</td> <td>   -0.376     0.016</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>income_ec</th>          <td>   -0.0636</td> <td>    0.113</td> <td>   -0.562</td> <td> 0.574</td> <td>   -0.286     0.158</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>income_er</th>          <td>   -0.0969</td> <td>    0.108</td> <td>   -0.901</td> <td> 0.368</td> <td>   -0.308     0.114</td>\n",
       "</tr>\n",
       "</table>"
      ],
      "text/plain": [
       "<class 'statsmodels.iolib.summary.Summary'>\n",
       "\"\"\"\n",
       "                     Multinomial Logit Model Regression Results                    \n",
       "===================================================================================\n",
       "Dep. Variable:                      choice   No. Observations:                  900\n",
       "Model:             Multinomial Logit Model   Df Residuals:                      890\n",
       "Method:                                MLE   Df Model:                           10\n",
       "Date:                     Mon, 14 Mar 2016   Pseudo R-squ.:                   0.306\n",
       "Time:                             18:51:14   Pseudo R-bar-squ.:               0.299\n",
       "converged:                           False   Log-Likelihood:             -1,005.889\n",
       "                                             LL-Null:                    -1,448.494\n",
       "======================================================================================\n",
       "                         coef    std err          z      P>|z|      [95.0% Conf. Int.]\n",
       "--------------------------------------------------------------------------------------\n",
       "ASC: gc                2.0552      0.486      4.225      0.000         1.102     3.008\n",
       "ASC: gr                1.1416      0.518      2.203      0.028         0.126     2.157\n",
       "ASC: ec                1.9545      0.704      2.778      0.005         0.576     3.333\n",
       "ASC: er                2.3056      0.624      3.695      0.000         1.083     3.528\n",
       "installation_costs    -0.0015      0.001     -2.466      0.014        -0.003    -0.000\n",
       "operating_costs       -0.0070      0.002     -4.479      0.000        -0.010    -0.004\n",
       "income_gc             -0.0718      0.089     -0.809      0.419        -0.246     0.102\n",
       "income_gr             -0.1798      0.100     -1.796      0.073        -0.376     0.016\n",
       "income_ec             -0.0636      0.113     -0.562      0.574        -0.286     0.158\n",
       "income_er             -0.0969      0.108     -0.901      0.368        -0.308     0.114\n",
       "======================================================================================\n",
       "\"\"\""
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Create the model specification\n",
    "model_5_spec = OrderedDict()\n",
    "model_5_names = OrderedDict()\n",
    "\n",
    "model_5_spec[\"intercept\"] = range(1, 5)\n",
    "model_5_names[\"intercept\"] = [\"ASC: {}\".format(x) \n",
    "                              for x in [\"gc\", \"gr\", \"ec\", \"er\"]]\n",
    "\n",
    "model_5_spec[\"installation_costs\"] = [range(1, 6)]\n",
    "model_5_names[\"installation_costs\"] = [\"installation_costs\"]\n",
    "\n",
    "model_5_spec[\"operating_costs\"] = [range(1, 6)]\n",
    "model_5_names[\"operating_costs\"] = [\"operating_costs\"]\n",
    "\n",
    "model_5_spec[\"income\"] = range(1, 5)\n",
    "model_5_names[\"income\"] = [\"income_{}\".format(x)\n",
    "                           for x in [\"gc\", \"gr\", \"ec\", \"er\"]]\n",
    "\n",
    "# Create an instance of the MNL model class\n",
    "model_5 = pl.create_choice_model(data=long_heating_df,\n",
    "                                 alt_id_col=heating_alt_id,\n",
    "                                 obs_id_col=heating_obs_id_col,\n",
    "                                 choice_col=heating_choice_column,\n",
    "                                 specification=model_5_spec,\n",
    "                                 model_type=\"MNL\",\n",
    "                                 names=model_5_names)\n",
    "\n",
    "# Estimate the given model, starting from a point of all zeros\n",
    "# as the initial values.\n",
    "model_5.fit_mle(np.zeros(10))\n",
    "\n",
    "# Look at the estimation summaries\n",
    "model_5.get_statsmodels_summary()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Compare with mlogit\n",
    "Look at the output from mlogit:\n",
    "<pre>\n",
    "Call:\n",
    "mlogit(formula = depvar ~ oc + ic | income, data = H, reflevel = \"hp\", \n",
    "    method = \"nr\", print.level = 0)\n",
    "\n",
    "Frequencies of alternatives:\n",
    "      hp       ec       er       gc       gr \n",
    "0.055556 0.071111 0.093333 0.636667 0.143333 \n",
    "\n",
    "nr method\n",
    "6 iterations, 0h:0m:0s \n",
    "g'(-H)^-1g = 6.27E-06 \n",
    "successive function values within tolerance limits \n",
    "\n",
    "Coefficients :\n",
    "                  Estimate  Std. Error t-value  Pr(>|t|)    \n",
    "ec:(intercept)  1.95445797  0.70353833  2.7780 0.0054688 ** \n",
    "er:(intercept)  2.30560852  0.62390478  3.6954 0.0002195 ***\n",
    "gc:(intercept)  2.05517018  0.48639682  4.2253 2.386e-05 ***\n",
    "gr:(intercept)  1.14158139  0.51828845  2.2026 0.0276231 *  \n",
    "oc             -0.00696000  0.00155383 -4.4792 7.491e-06 ***\n",
    "ic             -0.00153534  0.00062251 -2.4664 0.0136486 *  \n",
    "ec:income      -0.06362917  0.11329865 -0.5616 0.5743846    \n",
    "er:income      -0.09685787  0.10755423 -0.9005 0.3678281    \n",
    "gc:income      -0.07178917  0.08878777 -0.8085 0.4187752    \n",
    "gr:income      -0.17981159  0.10012691 -1.7958 0.0725205 .  \n",
    "---\n",
    "Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1\n",
    "\n",
    "Log-Likelihood: -1005.9\n",
    "McFadden R^2:  0.01598 \n",
    "Likelihood ratio test : chisq = 32.67 (p.value = 1.2134e-05)\n",
    "</pre>\n",
    "\n",
    "As before, pyLogit and mlogit agree on all calculated values except for the p-values and McFadden's $R^2$."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}

<MSG> Staying updated.

<DFF> @@ -77,7 +77,7 @@
    "outputs": [],
    "source": [
     "# Load the Heating data, noting that the data is in wide data format\n",
-    "wide_heating_df = pd.read_csv(\"heating_data_r.csv\")\n"
+    "wide_heating_df = pd.read_csv(\"../data/heating_data_r.csv\")\n"
    ]
   },
   {
@@ -633,7 +633,7 @@
      "text": [
       "Log-likelihood at zero: -1,448.4941\n",
       "Initial Log-likelihood: -1,448.4941\n",
-      "Estimation Time: 0.37 seconds.\n",
+      "Estimation Time: 0.39 seconds.\n",
       "Final log-likelihood: -1,095.2371\n"
      ]
     },
@@ -660,10 +660,10 @@
        "  <th>Method:</th>                  <td>MLE</td>           <th>  Df Model:          </th>      <td>2</td>    \n",
        "</tr>\n",
        "<tr>\n",
-       "  <th>Date:</th>             <td>Mon, 14 Mar 2016</td>     <th>  Pseudo R-squ.:     </th>    <td>0.244</td>  \n",
+       "  <th>Date:</th>             <td>Tue, 30 Aug 2016</td>     <th>  Pseudo R-squ.:     </th>    <td>0.244</td>  \n",
        "</tr>\n",
        "<tr>\n",
-       "  <th>Time:</th>                 <td>18:51:13</td>         <th>  Pseudo R-bar-squ.: </th>    <td>0.242</td>  \n",
+       "  <th>Time:</th>                 <td>13:09:07</td>         <th>  Pseudo R-bar-squ.: </th>    <td>0.242</td>  \n",
        "</tr>\n",
        "<tr>\n",
        "  <th>converged:</th>              <td>True</td>           <th>  Log-Likelihood:    </th> <td>-1,095.237</td>\n",
@@ -692,8 +692,8 @@
        "Dep. Variable:                      choice   No. Observations:                  900\n",
        "Model:             Multinomial Logit Model   Df Residuals:                      898\n",
        "Method:                                MLE   Df Model:                            2\n",
-       "Date:                     Mon, 14 Mar 2016   Pseudo R-squ.:                   0.244\n",
-       "Time:                             18:51:13   Pseudo R-bar-squ.:               0.242\n",
+       "Date:                     Tue, 30 Aug 2016   Pseudo R-squ.:                   0.244\n",
+       "Time:                             13:09:07   Pseudo R-bar-squ.:               0.242\n",
        "converged:                            True   Log-Likelihood:             -1,095.237\n",
        "                                             LL-Null:                    -1,448.494\n",
        "======================================================================================\n",
@@ -743,9 +743,9 @@
       "\n",
       "Number of Parameters                                          2\n",
       "Number of Observations                                      900\n",
-      "Null Log-Likelihood                                   -1448.494\n",
-      "Fitted Log-Likelihood                                 -1095.237\n",
-      "Rho-Squared                                           0.2438788\n",
+      "Null Log-Likelihood                                    -1448.49\n",
+      "Fitted Log-Likelihood                                  -1095.24\n",
+      "Rho-Squared                                            0.243879\n",
       "Rho-Bar-Squared                                        0.242498\n",
       "Estimation Message        Optimization terminated successfully.\n",
       "dtype: object\n",
@@ -823,7 +823,7 @@
      "text": [
       "Log-likelihood at zero: -1,448.4941\n",
       "Initial Log-likelihood: -1,448.4941\n",
-      "Estimation Time: 0.26 seconds.\n",
+      "Estimation Time: 0.29 seconds.\n",
       "Final log-likelihood: -1,248.7019\n"
      ]
     },
@@ -842,10 +842,10 @@
        "  <th>Method:</th>                  <td>MLE</td>           <th>  Df Model:          </th>      <td>1</td>    \n",
        "</tr>\n",
        "<tr>\n",
-       "  <th>Date:</th>             <td>Mon, 14 Mar 2016</td>     <th>  Pseudo R-squ.:     </th>    <td>0.138</td>  \n",
+       "  <th>Date:</th>             <td>Tue, 30 Aug 2016</td>     <th>  Pseudo R-squ.:     </th>    <td>0.138</td>  \n",
        "</tr>\n",
        "<tr>\n",
-       "  <th>Time:</th>                 <td>18:51:13</td>         <th>  Pseudo R-bar-squ.: </th>    <td>0.137</td>  \n",
+       "  <th>Time:</th>                 <td>13:09:07</td>         <th>  Pseudo R-bar-squ.: </th>    <td>0.137</td>  \n",
        "</tr>\n",
        "<tr>\n",
        "  <th>converged:</th>              <td>True</td>           <th>  Log-Likelihood:    </th> <td>-1,248.702</td>\n",
@@ -871,8 +871,8 @@
        "Dep. Variable:                      choice   No. Observations:                  900\n",
        "Model:             Multinomial Logit Model   Df Residuals:                      899\n",
        "Method:                                MLE   Df Model:                            1\n",
-       "Date:                     Mon, 14 Mar 2016   Pseudo R-squ.:                   0.138\n",
-       "Time:                             18:51:13   Pseudo R-bar-squ.:               0.137\n",
+       "Date:                     Tue, 30 Aug 2016   Pseudo R-squ.:                   0.138\n",
+       "Time:                             13:09:07   Pseudo R-bar-squ.:               0.137\n",
        "converged:                            True   Log-Likelihood:             -1,248.702\n",
        "                                             LL-Null:                    -1,448.494\n",
        "======================================================================================\n",
@@ -972,7 +972,7 @@
      "name": "stderr",
      "output_type": "stream",
      "text": [
-      "/Users/timothyb0912/anaconda/lib/python2.7/site-packages/scipy/optimize/_minimize.py:382: RuntimeWarning: Method BFGS does not use Hessian information (hess).\n",
+      "/Users/timothyb0912/anaconda/lib/python2.7/site-packages/scipy/optimize/_minimize.py:385: RuntimeWarning: Method BFGS does not use Hessian information (hess).\n",
       "  RuntimeWarning)\n"
      ]
     },
@@ -991,10 +991,10 @@
        "  <th>Method:</th>                  <td>MLE</td>           <th>  Df Model:          </th>      <td>6</td>    \n",
        "</tr>\n",
        "<tr>\n",
-       "  <th>Date:</th>             <td>Mon, 14 Mar 2016</td>     <th>  Pseudo R-squ.:     </th>    <td>0.304</td>  \n",
+       "  <th>Date:</th>             <td>Tue, 30 Aug 2016</td>     <th>  Pseudo R-squ.:     </th>    <td>0.304</td>  \n",
        "</tr>\n",
        "<tr>\n",
-       "  <th>Time:</th>                 <td>18:51:14</td>         <th>  Pseudo R-bar-squ.: </th>    <td>0.300</td>  \n",
+       "  <th>Time:</th>                 <td>13:09:07</td>         <th>  Pseudo R-bar-squ.: </th>    <td>0.300</td>  \n",
        "</tr>\n",
        "<tr>\n",
        "  <th>converged:</th>              <td>False</td>          <th>  Log-Likelihood:    </th> <td>-1,008.229</td>\n",
@@ -1035,8 +1035,8 @@
        "Dep. Variable:                      choice   No. Observations:                  900\n",
        "Model:             Multinomial Logit Model   Df Residuals:                      894\n",
        "Method:                                MLE   Df Model:                            6\n",
-       "Date:                     Mon, 14 Mar 2016   Pseudo R-squ.:                   0.304\n",
-       "Time:                             18:51:14   Pseudo R-bar-squ.:               0.300\n",
+       "Date:                     Tue, 30 Aug 2016   Pseudo R-squ.:                   0.304\n",
+       "Time:                             13:09:07   Pseudo R-bar-squ.:               0.300\n",
        "converged:                           False   Log-Likelihood:             -1,008.229\n",
        "                                             LL-Null:                    -1,448.494\n",
        "======================================================================================\n",
@@ -1200,7 +1200,7 @@
      "text": [
       "Log-likelihood at zero: -1,448.4941\n",
       "Initial Log-likelihood: -1,448.4941\n",
-      "Estimation Time: 0.04 seconds.\n",
+      "Estimation Time: 0.02 seconds.\n",
       "Final log-likelihood: -1,010.1975\n"
      ]
     },
@@ -1219,10 +1219,10 @@
        "  <th>Method:</th>                  <td>MLE</td>           <th>  Df Model:          </th>      <td>6</td>    \n",
        "</tr>\n",
        "<tr>\n",
-       "  <th>Date:</th>             <td>Mon, 14 Mar 2016</td>     <th>  Pseudo R-squ.:     </th>    <td>0.303</td>  \n",
+       "  <th>Date:</th>             <td>Tue, 30 Aug 2016</td>     <th>  Pseudo R-squ.:     </th>    <td>0.303</td>  \n",
        "</tr>\n",
        "<tr>\n",
-       "  <th>Time:</th>                 <td>18:51:14</td>         <th>  Pseudo R-bar-squ.: </th>    <td>0.298</td>  \n",
+       "  <th>Time:</th>                 <td>13:09:08</td>         <th>  Pseudo R-bar-squ.: </th>    <td>0.298</td>  \n",
        "</tr>\n",
        "<tr>\n",
        "  <th>converged:</th>              <td>False</td>          <th>  Log-Likelihood:    </th> <td>-1,010.198</td>\n",
@@ -1263,8 +1263,8 @@
        "Dep. Variable:                      choice   No. Observations:                  900\n",
        "Model:             Multinomial Logit Model   Df Residuals:                      894\n",
        "Method:                                MLE   Df Model:                            6\n",
-       "Date:                     Mon, 14 Mar 2016   Pseudo R-squ.:                   0.303\n",
-       "Time:                             18:51:14   Pseudo R-bar-squ.:               0.298\n",
+       "Date:                     Tue, 30 Aug 2016   Pseudo R-squ.:                   0.303\n",
+       "Time:                             13:09:08   Pseudo R-bar-squ.:               0.298\n",
        "converged:                           False   Log-Likelihood:             -1,010.198\n",
        "                                             LL-Null:                    -1,448.494\n",
        "============================================================================================\n",
@@ -1395,10 +1395,10 @@
        "  <th>Method:</th>                  <td>MLE</td>           <th>  Df Model:          </th>     <td>10</td>    \n",
        "</tr>\n",
        "<tr>\n",
-       "  <th>Date:</th>             <td>Mon, 14 Mar 2016</td>     <th>  Pseudo R-squ.:     </th>    <td>0.306</td>  \n",
+       "  <th>Date:</th>             <td>Tue, 30 Aug 2016</td>     <th>  Pseudo R-squ.:     </th>    <td>0.306</td>  \n",
        "</tr>\n",
        "<tr>\n",
-       "  <th>Time:</th>                 <td>18:51:14</td>         <th>  Pseudo R-bar-squ.: </th>    <td>0.299</td>  \n",
+       "  <th>Time:</th>                 <td>13:09:08</td>         <th>  Pseudo R-bar-squ.: </th>    <td>0.299</td>  \n",
        "</tr>\n",
        "<tr>\n",
        "  <th>converged:</th>              <td>False</td>          <th>  Log-Likelihood:    </th> <td>-1,005.889</td>\n",
@@ -1451,8 +1451,8 @@
        "Dep. Variable:                      choice   No. Observations:                  900\n",
        "Model:             Multinomial Logit Model   Df Residuals:                      890\n",
        "Method:                                MLE   Df Model:                           10\n",
-       "Date:                     Mon, 14 Mar 2016   Pseudo R-squ.:                   0.306\n",
-       "Time:                             18:51:14   Pseudo R-bar-squ.:               0.299\n",
+       "Date:                     Tue, 30 Aug 2016   Pseudo R-squ.:                   0.306\n",
+       "Time:                             13:09:08   Pseudo R-bar-squ.:               0.299\n",
        "converged:                           False   Log-Likelihood:             -1,005.889\n",
        "                                             LL-Null:                    -1,448.494\n",
        "======================================================================================\n",
@@ -1573,7 +1573,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython2",
-   "version": "2.7.11"
+   "version": "2.7.12"
   }
  },
  "nbformat": 4,

Staying updated.