Q: Index gymnastics inside a Theano function I am using Theano to implement a neural n-gram language model along the lines of Bengio et al 2003. This model uses a distributed representation for words, and I'm having trouble writing a symbolic expression that allows me to take the gradient with respect to the word representation vectors.
Following the notation in the paper, I have a word representation matrix C of size V x m, where V is the vocabulary size and m is the dimensionality of the word embedding. Each row of C is a vector representation of a word.
My training data consists of n-grams drawn from a corpus. Let's say I let n = 3. Then I am trying to estimate P(wt|wt-1, wt-2). A neural network estimates this probability by using the concatenated embedding vectors for wt-1 and wt-2 to predict wt via a non-linear function. (See the paper for details.) Each word is represented by an index into a vocabulary which also indexes its representation row in C. If these indexes are i1, i2, and i3, I am trying to write a Theano expression for.
f(i_3, C[i_1].C[i_2])

where f contains a hidden layer and a non-linear function, and C[i_1].C[i_2] is the concatenation of the arrays C[i_1] and C[i_2]. The first thing I have to do is write a symbolic Theano expression for C[i_1].C[i_2]. Also, this function needs to take not just a single training instance, but a mini-batch of multiple training instances.
I know how to do this if I'm working directly with numpy matricies instead of abstract Theano expressions. For example, if C is shared, and X is a N x n - 1 minibatch of N training vectors of word indexes, I can look up the concatenated vectors like so:
C.get_value()[X].reshape(X.shape[0], -1)

(A bit of index gymnastics I learned elsewhere on StackOverflow.)
When I try to compile this expression into a Theano function, however, I run into errors.
X_var = T.lmatrix('X_var')
function([X_var], C[X_var].reshape(X_var.shape[0], -1))

The preceding gives me this error
Exception: ('The following error happened while compiling the node'
, Reshape{-1 (AdvancedSubtensor1.0, InplaceDimShuffle{x}.0), 
'\n', "Compilation failed (return status=1): 
/Users/williammcneill/.theano/compiledir_Darwin-13.3.0-x86_64-i386-64bit-i386-2.7.6-64/tmpOlUO0n/mod.cpp:300:31: error: 
'new_dims' declared as an array with a negative size.
             npy_intp new_dims[-1];.
                               ^~. 1 error generated.. ", '[Reshape{-1}(<TensorType(float64, matrix)>, <TensorType(int64, (True,))>)]')

I think this means that the index trick of putting -1 as the final reshape parameter is not supported by the Theano compiler.
The equivalent command gives a different error.
function([X_var], C[X_var].reshape(X_var.shape[0], X_var.shape[1]*X_var.shape[2]))

ValueError: Expected ndim to be an integer, is <class 'theano.tensor.var.TensorVariable'>

I need to write the symbolic expression for f so that I can take its gradient with respect to C. Can anyone help me do this?
Alternately, can someone point me to example Theano code that works with word embeddings. All the tutorial material I've found has been for writing neural nets over image data, but I haven't seen any examples of how to do distributed representations.

A: Well, I'm an idiot. Kinda. I was missing an extra pair of parentheses around my reshape argument. The following works.
function([X_var], C[X_var].reshape((X_var.shape[0], -1)))

It's confusing though because the reshape method of an array will take either two arguments like I have above or a tuple like I have in the answer, but Theano will only compile the latter.
