Human and Machine Intelligence

Assignments 8 and 9

Assignment 9 Due: 12/18/2013

The
goal of Assignments 8 and 9 is to apply your probabilistic modeling
skills to an
interesting problem domain. Either you will propose a domain of your
own, or I'll suggest a problem to work on.

For
assignment 8, I'd like you to specify a domain or problem you'd like to
tackle using probabilistic models, and to propose a model you'll use to
do this exploration. Hopefully you can find a problem from your own
research area that is amenable to a probabilistic modeling approach.
I'd like you to do more than run an existing model (e.g., topic
model). Hopefully you will customize models to your particular problem
domain. I'd also like to see you write your own code to implement
a model and not use prepackaged software.

I obtained data from a group of children playing a
battleship number line game. Try it out
(with 'whole numbers' and 'fractions'). I will give you data from the
first portion of their play and will ask you to predict the last
portion of their play. At the end of the semester, we'll have a
competition and you will each submit your predictions and I'll evaluate
them. I'll spend some time in class talking about this data set
so that you understand the excel
spreadsheet with the data.

You are to write up a page or two which
includes:

I want you to provide enough information that I understand the specific
approach you're going to take. I will try to give you rapid
feedback to help guide you in your implementation and evaluation. Please submit to me via email.

- a description of the domain [include this even if you are using the battleship data, so that I am confident that you understand the game]
- a description of the data set you have to experiment with, in detail including: number of data points and a complete list of features that you will include in your model
- A description of your modeling approach. If you are using the battleship data, you may want to compare several models; if so, describe each model. If you are using your own data, you must compare at least two models in order to evaluate the relative benefits of the different approaches. (You could propose one model that is simpler and one that is more complex; one model that is hierarchical and one that isn't; etc.)
- details of the model, e.g., the generative model assumptions
- your methodology for experimentation: how will you set hyperparameters? what sort of evaluation will you use (e.g., a single split of the data into training and testing components, vs. cross validation)

Expand on your write up for Assignment 8 and hand
in a complete report on your simulations. In addition to the
points above, include a summary of your results, with figures and
tables as appropriate, and some conclusions. I would like this to
be as close as you can come to writing a conference paper describing
novel research. Please submit to
me via email.