You are encouraged to publicize your results on the class project through the newly-created leaderboard wiki, which you can find here.
The wiki is editable by anybody with a Github account, so feel free to add your results by yourself. Here are the instructions, taken from the wiki:
Use this wiki to publicize your results on the Dogs vs. Cats class project.
Every time you get a better result on the challenge, you can insert an entry to the list below (at the appropriate place please, so that the list stays sorted by test error rate). Make sure to include a link to a blog post detailing how you achieved that result. The format for entries is as follows:
<test error rate> (<train error rate>, <valid error rate>): <short description> (<link to blog post>)
IFT6266 - H2015 Representation Learning 
Vincent, is there a way to get more detailed performance results from Pylearn2 (such as confusion matrix) ?
I am pretty sure that it could significantly help to improve our algorithm for this (or any other future project)
LikeLike
Not that I’m aware of.
It’s not that difficult to create your own script that does that, though (provided you’re already familiar enough with Pylearn2 and Theano). I just pushed two scripts on my IFT6266H15 repo which compute error rates and confusion matrices respectively:
https://github.com/vdumoulin/ift6266h15/blob/master/code/pylearn2/scripts/dogs_vs_cats_error.py
and
https://github.com/vdumoulin/ift6266h15/blob/master/code/pylearn2/scripts/dogs_vs_cats_confusion.py
You can use these scripts as a starting point to compute fancier performance metrics.
LikeLiked by 2 people
Now I see that we can use .fprop() (ie. Y_hat = model.fprop(X) )
to get the prediction from a trained classifier.
Thank you for this.
Maor.
LikeLike
Vincent,
Is there a way to predict how much GPU memory a certain network configuration should take using Pylearn2 (before getting the actual allocation error)?
Thanks,
Maor.
LikeLike
Not that I’m aware of. I usually try whatever hyperparameters I chose and keep decreasing the batch size until the allocation error disappears.
You could count how many parameters you have and multiply that by either 32 or 64 bits (depending on the value of floatX you set) to get a rough estimate, but additional memory is used to store intermediary results, so you should treat that as a lower bound.
LikeLiked by 1 person
Thank you very much again. I think this could make a great feature/tool for pylearn2, as we are always constrained to fit in the gpu’s memory.
LikeLike