I also ran an experiment on the cluster, so I used a different architecture, So I called this experiment 2, however the image max size limiting method is still RandomFixedSizeCrop
In this experiment, I’m using a 3-convolution layered CNN
The configurations are as follows:

num_epochs= 100 early stopped
image_shape = (128,128)
filter_sizes = [(5,5),(5,5),(5,5),(4,4),(4,4),(4,4)]
feature_maps = [20,40,60,80,100,120]
pooling_sizes = [(2,2),(2,2),(2,2),(2,2),(2,2),(2,2)]
mlp_hiddens = [1000]
output_size = 2
weights_init=Uniform(width=0.2)
step_rule=Adam()
max_image_dim_limit_method= RandomFixedSizeCrop
dataset_processing = rescale to 128*128

The result is after 100 epochs is that training error=19.57%, validation error=19.91%

TRAINING HAS BEEN FINISHED:

Training status:
batch_interrupt_received: False
epoch_interrupt_received: False
epoch_started: False
epochs_done: 100
iterations_done: 31300
received_first_batch: True
resumed_from: None
training_started: True
Log records from the iteration 31300:
saved_to: (‘catsVsDogs128.pkl’,)
time_read_data_this_epoch: 1.25491786003
time_read_data_total: 453.110922098
time_train_this_epoch: 163.532570601
time_train_total: 16410.0936284
train_cost: 0.408693790436
train_error_rate: 0.195686891675
train_total_gradient_norm: 4.36319255829
training_finish_requested: True
training_finished: True
valid_cost: 0.452142506838
valid_error_rate: 0.199169307947
valid_error_rate2: 0.199169307947

I kind of understand why the learning is kind of slow and why there’s no overfitting phenomen in 1.0 and 1.01 because I used RandomFixedSizeCrop to limit the max size of image to 128*128. This might crop the unimportant area area and label it as cats/dogs thus create some noises…
In next experiments, inspired by Florian’s approach, I will try to modify the MinimumImageDimension function so as to limit the maximum Image dimension.

The same datset pre-processing with more layers does help to boost performance, and the last convolution layer has 120 feature maps, which feeds more features to the MLP classifier.

In this experiment, I also used Adam() as learning rule, instead of fixed learning rate, this might also helps the performance to improve.