Hi,
I’ve tried another time the same configuration as in experiment 1, still with random initialization of the weights, just to add the Bokeh plotting, but the results seems to be worse than experiment 1
In this experiment, I’m using a 3-convolution layered CNN
The configurations are as follows:

num_epochs= 100 
image_shape = (128,128)
filter_sizes = [(5,5),(5,5),(5,5)]
feature_maps = [20,50,80]
pooling_sizes = [(2,2),(2,2),(2,2)]
mlp_hiddens = [1000]
output_size = 2
weights_init=Uniform(width=0.2)
step_rule=Scale(learning_rate=0.1)
max_image_dim_limit_method= random crop
dataset_processing = rescale to 128*128

The result is after 100 epochs is that training error=35%, validation error= 35%

I kind of understand why the learning is kind of slow and why there’s no overfitting phenomen in 1.0 and 1.01 because I used RandomFixedSizeCrop to limit the max size of image to 128*128. This might crop the unimportant area area and label it as cats/dogs thus create some noises…
In next experiments, inspired by Florian’s approach, I will try to modify the MinimumImageDimension function so as to limit the maximum Image dimension.