Almost a year ago, I revisited the Kaggle version of the Hand Written Digit Recognition problem, the link to that post is here. At that time, my go to language is R, since the majority of friends around me use R as well. This summer, I evidently switched back to use python as my primary language to do almost everything, because it is just so efficient.

So, here is a convolutional neural network using Keras to tackle this problem again, in less than 100 lines of code you can get a convolutional neural network and obtain 99% accuracy on the Kaggle leaderboard.

A quick note about training time, it took close to 9 minutes to be trained on my laptop with GeForce GTX 970M chip. You can increase the number of epochs and run it by yourself, it should be able to lead to better results.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 |
import pandas as pd import numpy as np from keras.models import Sequential from keras.layers.core import Dense, Dropout, Activation, Flatten from keras.layers.convolutional import Convolution2D, MaxPooling2D from keras.utils import np_utils np.random.seed(36) img_rows, img_cols = 28, 28 # input image dimensions batch_size = 128 # Number of images used in each optimization step nb_classes = 10 # 0 - 9 total ten digit nb_epoch = 35 # Number of times the whole data is used to learn # Read the train and test datasets train = pd.read_csv("data/train.csv") test = pd.read_csv("data/test.csv") # seperate out label Y = train["label"] train.drop(labels = "label", axis = 1, inplace = True) # convert data to np array train = train.values test = test.values # Reshape the data to be used by a Theano CNN. Shape is # (nb_of_samples, nb_of_color_channels, img_width, img_heigh) X_train = train.reshape(train.shape[0], 1, img_rows, img_cols) X_test = test.reshape(test.shape[0], 1, img_rows, img_cols) # Min-Max Scale X_train = X_train.astype('float32') / 255 X_test = X_test.astype('float32') / 255 # convert class vectors to binary class matrices (ie one-hot vectors) Y_train = np_utils.to_categorical(Y, nb_classes) def make_CNN_model(): model = Sequential() model.add(Convolution2D(12, 5, 5, border_mode='valid',input_shape=(1, img_rows, img_cols))) model.add(Activation('relu')) model.add(MaxPooling2D(pool_size=(2, 2))) model.add(Dropout(0.15)) model.add(Convolution2D(24, 5, 5)) model.add(Activation('relu')) model.add(MaxPooling2D(pool_size=(2, 2))) model.add(Dropout(0.15)) model.add(Flatten()) model.add(Dense(180)) model.add(Activation('relu')) model.add(Dropout(0.5)) model.add(Dense(100)) model.add(Activation('relu')) model.add(Dropout(0.5)) model.add(Dense(nb_classes)) model.add(Activation('softmax')) model.compile(loss='categorical_crossentropy', optimizer='adadelta', metrics=["accuracy"]) return model cnn_model = make_CNN_model() # train the model cnn_model.fit( X_train, Y_train, batch_size = batch_size, nb_epoch = nb_epoch, verbose = 1 ) # Predict the label for test set pred = cnn_model.predict_classes(X_test) # create a submission file np.savetxt( 'submission.csv', np.c_[range(1,len(pred)+1),pred], delimiter = ',', header = 'ImageId,Label', comments = '', fmt = '%d' ) |