There are many scripts in the world, several of which are used by hundreds of millions of people. Handwrittencharacter recognition studies of several of these scripts arefound in the literature. Different hand-crafted feature sets havebeen used in these recognition studies. However, convolutionalneural network (CNN) has recently been used as an efficientunsupervised feature vector extractor. Although such a networkcan be used as a unified framework for both feature extractionand classification, it is more efficient as a feature extractor than asa classifier. In the present study, we performed certain amount of training of a 5-layer CNN for a moderately large class characterrecognition problem. We used this CNN trained for a larger classrecognition problem towards feature extraction of samples of several smaller class recognition problems. In each case, a distinctSupport Vector Machine (SVM) was used as the correspondingclassifier. In particular, the CNN of the present study is trainedusing samples of a standard 50-class Bangla basic characterdatabase and features have been extracted for 5 different 10-classnumeral recognition problems of English, Devanagari, Bangla,Telugu and Oriya each of which is an official Indian script.Recognition accuracies are comparable with the state-of-the-art