Benchmark of Different Gradient Descents in OCR

Marjan Kuchaki Rafsanjani; Masoud Pourshaban

doi:10.2478/cait-2014-0024

.blurhash-client-img { display: none !important; }

Benchmark of Different Gradient Descents in OCR

Cybernetics and Information Technologies

Volume 14 (2014): Issue 2 (June 2014)

By: Marjan Kuchaki Rafsanjani and Masoud Pourshaban

Open Access

|Jul 2014

Abstract

In this paper we implement six different learning algorithms in Optical Character Recognition (OCR) problem and achieve the criteria of end-time, number of iterations, train-set performance, test-set performance, validate-set performance and overall performance of these methods and compare them. Finally, we show the advantages and disadvantages of each method.

References

1. Broumandnia, A., J. Shanbehzadeh. Fast Zernike Wavelet Moments for Farsi Character Recognition. - Image and Vision Computing, Vol. 25, 2007, 717-726.10.1016/j.imavis.2006.05.014
Search in Google Scholar
2. Morita, M. E., E. Lethelier, A. E. Yacoubi, F. Bortolozzi, R. Sabourin, P. Pontifícia. Recognition of Handwritten Dates on Bankchecks Using an HMM Approach. - In: 13th Brazilian Symposium on Computer Graphics and Image Processing, Gramado, Brazil, 2000.
Search in Google Scholar
3. Pan, X., X. Ye, S. Zhang. A Hybrid Method for Robust Car Plate Character Recognition. -Engineering Applications of Artificial Intelligence, Vol. 18, 2005, No 8, 963-972.10.1016/j.engappai.2005.03.011
Search in Google Scholar
4. AI-A l a w i, R. A Hybrid w-Tuple Neuro-Fuzzy Classifier for Handwritten Numerals Recognition. - In: IEEE International Joint Conference on Neural Networks, Budapest, Hungary, 2004.
Search in Google Scholar
5. Amin, A. Recognition of Printed Arabic Text Based on Global Features and Decision Tree Learning Techniques. - Pattern Recognition, Vol. 33, 2000, No 8, 1309-1323.10.1016/S0031-3203(99)00114-4
Search in Google Scholar
6. N a g ab hushan, P., R. M. P a i. Modified Region Decomposition Method and Optimal Depth Decision Tree in the Recognition of Non-Uniform Sized Characters - An Experimentation with Kannada Characters. - Pattern Recognition Letters, Vol. 20, 1999, No 14, 1467-1475.10.1016/S0167-8655(99)00058-6
Search in Google Scholar
7. C h e o k, A. D., Z. J i a n, E. S. C h n g. Efficient Mobile Phone Chinese Optical Character Recognition Systems by Use of Heuristic Fuzzy Rules and Bigram Markov Language Models. - Applied Soft Computing, Vol. 8, 2008, No 2, 1005-1017.10.1016/j.asoc.2007.02.013
Search in Google Scholar
8. Chi, Z., M. Suters, H. Yan. Handwritten Digit Recognition Using Combined ID3-Derived Fuzzy Rules and Markov Chains. - Pattern Recognition, Vol. 29, 1996, No 11, 1821-1833.10.1016/0031-3203(96)00040-4
Search in Google Scholar
9. Hanmandlua, M., K. R. M. Mohanb, S. Chakrabortyc, S. Goyald. Unconstrained Handwritten Character Recognition Based on Fuzzy Logic. - Pattern Recognition, Vol. 36, 2003, 603-623.10.1016/S0031-3203(02)00069-9
Search in Google Scholar
10. Patil, P. M., T. R. Sontakke. Rotation, Scale and Translation Invariant Handwritten Devanagari Numeral Character Recognition Using General Fuzzy Neural Network. - Pattern Recognition, Vol. 40, 2007, No 7, 2110-2117.10.1016/j.patcog.2006.12.018
Search in Google Scholar
11. Goltsev, A., D. Rachkovskij. Combination of the Assembly Neural Network with a Perceptron for Recognition of Handwritten Digits Arranged in Numeral Strings. - Pattern Recognition, Vol. 38, 2005, No 3, 315-322.10.1016/j.patcog.2004.09.001
Search in Google Scholar
12. Lauer, F., C. Y. Suen, G. Bloch. A Trainable Feature Extractor for Handwritten Digit Recognition. - Pattern Recognition, Vol. 40, 2007, No 6, 1816-1824.10.1016/j.patcog.2006.10.011
Search in Google Scholar
13. Camastra, F. A SVM-Based Cursive Character Recognizer. - Pattern Recognition, Vol. 40, 2007, No 12, 3721-3727.10.1016/j.patcog.2007.03.014
Search in Google Scholar
14. Malon, C., S. Uchida, M. Suzuki. Mathematical Symbol Recognition with Support Vector Machines. - Pattern Recognition Letters, Vol. 29, 2008, No 9, 1326-1332.10.1016/j.patrec.2008.02.005
Search in Google Scholar
15. Amor, N. B., N. E. B. A mara. Multifont Arabic Character Recognition Using Hough Transform and Hidden Markov Models. - In: Fourth International Symposium on Image and Signal Processing and Analysis, Zagreb, Croatia, 2005.10.1109/ISPA.2005.195424
Search in Google Scholar
16. Bunke, H., M. Roth, E. Schukat-Talamazzini. Off-Line Cursive Handwriting Recognition Using Hidden Markov Models. - Pattern Recognition, Vol. 28, 1995, No 1, 399-1413.10.1016/0031-3203(95)00013-P
Search in Google Scholar
17. Khorsheed, M. S. Recognising Handwritten Arabic Manuscripts Using a Single Hidden Markov Model. - Pattern Recognition Letters, Vol. 24, 2003, No 14, 2235-2242.10.1016/S0167-8655(03)00050-3
Search in Google Scholar
18. Theeramunkong, T., C. Wongtapan. Off-Line Isolated Handwritten Thai OCR Using Island-Based Projection with w-Gram Model and Hidden Markov Models. - Information Processing and Management, Vol. 41, 2005, No 1, 139-160.10.1016/j.ipm.2004.04.011
Search in Google Scholar
19. Nagabhushan, P., R. M. Pai. Modified Region Decomposition Method and Optimal Depth Decision Tree in the Recognition of Non-Uniform Sized Characters - An Experimentation with Kannada Characters. - Pattern Recognition Letters, Vol. 20, 1999, No 14, 1467-1475.10.1016/S0167-8655(99)00058-6
Search in Google Scholar
20. Ping, Z., C. Lihui. A Novel Feature Extraction Method and Hybrid Tree Classification for Handwritten Numeral Recognition. - Pattern Recognition Letters, Vol. 23, 2002, No 1-3, 4556.10.1016/S0167-8655(01)00088-5
Search in Google Scholar
21. Oliveira, L. S., F. Bortolozzi, C. Y. Suen. Automatic Recognition of Handwritten Numerical Strings: A Recognition and Verification Strategy. - IEEE Transactions on Pattern Recognition and Machine Intelligence, Vol. 24, 2001, No 11, 1448-1456.10.1109/TPAMI.2002.1046154
Search in Google Scholar
22. Mohiuddin, K. M., J. Mao. A Comprehensive Study of Different Classifiers for HandPrinted Character Recognition. - Pattern Recognition, Practice IV, 1994, 437-448.10.1016/B978-0-444-81892-8.50043-2
Search in Google Scholar
23. Koerich, A. L. Unconstrained Handwritten Character Recognition Using Different Classification Strategies. - In: International Workshop on Artificial Neural Networks in Pattern Recognition (ANNPR), 2003.
Search in Google Scholar
24. Kim, H., K. K. Kim, C. Y. Suen. Hybrid Schemes of Homogeneous and Heterogeneous Classifiers for Cursive Word Recognition. - In: Proceedings of Seventh International Workshop on Frontiers in Handwritten Recognition, Amsterdam, 2000, 433-442.
Search in Google Scholar
25. Al-Omari, F. A., O. Al-Jarrah. Handwritten Indian Numerals Recognition System Using Probabilistic Neural Networks. - Advanced Engineering Informatics, Vol. 18, 2004, No 1, 916.10.1016/j.aei.2004.02.001
Search in Google Scholar
26. Pal, U., B. B. Chaudhuri. Indian Script Character Recognition: A Survey. - Pattern Recognition, Vol. 37, 2004, No 9, 1887-1899.10.1016/j.patcog.2004.02.003
Search in Google Scholar
27. Hung, K.-Y., R. W. P. L uk, D. S. Yeung, K. F. L. Chung, W. Shu. A Multiple Classifier Approach to Detect Chinese Character Recognition Errors. - Pattern Recognition, Vol. 38, 2005, No 5, 723-738.10.1016/j.patcog.2004.09.004
Search in Google Scholar
28. Li, Y.-X., C. L. Tan, X. Ding, C. Liu. Contextual Post-Processing Based on the Confusion Matrix in Offline Handwritten Chinese Script Recognition. - Pattern Recognition, Vol. 37, 2004, No 9, 1901-1912.10.1016/j.patcog.2004.03.002
Search in Google Scholar
29. Su, T.-H., T.-W. Zhang, D.-J. Guan, H.-J. Huang. Off-Line Recognition of Realistic Chinese Handwriting Using Segmentation-Free Strategy. - Pattern Recognition, Vol. 42, 2009, No 1, 167-182.10.1016/j.patcog.2008.05.012
Search in Google Scholar
30. Zhao, S., Z. Chi, P. Shi, H. Yan. Two-Stage Segmentation of Unconstrained Handwritten Chinese Characters. - Pattern Recognition, Vol. 36, 2003, No 1, 145-156.10.1016/S0031-3203(02)00041-9
Search in Google Scholar
31. Itoh, N. Japanese Language Model Based on Bigrams and Its Application to On-Line Character Recognition. - Pattern Recognition, Vol. 28, 1995, No 2, 135-141.10.1016/0031-3203(94)E0053-N
Search in Google Scholar
32. Kimura, F., T. Wakabayashi, S. Tsuruoka, Y. Miyake. Improvement of Handwritten Japanese Character Recognition Using Weighted Direction Code Histogram. -Pattern Recognition, Vol. 30, 1997, No 8, 1329-1337.10.1016/S0031-3203(96)00153-7
Search in Google Scholar
33. Srihari, S. N., T. Hong, G. Srikantan. Machine-Printed Japanese Document Recognition. - Pattern Recognition, Vol. 30, 1997, No 8, 1301-1313.10.1016/S0031-3203(96)00168-9
Search in Google Scholar
34. Al-Muhtaseb, H. A., S. A. Mahmo ud, R. S. Qahwaj i. Recognition of Off-Line Printed Arabic Text Using Hidden Markov Models. - Signal Processing, Vol. 88, 2008, No 12, 2902-2912.10.1016/j.sigpro.2008.06.013
Search in Google Scholar
35. Azmi, R., E. Kabir. A New Segmentation Technique for Omnifont Farsi Text. - Pattern Recognition Letters, Vol. 22, 2001, 97-104.10.1016/S0167-8655(00)00086-6
Search in Google Scholar
36. Dehghan, M., K. Faez, M. Ahmadi, M. Shridha r. Unconstrained Farsi Handwritten Word Recognition Using Fuzzy Vector Quantization and Hidden Markov Models. - Pattern Recognition Letters, 2001, 209-214.10.1016/S0167-8655(00)00090-8
Search in Google Scholar
37. Mozaffari, S., K. Faez, V. Märgner, H. El-Abed. Lexicon Reduction Using Dots for Off-Line Farsi/Arabic Handwritten Word Recognition. - Pattern Recognition Letters, Vol. 29, 2008, No 6, 724-734.10.1016/j.patrec.2007.11.009
Search in Google Scholar
38. Solimanpour, F., J. Sadri, C. Y. Suen. Standard Databases for Recognition of Handwritten Digits, Numerical Strings, Legal Amounts, Letters and Dates in Farsi Language. - In: 10th Int'l Workshop on Frontiers of Handwriting Recognition, La Baule, France, 2006.
Search in Google Scholar
39. Ziaratban, M., K. Faez, F. Faradji. Language-Based Feature Extraction Using Template-Matching in Farsi/Arabic Handwritten Numeral Recognition. - In: 9th Int'l Conference on Document Analysis and Recognition, Curitiba, Brazil, 2007.10.1109/ICDAR.2007.4405576
Search in Google Scholar
40. B lumen stein, M., X. Y. Liu, B. Verm a. A Modified Direction Feature for Cursive Character Recognition. - In: IEEE International Joint Conference on Neural Networks, Vol. 4, 2000, 2983-2987.
Search in Google Scholar
41. Fitzgerald, J. A., F. Geiselbrechtinger, T. Kechadi. Application of Fuzzy Logic to Online Recognition of Handwritten Symbols. - In: Proceedings of Ninth International Workshop on Frontiers in Handwritten Recognition, 2004, 395-400.
Search in Google Scholar
42. Fletcher, R., C. M. Reeves. Function Minimization by Conjugate Gradients. - Computer Journal, Vol. 7, 2004, 149-154.10.1093/comjnl/7.2.149
Search in Google Scholar
43. Hagan, M. T., H. B. Demuth, M. H. Be ale. Neural Network Design. Boston, MA, PWS Publishing, 1996.
Search in Google Scholar
44. Powell, M. J. D. Restart Procedures for the Conjugate Gradient Method. - Mathematical Programming, Vol. 12, 1977, 241-254.10.1007/BF01593790
Search in Google Scholar
45. Beale, E. M. L. A Derivation of Conjugate Gradients. - In: F. A. Lootsma, Ed. Numerical Methods for Nonlinear Optimization. London, Academic Press, 1972.
Search in Google Scholar
46. Mo l l e r, M. F. A Scaled Conjugate Gradient Algorithm for Fast Supervised Learning. - Neural Networks, Vol. 6, 1993, 525-533.10.1016/S0893-6080(05)80056-5
Search in Google Scholar
47. Battiti, R. First and Second Order Methods for Learning: Between Steepest Descent and Newton's Method. - Neural Computation, Vol. 2, 1992, No 4, 141-166.10.1162/neco.1992.4.2.141
Search in Google Scholar
48. Sexton, A. P., V. Sorge. Database-Driven Mathematical Character Recognition, Graphics Recognition, Algorithms and Applications (GREC). - In: Lecture Notes in Computer Science (LNCS), Hong Kong, 2006, 206-217.10.1007/11767978_20
Search in Google Scholar
49. Riedmiller, M., H. Braun. A Direct Adaptive Method for Faster Backpropagation Learning: The RPROP Algorithm. - In: Proceedings of IEEE International Conference on Neural Networks, 1993.
Search in Google Scholar
50. De Campos, T. E., B. R. Babu, M. Varma. Character Recognition in Natural Images. - In: Proceedings of the International Conference on Computer Vision Theory and Applications (VISAPP), Lisbon, Portugal, February 2009.
Search in Google Scholar

Articles in this issue

DOI: https://doi.org/10.2478/cait-2014-0024 | Journal eISSN: 1314-4081 | Journal ISSN: 1311-9702

Journal RSS Feed

Language: English

Page range: 114 - 126

Published on: Jul 15, 2014

Published by: Bulgarian Academy of Sciences, Institute of Information and Communication Technologies

In partnership with: Paradigm Publishing Services

Publication frequency: 4 issues per year

Keywords:

Learning algorithm,

neural network,

Optical Character Recognition (OCR),

pattern recognition

Related subjects:

Computer sciences,

Information technology

© 2014 Marjan Kuchaki Rafsanjani, Masoud Pourshaban, published by Bulgarian Academy of Sciences, Institute of Information and Communication Technologies
This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 3.0 License.

Volume 14 (2014): Issue 2 (June 2014)