О применении искусственного интеллекта в распознавании текстов

Кисилев Игорь Николаевич

�� 004.8+930.25

��

�� -��

��

�. ��, ��

[email protected]

��

� ��

�� . �� .

�� : �� , �� , �� .

�� : �� .�. � �� // �� . 2024. � 1. �. 84�95.

Original article

ON THE AI APPLICATION IN TEXT RECOGNITION

The main stages of archival documents recognition using artificial intelligence are presented. Examples of recognition projects and used technological tools are provided.

Key words: archival documents, artificial intelligence, text recognition.

For citation: Kiselev I.N. On the AI application in text recognition. Vestnik VNIIDAD = Herald of VNIIDAD. 2024;1:84�95. (In Russian).

�� (��) � �� , �� .

�� , �� ,�� .

�� , �� , �� , �� , �� , ��, ��.

� �� : �� , ��, �� , �� , �� .�. � �� , �� , �� .

�� , �� .

� 2019 �� (��) �� : �� 14-� �� (��) ��: �� [1]. � �� .

�� , �� . � �� . �� (� �� ). �� -��, �� .

�� , �� (�� ) �� , �� .

�� :

�� ;

�� (�� );

�� ;

�� (�� );

�� .

�� , �� .

�� (��) ��. ��, �� .

�� , �� .�., �� . �� , �� .

�� (�� ) �� -��. �� . �� [2, 3].

�� (��, �� .�.) �� , �� .

�� , �� [4]. �� . �� . �� [5, 6] �� .

�� . �� , �� , �� .

�� , �.�. �� . �� . �� , �� , �� . �� ground truth (� �� , �� .).

�� . � �� , �� , �� , ��, ��, � �� .

�� 5�000 �� 15�000 �� (�� 25�75 ��) �� [7].

�� , �� [8].

�� , �� . �� (�� ), � �� . �� . �� (� ��) �� , �.�. � �� . �� (Character Error Rate, CER) � �� .

�� CER �� 2 �� 8% �� 0,5 �� 2% �� .

�� .

�� ReadCoop �� 134 �� 1, �� Transkribus. �� 頖 �� , � �� (��). �� CER, �� . �� CER �� , �� . �� XX ��2, �� XIX��3 ��.

�� , �� , �� . �� PyLaia4, Kraken5 [9], Calamari [10], Tesseract6. �� GitHub � �� .

� �� , �� . �� , �� , �� .

�� . �� , �� . �� , �� , �� . �� , �� , �� .

�� : �� ; �� ; �� , (�� ); �� ; �� .

�� READ-COOP7, �� 2019 �� Transkribus8. �� 1700 �� [11].

�� [12], �� 54 ��.

� 2020 �� 9. � 2024 �� -�� .

�� TEKLIA10, eScriptorium11 [13], OCR4all12.

�� . �� Transkribus �� , �� , �� .

�.�. �� (P.B. Str?bel) �� , � �� [14].

�� , �� .

�� Balsac13 [15, 16] � �� XIX �� XX ��. �� (��), �� (�� 44�742 �� 1�985 ��). �� , �� (��, ��, ��, ��) � ��; �� , �� .

�� -��14 �� . �� 85% ��.

�� , �� 18 ��. �� , �� XIII�� [17].

�� , �� : �� XV �� [18], �� 1868�1902 �� [19], �� [20], �� [21] � �� .

� �� .

��-�� : �� [22�24]. �� Digital ��15. �� I�16 �� 192�� .

�� , �� XVIII �� 1919 �� [25] �� . �� .

�� , �� . �� . �� , �� , � � �� , � �� .

�� . �� . �� , �� , �� 17, �� , �� 18.

�� (International Association for Pattern Recognition, IAPR)19.

�� 20, �� 21.

�� , �� (International conference on frontiers in handwriting recognition, ICFHR), �� (International Conference on Document Analysis and Recognition, ICDAR). �� , �� .

�� -�� InterPARES Trust AI (2021�2026). �� , �� , �� , �� -�� (�� 101 ��), �� (124 ��)22.

�� . � �� , �� , �� , �� .

� �� , �� (� ��) ��. �� , �� . �� , �� , �� , �� .

��

1. Seles A. A brave new world: artificial intelligence and archives [� �� : �� ] // 14th EASTICA General Conference and Seminar. 2019. URL: https://www.archives.go.jp/english/news/pdf/20191125_27e_01.pdf (�� : 24.11.2023).

2. Calvo-Zaragozaa J., Gallego A.-J. A selectional auto-encoder approach for document image binarization [�� ] // Published in Pattern Recognition. 2018. URL: https://arxiv.org/pdf/1706.10241.pdf (�� : 24.11.2023).

3. Westphal F., Lavesson N., Grahn H. Document image binarization using recurrent neural networks [�� ] // Proceedings � 13th IAPR International Workshop on Document Analysis Systems, DAS 2018. P. 263�268. URL: http://www.diva-portal.org/smash/get/diva2:1231250/FULLTEXT01.pdf (�� : 24.11.2023).

4. Xu Y., Yin F., Zhang Z., Liu C.-L. et al. Multi-task layout analysis for historical handwritten documents using fully convolutional networks [�� ] // Twenty-Seventh International Joint Conference on Artificial Intelligence (IJCAI). 2018. URL: https://www.researchgate.net/publication/326201381_Multi-task_Layout_Analysis_for_Historical_Handwritten_Documents_Using_Fully_Convolutional_Networks (�� : 24.11.2023).

5. Fischer A., W?thrich M., Liwicki M., Frinken V. Automatic transcription of handwritten medieval documents [�� ] // Conference: Proc. 15th Int. Conf. on Virtual Systems and Multimedia (VSMM�09). 2009. URL: https://www.researchgate.net/publication/228370463 (�� : 24.11.2023).

6. Zhao L., Wu Z., Wu X., Wilsbacher G., Wang S. Background-insensitive scene text recognition with text semantic segmentation [�� ] // Computer Vision � ECCV 2022. ECCV 2022. Lecture Notes in Computer Science. Vol. 13685. URL: https://www.ecva.net/papers/eccv_2022/papers_ECCV/papers/136850161.pdf (�� : 24.11.2023).

7. Model Training [�� ] // READ-COOP SCE. URL: https://readcoop.eu/glossary/model-training/ (�� : 24.11.2023).

8. Cl?rice T. Ground-truth free evaluation of HTR on old french and latin medieval literary manuscripts [�� HTR �� ] // CHR 2022: Computational Humanities Research Conference, December 12�14, 2022, Antwerp, Belgium. URL: https://ceur-ws.org/Vol-3290/long_paper2081.pdf (�� : 24.11.2023).

9. Kiessling B. Kraken � a Universal Text Recognizer for the Humanities [Kraken � �� ] // DataverseNL. 2019. V2. URL: https://dh-abstracts.library.virginia.edu/works/9912 (�� : 24.11.2023).

10. Wick C.,�Reul C., Puppe F. Calamari ? a high-performance Tensorflow-based deep learning package for optical character recognition [Calamari � �� Tensorflow �� ] // Digital Humanities Quarterly. 2020. Vol. 14. � 2. URL: http://www.digitalhumanities.org/dhq/vol/14/2/000451/000451.html (�� : 24.11.2023).

11. Nockels J.,Gooding P., Ames S., Terras M. Understanding the application of handwritten text recognition technology in heritage contexts: a systematic review of Transkribus in published research [�� : �� Transkribus � �� ] // Archival Science. 2022. 22. P. 367�392. URL: https://doi.org/10.1007/s10502-022-09397-0 (�� : 24.11.2023).

12. Muehlberger G. Transforming scholarship in the archives through handwritten text recognition Transkribus as a case study [�� Transkribus �� ] // Journal of Documentation. 2018. Vol. 75. Issue 5. P. 954-976.

13. Stokes P.A., Kiessling B. The eScriptorium VRE for manuscript cultures [eScriptorium VRE �� ] // Classics. 2021. Vol. 18. URL: https://classics-at.chs.harvard.edu/classics18-stokes-kiessling-stokl-ben-ezra-tissot-gargem/ (�� : 24.11.2023).

14. Str?bel P.B. Flexible Techniques for Automatic Text Recognition of Historical Documents [�� ] // University of Zurich. 2023. URL: https://www.researchgate.net/publication/372501015_Flexible_Techniques_for_Automatic_Text_Recognition_of_Historical_Documents (�� : 24.11.2023).

15. Maarand M. BALSAC project registers have been processed! [�� BALSAC ��!] // Teklia. 2022. URL: https://teklia.com/blog/202202-balsac/ (�� : 24.11.2023).

16. Tarride S. Large Scale Genealogical information extraction from handwritten Quebec Parish Records [�� ] // Research square. 2022. URL: https://doi.org/10.21203/rs.3.rs-2260181/v1 (�� : 24.11.2023).

17. Firmani D., Maiorino M., Merialdo P., Nieddu E. Towards knowledge discovery from the Vatican Secret Archives. In codice ratio � Episode 1: Machine transcription of the manuscripts [�� . � �� 1: �� ] // Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (KDD �18). ACM, New York, NY, USA. 2018. URL: https://arxiv.org/abs/1803.03200 (�� : 24.11.2023).

18. Ertl T., Schmidle W., Helmchen J., Duval T. Mapping. Medieval Vienna: The Social Topography of Vienna in the 15th Century [��. �� : �� XV ��] // Freie Universitat, Berlin. URL: https://www.geschkult.fu-berlin.de/e/fmi/bereiche/mittelalter/ab_ertl/Mapping-Vienna.html (�� : 24.11.2023).

19. Karsvall O. Maskintolkning av handskrivna�k?llmaterial [�� ]. RIKSARKIVET. URL: https://riksarkivet.se/htr (�� : 24.11.2023).

20. How to make a complete collection accessible with Transkribus. A best-practice example from the Tyrolean State Archives [�� Transkribus. �� ] // READ-COOP SCE. URL: https://readcoop.eu/success-stories/grundbuchblaetter/ (�� : 24.11.2023).

21. Kermorvant C. Belfort city archives: a pilot project for automatic recognition of city council registers [�� : �� ] // Teklia. 2022. URL: https://teklia.com/blog/202211-belfort-en/ (�� : 24.11.2023).

22. �� ., �� ., �� ., �� . �� : �� // �� . 2020. � 4. �. 64�71. URL: https://portal.historyrussia.org/img/news/VP_4.2020.pdf#page=33 (�� : 24.11.2023).

23. �� : �� // �� . 2021. URL: https://historyrussia.org/sobytiya/ideya-proekta-petru-prishlas-by-po-dushe.html (�� : 24.11.2023).

24. �� .�. �� // �� . 2020. � 4. �. 74�75.

25. �� . �� // ��. 2023. URL: https://stimul.online/articles/science-and-technology/neyroset-pomozhet-prochitat-starinnye-rukopisi/ (�� : 24.11.2023).

References

iНе можете найти то, что вам нужно? Попробуйте сервис подбора литературы.

1. Seles A. A brave new world: artificial intelligence and archives. 14th EASTICA Gene�ral Conference and Seminar. 2019. URL: https://www.archives.go.jp/english/news/pdf/20191125_27e_01.pdf (accessed: 24.11.2023).

2. Calvo-Zaragozaa J., Gallego A.-J. A selectional auto-encoder approach for document image binarization. Published in Pattern Recognition. 2018. URL: https://arxiv.org/pdf/1706.10241.pdf (accessed: 24.11.2023).

3. Westphal F., Lavesson N., Grahn H. Document image binarization using recurrent neural networks. Proceedings � 13th IAPR International Workshop on Document Analysis Systems, DAS 2018. P. 263�268. URL: http://www.diva-portal.org/smash/get/diva2:1231250/FULLTEXT01.pdf (accessed: 24.11.2023).

4. Xu Y., Yin F., Zhang Z., Liu C.-L. et al. Multi-task layout analysis for historical handwritten documents using fully convolutional networks. Twenty-Seventh International Joint Confe�rence on Artificial Intelligence (IJCAI). 2018. URL: https://www.researchgate.net/publication/326201381_Multi-task_Layout_Analysis_for_Historical_Handwritten_Documents_Using_Fully_Convolutional_Networks (accessed: 24.11.2023).

5. Fischer A., W?thrich M., Liwicki M., Frinken V. Automatic transcription of handwritten medieval documents. Conference: Proc. 15th Int. Conf. on Virtual Systems and Multimedia (VSMM�09). 2009. URL: https://www.researchgate.net/publication/228370463 (accessed: 24.11.2023).

6. Zhao L., Wu Z., Wu X., Wilsbacher G., Wang S. Background-insensitive scene text recognition with text semantic segmentation. Computer Vision � ECCV 2022. ECCV 2022. Lecture Notes in Computer Science. Vol. 13685. URL: https://www.ecva.net/papers/eccv_2022/papers_ECCV/papers/136850161.pdf (accessed: 24.11.2023).

7. Model Training. READ-COOP SCE. URL: https://readcoop.eu/glossary/model-training/ (accessed: 24.11.2023).

8. Cl?rice T. Ground-truth free evaluation of HTR on old French and Latin medieval literary manuscripts. CHR 2022: Computational Humanities Research Conference, December 12�14, 2022, Antwerp, Belgium. URL: https://ceur-ws.org/Vol-3290/long_paper2081.pdf (accessed: 24.11.2023).

9. Kiessling B. Kraken � a Universal Text Recognizer for the Humanities. DataverseNL. 2019. V2. URL: https://dh-abstracts.library.virginia.edu/works/9912 (accessed: 24.11.2023).

10. Wick C., Reul C., Puppe F. Calamari ? A High-Performance Tensorflow-based Deep Lear�ning Package for Optical Character Recognition. Digital Humanities Quarterly. 2020;2(14). URL: http://www.digitalhumanities.org/dhq/vol/14/2/000451/000451.html (accessed: 24.11.2023).

11. Nockels J., Gooding P., Ames S., Terras M. Understanding the application of handwritten text recognition technology in heritage contexts: a systematic review of Transkribus in published research. Archival Science. 2022;22:367�392. URL: https://doi.org/10.1007/s10502-022-09397-0 (accessed: 24.11.2023).

12. Muehlberger G. Transforming scholarship in the archives through handwritten text recognition Transkribus as a case study. Journal of Documentation. 2018;5(75):954-976.

13. Stokes P.A., Kiessling B. The eScriptorium VRE for manuscript cultures. Classics. 2021. Vol. 18. URL: https://classics-at.chs.harvard.edu/classics18-stokes-kiessling-stokl-ben-ezra-tissot-gargem/ (accessed: 24.11.2023).

14. Str?bel P.B. Flexible Techniques for Automatic Text Recognition of Historical Documents. University of Zurich. 2023. URL: https://www.researchgate.net/publication/372501015_Flexible_Techniques_for_Automatic_Text_Recognition_of_Historical_Documents (accessed: 24.11.2023).

15. Maarand M. BALSAC project registers have been processed! Teklia. 2022. URL: https://teklia.com/blog/202202-balsac/ (accessed: 24.11.2023).

16. Tarride S. Large Scale Genealogical information extraction from handwritten Quebec Parish Records. Research square. 2022. URL: https://doi.org/10.21203/rs.3.rs-2260181/v1 (accessed: 24.11.2023).

17. Firmani D., Maiorino M., Merialdo P., Nieddu E. Towards knowledge discovery from the Vatican Secret Archives. In codice ratio � Episode 1: Machine transcription of the manuscripts. Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (KDD �18). ACM, New York, NY, USA. 2018. URL: https://arxiv.org/abs/1803.03200 (accessed: 24.11.2023).

18. Ertl T., Schmidle W., Helmchen J., Duval T. Mapping. Medieval Vienna: The Social Topography of Vienna in the 15th Century. Freie Universitat, Berlin. URL: https://www.geschkult.fu-berlin.de/e/fmi/bereiche/mittelalter/ab_ertl/Mapping-Vienna.html (accessed: 24.11.2023).

19. Karsvall O. Maskintolkning av handskrivna k?llmaterial. RIKSARKIVET. URL: https://riksarkivet.se/htr (accessed: 24.11.2023). (In Swedish).

20. How to make a complete collection accessible with Transkribus. A best-practice example from the Tyrolean State Archives. READ-COOP SCE. URL: https://readcoop.eu/success-stories/grundbuchblaetter/ (accessed: 24.11.2023).

21. Kermorvant C. Belfort city archives: a pilot project for automatic recognition of city council registers. Teklia. 2022. URL: https://teklia.com/blog/202211-belfort-en/ (accessed: 24.11.2023).

22. Bazarova T., Dimitrov D., Potanin M., Proskuryakova M. Raspoznat` i transkribirovat`: avtografy` Petra Velikogo i texnologii iskusstvennogo intellekta [Recognize and transcribe: autographs of Peter the Great and artificial intelligence technologies]. Voronczovo pole = Vorontsovo Pole. 2020;4:64�71. URL: https://portal.historyrussia.org/img/news/VP_4.2020.pdf#page=33 (accessed: 24.11.2023). (In Russian).

23. Vladimir Arakcheev: Ideya proekta Petru prishlas` by` po dushe [Vladimir Arakcheev: Peter would have liked the idea of the project]. Russian Historical Society. 2021. URL: https://historyrussia.org/sobytiya/ideya-proekta-petru-prishlas-by-po-dushe.html (accessed: 24.11.2023). (In Russian).

24. Sirenov A.V. Zdes` my` vidim dejstvitel`no nauchny`j metod [Here we see a truly scientific method]. Voronczovo pole = Vorontsovo Pole. 2020;4:74�75. (In Russian).

25. Andreev A. Nejroset` pomozhet prochitat` starinny`e rukopisi [A neural network will help read ancient manuscripts]. Stimul. 2023. URL: https://stimul.online/articles/science-and-technology/neyroset-pomozhet-prochitat-starinnye-rukopisi/ (accessed: 24.11.2023). (In Russian).

��

�� , �� , �� -�� (��), ��, �� .

INFORMATION ABOUT THE AUTHORS

Igor N. Kiselev, PhD (in history), senior researcher of Archival Science Department of the All-Russian Scientific and Research Institute for Records and Archives Management (VNIIDAD), Moscow, Russian Federation.

1 Public AI models in Transkribus. URL: https://readcoop.eu/transkribus/public-models/ (�� : 24.11.2023).

2 Russian Handwriting early 20th century. URL: https://readcoop.eu/model/russian-handwriting-early-20th-century/ (�� : 24.11.2023).

3 Russian Civil Records late XIX cent. URL: https://readcoop.eu/model/russian-civil-records-late-xix-cent/ (�� : 24.11.2023).

4 PyLaia. Pattern Recognition and Human Language Technology (PRHLT) Research Center. URL: https://github.com/jpuigcerver/pylaia (�� : 24.11.2023).

5 Kraken. Mittagessen. URL: https://kraken.re/main/index.html (�� : 30.09.2023); Kraken. RESILIENCE project. URL: https://github.com/mittagessen/kraken (�� : 30.09.2023); Training kraken. Mittagessen. URL: https://kraken.re/main/training.html#evaluation-and-validation (�� : 24.11.2023).

6 Tesseract-ocr. URL: https://github.com/tesseract-ocr/ (�� : 24.11.2023); Tesseract User Manual. URL: https://tesseract-ocr.github.io/tessdoc/#external-projects (�� : 24.11.2023).

7 We revolutionise Access to Historical Document. URL: https://readcoop.eu (�� : 24.11.2023).

8 Transkribus. URL: https://readcoop.eu/transkribus (�� : 24.11.2023).

9 Packages & Plans. URL: https://readcoop.eu/transkribus/credits/ (�� : 24.11.2023).

10 Automatic Document Processing with AI. Teklia. URL: https://teklia.com (�� : 24.11.2023).

11 Stokes P. RESILIENCE Tool: eScriptorium. RESILIENCE. 2020. URL: https://www.resilience-ri.eu/blog/resilience-tool-escriptorium/ (�� : 24.11.2023).

12 OCR4all: Optical Character Recognition (and more) for everyone. Centre for Philology and Digitality. URL: https://www.ocr4all.org (�� : 24.11.2023).

13 Balsac. Teklia. 2023. URL: https://teklia.com/research/projects/balsac/details/ (�� : 24.11.2023).

14 AI for cataloguing at the Sainte Genevi?ve library. Teklia. 2023. URL: https://teklia.com/blog/cataloguing-with-AI-at-BSG/ (�� : 24.11.2023).

15 Digital ϸ�� // Sber AI. URL: https://projects.tib.eu/en/viva/projekt/; https://fusionbrain.ai/digital-petr (�� : 24.11.2023).

16 �� I: �� // ��. URL: https://peterscript.historyrussia.org/documents (�� : 24.11.2023).

17 ��., ��,�DIVAHisDB Dataset of Medieval Manuscripts. University of Fribourg. URL: https://www.unifr.ch/inf/diva/en/research/software-data/diva-hisdb.html (�� : 24.11.2023).

18 ��., ��, About Zenodo. CERN data centre & Invenio. URL: https://about.zenodo.org (�� : 24.11.2023).

19 IARP. URL: https://iapr.org (�� : 24.11.2023).

20 IAPR Newsletter. IARP. URL: https://iapr.org/articles/newsletter (�� : 24.11.2023).

21 International Journal on Document Analysis and Recognition (IJDAR). Springer. URL: https://www.springer.com/journal/10032/ (�� : 24.11.2023).

22 Research Dissemination. InterPARES TRUST AI. URL: https://interparestrustai.org/trust/research_dissemination (�� : 24.11.2023).

---------------

------------------------------------------------------------

---------------

------------------------------------------------------------

О применении искусственного интеллекта в распознавании текстов Текст научной статьи по специальности «История и археология»

Аннотация научной статьи по истории и археологии, автор научной работы — Кисилев Игорь Николаевич

Похожие темы научных работ по истории и археологии , автор научной работы — Кисилев Игорь Николаевич

On the AI application in text recognition

Текст научной работы на тему «О применении искусственного интеллекта в распознавании текстов»