COMPARISON OF ABILITIES OF NEURAL NETWORK ARCHITECTURE IN ARTWORK RESEARCH
DOI:
https://doi.org/10.32782/2411-3034-2024-35-24Keywords:
neural networks, artificial intelligence, computational art analysis, interdisciplinary approach, image generationAbstract
The article explores the possibilities of applying neural network algorithms, particularly convolutional neural networks (CNNs), generative adversarial networks (GANs), and transformers, for the analysis of artworks. The research methodology is based on an analytical approach to global research and publications, as well as experiments and tests in a technological laboratory setting. Results. A comparison of the advantages and limitations of three neural network architectures is conducted, and accuracy metrics in various tasks and conditions are analyzed. The author’s own experimental results using some neural network architectures are presented, along with an overview of examples of their application in art studies. Conclusions. It is shown that convolutional neural networks are optimal for classification, attribution, and finding similar works; generative adversarial networks are more suitable for generating new images, stylization, and restoring damage; the transformer architecture is effective for analyzing composition, semantics, and context. Emphasis is placed on the fact that the choice of a relevant architecture depends on the specifics of the task, the researcher’s available resources , and the quality and quantity of data.
References
Generative adversarial networks / I. Goodfellow et al. Communications of the ACM. 2020. Vol. 63, No. 11. P. 139–144. URL: https://doi.org/10.1145/3422622
Karpathy A., Fei-Fei L. Deep visual-semantic alignments for generating image descriptions. IEEE machine intelligence. 2017. Vol. 39, No. 4. P. 664–676. URL: https://doi.org/10.1109/tpami.2016.2598339
Bengio Y., Lecun Y., Hinton G. Deep learning for AI. Communications of the ACM. 2021. Vol. 64, no. 7. P. 58–65. URL: https://doi.org/10.1145/3448250
Sandoval C., Pirogova E., Lech M. Two-Stage deep learning approach to the classification of fine-art paintings. IEEE access. 2019. Vol. 7. P. 41770–41781. URL: https://doi.org/10.1109/access.2019.2907986
Lecoutre A. Recognizing art style automatically in painting with deep learning. Proceedings of the ninth asian conference on machine learning, 16 June 2024. P. 327–342. URL: https://proceedings.mlr.press/v77/lecoutre17a.html.
Sanakoyeu A., Kotovenko D., Lang S. A style-aware content loss for real-time HD style transfer. Arxiv. 2018. URL: https://doi.org/10.48550/arXiv.1807.10201.
Trach Y. Artificial intelligence as a tool for creating and analysing works of art. Culture and arts in the modern world. 2021. No. 22. P. 164–173. URL: https://doi.org/10.31866/2410-1915.22.2021.235907
Sovhyra T. Artificial intelligence and issue of authorship and uniqueness for works of art (technological research of the next rembrandt). Culture and arts in the modern world. 2021. No. 22. P. 156–163. URL: https://doi.org/10.31866/2410-1915.22.2021.235903
Volynets V. The impact of artificial intelligence on contemporary art: opportunities and challenges. Digital platform: information technologies in sociocultural sphere. 2023. Vol. 6, No. 1. P. 21–31. URL: https://doi.org/10.31866/2617-796x.6.1.2023.283933
Потапенко В. Нейронні мережі для розпізнавання об’єктів : дипломна робота на здобуття ступеня магістра спеціальності “Комп’ютерні науки”. Київ, 2022. 84 с. URL: https://er.nau.edu.ua/handle/NAU/57740.
Мартиненко А. Методи і моделі організації, обробки та аналізу даних в інтелектуальній системі підтримки прийняття рішень при ідентифікації творів живопису : дис. … доктора філософії. Дніпро, 2023. 156 с. URL: https://ir.nmu.org.ua/handle/123456789/165588.
Mcculloch W., Pitts W. A logical calculus of the ideas immanent in nervous activity. Bulletin of mathematical biology. 1990. Vol. 52, No. 1–2. P. 99–115. URL: https://www.cs.cmu.edu/~epxing/Class/10715/reading/McCulloch.and.Pitts.pdf
Rosenblatt F. The perceptron: a probabilistic model for information storage and organization in the brain. Psychological review. 1958. Vol. 65, No. 6. P. 386–408. URL: https://doi.org/10.1037/h0042519
Rumelhart D. E., Hinton G. E., Williams R. J. Learning representations by back-propagating errors. Nature. 1986. Vol. 323, No. 6088. P. 533–536. URL: https://doi.org/10.1038/323533a0
Досвід FILM.UA Design: практичні кейси використання нейромереж і ШІ в українському кіно та серіалах / Анна Давидова, Анастасія Рахманіна, Юлія Козиряцька. Media Business Reports. URL: https://mbr.com.ua/uk/news/technology/4959-sannikov-ai Дата публікації: 4.05.2023 (дата звернення: 15.03.2024).
Воркшоп від Ради молодих учених КНУКіМ. КНУКіМ – Київський національний університет культури і мистецтв. URL: https://knukim.edu.ua/vorkshop-vid-rady-molodyh-uchenyhknukim-2/ Дата публікації: 22.02.2024 (дата звернення: 15.03.2024).
Санніков Є. Інноваційні методи у продакшні контенту. AI для генерації зображень та відео, deepfake, нейронне сканування, віртуальні студії. Міжнародний фестиваль актуальної анімації та медіа-мистецтва LINOLEUM. URL: https://linoleumfest.com/uk/program/lectortium/events/innovations-production. Дата публікації: 25.09.2023 (дата звернення: 18.06.2024).
Ahrneteg J. Semantic segmentation of historical document images using recurrent neural networks. Karlskrona, Sweden, 2019. 52 p. URL: https://www.diva-portal.org/smash/get/diva2:1332073/FULLTEXT01.pdf.
GitHub – Grimwan/ReccurentNeuralDSS: ReccurentNeuralNetwork. GitHub. URL: https://github.com/Grimwan/ReccurentNeuralDSS (дата звернення: 18.06.2024).
Convolutional neural network style transfer towards Chinese paintings / J. Sheng, C. Song, J. Wang, Y. Han. IEEE access. 2017. Vol. 20. P. 1–9. URL: https://doi.org/10.1109/access.2019.2952616 (date of access: 16.06.2024).
Review of deep learning: concepts, CNN architectures, challenges, applications, future directions / L. Alzubaidi et al. Journal of big data. 2021. Vol. 8, no. 1. URL: https://doi.org/10.1186/s40537-021-00444-8 (date of access: 16.06.2024).
Gao X., Tian Y., Qi Z. RPD-GAN: learning to draw realistic paintings with generative adversarial network. IEEE transactions on image processing. 2020. Vol. 29. URL: https://doi.org/10.1109/tip.2020.3018856 (date of access: 16.06.2024).
Guarnera L., Giudice O., Battiato S. Fighting deepfake by exposing the convolutional traces on images. IEEE access. 2020. Vol. 8. URL: https://doi.org/10.1109/access.2020.3023037 (date of access: 16.06.2024).
An image is worth 16x16 words: transformers for image recognition at scale / Dosovitskiy A. et al. Arxiv. 2021. URL: https://arxiv.org/abs/2010.11929 Дата публікації: 22.10.2020 (дата звернення: 18.06.2024).
Neural Network Framework. PyTorch. URL: https://pytorch.org/ Дата публікації: 22.10.2020 (дата звернення: 18.06.2024).
Google Colab AI platform. Google Colab. URL: https://colab.research.google.com/ (дата звернення: 18.06.2024).
Find open datasets and machine learning projects. Kaggle: Your Machine Learning Community. URL: https://www.kaggle.com/datasets (дата звернення: 18.06.2024).
