Automatic Image Description Generation Using Deep Learning Techniques

Authors

  • Jinal Butala Research Scholar, Computer Engineering Department, Dharamsinh Desai University, Nadiad, India
  • Dr. Brijesh Bhatt Associate Professor, Computer Engineering Department, Dharamsinh Desai University, Nadiad, India

DOI:

https://doi.org/10.37591/jomccmn.v6i1.2054

Keywords:

Convolutional neural network, Deep learning, Alexnet, Artificial intelligence, Cascade object detector, Computer vision toolbox

Abstract

Automatic Image Description Generation methods are extremely useful for image retrieval, search and organization. Previous approaches either use the existing labelled dataset to compose sentences or compose a new description for the test image by exploiting available descriptions of the training images. In practice these methods have limited accuracy, hence if the most important objects in an image cannot be identified, then they cannot generate valid description. Another difficulty lies within the final description generation step; it is crucial to generate grammatically correct sentences.

In this paper, we propose a two-stage framework. In the first stage we predict image objects using convolutional neural network (CNN) network Alexnet1 and the second stage of our framework generates caption for the image using these objects. We segment images and then apply alexnet on each segment one by one and identify the objects present in image. Later, construct sentences using Computer Vision System Toolbox Cascade object detector (COD). Using this technique to identify image classes, we obtain BLEU score to be 51.96 for Alexnet based approach and 46.01 for CNN based approach. Note that BLEU2 (bilingual evaluation understudy) is an algorithm for evaluating the quality of machine-translated text.

Downloads

Published

2019-06-13

Issue

Vol. 6 No. 1 (2019)

Section

Research Articles

License

Declaration and Copyright Transfer Form

(to be completed by authors)

I/ We, the undersigned author(s) of the submitted manuscript, hereby declare, that the above manuscript which is submitted for publication in the STM Journals(s), is not published already in part or whole (except in the form of abstract) in any journal or magazine for private or public circulation, and, is not under consideration of publication elsewhere.

  • I/We will not withdraw the manuscript after 1 week of submission as I have read the Author Guidelines and will adhere to the guidelines.
  • I/We Author(s ) have niether given nor will give this manuscript elsewhere for publishing after submitting in STM Journal(s).
  • I/ We have read the original version of the manuscript and am/ are responsible for the thought contents embodied in it. The work dealt in the manuscript is my/ our own, and my/ our individual contribution to this work is significant enough to qualify for authorship.
  •  I/We also agree to the authorship of the article in the following order:

Author’s name                                    

 

1. ________________

2. ________________

3. ________________

4. ________________

 

We  Author(s) tick this box and would request you to consider it as our signature as we agree to the terms of this Copyright Notice, which will apply to this submission if and when it is published by this journal.