Computer Vision based Automation System for Detecting Objects

Full Text (PDF, 382KB), PP.68-74

Views: 0 Downloads: 0


Rajibul Anam 1,* Mostafizur Rahman 1 Mohammad Obaidul Haque 1 Md. Syeful Islam 1

1. Samsung R&D Institute Bangladesh, Dhaka, 1205, Bangladesh

* Corresponding author.


Received: 1 May 2015 / Revised: 5 Aug. 2015 / Accepted: 11 Sep. 2015 / Published: 8 Nov. 2015

Index Terms

Automation, test case, blackbox testing, vision based, window application


Software Quality Assurance Testing time computer vision based automation tools are used to test the window based application and window based application is combined of many objects. Among them most of the automation tool detect window objects by comparing images. Most of the objects are visible in the window screen but some objects which are not visible to the screen at the first time. Proper interaction with the window application hidden objects get visible to the screen like dropdown list item, editor text object, list box item and slider. With the automation tools these hidden objects cannot be searched directly. In this paper proposes some methods which will enhance the automation tools to access the window application hidden objects.

Cite This Paper

Rajibul Anam, Mostafizur Rahman, Mohammad Obaidul Haque, Md. Syeful Islam, "Computer Vision based Automation System for Detecting Objects", International Journal of Intelligent Systems and Applications(IJISA), vol.7, no.12, pp.68-74, 2015. DOI:10.5815/ijisa.2015.12.07


[1]S. Inderjeet, and T. Bindia, "Comparative Analysis of Open Source Automated Software Testing Tools: Selenium, Sikuli and Watir", International Journal of Information & Computation Technology, vol. 4, pp. 1507-1518, 2014.
[2]K. Dea-Kwang, and L. Lee-Sub, "Reverse Engineering from Exploratory Testing to Specification-based Testing", International Journal of Software Engineering and Its Applications, vol. 8(11), pp. 197-208, 2014.
[3]L. Maurizio, S. Andrea, R. Filippo, and T. Paolo, "Automated Generation of Visual Web Tests from DOM-based Web Tests", ACM/SIGAPP Symposium on Applied Computing, April, 2015.
[4]M. Leotta, A. Stocco, F. Ricca, P. Tonella, "PESTO: A Tool for Migrating DOM-Based to Visual Web Tests", ACM/SIGAPPSymposium on Applied Computing, April, 2015.
[5]L. Capocchi, J.F. Santucci, T. Ville, "Software Test Automation using DEVSimPy Environment", International Conference on Principles of Advanced Discrete Simulation, May, 2013.
[6]Borjesson, and F. Robert, "Automated System Testing using Visual GUI Testing Tools: A Comparative Study in Industry", "Borjesson2012visual", 2012.
[7]J. Hyunjun, L. Sukhoon, B. Doo-Kwon, "An Image Comparing-based GUI Software Testing Automation System", World Congress in Computer Science, Computer Engineering, and Applied Computing, 2012.
[8]K. Pragya, "Ameliorating the image matching algorithm of Sikuli using Artificial Neural Networks", International Journal of Computer Science & Communication, vol. 5, pp. 1-4, 2014.
[9], 12-Jan-2014.
[10]C. Tsung-Hsiang, "Using graphical representation of user interfaces as visual references", The 24th annual ACM symposium adjunct on User interface software and technology, pp. 27-30, 2011.
[11]S. L. M. Jeffrey, "User interface computation as a contextualized approach for introductory computing instruction", The 9th Annual International ACM Conference on International Computing Education Research, pp. 179-180, 2013.
[12]V. Andriychenko, L. Ying-dar, C. T. National, "Automatic Functionality and Stability Testing Through GUI of Handheld Devices", CiteSeerx, 2011.
[13]C. Tsung-Hsiang, Y. Tom, C. M. Robert, "GUI testing using computer vision", CHI 10th Conference on Human Factors in Computing Systems, pp. 1535-1544, 2010.
[14]Y. Tom, C. Tsung-Hsiang, C. M. Robert, "Sikuli: using GUI screenshots for search and automation", The 22nd annual ACM symposium on User interface software and technology, pp. 183-192, 2009.
[15]http://robotframework, 10-Nov-2014.
[16], 12-Nov-2014.
[17], 14-Nov-2014.
[18], 16-Nov-2014.