AutomatedGuiTesting

This page is intended to collect information about automated GUI testing for EMC2.

The intent is to come up to automatically test a GUI in 'headless mode', and compare the screen result against a pre-recorded screenshot.

There isnt a whole lot of free and good tools for that purpose. The android tool is nice as it allows to record a session, start the program, record interactions, compare the screenshot, and exit. Unfortunately the components are very old and unsuitable - xscope is too limited and cant deal with OpenGL? windows properly. It is a proxy sitting between X server and application; if it cant relay events the application fails. Since it is a proxy it doesnt see window manager events as well.

Nowadays tools mostly work directly with the X server through the XTEST and record extensions, and the application connects directly to the X server as usual; the XTEST and record extensions allow recording and inserting of X events. Some tools also use ATSPI, which I didnt get to work on ubuntu (http://en.wikipedia.org/wiki/Assistive_Technology_Service_Provider_Interface).

After some googling and dead ends, the most promising route seems to be a combination of

- Xvfb - virtual framebuffer X server - lets one run an X application headless and get at the screen content

- Xnee: looks promising : http://itupw056.itu.chalmers.se/xnee/nightly-dists/ , http://itupw056.itu.chalmers.se/xnee/doc/xnee.html - needs to be installed from source

Those could be glued together like so (this is theory - I didnt try the full flow):

- record a session cnee and take a screenshot without window manager decoration (eg at left top corner)

- package session and screenshot as test

running the test:

- start Xvfb

- start GUI under test to connect to Xvfb

- replay cnee session

- take an Xvfb screen dump and compare to packaged screenshot

Problems:

- Xvfb is a very limited environment - the screenshots lack detail and fields

- recording without window manager interaction is icky

- recording with cnee is very lowlevel although there is a semi-helpful gnee gtk app

Xnee can record and playback touchy and axis sessions. This records the first 3000 events (just move the mouse until cnee stops)

  env DISPLAY=:0 cnee --record --events-to-record 3000 --mouse --keyboard -o /tmp/axis.xns

This replays the session - application must be in exactly same starting state (buttons etc):

  env DISPLAY=:0 cnee --no-synchronise --replay --file /tmp/axis.xns

Xnee build notes:

   sudo  apt-get install libxtst-dev libpanel-applet2-dev
   wget http://itupw056.itu.chalmers.se/xnee/nightly-dists/xnee-cvs-20111119.tar.gz
   tar zxvf /tmp/xnee-cvs-20111119.tar.gz 
   cd xnee-cvs-20111119/
   configure
   make
   sudo make install

- Xvfb: both axis and touchy in principle can run under Xvfb as follows:

 Xvfb :1 -fbdir /tmp/ &
 emc2-dev/configs/sim$ env DISPLAY=localhost:1 emc touchy.ini &
 cd /tmp
 xwud -in Xvfb_screen0

The output is terminally ugly and missing detail, can maybe improved with proper Xvfb options

- computing xwd visual differences:

 #!/bin/bash
 if [ ! $# -eq  2 ] ; then 
   echo usage: wdiff window1.xwd window2.xwd
   exit 2
 fi
 if cmp $1 $2 ; then
    	echo files dont differ
	exit 0
 else
	composite $1 $2 -compose difference  /tmp/difference.jpg
	eog /tmp/difference.jpg
	#rm -f /tmp/difference.jpg
	exit 1
 fi

- android: see http://drdobbs.com/tools/184404691 and http://web.archive.org/web/20040604025740/http://www.wildopensource.com/larry-projects/android.html

can record a Touchy (gtk) session. Cannot record an Axis session (probably OpenGL? related - loops with a PutImage? request). very limited - uses a proxy between client and server, which fails to record/playback window manager events

 Android: http://web.archive.org/web/20040604025740/http://www.wildopensource.com/larry-projects/android.html

Other related utilities

• xmacro: http://xmacro.sourceforge.net/