On Fri, 2012-04-13 at 15:28 +0200, Patrick Ohly wrote:
This is just a quick-and-dirty solution, not committed anywhere.
Shall I
do the base work for a few tests and then let you continue with adding
the rest?
Below are the commits, already merged into master after a quick test
run:
http://syncev.meego.com/2012-04-16-12-37_lucid-amd64_prebuilt-amd64_testi...
I started with your initial commit and then included the changes that we
were discussing, plus a fix for a failure still visible in that test
run.
The reason is that your scanFiles() implementation is slightly different
from the C++ version. In contrast to the C++ version it includes lines
like:
sources/addressbook/config.ini:# ActiveSync Address Book = eas-contacts
sources/addressbook/config.ini:# ActiveSync Events = eas-events
sources/addressbook/config.ini:# ActiveSync Todos = eas-todos
sources/addressbook/config.ini:# ActiveSync Memos = eas-memos
These need to be excluded by the comparison because these comments can
change, depending on how the binaries were compiled or run. Your
reference text in the ScheduleWorld config includes one particular
variation and therefore it worked for me when testing manually, but the
nightly testing differs.
A good test for the assertEqualDiff() :-) See:
http://syncev.meego.com/2012-04-16-12-37_lucid-amd64_prebuilt-amd64_testi...
With everything in place, do you still want to convert some more tests?
I think one commit per converted test would be good, with C++ removed
and Python added in the same test. I expect that most tests can and
should be moved. The only reason for keeping a test in the C++ code
would be if it explicitly tests some aspect that cannot be tested when
going through D-Bus - not sure whether there is any such case.
commit 2925b97bac45ae4dc4dac92b04b91683cad29ddc
Author: Krzesimir Nowak <krnowak(a)openismus.com>
Date: Thu Apr 12 15:45:02 2012 +0200
D-Bus Testing: add Cmdline tests framework
This is a translation of the corresponding C++ TestCmdline from
Cmdline.cpp into Python.
commit aa562e5ef97eba78b7c45ab095df5071134840dc
Author: Patrick Ohly <patrick.ohly(a)intel.com>
Date: Mon Apr 16 09:56:50 2012 +0200
D-Bus Testing: added assertEqualDiff()
Replaces the diffStrings() from the previous commit. Renamed it
because it really is specifying an assertion and moved it to
DBusUtil because it may be useful outside of the command line
testing.
Also made sure that the diff is part of the assertion because then it
will be repeated when listing the failure.
commit 5389b8eeeddc328e29a86c91aec8606ac3fb9741
Author: Patrick Ohly <patrick.ohly(a)intel.com>
Date: Mon Apr 16 10:54:52 2012 +0200
D-Bus Testing: ensure that we have assertRegexpMatches()
unittest.assertRegexpMatches() is very useful for command line
output testing. But it is not yet available in Ubuntu Lucid,
the current base line for SyncEvolution releases. Therefore
provide a fallback implementation.
commit 0e63bc8e59b53a8821beff9f29d061cae48a66fc
Author: Patrick Ohly <patrick.ohly(a)intel.com>
Date: Mon Apr 16 17:26:12 2012 +0200
D-Bus Testing: fixed scanFiles()
The function must return all property assignments, including those
which are commented out. The Python version accidentally included some
comment lines which looked like assignments, partly because it ignored
leading spaces, partly because it looked anywhere in the line for an
equal sign.
Fixed by using a regex for the matching.
commit 67ce74b57fcb51f57b8cf29e353e13da16d60b8b
Author: Patrick Ohly <patrick.ohly(a)intel.com>
Date: Mon Apr 16 12:13:14 2012 +0200
D-Bus Testing: eliminate TestCmdline.testdir
All D-Bus tests already have their private xdg_root = 'temp-test-dbus'
which is cleared before running each test. Use it also for TestCmdline
instead of creating our own temporary directory, for the sake of
consistency.
Another advantage is that the syncevo-dbus-server is running with that
same XDG root and reading/writing files in it automatically. The
command line client currently *does* send its XDG env variables and
the server uses them, but perhaps that feature will have to be
removed, because it has a design mistake (*), so don't depend on it.
(*) If different clients and the auto-syncing code in the server are
allowed to use different config roots, the 'config name' alone no
longer is a unique identifier for configs.
commit 7c2de4ed5199f9c971d1474595f5a04988c1eac3
Author: Patrick Ohly <patrick.ohly(a)intel.com>
Date: Mon Apr 16 12:58:52 2012 +0200
D-Bus Testing: added running of command line tool
The 'syncevolution' command line tool is started inside the same D-Bus
session as the 'syncevo-dbus-server' that is started before running
each test. They use the same XDG env variables.
The test-dbus.py script is able to watch and report all D-Bus traffic
between tool and server.
Because tests are meant to be realistic, SYNCEVOLUTION_DEBUG has to be
optional and must be turned off for all tests which test for real
output as seen by users.
Every single invocation of runCmdline() should check stdout and
stderr, to detect unexpected output. The return code is already tested
by runCmdline() itself, unless explicitly turned off (may be useful for
tests which expect certain kind of failures).
Ideally the entire string gets tested, either via an exact string
comparison or a full regex match. Note that assertRegexpMatches() only
does a search, so use ^ and $ to ensure full match.
If (and only if) the command is expected to produce output on both
stdout and stderr where the order matters (for example, syncing), then
stdout and stderr should be combined into one stream. Otherwise the
order cannot be tested.
commit 470f0e7d7a56b96adff21e5e918c58ee0a9e3420
Author: Patrick Ohly <patrick.ohly(a)intel.com>
Date: Mon Apr 16 13:02:23 2012 +0200
D-Bus Testing: added testSetupScheduleWorld
The test is a straight copy from Cmdline.cpp. Because the hard-coded
default SSL certificate path is not known in test-dbus.py, it has to
extract that string by printing a template once with the command line
tool (only the command line exposes the comments, normal D-Bus API
doesn't) and finding it in the output.
This allows copying the C++ tests without making them less strict by
filtering out the SSL certificate path before the result comparisons.
commit a27f2af580a00d81107f69bc5802c079cef23766
Author: Patrick Ohly <patrick.ohly(a)intel.com>
Date: Mon Apr 16 14:38:34 2012 +0200
D-Bus Testing: work around Python 2.6 subprocess bug
Explicitly pass an environment when running the command line
tool. Otherwise subprocess.Popen() from Python 2.6 uses not
os.environ (which would be okay) but rather the environment passed to
a previous call to subprocess.Popen() (which will fail if the previous
test ran with an environment which had SYNCEVOLUTION_DEBUG set).
--
Best Regards, Patrick Ohly
The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.