On Fr, 2012-04-13 at 17:27 +0200, Krzesimir Nowak wrote:
2012/4/13 Patrick Ohly <patrick.ohly(a)intel.com>:
> On Fr, 2012-04-13 at 14:38 +0200, Patrick Ohly wrote:
>> On Do, 2012-04-12 at 15:59 +0200, Krzesimir Nowak wrote:
>> > Some of the tests are implemented in following branch:
>> >
https://meego.gitorious.org/~krnowak/meego-middleware/krnowaks-syncevolut...
>> >
>> > Commits are not so informative - will do it later.
>>
>> And they are also incomplete, aren't they? stdout and stderr from
>> running the command line tool are captured, but they are not checked
>> anywhere.
>
> The way how they are captured doesn't allow checking the right order of
> output via stdout and stderr, because they are stored in separate
> strings. But I am hard-pressed to suggest a better solution: one could
> write a custom communicate() method which uses two pipes and copies into
> one string for each stream and one combined string.
>
> But that suffers from race conditions. If output is ready in both
> streams when the parents starts to collect it, then the information
> about which output is older is already lost.
>
> I'm inclined to forgo testing the stdout/stderr distinction in most
> tests: use the same pipe for both (so that we can test the order and
> thus how it really looks to users) in most tests and then write a few
> tests which check for stdout/stderr usage (and ignore the order).
I wonder if there is some crazy platform where stdout is buffered so
long, that unbuffered stderr shows up first even if something earlier
was printed into stdout. I guess that such tests are hardly reliable
as an order of messages is guaranteed for one stream, but not between
two streams.
My idea would be to print usage messages to different streams
depending on situation. That is - if I call "syncevolution --help"
then it would print usage to stdout. But if I call "syncevolution
--template" then it would print usage to stderr with some "[ERROR]
missing parameter for 'template' option" following it.
Are there any other cases when we want to have messages in stdout and
stderr in some order?
Yes, running a sync. At least the way it is implemented now, "normal"
output is via stdout and all meta information is via stderr. In fact, it
might be the only example where it really matters, so my proposal to
test with two streams most of the time and then check one of them for
empty should work okay.
> Actually, any test which expects output only via one pipe can
use two
> pipes, because order doesn't matter when one pipe never has data.
>
> Below my patch to make the tests work for me:
> 1. I had to set SYNCEVOLUTION_TEMPLATE_DIR before invoking
> test-dbus.py, otherwise os.environ["SYNCEVOLUTION_TEMPLATE_DIR"]
> fails. This really should be rethought.
Yes, because there is a test (e.g. testMatchTemplate) which wants
different directory ("$(srcdir)/test/testcases/templates"). Maybe
copying some files/directories during build to $(builddir)/src would
suffice. That way we can specify SYNCEVOLUTION_TEMPLATE_DIR with
relative directories instead (like C++ tests are doing it).
test-dbus.py already expects certain files to be available. Adding
certain templates to that would be easy. But then
SYNCEVOLUTION_TEMPLATE_DIR would only have to be set for that test and
by that test itself, not by the user of test-dbus.py.
> 3. Output checking only gives "normal" results
when
> syncevo-dbus-server is run without SYNCEVOLUTION_DEBUG.
> @property('debug', False) achieves that (better name welcome).
I just added a function stripping all lines having "[DEBUG" prefix.
The original output with debugs might be useful (maybe print it when
some test fails).
I'd rather run the command line and the daemon exactly as the user does.
> This is just a quick-and-dirty solution, not committed anywhere.
Shall I
> do the base work for a few tests and then let you continue with adding
> the rest?
Ok. I have some tests already written, but I will adapt them to your
changes. I still have to rebase them to your ccs-pohly-cmdline branch,
No, let's finish the tests against master. That way we have a sound
baseline before starting to test the re-written server.
--
Best Regards, Patrick Ohly
The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.