On Thu, Mar 30, 2017 at 02:16:00PM +0800, Xiong Zhou wrote:
Ccing Eryu
On Wed, Mar 29, 2017 at 02:12:25PM -0700, Dan Williams wrote:
> On Wed, Mar 29, 2017 at 2:04 PM, Jeff Moyer <jmoyer(a)redhat.com> wrote:
> > Dan Williams <dan.j.williams(a)intel.com> writes:
> >
> >>>>>>> Can we stop with this kernel version checking, please?
Test to see if
> >>>>>>> you can create a device dax instance. If not, skip the
test. If so,
> >>>>>>> and if you have a kernel that isn't fixed, so be
it, you'll get
> >>>>>>> failures.
> >>>>>>
> >>>>>> I'd rather not. It helps me keep track of what went in
where. If you
> >>>>>> want to run all the tests on a random kernel just do:
> >>>>>>
> >>>>>> KVER="4.11.0" make check
> >>>>>
> >>>>> This, of course, breaks completely with distro kernels.
> >>>>
> >>>> Why does this break distro kernels? The KVER variable overrides
"uname -r"
> >>>
> >>> Because some features may not exist in the distro kernels. It's
the
> >>> same problem you outline with xfstests, below.
> >>>
> >>
> >> Right, but you started off with suggesting that just running the test
> >> and failing was ok, and that's the behavior you get with KVER=.
> >
> > Well, the goal was to be somewhat smart about it, by not even trying to
> > test things that aren't implemented or configured into the current
> > kernel (such as device dax, for example). Upon thinking about it
> > further, I don't think that gets us very far. However, that does raise
> > a use case that is not distro-specific. If you don't enable device dax,
> > your test suite will still try to run those tests. ;-)
>
> The other part of the equation is that I'm lazy and don't want to do
> the extra work of validating the environment for each test. So just do
> a quick version check and if the test fails you get to figure out what
> configuration you failed to enable. The most common case being that
> you failed to install the nfit_test modules in which case we do have a
> capability check for that.
>
> >>>>> You don't see this kind of checking in xfstests, for
example. git
> >>>>> helps you keep track of what changes went in where (see git
describe
> >>>>> --contains). It's weird to track that via your test
harness. So, I
> >>>>> would definitely prefer to move to a model where we check for
> >>>>> features instead of kernel versions.
> >>>>
> >>>> I see this as a deficiency of xfstests. We have had to go through
and
> >>>> qualify and track each xfstest and why it may fail with random
> >>>> combinations of kernel, xfsprogs, or e2fsprogs versions. I'd
much
> >>>> prefer that upstream xfstests track the minimum versions of
components
> >>>> to make a given test pass so we can stop doing it out of tree.
> >>>
> >>> Yes, that's a good point. I can't think of a good compromise,
either.
> >>
> >> Maybe we can at least get our annotated blacklist upstream so other
> >> people can start contributing to it?
> >
> > Are you referring to xfstests? Yeah, that's a good idea. Right now I
> > just add tests to the dangerous group as I encounter known issues. ;-)
> > So, my list probably isn't very helpful in its current form.
>
> Yes, xfstests. We have entries in our blacklist like this:
Just a thought.
How about:
0, Adding infrastructure to teach xfstests querying know issues
before reporting pass or fail.
1, Formatted known issue files. xfstests may maintain some in tree,
while people can have theire own in hand.
I've posted similar RFC patch to fstests before.
[RFC PATCH] fstests: add known issue support
https://www.spinics.net/lists/fstests/msg02344.html
And Dave rejected it:
https://www.spinics.net/lists/fstests/msg02345.html
So the short answer is: xfstests should only control whether a test
should be run or not, the known issues information should be maintained
outside the test harness.
And I tend to agree with Dave now :)
Questions:
0, Tracking components, like kernel, xfstests, e2fsprogs, xfsprogs,
fio, etc
1, Tracking versions of components, different distributions,
upstream versions.
2, Versions of xfstests itself. We don't have many tags now,
however if we start to do this, it will not be an issue.
3, Tracking architectures/platforms, x86_64, ppc64, etc
4, Is this matrix too big ? :)
5, Is this failure now the same faliure that I've got from
the known issues?
I know the results handling of xfstests need more work, and the progress
is slow, but we do have some processes, e.g. configurable $RESULTDIR,
recent tools from Eric to compare failures across runs
(tools/compare-failures).
I've been thinking about Dave's suggestions for a long time, but due to
the complexity of known issues (see above questions :) and other
tasks/jobs, I'm not able to work out a solution yet. I'll look into it
again, thanks for the reminder :)
Thanks,
Eryu
Thanks,
Xiong
>
> # needs xfsprogs fix
> # c8dc42356142 xfs_db: fix the 'source' command when passed as a -c option
> # Last checked:
> # - xfsprogs-4.9.0-1.fc25.x86_64
> xfs/138
>
> # needs xfsprogs fix
> # 3297e0caa25a xfs: replace xfs_mode_to_ftype table with switch statement
> # Last checked:
> # - xfsprogs-4.9.0-1.fc25.x86_64
> xfs/348
>
> # see "[BUG] generic/232 hangs on XFS DAX mount" thread on xfs mailing
> # list
> generic/232
>
> # failed on emulated pmem without dax, may be impacted by the same fix
> # as the one for generic/270. The generic/270 failure is tracked in this
> # thread on the xfs mailing list: "XFS kernel BUG during generic/270
> # with v4.10". Re-test on v4.11 with fa7f138 ("xfs: clear delalloc and
> # cache on buffered write failure")
> generic/269
> generic/270
> _______________________________________________
> Linux-nvdimm mailing list
> Linux-nvdimm(a)lists.01.org
>
https://lists.01.org/mailman/listinfo/linux-nvdimm