+check_min_kver() +{ + local ver="$1" + : "${KVER:=$(uname -r)}" + + [ -n "$ver" ] || return 1 + [[ "$ver" == "$(echo -e "$ver\n$KVER" | sort -V | head -1)" ]] +} + +check_min_kver "4.11" || { echo "kernel $KVER may lack latest device-dax fixes"; exit $rc; }

+ +set -e +trap 'err $LINENO' ERR + +if ! fio --enghelp | grep -q "dev-dax"; then + echo "fio lacks dev-dax engine" + exit 77 +fi + +dev=$(./dax-dev) +for align in 4k 2m 1g +do + json=$($NDCTL create-namespace -m dax -a $align -f -e $dev) + chardev=$(echo $json | jq -r ". | select(.mode == \"dax\") | .daxregion.devices[0].chardev") + if [ align = "1g" ]; then + bs="1g" + else + bs="2m" + fi

+ + cat > fio.job <<- EOF + [global] + ioengine=dev-dax + direct=0 + filename=/dev/${chardev} + verify=crc32c + bs=${bs} + + [write] + rw=write + runtime=5 + + [read] + stonewall + rw=read + runtime=5 + EOF + + rc=1 + fio fio.job 2>&1 | tee fio.log + + if grep -q "fio.*got signal" fio.log; then + echo "test/device-dax-fio.sh: failed with align: $align" + exit 1 + fi + + # revert namespace to raw mode + json=$($NDCTL create-namespace -m raw -f -e $dev) + mode=$(echo $json | jq -r ".mode") + [ $mode != "memory" ] && echo "fail: $LINENO" && exit 1 +done + +exit 0

On Wed, Mar 29, 2017 at 1:02 PM, Jeff Moyer <jmoyer(a)redhat.com&gt; wrote: > Dan Williams <dan.j.williams(a)intel.com&gt; writes: > >> +check_min_kver() >> +{ >> + local ver="$1" >> + : "${KVER:=$(uname -r)}" >> + >> + [ -n "$ver" ] || return 1 >> + [[ "$ver" == "$(echo -e "$ver\n$KVER" | sort -V | head -1)" ]] >> +} >> + >> +check_min_kver "4.11" || { echo "kernel $KVER may lack latest device-dax fixes"; exit $rc; } > > Can we stop with this kernel version checking, please? Test to see if > you can create a device dax instance. If not, skip the test. If so, > and if you have a kernel that isn't fixed, so be it, you'll get > failures. I'd rather not. It helps me keep track of what went in where. If you want to run all the tests on a random kernel just do: KVER="4.11.0" make check

>> + >> +set -e >> +trap 'err $LINENO' ERR >> + >> +if ! fio --enghelp | grep -q "dev-dax"; then >> + echo "fio lacks dev-dax engine" >> + exit 77 >> +fi >> + >> +dev=$(./dax-dev) >> +for align in 4k 2m 1g >> +do >> + json=$($NDCTL create-namespace -m dax -a $align -f -e $dev) >> + chardev=$(echo $json | jq -r ". | select(.mode == \"dax\") | .daxregion.devices[0].chardev") >> + if [ align = "1g" ]; then >> + bs="1g" >> + else >> + bs="2m" >> + fi > > I'm not sure the blocksize even matters. > iirc it affects the alignment of the mmap() request. So for example bs=4k should fail when the alignment is larger.

On Wed, Mar 29, 2017 at 1:19 PM, Jeff Moyer <jmoyer(a)redhat.com&gt; wrote: > Dan Williams <dan.j.williams(a)intel.com&gt; writes: > >> On Wed, Mar 29, 2017 at 1:02 PM, Jeff Moyer <jmoyer(a)redhat.com&gt; wrote: >>> Dan Williams <dan.j.williams(a)intel.com&gt; writes: >>> >>>> +check_min_kver() >>>> +{ >>>> + local ver="$1" >>>> + : "${KVER:=$(uname -r)}" >>>> + >>>> + [ -n "$ver" ] || return 1 >>>> + [[ "$ver" == "$(echo -e "$ver\n$KVER" | sort -V | head -1)" ]] >>>> +} >>>> + >>>> +check_min_kver "4.11" || { echo "kernel $KVER may lack latest device-dax fixes"; exit $rc; } >>> >>> Can we stop with this kernel version checking, please? Test to see if >>> you can create a device dax instance. If not, skip the test. If so, >>> and if you have a kernel that isn't fixed, so be it, you'll get >>> failures. >> >> I'd rather not. It helps me keep track of what went in where. If you >> want to run all the tests on a random kernel just do: >> >> KVER="4.11.0" make check > > This, of course, breaks completely with distro kernels. Why does this break distro kernels? The KVER variable overrides "uname -r"

> You don't see this kind of checking in xfstests, for example. git > helps you keep track of what changes went in where (see git describe > --contains). It's weird to track that via your test harness. So, I > would definitely prefer to move to a model where we check for > features instead of kernel versions. I see this as a deficiency of xfstests. We have had to go through and qualify and track each xfstest and why it may fail with random combinations of kernel, xfsprogs, or e2fsprogs versions. I'd much prefer that upstream xfstests track the minimum versions of components to make a given test pass so we can stop doing it out of tree.

>>>>> Can we stop with this kernel version checking, please? Test to see if >>>>> you can create a device dax instance. If not, skip the test. If so, >>>>> and if you have a kernel that isn't fixed, so be it, you'll get >>>>> failures. >>>> >>>> I'd rather not. It helps me keep track of what went in where. If you >>>> want to run all the tests on a random kernel just do: >>>> >>>> KVER="4.11.0" make check >>> >>> This, of course, breaks completely with distro kernels. >> >> Why does this break distro kernels? The KVER variable overrides "uname -r" > > Because some features may not exist in the distro kernels. It's the > same problem you outline with xfstests, below. > Right, but you started off with suggesting that just running the test and failing was ok, and that's the behavior you get with KVER=.

>>> You don't see this kind of checking in xfstests, for example. git >>> helps you keep track of what changes went in where (see git describe >>> --contains). It's weird to track that via your test harness. So, I >>> would definitely prefer to move to a model where we check for >>> features instead of kernel versions. >> >> I see this as a deficiency of xfstests. We have had to go through and >> qualify and track each xfstest and why it may fail with random >> combinations of kernel, xfsprogs, or e2fsprogs versions. I'd much >> prefer that upstream xfstests track the minimum versions of components >> to make a given test pass so we can stop doing it out of tree. > > Yes, that's a good point. I can't think of a good compromise, either. Maybe we can at least get our annotated blacklist upstream so other people can start contributing to it?

On Wed, Mar 29, 2017 at 2:04 PM, Jeff Moyer <jmoyer(a)redhat.com&gt; wrote: > Dan Williams <dan.j.williams(a)intel.com&gt; writes: > >>>>>>> Can we stop with this kernel version checking, please? Test to see if >>>>>>> you can create a device dax instance. If not, skip the test. If so, >>>>>>> and if you have a kernel that isn't fixed, so be it, you'll get >>>>>>> failures. >>>>>> >>>>>> I'd rather not. It helps me keep track of what went in where. If you >>>>>> want to run all the tests on a random kernel just do: >>>>>> >>>>>> KVER="4.11.0" make check >>>>> >>>>> This, of course, breaks completely with distro kernels. >>>> >>>> Why does this break distro kernels? The KVER variable overrides "uname -r" >>> >>> Because some features may not exist in the distro kernels. It's the >>> same problem you outline with xfstests, below. >>> >> >> Right, but you started off with suggesting that just running the test >> and failing was ok, and that's the behavior you get with KVER=. > > Well, the goal was to be somewhat smart about it, by not even trying to > test things that aren't implemented or configured into the current > kernel (such as device dax, for example). Upon thinking about it > further, I don't think that gets us very far. However, that does raise > a use case that is not distro-specific. If you don't enable device dax, > your test suite will still try to run those tests. ;-) The other part of the equation is that I'm lazy and don't want to do the extra work of validating the environment for each test. So just do a quick version check and if the test fails you get to figure out what configuration you failed to enable. The most common case being that you failed to install the nfit_test modules in which case we do have a capability check for that. >>>>> You don't see this kind of checking in xfstests, for example. git >>>>> helps you keep track of what changes went in where (see git describe >>>>> --contains). It's weird to track that via your test harness. So, I >>>>> would definitely prefer to move to a model where we check for >>>>> features instead of kernel versions. >>>> >>>> I see this as a deficiency of xfstests. We have had to go through and >>>> qualify and track each xfstest and why it may fail with random >>>> combinations of kernel, xfsprogs, or e2fsprogs versions. I'd much >>>> prefer that upstream xfstests track the minimum versions of components >>>> to make a given test pass so we can stop doing it out of tree. >>> >>> Yes, that's a good point. I can't think of a good compromise, either. >> >> Maybe we can at least get our annotated blacklist upstream so other >> people can start contributing to it? > > Are you referring to xfstests? Yeah, that's a good idea. Right now I > just add tests to the dangerous group as I encounter known issues. ;-) > So, my list probably isn't very helpful in its current form. Yes, xfstests. We have entries in our blacklist like this:

# needs xfsprogs fix # c8dc42356142 xfs_db: fix the 'source' command when passed as a -c option # Last checked: # - xfsprogs-4.9.0-1.fc25.x86_64 xfs/138 # needs xfsprogs fix # 3297e0caa25a xfs: replace xfs_mode_to_ftype table with switch statement # Last checked: # - xfsprogs-4.9.0-1.fc25.x86_64 xfs/348 # see "[BUG] generic/232 hangs on XFS DAX mount" thread on xfs mailing # list generic/232 # failed on emulated pmem without dax, may be impacted by the same fix # as the one for generic/270. The generic/270 failure is tracked in this # thread on the xfs mailing list: "XFS kernel BUG during generic/270 # with v4.10". Re-test on v4.11 with fa7f138 ("xfs: clear delalloc and # cache on buffered write failure") generic/269 generic/270 _______________________________________________ Linux-nvdimm mailing list Linux-nvdimm(a)lists.01.org https://lists.01.org/mailman/listinfo/linux-nvdimm

On Thu, Mar 30, 2017 at 02:16:00PM +0800, Xiong Zhou wrote: > Ccing Eryu > > On Wed, Mar 29, 2017 at 02:12:25PM -0700, Dan Williams wrote: > > On Wed, Mar 29, 2017 at 2:04 PM, Jeff Moyer <jmoyer(a)redhat.com&gt; wrote: > > > Dan Williams <dan.j.williams(a)intel.com&gt; writes: > > > > > >>>>>>> Can we stop with this kernel version checking, please? Test to see if > > >>>>>>> you can create a device dax instance. If not, skip the test. If so, > > >>>>>>> and if you have a kernel that isn't fixed, so be it, you'll get > > >>>>>>> failures. > > >>>>>> > > >>>>>> I'd rather not. It helps me keep track of what went in where. If you > > >>>>>> want to run all the tests on a random kernel just do: > > >>>>>> > > >>>>>> KVER="4.11.0" make check > > >>>>> > > >>>>> This, of course, breaks completely with distro kernels. > > >>>> > > >>>> Why does this break distro kernels? The KVER variable overrides "uname -r" > > >>> > > >>> Because some features may not exist in the distro kernels. It's the > > >>> same problem you outline with xfstests, below. > > >>> > > >> > > >> Right, but you started off with suggesting that just running the test > > >> and failing was ok, and that's the behavior you get with KVER=. > > > > > > Well, the goal was to be somewhat smart about it, by not even trying to > > > test things that aren't implemented or configured into the current > > > kernel (such as device dax, for example). Upon thinking about it > > > further, I don't think that gets us very far. However, that does raise > > > a use case that is not distro-specific. If you don't enable device dax, > > > your test suite will still try to run those tests. ;-) > > > > The other part of the equation is that I'm lazy and don't want to do > > the extra work of validating the environment for each test. So just do > > a quick version check and if the test fails you get to figure out what > > configuration you failed to enable. The most common case being that > > you failed to install the nfit_test modules in which case we do have a > > capability check for that. > > > > >>>>> You don't see this kind of checking in xfstests, for example. git > > >>>>> helps you keep track of what changes went in where (see git describe > > >>>>> --contains). It's weird to track that via your test harness. So, I > > >>>>> would definitely prefer to move to a model where we check for > > >>>>> features instead of kernel versions. > > >>>> > > >>>> I see this as a deficiency of xfstests. We have had to go through and > > >>>> qualify and track each xfstest and why it may fail with random > > >>>> combinations of kernel, xfsprogs, or e2fsprogs versions. I'd much > > >>>> prefer that upstream xfstests track the minimum versions of components > > >>>> to make a given test pass so we can stop doing it out of tree. > > >>> > > >>> Yes, that's a good point. I can't think of a good compromise, either. > > >> > > >> Maybe we can at least get our annotated blacklist upstream so other > > >> people can start contributing to it? > > > > > > Are you referring to xfstests? Yeah, that's a good idea. Right now I > > > just add tests to the dangerous group as I encounter known issues. ;-) > > > So, my list probably isn't very helpful in its current form. > > > > Yes, xfstests. We have entries in our blacklist like this: > > Just a thought. > > How about: > 0, Adding infrastructure to teach xfstests querying know issues > before reporting pass or fail. > 1, Formatted known issue files. xfstests may maintain some in tree, > while people can have theire own in hand. I've posted similar RFC patch to fstests before. [RFC PATCH] fstests: add known issue support https://www.spinics.net/lists/fstests/msg02344.html And Dave rejected it: https://www.spinics.net/lists/fstests/msg02345.html So the short answer is: xfstests should only control whether a test should be run or not, the known issues information should be maintained outside the test harness. And I tend to agree with Dave now :)

On Wed, Mar 29, 2017 at 1:19 PM, Jeff Moyer <jmoyer(a)redhat.com&gt; wrote: > Dan Williams <dan.j.williams(a)intel.com&gt; writes: > >> On Wed, Mar 29, 2017 at 1:02 PM, Jeff Moyer <jmoyer(a)redhat.com&gt; wrote: >>> Dan Williams <dan.j.williams(a)intel.com&gt; writes: >>> >>>> +check_min_kver() >>>> +{ >>>> + local ver="$1" >>>> + : "${KVER:=$(uname -r)}" >>>> + >>>> + [ -n "$ver" ] || return 1 >>>> + [[ "$ver" == "$(echo -e "$ver\n$KVER" | sort -V | head -1)" ]] >>>> +} >>>> + >>>> +check_min_kver "4.11" || { echo "kernel $KVER may lack latest device-dax fixes"; exit $rc; } >>> >>> Can we stop with this kernel version checking, please? Test to see if >>> you can create a device dax instance. If not, skip the test. If so, >>> and if you have a kernel that isn't fixed, so be it, you'll get >>> failures. >> >> I'd rather not. It helps me keep track of what went in where. If you >> want to run all the tests on a random kernel just do: >> >> KVER="4.11.0" make check > > This, of course, breaks completely with distro kernels. Why does this break distro kernels? The KVER variable overrides "uname -r"

On Thu, Mar 30, 2017 at 9:08 AM, Linda Knippers <linda.knippers(a)hpe.com&gt; wrote: > On 03/29/2017 04:30 PM, Dan Williams wrote: >> On Wed, Mar 29, 2017 at 1:19 PM, Jeff Moyer <jmoyer(a)redhat.com&gt; wrote: >>> Dan Williams <dan.j.williams(a)intel.com&gt; writes: >>> >>>> On Wed, Mar 29, 2017 at 1:02 PM, Jeff Moyer <jmoyer(a)redhat.com&gt; wrote: >>>>> Dan Williams <dan.j.williams(a)intel.com&gt; writes: >>>>> >>>>>> +check_min_kver() >>>>>> +{ >>>>>> + local ver="$1" >>>>>> + : "${KVER:=$(uname -r)}" >>>>>> + >>>>>> + [ -n "$ver" ] || return 1 >>>>>> + [[ "$ver" == "$(echo -e "$ver\n$KVER" | sort -V | head -1)" ]] >>>>>> +} >>>>>> + >>>>>> +check_min_kver "4.11" || { echo "kernel $KVER may lack latest device-dax fixes"; exit $rc; } >>>>> >>>>> Can we stop with this kernel version checking, please? Test to see if >>>>> you can create a device dax instance. If not, skip the test. If so, >>>>> and if you have a kernel that isn't fixed, so be it, you'll get >>>>> failures. >>>> >>>> I'd rather not. It helps me keep track of what went in where. If you >>>> want to run all the tests on a random kernel just do: >>>> >>>> KVER="4.11.0" make check >>> >>> This, of course, breaks completely with distro kernels. >> >> Why does this break distro kernels? The KVER variable overrides "uname -r" > > FYI - dax-errors.sh doesn't look at KVER. > Patches welcome :).

On Thu, Mar 30, 2017 at 10:06 AM, Linda Knippers <linda.knippers(a)hpe.com&gt; wrote: > > > On 3/30/2017 12:56 PM, Dan Williams wrote: [..] >> Patches welcome :). > > > You won't like my patch for that because I agree with Jeff. :-) > > Right now I'm more interested in seeing if I can modify the tests to not > require nfit_test. I've only looked at btt-check.sh but so far, it doesn't > look that hard. The point of nfit_test is that you can run them with worrying about risks to real data. So I don't want to see patches moving existing nfit_test tests to something else.