[PATCH] libnvdimm, dax: fix deletion
by Dan Williams
The ndctl unit tests discovered that the dax enabling omitted updates to
nd_detach_and_reset(). This routine clears device the configuration
when the namespace is detached. Without this clearing userspace may
assume that the device is in the process of being configured by another
agent in the system.
Signed-off-by: Dan Williams <dan.j.williams(a)intel.com>
---
drivers/nvdimm/claim.c | 23 +++++++++++++++++++++--
drivers/nvdimm/nd-core.h | 1 +
drivers/nvdimm/pfn_devs.c | 19 -------------------
3 files changed, 22 insertions(+), 21 deletions(-)
diff --git a/drivers/nvdimm/claim.c b/drivers/nvdimm/claim.c
index 5f53db59a058..8b2e3c4fb0ad 100644
--- a/drivers/nvdimm/claim.c
+++ b/drivers/nvdimm/claim.c
@@ -93,6 +93,25 @@ static bool is_idle(struct device *dev, struct nd_namespace_common *ndns)
return true;
}
+struct nd_pfn *to_nd_pfn_safe(struct device *dev)
+{
+ /*
+ * pfn device attributes are re-used by dax device instances, so we
+ * need to be careful to correct device-to-nd_pfn conversion.
+ */
+ if (is_nd_pfn(dev))
+ return to_nd_pfn(dev);
+
+ if (is_nd_dax(dev)) {
+ struct nd_dax *nd_dax = to_nd_dax(dev);
+
+ return &nd_dax->nd_pfn;
+ }
+
+ WARN_ON(1);
+ return NULL;
+}
+
static void nd_detach_and_reset(struct device *dev,
struct nd_namespace_common **_ndns)
{
@@ -106,8 +125,8 @@ static void nd_detach_and_reset(struct device *dev,
nd_btt->lbasize = 0;
kfree(nd_btt->uuid);
nd_btt->uuid = NULL;
- } else if (is_nd_pfn(dev)) {
- struct nd_pfn *nd_pfn = to_nd_pfn(dev);
+ } else if (is_nd_pfn(dev) || is_nd_dax(dev)) {
+ struct nd_pfn *nd_pfn = to_nd_pfn_safe(dev);
kfree(nd_pfn->uuid);
nd_pfn->uuid = NULL;
diff --git a/drivers/nvdimm/nd-core.h b/drivers/nvdimm/nd-core.h
index 4136c1a82539..6c42eda025f9 100644
--- a/drivers/nvdimm/nd-core.h
+++ b/drivers/nvdimm/nd-core.h
@@ -94,4 +94,5 @@ bool __nd_attach_ndns(struct device *dev, struct nd_namespace_common *attach,
ssize_t nd_namespace_store(struct device *dev,
struct nd_namespace_common **_ndns, const char *buf,
size_t len);
+struct nd_pfn *to_nd_pfn_safe(struct device *dev);
#endif /* __ND_CORE_H__ */
diff --git a/drivers/nvdimm/pfn_devs.c b/drivers/nvdimm/pfn_devs.c
index 04f71d6d304d..436191c47077 100644
--- a/drivers/nvdimm/pfn_devs.c
+++ b/drivers/nvdimm/pfn_devs.c
@@ -54,25 +54,6 @@ struct nd_pfn *to_nd_pfn(struct device *dev)
}
EXPORT_SYMBOL(to_nd_pfn);
-static struct nd_pfn *to_nd_pfn_safe(struct device *dev)
-{
- /*
- * pfn device attributes are re-used by dax device instances, so we
- * need to be careful to correct device-to-nd_pfn conversion.
- */
- if (is_nd_pfn(dev))
- return to_nd_pfn(dev);
-
- if (is_nd_dax(dev)) {
- struct nd_dax *nd_dax = to_nd_dax(dev);
-
- return &nd_dax->nd_pfn;
- }
-
- WARN_ON(1);
- return NULL;
-}
-
static ssize_t mode_show(struct device *dev,
struct device_attribute *attr, char *buf)
{
4 years, 5 months
[ndctl PATCH 0/5] Device DAX support
by Dan Williams
Extend libndctl with a libdaxctl helper library to enumerate the devices
that appear underneath a libnvdimm namespace in "dax" mode. libdaxctl
is meant to be independent from libndctl as device-dax instances may
arise from other memory ranges besides persistent memory. It is a
generic enumeration facility for performance / featured differentiated
memory ranges when the kernel arranges for them to be exposed as a
devices.
---
Dan Williams (5):
util: centralize logging
util: centralize sysfs helpers
test: fix 'pfn alignment too big' checks
daxctl: initial boilerplate
daxctl: enumeration api and unit test for "Device DAX"
Makefile.am | 2
Makefile.am.in | 3
configure.ac | 1
daxctl/Makefile.am | 28 ++
daxctl/lib/libdaxctl-private.h | 40 +++
daxctl/lib/libdaxctl.c | 353 ++++++++++++++++++++++++++++
daxctl/lib/libdaxctl.pc.in | 11 +
daxctl/lib/libdaxctl.sym | 32 +++
daxctl/libdaxctl.h | 64 +++++
ndctl/Makefile.am | 21 +-
ndctl/lib/libndctl-private.h | 65 ++---
ndctl/lib/libndctl-smart.c | 1
ndctl/lib/libndctl.c | 219 ++++--------------
ndctl/lib/libndctl.sym | 2
ndctl/libndctl.h.in | 3
test/Makefile.am | 4
test/libndctl.c | 500 ++++++++++++++++++++++++++++++++++------
util/log.c | 65 +++++
util/log.h | 69 ++++++
util/sysfs.c | 128 ++++++++++
util/sysfs.h | 40 +++
21 files changed, 1350 insertions(+), 301 deletions(-)
create mode 100644 daxctl/Makefile.am
create mode 100644 daxctl/lib/libdaxctl-private.h
create mode 100644 daxctl/lib/libdaxctl.c
create mode 100644 daxctl/lib/libdaxctl.pc.in
create mode 100644 daxctl/lib/libdaxctl.sym
create mode 100644 daxctl/libdaxctl.h
create mode 100644 util/log.c
create mode 100644 util/log.h
create mode 100644 util/sysfs.c
create mode 100644 util/sysfs.h
4 years, 5 months
[PATCH v4 0/6] Add alignment check for DAX mount
by Toshi Kani
When a partition is not aligned by 4KB, mount -o dax succeeds,
but any read/write access to the filesystem fails, except for
metadata update. Add alignment check to ext4, ext2, and xfs.
- Patch 1-2 add bdev_dax_supported() which performs all the checks
necessary for dax mount.
- Patch 3-5 change fillesystems to call bdev_dax_supported().
- Patch 6 is a cleanup to keep dax capability checks consistent.
v4:
- blkdev_dax_capable() is similar to bdev_dax_supported().
Manage them consistently. (Dan Williams, Dave Chinner)
v3:
- Remove boilerplate code from filesytems (Christoph Hellwig)
- Add a helper function to perform all checks (Dave Chinner)
v2:
- Use a helper function via ->direct_access for the check.
(Christoph Hellwig)
- Call bdev_direct_access() with sector 0 for the check.
(Boaz Harrosh)
---
Toshi Kani (6):
1/6 block: Add vfs_msg() interface
2/6 block: Add bdev_dax_supported() for dax mount checks
3/6 ext4: Add alignment check for DAX mount
4/6 ext2: Add alignment check for DAX mount
5/6 xfs: Add alignment check for DAX mount
6/6 block: Update blkdev_dax_capable() for consistency
---
block/ioctl.c | 30 ----------------
fs/block_dev.c | 96 ++++++++++++++++++++++++++++++++++++++++++++++++--
fs/ext2/super.c | 11 ++----
fs/ext4/super.c | 11 ++----
fs/xfs/xfs_super.c | 12 +++----
include/linux/blkdev.h | 13 +++++++
include/linux/fs.h | 8 -----
7 files changed, 116 insertions(+), 65 deletions(-)
4 years, 5 months
[PATCH v3 0/5] "Device DAX" for persistent memory
by Dan Williams
Changes since v2 [1]:
1/ Allow libnvdimm drivers to omit a ->remove() method (Johannes)
2/ Fix memory leak due to missing ida_destroy() in drivers/nvdimm/ and
drivers/dax/ (Johannes)
3/ Mark some dev_dbg() instances as dev_info() (Johannes)
4/ Clarify RCU usage in dax.c (Johannes), acked-by Paul.
---
Device DAX is the device-centric analogue of Filesystem DAX
(CONFIG_FS_DAX). It allows memory ranges to be allocated and mapped
without need of an intervening file system or being bound to block
device semantics.
Why "Device DAX"?
1/ As I mentioned at LSF [2] we are starting to see platforms with
performance and feature differentiated memory ranges. Environments like
high-performance-computing and usages like in-memory databases want
exclusive allocation of a memory range with zero conflicting
kernel/metadata allocations. For dedicated applications of high
bandwidth or low latency memory device-DAX provides a predictable direct
map mechanism.
Note that this is only for the small number of "crazy" applications that
are willing to re-write to get every bit of performance. For everyone
else we, Dave Hansen and I, are looking to add a mechanism to hot-plug
device-DAX ranges into the mm to get general memory management services
(oversubscribe / migration, etc) with the understanding that it may
sacrifice some predictability.
2/ For persistent memory there are similar applications that are willing
to re-write to take full advantage of byte-addressable persistence.
This mechanism satisfies those usages that only need a pre-allocated
file to mmap.
3/ It answers Dave Chinner's call to start thinking about pmem-native
solutions. Device DAX specifically avoids block-device and file system
conflicts.
[1]: https://lists.01.org/pipermail/linux-nvdimm/2016-May/005766.html
[2]: https://lwn.net/Articles/685107/
---
Dan Williams (5):
libnvdimm: stop requiring a driver ->remove() method
/dev/dax, pmem: direct access to persistent memory
/dev/dax, core: file operations and dax-mmap
Revert "block: enable dax for raw block devices"
libnvdimm: release ida resources
block/ioctl.c | 32 --
drivers/Kconfig | 2
drivers/Makefile | 1
drivers/dax/Kconfig | 26 ++
drivers/dax/Makefile | 4
drivers/dax/dax.c | 575 +++++++++++++++++++++++++++++++++++
drivers/dax/dax.h | 24 +
drivers/dax/pmem.c | 158 ++++++++++
drivers/nvdimm/bus.c | 9 -
drivers/nvdimm/core.c | 3
drivers/nvdimm/dimm_devs.c | 5
drivers/nvdimm/nd-core.h | 2
drivers/nvdimm/region_devs.c | 5
fs/block_dev.c | 96 ++----
include/linux/fs.h | 8
include/uapi/linux/fs.h | 1
mm/huge_memory.c | 1
mm/hugetlb.c | 1
tools/testing/nvdimm/Kbuild | 9 +
tools/testing/nvdimm/config_check.c | 2
20 files changed, 852 insertions(+), 112 deletions(-)
create mode 100644 drivers/dax/Kconfig
create mode 100644 drivers/dax/Makefile
create mode 100644 drivers/dax/dax.c
create mode 100644 drivers/dax/dax.h
create mode 100644 drivers/dax/pmem.c
4 years, 5 months
[PATCH] dax: Remove unused variable in __dax_pmd_fault()
by Jan Kara
Reported-by: Ross Zwisler <ross.zwisler(a)linux.intel.com>
Signed-off-by: Jan Kara <jack(a)suse.cz>
---
fs/dax.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
Ross spotted this small issue in my patch set so can you guys please merge this
fixup as well? Thanks!
diff --git a/fs/dax.c b/fs/dax.c
index a07202ab8f61..10f0303a92a5 100644
--- a/fs/dax.c
+++ b/fs/dax.c
@@ -1007,7 +1007,7 @@ int __dax_pmd_fault(struct vm_area_struct *vma, unsigned long address,
struct block_device *bdev;
pgoff_t size, pgoff;
sector_t block;
- int error, result = 0;
+ int result = 0;
bool alloc = false;
/* dax pmd mappings require pfn_t_devmap() */
--
2.6.6
4 years, 5 months
Reach Millions of FB Group Members
by MARK
Hello,
Do you want to advertise on facebook? We're here to help.
We wil manually post your product/logo/link on Facebook Groups and will
give you a full report with links of each live post where your advertisement
was posted.
http://www.buysocial.cn/detail.php?id=12
Regards
MARK
Unsubscribe option is available on the footer of our website
4 years, 5 months
Legacy NVDIMM-F over EFI
by Yigal Korman
Hi all,
I've got a couple of legacy 'type 12' NVDIMM-F modules in our lab I've
been successfully testing.
Recently I moved them to a new system where EFI bootloader is used
(CentOS 7 installed with EFI) and was surprised to see that they're
not identified as legacy 'type 12' persistent memory, are not caught
by the 'nd_pmem' driver and I don't get /dev/pmemX devices.
They show as 'type 7' in the E820 map and 'Persistent memory' in /proc/iomem.
Is this the intended behavior? shouldn't they be identified as legacy
if they don't have NFIT?
Thanks,
Yigal
4 years, 5 months
[PATCH 0/7 v4] DAX page fault locking
by Jan Kara
Hello,
this is my fourth attempt at DAX page fault locking rewrite. The patch set has
passed xfstests both with and without DAX mount option on ext4 and xfs for
me (including new xfstests for stressing DAX mmap behavior). Also all tests
are passing for Ross now. All the patches except for PMD fault handling ones
got reviewed by Ross so I think they are in a reasonably good shape. For now
I'm not aware of any outstanding issues so can we merge the patches through
NVDIMM tree please?
Changes since v3:
- split patch set into three - ext4 bits, cleanups & easy fixes, this part
- fixed missed wakeups on exceptional entry locks (Ross)
- fixed misaccounting of number of used entries in a radix tree node
- fixed bad interaction with workingset code
- other minor changes based on review comments
Changes since v2:
- lot of additional ext4 fixes and cleanups
- make PMD page faults depend on CONFIG_BROKEN instead of #if 0
- fixed page reference leak when replacing hole page with a pfn
- added some reviewed-by tags
- rebased on top of current Linus' tree
Changes since v1:
- handle wakeups of exclusive waiters properly
- fix cow fault races
- other minor stuff
General description
The basic idea is that we use a bit in an exceptional radix tree entry as
a lock bit and use it similarly to how page lock is used for normal faults.
That way we fix races between hole instantiation and read faults of the
same index. For now I have disabled PMD faults since there the issues with
page fault locking are even worse. Now that Matthew's multi-order radix tree
has landed, I can have a look into using that for proper locking of PMD faults
but first I want normal pages sorted out.
Honza
4 years, 5 months
[ndctl PATCH] ndctl: reorganize file structure, use per sub-directory makefiles
by Dan Williams
In preparation for daxctl / libdaxctl move the ndctl infrastructure to
its own sub-directory.
This also moves tests to have their own Makefile rather than continuing
to clutter up one top-level Makefile.am.
Signed-off-by: Dan Williams <dan.j.williams(a)intel.com>
---
Makefile.am | 171 ++++---------------------------------
Makefile.am.in | 41 +++++++++
configure.ac | 14 ++-
ndctl/Makefile.am | 75 ++++++++++++++++
ndctl/builtin-bat.c | 0
ndctl/builtin-create-nfit.c | 0
ndctl/builtin-help.c | 0
ndctl/builtin-list.c | 0
ndctl/builtin-read-labels.c | 0
ndctl/builtin-test.c | 0
ndctl/builtin-xable-region.c | 0
ndctl/builtin-xaction-namespace.c | 0
ndctl/builtin-zero-labels.c | 0
ndctl/builtin.h | 0
ndctl/lib/.gitignore | 0
ndctl/lib/libndctl-ars.c | 0
ndctl/lib/libndctl-private.h | 0
ndctl/lib/libndctl-smart.c | 0
ndctl/lib/libndctl.c | 0
ndctl/lib/libndctl.h.in | 0
ndctl/lib/libndctl.pc.in | 0
ndctl/lib/libndctl.sym | 0
ndctl/ndctl.c | 0
ndctl/ndctl.h | 0
ndctl/util/json-smart.c | 0
ndctl/util/json.c | 0
ndctl/util/json.h | 0
test/Makefile.am | 60 +++++++++++++
test/clear.sh | 2
test/create.sh | 2
test/dax-errors.sh | 6 +
test/dax.sh | 12 +--
test/mmap.sh | 4 -
33 files changed, 215 insertions(+), 172 deletions(-)
create mode 100644 Makefile.am.in
create mode 100644 ndctl/Makefile.am
rename builtin-bat.c => ndctl/builtin-bat.c (100%)
rename builtin-create-nfit.c => ndctl/builtin-create-nfit.c (100%)
rename builtin-help.c => ndctl/builtin-help.c (100%)
rename builtin-list.c => ndctl/builtin-list.c (100%)
rename builtin-read-labels.c => ndctl/builtin-read-labels.c (100%)
rename builtin-test.c => ndctl/builtin-test.c (100%)
rename builtin-xable-region.c => ndctl/builtin-xable-region.c (100%)
rename builtin-xaction-namespace.c => ndctl/builtin-xaction-namespace.c (100%)
rename builtin-zero-labels.c => ndctl/builtin-zero-labels.c (100%)
rename builtin.h => ndctl/builtin.h (100%)
rename lib/.gitignore => ndctl/lib/.gitignore (100%)
rename lib/libndctl-ars.c => ndctl/lib/libndctl-ars.c (100%)
rename lib/libndctl-private.h => ndctl/lib/libndctl-private.h (100%)
rename lib/libndctl-smart.c => ndctl/lib/libndctl-smart.c (100%)
rename lib/libndctl.c => ndctl/lib/libndctl.c (100%)
rename lib/ndctl/libndctl.h.in => ndctl/lib/libndctl.h.in (100%)
rename lib/libndctl.pc.in => ndctl/lib/libndctl.pc.in (100%)
rename lib/libndctl.sym => ndctl/lib/libndctl.sym (100%)
rename ndctl.c => ndctl/ndctl.c (100%)
rename ndctl.h => ndctl/ndctl.h (100%)
rename util/json-smart.c => ndctl/util/json-smart.c (100%)
rename util/json.c => ndctl/util/json.c (100%)
rename util/json.h => ndctl/util/json.h (100%)
create mode 100644 test/Makefile.am
diff --git a/Makefile.am b/Makefile.am
index c5c89f4d50c7..db07205b33a1 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -1,58 +1,18 @@
-EXTRA_DIST =
+include Makefile.am.in
+
+ACLOCAL_AMFLAGS = -I m4 ${ACLOCAL_FLAGS}
+SUBDIRS = . ndctl
if ENABLE_DOCS
-SUBDIRS = . Documentation
+SUBDIRS += Documentation
endif
-CLEANFILES =
-ACLOCAL_AMFLAGS = -I m4 ${ACLOCAL_FLAGS}
-AM_MAKEFLAGS = --no-print-directory
-
-AM_CPPFLAGS = \
- -include $(top_builddir)/config.h \
- -DSYSCONFDIR=\""$(sysconfdir)"\" \
- -DLIBEXECDIR=\""$(libexecdir)"\" \
- -DPREFIX=\""$(prefix)"\" \
- -DNDCTL_MAN_PATH=\""$(mandir)"\" \
- -I${top_srcdir}/lib/ndctl \
- -I${top_srcdir}/lib \
- -I${top_srcdir}/ \
- $(KMOD_CFLAGS) \
- $(UDEV_CFLAGS) \
- $(UUID_CFLAGS) \
- $(JSON_CFLAGS)
-
-AM_CFLAGS = ${my_CFLAGS} \
- -fvisibility=hidden \
- -ffunction-sections \
- -fdata-sections
+SUBDIRS += test
-AM_LDFLAGS = \
- -Wl,--gc-sections \
- -Wl,--as-needed
-
-BUILT_SOURCES = $(top_srcdir)/version.m4 $(top_srcdir)/lib/ndctl/libndctl.h
-$(top_srcdir)/lib/ndctl/libndctl.h: $(top_srcdir)/lib/ndctl/libndctl.h.in
- touch $(top_srcdir)/version.m4
+BUILT_SOURCES = $(top_srcdir)/version.m4
$(top_srcdir)/version.m4: FORCE
$(AM_V_GEN)$(top_srcdir)/git-version-gen
FORCE:
-SED_PROCESS = \
- $(AM_V_GEN)$(MKDIR_P) $(dir $@) && $(SED) \
- -e 's,@VERSION\@,$(VERSION),g' \
- -e 's,@prefix\@,$(prefix),g' \
- -e 's,@exec_prefix\@,$(exec_prefix),g' \
- -e 's,@libdir\@,$(libdir),g' \
- -e 's,@includedir\@,$(includedir),g' \
- < $< > $@ || rm $@
-
-%.pc: %.pc.in Makefile
- $(SED_PROCESS)
-
-LIBNDCTL_CURRENT=7
-LIBNDCTL_REVISION=0
-LIBNDCTL_AGE=1
-
noinst_SCRIPTS = rhel/ndctl.spec sles/ndctl.spec
CLEANFILES += $(noinst_SCRIPTS)
@@ -67,68 +27,12 @@ do_sles_subst = sed -e 's,VERSION,$(VERSION),g' \
-e 's,\(^License:.*GPL\)v2,\1-2.0,g' \
-e "s,LNAME,libndctl$$(($(LIBNDCTL_CURRENT) - $(LIBNDCTL_AGE))),g"
-rhel/ndctl.spec: ndctl.spec.in Makefile.am
+rhel/ndctl.spec: ndctl.spec.in Makefile.am version.m4
$(AM_V_GEN)$(MKDIR_P) rhel; $(do_rhel_subst) < $< > $@
-sles/ndctl.spec: ndctl.spec.in Makefile.am
+sles/ndctl.spec: ndctl.spec.in Makefile.am version.m4
$(AM_V_GEN)$(MKDIR_P) sles; cat sles/header $< | $(do_sles_subst) > $@
-pkginclude_HEADERS = lib/ndctl/libndctl.h
-lib_LTLIBRARIES = lib/libndctl.la
-
-lib_libndctl_la_SOURCES =\
- lib/libndctl-private.h \
- lib/libndctl.c
-lib_libndctl_la_LIBADD = $(UDEV_LIBS) $(UUID_LIBS) $(KMOD_LIBS)
-
-if ENABLE_ARS
-lib_libndctl_la_SOURCES += lib/libndctl-ars.c
-endif
-
-if ENABLE_SMART
-lib_libndctl_la_SOURCES += lib/libndctl-smart.c
-endif
-
-bin_PROGRAMS = ndctl
-
-ndctl_SOURCES = ndctl.c \
- builtin-create-nfit.c \
- builtin-xaction-namespace.c \
- builtin-xable-region.c \
- builtin-list.c \
- builtin-test.c \
- builtin-help.c \
- builtin-zero-labels.c \
- builtin-read-labels.c \
- util/parse-options.c \
- util/parse-options.h \
- util/usage.c \
- util/json.c \
- util/size.c \
- util/strbuf.c \
- util/wrapper.c \
- util/filter.c \
- test/core.c
-
-if ENABLE_SMART
-ndctl_SOURCES += util/json-smart.c
-endif
-
-if ENABLE_TEST
-ndctl_SOURCES += test/libndctl.c \
- test/dpa-alloc.c \
- test/parent-uuid.c
-endif
-
-if ENABLE_DESTRUCTIVE
-ndctl_SOURCES += test/blk_namespaces.c \
- test/pmem_namespaces.c \
- test/pcommit.c
-ndctl_SOURCES += builtin-bat.c
-endif
-
-ndctl_LDADD = lib/libndctl.la $(UUID_LIBS) $(KMOD_LIBS) $(JSON_LIBS)
-
noinst_LIBRARIES = libccan.a
libccan_a_SOURCES = \
ccan/str/str.h \
@@ -145,51 +49,12 @@ libccan_a_SOURCES = \
ccan/short_types/short_types.h \
ccan/endian/endian.h
-EXTRA_DIST += lib/libndctl.sym
-
-lib_libndctl_la_LDFLAGS = $(AM_LDFLAGS) \
- -version-info $(LIBNDCTL_CURRENT):$(LIBNDCTL_REVISION):$(LIBNDCTL_AGE) \
- -Wl,--version-script=$(top_srcdir)/lib/libndctl.sym
-lib_libndctl_la_DEPENDENCIES = ${top_srcdir}/lib/libndctl.sym
-
-pkgconfigdir = $(libdir)/pkgconfig
-pkgconfig_DATA = lib/libndctl.pc
-EXTRA_DIST += lib/libndctl.pc.in
-CLEANFILES += lib/libndctl.pc
-
-TESTS = test/libndctl test/dpa-alloc test/parent-uuid test/create.sh \
- test/clear.sh test/dax-errors.sh
-check_PROGRAMS = test/libndctl test/dpa-alloc test/parent-uuid test/dax-errors
-
-if ENABLE_DESTRUCTIVE
-TESTS += test/blk-ns test/pmem-ns test/pcommit
-check_PROGRAMS += test/blk-ns test/pmem-ns test/pcommit
-
-TESTS += test/dax-dev test/dax.sh test/mmap.sh
-check_PROGRAMS += test/dax-dev test/dax-pmd test/mmap
-endif
-
-test_libndctl_SOURCES = test/libndctl.c test/core.c
-test_libndctl_LDADD = lib/libndctl.la $(UUID_LIBS) $(KMOD_LIBS)
-
-test_pcommit_SOURCES = test/pcommit.c test/core.c
-test_pcommit_LDADD = lib/libndctl.la $(KMOD_LIBS)
-
-test_blk_ns_SOURCES = test/blk_namespaces.c test/core.c
-test_blk_ns_LDADD = lib/libndctl.la $(KMOD_LIBS)
-
-test_pmem_ns_SOURCES = test/pmem_namespaces.c test/core.c
-test_pmem_ns_LDADD = lib/libndctl.la $(KMOD_LIBS)
-
-test_dpa_alloc_SOURCES = test/dpa-alloc.c test/core.c
-test_dpa_alloc_LDADD = lib/libndctl.la $(UUID_LIBS) $(KMOD_LIBS)
-
-test_parent_uuid_SOURCES = test/parent-uuid.c test/core.c
-test_parent_uuid_LDADD = lib/libndctl.la $(UUID_LIBS) $(KMOD_LIBS)
-
-test_dax_dev_SOURCES = test/dax-dev.c test/core.c
-test_dax_dev_LDADD = lib/libndctl.la
-
-test_dax_pmd_SOURCES = test/dax-pmd.c
-test_mmap_SOURCES = test/mmap.c
-test_dax_errors_SOURCES = test/dax-errors.c
+noinst_LIBRARIES += libutil.a
+libutil_a_SOURCES = \
+ util/parse-options.c \
+ util/parse-options.h \
+ util/usage.c \
+ util/size.c \
+ util/strbuf.c \
+ util/wrapper.c \
+ util/filter.c
diff --git a/Makefile.am.in b/Makefile.am.in
new file mode 100644
index 000000000000..a27d95e15061
--- /dev/null
+++ b/Makefile.am.in
@@ -0,0 +1,41 @@
+EXTRA_DIST =
+CLEANFILES =
+
+AM_MAKEFLAGS = --no-print-directory
+
+AM_CPPFLAGS = \
+ -include $(top_builddir)/config.h \
+ -DSYSCONFDIR=\""$(sysconfdir)"\" \
+ -DLIBEXECDIR=\""$(libexecdir)"\" \
+ -DPREFIX=\""$(prefix)"\" \
+ -DNDCTL_MAN_PATH=\""$(mandir)"\" \
+ -I${top_srcdir}/ndctl/lib \
+ -I${top_srcdir}/ndctl \
+ -I${top_srcdir}/ \
+ $(KMOD_CFLAGS) \
+ $(UDEV_CFLAGS) \
+ $(UUID_CFLAGS) \
+ $(JSON_CFLAGS)
+
+AM_CFLAGS = ${my_CFLAGS} \
+ -fvisibility=hidden \
+ -ffunction-sections \
+ -fdata-sections
+
+AM_LDFLAGS = \
+ -Wl,--gc-sections \
+ -Wl,--as-needed
+
+SED_PROCESS = \
+ $(AM_V_GEN)$(MKDIR_P) $(dir $@) && $(SED) \
+ -e 's,@VERSION\@,$(VERSION),g' \
+ -e 's,@prefix\@,$(prefix),g' \
+ -e 's,@exec_prefix\@,$(exec_prefix),g' \
+ -e 's,@libdir\@,$(libdir),g' \
+ -e 's,@includedir\@,$(includedir),g' \
+ < $< > $@ || rm $@
+
+LIBNDCTL_CURRENT=7
+LIBNDCTL_REVISION=0
+LIBNDCTL_AGE=1
+
diff --git a/configure.ac b/configure.ac
index 1525cd4dd954..bc8e947e89f8 100644
--- a/configure.ac
+++ b/configure.ac
@@ -5,7 +5,7 @@ AC_INIT([ndctl],
[linux-nvdimm(a)lists.01.org],
[ndctl],
[https://github.com/pmem/ndctl])
-AC_CONFIG_SRCDIR([lib/libndctl.c])
+AC_CONFIG_SRCDIR([ndctl/lib/libndctl.c])
AC_CONFIG_AUX_DIR([build-aux])
AM_INIT_AUTOMAKE([
check-news
@@ -113,7 +113,7 @@ AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[
#ifdef HAVE_NDCTL_H
#include <linux/ndctl.h>
#else
- #include "ndctl.h"
+ #include "ndctl/ndctl.h"
#endif
]], [[
int x = ARS_STATUS_MASK;
@@ -132,7 +132,7 @@ AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[
#ifdef HAVE_NDCTL_H
#include <linux/ndctl.h>
#else
- #include "ndctl.h"
+ #include "ndctl/ndctl.h"
#endif
]], [[
int x = ND_CMD_CLEAR_ERROR;
@@ -151,7 +151,7 @@ AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[
#ifdef HAVE_NDCTL_H
#include <linux/ndctl.h>
#else
- #include "ndctl.h"
+ #include "ndctl/ndctl.h"
#endif
]], [[
int x = ND_DEVICE_DAX_PMEM;
@@ -170,7 +170,7 @@ AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[
#ifdef HAVE_NDCTL_H
#include <linux/ndctl.h>
#else
- #include "ndctl.h"
+ #include "ndctl/ndctl.h"
#endif
]], [[
int x = ND_SMART_HEALTH_VALID;
@@ -209,7 +209,7 @@ AC_CONFIG_COMMANDS([gen-libndctl.h],
-e s/HAVE_NDCTL_SMART/$enable_smart/ \
-e s/HAVE_NDCTL_CLEAR_ERROR/$enable_clear_err/ \
-e s/HAVE_NDCTL_DEV_DAX/$enable_dev_dax/ \
- < lib/ndctl/libndctl.h.in > lib/ndctl/libndctl.h
+ < ndctl/lib/libndctl.h.in > ndctl/lib/libndctl.h
]],
[[
enable_ars=$enable_ars
@@ -243,6 +243,8 @@ AC_SUBST([my_CFLAGS])
AC_CONFIG_HEADERS(config.h)
AC_CONFIG_FILES([
Makefile
+ ndctl/Makefile
+ test/Makefile
Documentation/Makefile
])
diff --git a/ndctl/Makefile.am b/ndctl/Makefile.am
new file mode 100644
index 000000000000..a89f95088565
--- /dev/null
+++ b/ndctl/Makefile.am
@@ -0,0 +1,75 @@
+include $(top_srcdir)/Makefile.am.in
+
+BUILT_SOURCES = lib/libndctl.h
+lib/libndctl.h: lib/libndctl.h.in
+ touch $(top_srcdir)/version.m4
+
+%.pc: %.pc.in Makefile
+ $(SED_PROCESS)
+
+pkginclude_HEADERS = lib/libndctl.h
+lib_LTLIBRARIES = lib/libndctl.la
+
+lib_libndctl_la_SOURCES =\
+ lib/libndctl.h \
+ lib/libndctl-private.h \
+ lib/libndctl.c
+lib_libndctl_la_LIBADD = $(UDEV_LIBS) $(UUID_LIBS) $(KMOD_LIBS)
+
+if ENABLE_ARS
+lib_libndctl_la_SOURCES += lib/libndctl-ars.c
+endif
+
+if ENABLE_SMART
+lib_libndctl_la_SOURCES += lib/libndctl-smart.c
+endif
+
+bin_PROGRAMS = ndctl
+
+ndctl_SOURCES = ndctl.c \
+ builtin-create-nfit.c \
+ builtin-xaction-namespace.c \
+ builtin-xable-region.c \
+ builtin-list.c \
+ builtin-test.c \
+ builtin-help.c \
+ builtin-zero-labels.c \
+ builtin-read-labels.c \
+ util/json.c
+
+if ENABLE_SMART
+ndctl_SOURCES += util/json-smart.c
+endif
+
+if ENABLE_TEST
+ndctl_SOURCES += $(top_srcdir)/test/libndctl.c \
+ $(top_srcdir)/test/dpa-alloc.c \
+ $(top_srcdir)/test/parent-uuid.c \
+ $(top_srcdir)/test/core.c
+endif
+
+if ENABLE_DESTRUCTIVE
+ndctl_SOURCES += $(top_srcdir)/test/blk_namespaces.c \
+ $(top_srcdir)/test/pmem_namespaces.c \
+ $(top_srcdir)/test/pcommit.c
+ndctl_SOURCES += builtin-bat.c
+endif
+
+ndctl_LDADD =\
+ lib/libndctl.la \
+ $(top_srcdir)/libutil.a \
+ $(UUID_LIBS) \
+ $(KMOD_LIBS) \
+ $(JSON_LIBS)
+
+EXTRA_DIST += lib/libndctl.sym
+
+lib_libndctl_la_LDFLAGS = $(AM_LDFLAGS) \
+ -version-info $(LIBNDCTL_CURRENT):$(LIBNDCTL_REVISION):$(LIBNDCTL_AGE) \
+ -Wl,--version-script=$(top_srcdir)/ndctl/lib/libndctl.sym
+lib_libndctl_la_DEPENDENCIES = lib/libndctl.sym
+
+pkgconfigdir = $(libdir)/pkgconfig
+pkgconfig_DATA = lib/libndctl.pc
+EXTRA_DIST += lib/libndctl.pc.in
+CLEANFILES += lib/libndctl.pc
diff --git a/builtin-bat.c b/ndctl/builtin-bat.c
similarity index 100%
rename from builtin-bat.c
rename to ndctl/builtin-bat.c
diff --git a/builtin-create-nfit.c b/ndctl/builtin-create-nfit.c
similarity index 100%
rename from builtin-create-nfit.c
rename to ndctl/builtin-create-nfit.c
diff --git a/builtin-help.c b/ndctl/builtin-help.c
similarity index 100%
rename from builtin-help.c
rename to ndctl/builtin-help.c
diff --git a/builtin-list.c b/ndctl/builtin-list.c
similarity index 100%
rename from builtin-list.c
rename to ndctl/builtin-list.c
diff --git a/builtin-read-labels.c b/ndctl/builtin-read-labels.c
similarity index 100%
rename from builtin-read-labels.c
rename to ndctl/builtin-read-labels.c
diff --git a/builtin-test.c b/ndctl/builtin-test.c
similarity index 100%
rename from builtin-test.c
rename to ndctl/builtin-test.c
diff --git a/builtin-xable-region.c b/ndctl/builtin-xable-region.c
similarity index 100%
rename from builtin-xable-region.c
rename to ndctl/builtin-xable-region.c
diff --git a/builtin-xaction-namespace.c b/ndctl/builtin-xaction-namespace.c
similarity index 100%
rename from builtin-xaction-namespace.c
rename to ndctl/builtin-xaction-namespace.c
diff --git a/builtin-zero-labels.c b/ndctl/builtin-zero-labels.c
similarity index 100%
rename from builtin-zero-labels.c
rename to ndctl/builtin-zero-labels.c
diff --git a/builtin.h b/ndctl/builtin.h
similarity index 100%
rename from builtin.h
rename to ndctl/builtin.h
diff --git a/lib/.gitignore b/ndctl/lib/.gitignore
similarity index 100%
rename from lib/.gitignore
rename to ndctl/lib/.gitignore
diff --git a/lib/libndctl-ars.c b/ndctl/lib/libndctl-ars.c
similarity index 100%
rename from lib/libndctl-ars.c
rename to ndctl/lib/libndctl-ars.c
diff --git a/lib/libndctl-private.h b/ndctl/lib/libndctl-private.h
similarity index 100%
rename from lib/libndctl-private.h
rename to ndctl/lib/libndctl-private.h
diff --git a/lib/libndctl-smart.c b/ndctl/lib/libndctl-smart.c
similarity index 100%
rename from lib/libndctl-smart.c
rename to ndctl/lib/libndctl-smart.c
diff --git a/lib/libndctl.c b/ndctl/lib/libndctl.c
similarity index 100%
rename from lib/libndctl.c
rename to ndctl/lib/libndctl.c
diff --git a/lib/ndctl/libndctl.h.in b/ndctl/lib/libndctl.h.in
similarity index 100%
rename from lib/ndctl/libndctl.h.in
rename to ndctl/lib/libndctl.h.in
diff --git a/lib/libndctl.pc.in b/ndctl/lib/libndctl.pc.in
similarity index 100%
rename from lib/libndctl.pc.in
rename to ndctl/lib/libndctl.pc.in
diff --git a/lib/libndctl.sym b/ndctl/lib/libndctl.sym
similarity index 100%
rename from lib/libndctl.sym
rename to ndctl/lib/libndctl.sym
diff --git a/ndctl.c b/ndctl/ndctl.c
similarity index 100%
rename from ndctl.c
rename to ndctl/ndctl.c
diff --git a/ndctl.h b/ndctl/ndctl.h
similarity index 100%
rename from ndctl.h
rename to ndctl/ndctl.h
diff --git a/util/json-smart.c b/ndctl/util/json-smart.c
similarity index 100%
rename from util/json-smart.c
rename to ndctl/util/json-smart.c
diff --git a/util/json.c b/ndctl/util/json.c
similarity index 100%
rename from util/json.c
rename to ndctl/util/json.c
diff --git a/util/json.h b/ndctl/util/json.h
similarity index 100%
rename from util/json.h
rename to ndctl/util/json.h
diff --git a/test/Makefile.am b/test/Makefile.am
new file mode 100644
index 000000000000..8f0115387fab
--- /dev/null
+++ b/test/Makefile.am
@@ -0,0 +1,60 @@
+include $(top_srcdir)/Makefile.am.in
+
+TESTS =\
+ libndctl \
+ dpa-alloc \
+ parent-uuid \
+ create.sh \
+ clear.sh \
+ dax-errors.sh
+
+check_PROGRAMS =\
+ libndctl \
+ dpa-alloc \
+ parent-uuid \
+ dax-errors
+
+if ENABLE_DESTRUCTIVE
+TESTS +=\
+ blk-ns \
+ pmem-ns \
+ pcommit \
+ dax-dev \
+ dax.sh \
+ mmap.sh
+
+check_PROGRAMS +=\
+ blk-ns \
+ pmem-ns \
+ pcommit \
+ dax-dev \
+ dax-pmd \
+ mmap
+endif
+
+LIBNDCTL_LIB = $(top_srcdir)/ndctl/lib/libndctl.la
+
+libndctl_SOURCES = libndctl.c core.c
+libndctl_LDADD = $(LIBNDCTL_LIB) $(UUID_LIBS) $(KMOD_LIBS)
+
+pcommit_SOURCES = pcommit.c core.c
+pcommit_LDADD = $(LIBNDCTL_LIB) $(KMOD_LIBS)
+
+blk_ns_SOURCES = blk_namespaces.c core.c
+blk_ns_LDADD = $(LIBNDCTL_LIB) $(KMOD_LIBS)
+
+pmem_ns_SOURCES = pmem_namespaces.c core.c
+pmem_ns_LDADD = $(LIBNDCTL_LIB) $(KMOD_LIBS)
+
+dpa_alloc_SOURCES = dpa-alloc.c core.c
+dpa_alloc_LDADD = $(LIBNDCTL_LIB) $(UUID_LIBS) $(KMOD_LIBS)
+
+parent_uuid_SOURCES = parent-uuid.c core.c
+parent_uuid_LDADD = $(LIBNDCTL_LIB) $(UUID_LIBS) $(KMOD_LIBS)
+
+dax_dev_SOURCES = dax-dev.c core.c
+dax_dev_LDADD = $(LIBNDCTL_LIB)
+
+dax_pmd_SOURCES = dax-pmd.c
+mmap_SOURCES = mmap.c
+dax_errors_SOURCES = dax-errors.c
diff --git a/test/clear.sh b/test/clear.sh
index 16b0cbf3a71c..7765c10078e0 100755
--- a/test/clear.sh
+++ b/test/clear.sh
@@ -1,6 +1,6 @@
#!/bin/bash -x
DEV=""
-NDCTL="./ndctl"
+NDCTL="../ndctl/ndctl"
BUS="-b nfit_test.0"
BUS1="-b nfit_test.1"
json2var="s/[{}\",]//g; s/:/=/g"
diff --git a/test/create.sh b/test/create.sh
index b190eccfc243..b0dcdb7da2bd 100755
--- a/test/create.sh
+++ b/test/create.sh
@@ -1,6 +1,6 @@
#!/bin/bash -x
DEV=""
-NDCTL="./ndctl"
+NDCTL="../ndctl/ndctl"
BUS="-b nfit_test.0"
json2var="s/[{}\",]//g; s/:/=/g"
SECTOR_SIZE="4096"
diff --git a/test/dax-errors.sh b/test/dax-errors.sh
index 51fe082f0de4..2a498f11e089 100755
--- a/test/dax-errors.sh
+++ b/test/dax-errors.sh
@@ -1,7 +1,7 @@
#!/bin/bash -x
DEV=""
-NDCTL="./ndctl"
+NDCTL="../ndctl/ndctl"
BUS="-b nfit_test.0"
BUS1="-b nfit_test.1"
MNT=test_dax_mnt
@@ -83,8 +83,8 @@ echo $start_sect 8 > /sys/block/$blockdev/badblocks
dd if=$MNT/$FILE of=/dev/null iflag=direct bs=4096 count=1 && err $LINENO || true
# run the dax-errors test
-test -x test/dax-errors
-test/dax-errors $MNT/$FILE
+test -x ./dax-errors
+./dax-errors $MNT/$FILE
# TODO: disable this check till we have clear-on-write in the kernel
#if read sector len < /sys/block/$blockdev/badblocks; then
diff --git a/test/dax.sh b/test/dax.sh
index 9d2c27225653..8db436a869b5 100755
--- a/test/dax.sh
+++ b/test/dax.sh
@@ -1,7 +1,7 @@
#!/bin/bash
MNT=test_dax_mnt
FILE=image
-NDCTL="./ndctl"
+NDCTL="../ndctl/ndctl"
json2var="s/[{}\",]//g; s/:/=/g"
blockdev=""
@@ -21,13 +21,13 @@ set -e
mkdir -p $MNT
trap 'err $LINENO' ERR
-blockdev=$(basename $(test/dax-dev))
+blockdev=$(basename $(./dax-dev))
dev=$(basename $(readlink -f /sys/block/$(basename $blockdev)/device))
mkfs.ext4 /dev/$blockdev
mount /dev/$blockdev $MNT -o dax
fallocate -l 1GiB $MNT/$FILE
-test/dax-pmd $MNT/$FILE
+./dax-pmd $MNT/$FILE
umount $MNT
# convert pmem to put the memmap on the device
@@ -39,7 +39,7 @@ eval $(echo $json | sed -e "$json2var")
mkfs.ext4 /dev/$blockdev
mount /dev/$blockdev $MNT -o dax
fallocate -l 1GiB $MNT/$FILE
-test/dax-pmd $MNT/$FILE
+./dax-pmd $MNT/$FILE
umount $MNT
json=$($NDCTL create-namespace -m raw -f -e $dev)
@@ -49,7 +49,7 @@ eval $(echo $json | sed -e "$json2var")
mkfs.xfs -f /dev/$blockdev
mount /dev/$blockdev $MNT -o dax
fallocate -l 1GiB $MNT/$FILE
-test/dax-pmd $MNT/$FILE
+./dax-pmd $MNT/$FILE
umount $MNT
# convert pmem to put the memmap on the device
@@ -60,7 +60,7 @@ eval $(echo $json | sed -e "$json2var")
mkfs.xfs -f /dev/$blockdev
mount /dev/$blockdev $MNT -o dax
fallocate -l 1GiB $MNT/$FILE
-test/dax-pmd $MNT/$FILE
+./dax-pmd $MNT/$FILE
umount $MNT
# revert namespace to raw mode
diff --git a/test/mmap.sh b/test/mmap.sh
index 745962b0c059..e66fb1369434 100755
--- a/test/mmap.sh
+++ b/test/mmap.sh
@@ -2,7 +2,7 @@
MNT=test_mmap_mnt
FILE=image
DEV=""
-TEST=test/mmap
+TEST=./mmap
err() {
rc=1
@@ -46,7 +46,7 @@ set -e
mkdir -p $MNT
trap 'err $LINENO' ERR
-DEV=$(test/dax-dev)
+DEV=$(./dax-dev)
mkfs.ext4 $DEV
mount $DEV $MNT -o dax
4 years, 5 months
Re: [PATCH v2 2/3] /dev/dax, core: file operations and dax-mmap
by Dan Williams
On Wed, May 18, 2016 at 10:12 AM, Paul E. McKenney
<paulmck(a)linux.vnet.ibm.com> wrote:
> On Wed, May 18, 2016 at 11:15:11AM +0200, Hannes Reinecke wrote:
>> On 05/18/2016 11:10 AM, Paul Mackerras wrote:
>> > On Wed, May 18, 2016 at 10:07:19AM +0200, Hannes Reinecke wrote:
>> >> On 05/18/2016 12:19 AM, Dan Williams wrote:
>> >>> On Tue, May 17, 2016 at 3:57 AM, Johannes Thumshirn <jthumshirn(a)suse.de> wrote:
>> >>>> On Sat, May 14, 2016 at 11:26:29PM -0700, Dan Williams wrote:
>> >>>>> The "Device DAX" core enables dax mappings of performance / feature
>> >>>>> differentiated memory. An open mapping or file handle keeps the backing
>> >>>>> struct device live, but new mappings are only possible while the device
>> >>>>> is enabled. Faults are handled under rcu_read_lock to synchronize
>> >>>>> with the enabled state of the device.
>> >>>>>
>> >>>>> Similar to the filesystem-dax case the backing memory may optionally
>> >>>>> have struct page entries. However, unlike fs-dax there is no support
>> >>>>> for private mappings, or mappings that are not backed by media (see
>> >>>>> use of zero-page in fs-dax).
>> >>>>>
>> >>>>> Mappings are always guaranteed to match the alignment of the dax_region.
>> >>>>> If the dax_region is configured to have a 2MB alignment, all mappings
>> >>>>> are guaranteed to be backed by a pmd entry. Contrast this determinism
>> >>>>> with the fs-dax case where pmd mappings are opportunistic. If userspace
>> >>>>> attempts to force a misaligned mapping, the driver will fail the mmap
>> >>>>> attempt. See dax_dev_check_vma() for other scenarios that are rejected,
>> >>>>> like MAP_PRIVATE mappings.
>> >>>>>
>> >>>>> Cc: Jeff Moyer <jmoyer(a)redhat.com>
>> >>>>> Cc: Christoph Hellwig <hch(a)lst.de>
>> >>>>> Cc: Andrew Morton <akpm(a)linux-foundation.org>
>> >>>>> Cc: Dave Hansen <dave.hansen(a)linux.intel.com>
>> >>>>> Cc: Ross Zwisler <ross.zwisler(a)linux.intel.com>
>> >>>>> Signed-off-by: Dan Williams <dan.j.williams(a)intel.com>
>> >>>>> ---
>> >>>>> drivers/dax/Kconfig | 1
>> >>>>> drivers/dax/dax.c | 316 +++++++++++++++++++++++++++++++++++++++++++++++++++
>> >>>>> mm/huge_memory.c | 1
>> >>>>> mm/hugetlb.c | 1
>> >>>>> 4 files changed, 319 insertions(+)
>> >>>>>
>> >>>>> diff --git a/drivers/dax/Kconfig b/drivers/dax/Kconfig
>> >>>>> index 86ffbaa891ad..cedab7572de3 100644
>> >>>>> --- a/drivers/dax/Kconfig
>> >>>>> +++ b/drivers/dax/Kconfig
>> >>>>> @@ -1,6 +1,7 @@
>> >>>>> menuconfig DEV_DAX
>> >>>>> tristate "DAX: direct access to differentiated memory"
>> >>>>> default m if NVDIMM_DAX
>> >>>>> + depends on TRANSPARENT_HUGEPAGE
>> >>>>> help
>> >>>>> Support raw access to differentiated (persistence, bandwidth,
>> >>>>> latency...) memory via an mmap(2) capable character
>> >>>>> diff --git a/drivers/dax/dax.c b/drivers/dax/dax.c
>> >>>>> index 8207fb33a992..b2fe8a0ce866 100644
>> >>>>> --- a/drivers/dax/dax.c
>> >>>>> +++ b/drivers/dax/dax.c
>> >>>>> @@ -49,6 +49,7 @@ struct dax_region {
>> >>>>> * @region - parent region
>> >>>>> * @dev - device backing the character device
>> >>>>> * @kref - enable this data to be tracked in filp->private_data
>> >>>>> + * @alive - !alive + rcu grace period == no new mappings can be established
>> >>>>> * @id - child id in the region
>> >>>>> * @num_resources - number of physical address extents in this device
>> >>>>> * @res - array of physical address ranges
>> >>>>> @@ -57,6 +58,7 @@ struct dax_dev {
>> >>>>> struct dax_region *region;
>> >>>>> struct device *dev;
>> >>>>> struct kref kref;
>> >>>>> + bool alive;
>> >>>>> int id;
>> >>>>> int num_resources;
>> >>>>> struct resource res[0];
>> >>>>> @@ -150,6 +152,10 @@ static void destroy_dax_dev(void *_dev)
>> >>>>>
>> >>>>> dev_dbg(dev, "%s\n", __func__);
>> >>>>>
>> >>>>> + /* disable and flush fault handlers, TODO unmap inodes */
>> >>>>> + dax_dev->alive = false;
>> >>>>> + synchronize_rcu();
>
> If you need to wait for fault handlers, you need synchronize_sched()
> instead of synchronize_rcu(). Please note that synchronize_rcu() is
> guaranteed to wait only for tasks that have done rcu_read_lock() to reach
> the corresponding rcu_read_unlock(). In contrast, synchronize_sched()
> is guaranteed to wait for any non-idle preempt-disable region of code
> to complete, regardless of exactly what is disabling preemptiong.
>
> And the "non-idle" is not an idle qualifier. If you need to wait on fault
> handlers that somehow occur from an idle hardware thread, you will need
> those fault handlers to do rcu_irq_enter() on entry and rcu_irq_exit()
> on exit. (My guess is that you cannot take faults in the idle loop,
> but I have learned not to trust such guesses all that far.)
>
> And last, but definitely not least, synchronize_sched() waits only
> for pre-existing preempt-disable regions of code. So if you do
> synchronize_sched(), and immediately after a fault handler starts,
> synchronize_sched() won't necessarily wait on it. However, you -are-
> guaranteed that synchronize_shced() -will- wait for any fault handler
> that might possibly see dax_dev->alive with a non-false value.
>
> Are these the guarantees you are looking for? (Yes, I did recently
> watch "A New Hope". Why do you ask?)
Spoken like a true rcu-Jedi.
So in this case the fault handlers are indeed running under
rcu_read_lock(), and I can't fathom how these faults would trigger
from an idle thread...
>
>> >>>>> +
>> >>>>
>> >>>> IIRC RCU is only protecting a pointer, not the content of the pointer, so this
>> >>>> looks wrong to me.
>
> RCU can be, and usually is, used to protect pointers, but it can be and
> sometimes is used for other things as well. At its core, RCU is about
> waiting for pre-existing RCU readers to complete.
>
>> >>> The driver is using RCU to guarantee that all currently running fault
>> >>> handlers have either completed or will see the new state of ->alive
>> >>> when they start. Reference counts are protecting the actual dax_dev
>> >>> object.
>> >>>
>> >> Hmm.
>> >> This is the same 'creative' RCU usage Mike Snitzer has been trying
>> >> when trying to improve device-mapper performance.
>
> To repeat, unless all your fault handlers begin with rcu_read_lock()
> and end with rcu_read_unlock(), and as long as you don't care about not
> waiting for fault handlers that are currently executing just before
> the rcu_read_lock() and just after the rcu_read_unlock(), you need
> synchronize_sched() rather than synchronize_rcu() for this job.
>
>> >> >From my understanding RCU is protecting the _pointer_, not the
>> >> values of the structure pointed to.
>> >> IOW we are guaranteed to have a valid pointer at any time.
>> >> But at the same time _no_ guarantee is made about the _contents_ of
>> >> the structure.
>> >> It might well be that using 'synchronize_rcu' giving you similar
>> >> results (as synchronize_rcu() is essentially waiting a SMP grace
>> >> period, after which all CPUs should be seeing the update).
>> >> However, I haven't been able to find that this is a guaranteed
>> >> behaviour.
>> >> So from my understanding you have to use locking primitives
>> >> protecting the contents of the structure or exchange the _entire_
>> >> structure if you want to rely on RCU here.
>> >>
>> >> Can we get some clarification here?
>
> Maybe... What exactly is your synchronization design needing here?
>
>> >> Paul?
>> >
>> > I think you want the other Paul, Paul McKenney.
>> >
>> I think you are in fact right.
>> Sorry for the Paul-confusion :-)
>
> Did I keep my end of the confusion up? ;-)
Yes, I think we're good, but please double check I am not mistaken in
the following clarification comment:
@@ -150,6 +152,16 @@ static void destroy_dax_dev(void *_dev)
dev_dbg(dev, "%s\n", __func__);
+ /*
+ * Note, rcu is not protecting the liveness of dax_dev, rcu is
+ * ensuring that any fault handlers that might have seen
+ * dax_dev->alive == true, have completed. Any fault handlers
+ * that start after synchronize_rcu() has started will abort
+ * upon seeing dax_dev->alive == false.
+ */
+ dax_dev->alive = false;
+ synchronize_rcu();
+
get_device(dev);
device_unregister(dev);
ida_simple_remove(&dax_region->ida, dax_dev->id);
@@ -173,6 +185,7 @@ int devm_create_dax_dev(struct dax_region
*dax_region, struct resource *re
4 years, 5 months
