12:52 a.m.
On 02/11/2015 01:33 AM, Roger Pao wrote:
adrbd correctly found all NVDIMMs and presented a block device for
them. It was simply plug and play.
Requiring the user to look at dmesg output for the location of the NVDIMMs, creating a
map= for pmem or the equivalent for prd is a bit complex for an end user.
It's worse if they have to hack the kernel's e820 handling of NVDIMMs. I assume
pmem/prd will have an easier to use solution in the future.
On Tue, Feb 10, 2015 at 3:05 PM, Dan Williams <dan.j.williams(a)intel.com
<mailto:dan.j.williams@intel.com>> wrote:
On Tue, Feb 10, 2015 at 2:56 PM, Roger C. Pao (Enmotus)
<rcpao.enmotus(a)gmail.com <mailto:rcpao.enmotus@gmail.com>> wrote:
> Thanks Dan.
>
> Unfortunately, this tells me that NVDIMM support in Linux still has a way to
> go before it can be used by our customers.
>
> At least these two implementations (pmem and prd) are publicly available
> open source code unlike adrbd.
>
> I will continue using the prd driver for now as I have it working in a md
> RAID0 with /dev/pmem0 and /dev/sdb (a disk). The md RAID0 is only to verify
> the block device interface works for what we need it to do. We will
> continue developing our products with the assumption that /dev/pmem0 will
> behave similar to any other /dev/sdx in the system.
>
> Regardless of the implementation you finally decide to use, I recommend
> adding a driver parameter to disable the call to
> request_mem_region_exclusive()
That's dangerous. Especially when we have kernel command line and
kconfig options for defining ranges, we do not want a driver
scribbling on random locations that it may not own.
I'm working on a patch that will ignore the failure of request_mem_region_exclusive()
and continue, in this case the driver tests the first and last word of the region to
see if it is actually accessible.
In the same tree I will put a patch that fixes e820 handling of type-12 memory, the
guys in the lab ran with this patch for a long time and did not tell me. Note that
the tree I have at pmem already has patches to mm. Though it could be compiled out of
tree for an STD Kernel if you set BLK_DEV_PMEM_USE_PAGES=n.
It will take me a couple of days to actually test everything though (mainly because
I moved to a 3.20-rc base tree).
Thanks
Boaz
12:54 a.m.
New subject: [Linux-nvdimm] modprobe: ERROR: could not insert 'pmem': Invalid argument
On 02/11/2015 10:52 AM, Boaz Harrosh wrote:
On 02/11/2015 01:33 AM, Roger Pao wrote:
> adrbd correctly found all NVDIMMs and presented a block device for them. It was
simply plug and play.
>
> Requiring the user to look at dmesg output for the location of the NVDIMMs, creating
a map= for pmem or the equivalent for prd is a bit complex for an end user.
>
> It's worse if they have to hack the kernel's e820 handling of NVDIMMs. I
assume pmem/prd will have an easier to use solution in the future.
>
> On Tue, Feb 10, 2015 at 3:05 PM, Dan Williams <dan.j.williams(a)intel.com
<mailto:dan.j.williams@intel.com>> wrote:
>
> On Tue, Feb 10, 2015 at 2:56 PM, Roger C. Pao (Enmotus)
> <rcpao.enmotus(a)gmail.com <mailto:rcpao.enmotus@gmail.com>> wrote:
> > Thanks Dan.
> >
> > Unfortunately, this tells me that NVDIMM support in Linux still has a way
to
> > go before it can be used by our customers.
> >
> > At least these two implementations (pmem and prd) are publicly available
> > open source code unlike adrbd.
> >
> > I will continue using the prd driver for now as I have it working in a md
> > RAID0 with /dev/pmem0 and /dev/sdb (a disk). The md RAID0 is only to
verify
> > the block device interface works for what we need it to do. We will
> > continue developing our products with the assumption that /dev/pmem0 will
> > behave similar to any other /dev/sdx in the system.
> >
> > Regardless of the implementation you finally decide to use, I recommend
> > adding a driver parameter to disable the call to
> > request_mem_region_exclusive()
>
> That's dangerous. Especially when we have kernel command line and
> kconfig options for defining ranges, we do not want a driver
> scribbling on random locations that it may not own.
>
>
I'm working on a patch that will ignore the failure of
request_mem_region_exclusive()
and continue, in this case the driver tests the first and last word of the region to
see if it is actually accessible.
In the same tree I will put a patch that fixes e820 handling of type-12 memory, the
guys in the lab ran with this patch for a long time and did not tell me. Note that
the tree I have at pmem already has patches to mm. Though it could be compiled out of
tree for an STD Kernel if you set BLK_DEV_PMEM_USE_PAGES=n.
It will take me a couple of days to actually test everything though (mainly because
I moved to a 3.20-rc base tree).
Thanks
Boaz
Ha, I forgot to ask do you have source code of adrbd?
Thanks
Boaz
7:48 a.m.
New subject: [Linux-nvdimm] modprobe: ERROR: could not insert 'pmem': Invalid argument
-----Original Message-----
From: Linux-nvdimm [mailto:linux-nvdimm-bounces@lists.01.org] On Behalf Of Boaz Harrosh
Sent: Wednesday, February 11, 2015 1:55 AM
To: Roger Pao; Williams, Dan J
Cc: linux-nvdimm(a)lists.01.org
Subject: Re: [Linux-nvdimm] modprobe: ERROR: could not insert 'pmem': Invalid
argument
On 02/11/2015 10:52 AM, Boaz Harrosh wrote:
> On 02/11/2015 01:33 AM, Roger Pao wrote:
>> adrbd correctly found all NVDIMMs and presented a block device for them. It was
simply plug and play.
>>
>> Requiring the user to look at dmesg output for the location of the NVDIMMs,
creating a map= for pmem or the equivalent for prd is
a bit complex for an end user.
>>
>> It's worse if they have to hack the kernel's e820 handling of NVDIMMs.
I assume pmem/prd will have an easier to use solution in
the future.
>>
>> On Tue, Feb 10, 2015 at 3:05 PM, Dan Williams <dan.j.williams(a)intel.com
<mailto:dan.j.williams@intel.com>> wrote:
>>
>> On Tue, Feb 10, 2015 at 2:56 PM, Roger C. Pao (Enmotus)
>> <rcpao.enmotus(a)gmail.com <mailto:rcpao.enmotus@gmail.com>>
wrote:
>> > Thanks Dan.
>> >
>> > Unfortunately, this tells me that NVDIMM support in Linux still has a
way to
>> > go before it can be used by our customers.
>> >
>> > At least these two implementations (pmem and prd) are publicly
available
>> > open source code unlike adrbd.
>> >
>> > I will continue using the prd driver for now as I have it working in a
md
>> > RAID0 with /dev/pmem0 and /dev/sdb (a disk). The md RAID0 is only to
verify
>> > the block device interface works for what we need it to do. We will
>> > continue developing our products with the assumption that /dev/pmem0
will
>> > behave similar to any other /dev/sdx in the system.
>> >
>> > Regardless of the implementation you finally decide to use, I
recommend
>> > adding a driver parameter to disable the call to
>> > request_mem_region_exclusive()
>>
>> That's dangerous. Especially when we have kernel command line and
>> kconfig options for defining ranges, we do not want a driver
>> scribbling on random locations that it may not own.
>>
>>
>
> I'm working on a patch that will ignore the failure of
request_mem_region_exclusive()
> and continue, in this case the driver tests the first and last word of the region
to
> see if it is actually accessible.
>
> In the same tree I will put a patch that fixes e820 handling of type-12 memory, the
> guys in the lab ran with this patch for a long time and did not tell me. Note that
> the tree I have at pmem already has patches to mm. Though it could be compiled out
of
> tree for an STD Kernel if you set BLK_DEV_PMEM_USE_PAGES=n.
>
> It will take me a couple of days to actually test everything though (mainly because
> I moved to a 3.20-rc base tree).
>
> Thanks
> Boaz
>
Ha, I forgot to ask do you have source code of adrbd?
Boaz,
While there are no issues getting you the source of ADRBD, it's probably
counterproductive. It's a bit of history. There is no official memory type in the APCI
spec for NVDIMM yet. Originally we set it for type 6 as an experiment but then type 6 was
taken. Then we basically said screw it and set it for type 12 until something official
happens. The ADRBD code basically puts in a hack during the memory initialization code to
claim all type 12 memory as special reserved type and exports a platform device with the
resources. When the driver probes, it picks up the platform device and exposes the memory
ranges as block devices. Going forward we want to move towards pmem and away from adrbd as
the hacks are not upstreamable (therefore not distributed to the wider public). And there
really isn't any reason we can't use pmem as the common code base. ADRBD was
always provided as reference code more than anything else. The bottom line is pmem is
where we are going and we should focus on improving that and get that to work for all
NVDIMM types.
8:32 a.m.
New subject: [Linux-nvdimm] modprobe: ERROR: could not insert 'pmem': Invalid argument
On 02/11/2015 05:48 PM, Jiang, Dave wrote:
<>
Boaz,
While there are no issues getting you the source of ADRBD, it's
probably counterproductive. It's a bit of history. There is no
official memory type in the APCI spec for NVDIMM yet. Originally we
set it for type 6 as an experiment but then type 6 was taken. Then we
basically said screw it and set it for type 12 until something
official happens. The ADRBD code basically puts in a hack during the
memory initialization code to claim all type 12 memory as special
reserved type and exports a platform device with the resources. When
the driver probes, it picks up the platform device and exposes the
memory ranges as block devices. Going forward we want to move towards
pmem and away from adrbd as the hacks are not upstreamable (therefore
not distributed to the wider public). And there really isn't any
reason we can't use pmem as the common code base. ADRBD was always
provided as reference code more than anything else. The bottom line
is pmem is where we are going and we should focus on improving that
and get that to work for all NVDIMM types.
Hi Sir Dave
Not sure if you have noticed, I kind of father pmem and am actively
maintaining my own advanced version of this driver.
We have our own changes to e820.c to support type 12. I intend
to eventually push this change to Kernel.
Because APCI spec, or not, There are out there, (And here in our lab)
type 12 DDR3 NvDIMMs and that is an hard cold fact. ACPI comity better
black list this type, yes I dare them to define a type-12 for something
else. I understand this is considered bad politics, but life is stronger
than any comity or politics and these chips are out there because users
wants them and already use them, and type 12 is good as any.
We just used "reserved" for type 12 right now, but I want to define
a "DDR3 NvDIMM" new type and constant (12). (I'd love if DDR4 would
use the de-fecto STD type 12 as well and I'll change the name to
"NVDIMM")
With this very small definition to e820.c, and the current pmem.c
driver. We are able to auto load any and multiple NvDIMMs with an
help of an autoload_pmem script. (actually autoload_pmem also
works without the changes to e820.c)
Yes please send me the ADRBD source code I want it as a reference
code that has some experience in the field with some of the vendors.
All I can test with is the two vendors I have, so it will give me
more confidence that I have not missed some details with current chips.
From what I understand they all use type-12 and this comes up as
"busy"
in the call to request_mem_region_exclusive(), with current code.
All I'm changing is to make it start as "available" at startup.
Even with no changes to e820.c I can just ignore the failure of
request_mem_region_exclusive() and continue to use the chips. As is.
I promise to not publish the driver or anything like that. I only
need it as reference code.
Thanks in advance
Boaz
9:51 a.m.
New subject: [Linux-nvdimm] modprobe: ERROR: could not insert 'pmem': Invalid argument
On Wed, Feb 11, 2015 at 8:32 AM, Boaz Harrosh <boaz(a)plexistor.com> wrote:
Because APCI spec, or not, There are out there, (And here in our lab)
type 12 DDR3 NvDIMMs and that is an hard cold fact. ACPI comity better
black list this type, yes I dare them to define a type-12 for something
else. I understand this is considered bad politics, but life is stronger
than any comity or politics and these chips are out there because users
wants them and already use them, and type 12 is good as any.
https://lists.01.org/mailman/listinfo/linux-nvdimm
I can confirm three NVDIMM vendors use Type 12. I had to send back the
fourth vendor's NVDIMMs due to reboot issues. Vendors' names redacted due
to NDAs.
a
=~=~=~=~=~=~=~=~=~=~=~= PuTTY log 2015.02.11 09:07:07
=~=~=~=~=~=~=~=~=~=~=~=
Copyright (c) 2013-2015 Enmotus, Inc. All rights reserved.
[2015-01-22 12:13:29]
E820 Memory Map
Entry Nr Base Address Length Type Ext Attr
-------- ---------------- ---------------- -------- --------
00000001 0000000000000000 000000000009ac00 00000001 00000001
00000002 000000000009ac00 0000000000005400 00000002 00000001
00000003 00000000000e0000 0000000000020000 00000002 00000001
00000004 0000000000100000 00000000732fd000 00000001 00000001
00000005 00000000733fd000 000000000a857000 00000002 00000001
00000006 000000007dc54000 00000000002fa000 00000004 00000001
00000007 000000007df4e000 00000000007a0000 00000002 00000001
00000008 000000007e6ee000 0000000000030000 00000004 00000001
00000009 000000007e71e000 0000000000c3e000 00000002 00000001
0000000a 000000007f35c000 00000000004a4000 00000004 00000001
0000000b 0000000180000000 0000000300000000 00000001 00000001
0000000c 0000000480000000 0000000100000000 0000000c 00000001
0000000d 0000000080000000 0000000010000000 00000002 00000001
0000000e 00000000fed1c000 0000000000024000 00000002 00000001
b
=~=~=~=~=~=~=~=~=~=~=~= PuTTY log 2015.02.11 09:19:02
=~=~=~=~=~=~=~=~=~=~=~=
Copyright (c) 2013-2015 Enmotus, Inc. All rights reserved.
[2015-01-22 12:13:29]
E820 Memory Map
Entry Nr Base Address Length Type Ext Attr
-------- ---------------- ---------------- -------- --------
00000001 0000000000000000 000000000009ac00 00000001 00000001
00000002 000000000009ac00 0000000000005400 00000002 00000001
00000003 00000000000e0000 0000000000020000 00000002 00000001
00000004 0000000000100000 00000000732fd000 00000001 00000001
00000005 00000000733fd000 000000000a857000 00000002 00000001
00000006 000000007dc54000 00000000002fa000 00000004 00000001
00000007 000000007df4e000 00000000007a0000 00000002 00000001
00000008 000000007e6ee000 0000000000030000 00000004 00000001
00000009 000000007e71e000 0000000000c3e000 00000002 00000001
0000000a 000000007f35c000 00000000004a4000 00000004 00000001
0000000b 0000000100000000 0000000380000000 00000001 00000001
0000000c 0000000480000000 0000000100000000 0000000c 00000001
0000000d 0000000080000000 0000000010000000 00000002 00000001
0000000e 00000000fed1c000 0000000000024000 00000002 00000001
c
=~=~=~=~=~=~=~=~=~=~=~= PuTTY log 2015.02.11 09:42:31
=~=~=~=~=~=~=~=~=~=~=~=
Copyright (c) 2013-2015 Enmotus, Inc. All rights reserved.
[2015-02-11 09:31:25]
E820 Memory Map
Entry Nr Base Address Length Type Ext Attr
-------- ---------------- ---------------- -------- --------
00000001 0000000000000000 000000000009ac00 00000001 00000001
00000002 000000000009ac00 0000000000005400 00000002 00000001
00000003 00000000000e0000 0000000000020000 00000002 00000001
00000004 0000000000100000 00000000732fd000 00000001 00000001
00000005 00000000733fd000 000000000a857000 00000002 00000001
00000006 000000007dc54000 00000000002fa000 00000004 00000001
00000007 000000007df4e000 00000000007a0000 00000002 00000001
00000008 000000007e6ee000 0000000000030000 00000004 00000001
00000009 000000007e71e000 0000000000c3e000 00000002 00000001
0000000a 000000007f35c000 00000000004a4000 00000004 00000001
0000000b 0000000100000000 0000000380000000 00000001 00000001
0000000c 0000000480000000 0000000200000000 0000000c 00000001
0000000d 0000000080000000 0000000010000000 00000002 00000001
0000000e 00000000fed1c000 0000000000024000 00000002 00000001
2:56 p.m.
New subject: [Linux-nvdimm] modprobe: ERROR: could not insert 'pmem': Invalid argument
On Wed, Feb 11, 2015 at 8:32 AM, Boaz Harrosh <boaz(a)plexistor.com> wrote:
Because APCI spec, or not, There are out there, (And here in our lab)
type 12 DDR3 NvDIMMs and that is an hard cold fact. ACPI comity better
black list this type, yes I dare them to define a type-12 for something
else. I understand this is considered bad politics, but life is stronger
than any comity or politics and these chips are out there because users
wants them and already use them, and type 12 is good as any.
https://lists.01.org/mailman/listinfo/linux-nvdimm
I can confirm three NVDIMM vendors use Type 12. I had to send back the
fourth vendor's NVDIMMs due to reboot issues. Vendors' names redacted due
to NDAs.
a 4GB
=~=~=~=~=~=~=~=~=~=~=~= PuTTY log 2015.02.11 09:07:07
=~=~=~=~=~=~=~=~=~=~=~=
Copyright (c) 2013-2015 Enmotus, Inc. All rights reserved.
[2015-01-22 12:13:29]
E820 Memory Map
Entry Nr Base Address Length Type Ext Attr
-------- ---------------- ---------------- -------- --------
00000001 0000000000000000 000000000009ac00 00000001 00000001
00000002 000000000009ac00 0000000000005400 00000002 00000001
00000003 00000000000e0000 0000000000020000 00000002 00000001
00000004 0000000000100000 00000000732fd000 00000001 00000001
00000005 00000000733fd000 000000000a857000 00000002 00000001
00000006 000000007dc54000 00000000002fa000 00000004 00000001
00000007 000000007df4e000 00000000007a0000 00000002 00000001
00000008 000000007e6ee000 0000000000030000 00000004 00000001
00000009 000000007e71e000 0000000000c3e000 00000002 00000001
0000000a 000000007f35c000 00000000004a4000 00000004 00000001
0000000b 0000000180000000 0000000300000000 00000001 00000001
0000000c 0000000480000000 0000000100000000 0000000c 00000001
0000000d 0000000080000000 0000000010000000 00000002 00000001
0000000e 00000000fed1c000 0000000000024000 00000002 00000001
b 4GB
=~=~=~=~=~=~=~=~=~=~=~= PuTTY log 2015.02.11 09:19:02
=~=~=~=~=~=~=~=~=~=~=~=
Copyright (c) 2013-2015 Enmotus, Inc. All rights reserved.
[2015-01-22 12:13:29]
E820 Memory Map
Entry Nr Base Address Length Type Ext Attr
-------- ---------------- ---------------- -------- --------
00000001 0000000000000000 000000000009ac00 00000001 00000001
00000002 000000000009ac00 0000000000005400 00000002 00000001
00000003 00000000000e0000 0000000000020000 00000002 00000001
00000004 0000000000100000 00000000732fd000 00000001 00000001
00000005 00000000733fd000 000000000a857000 00000002 00000001
00000006 000000007dc54000 00000000002fa000 00000004 00000001
00000007 000000007df4e000 00000000007a0000 00000002 00000001
00000008 000000007e6ee000 0000000000030000 00000004 00000001
00000009 000000007e71e000 0000000000c3e000 00000002 00000001
0000000a 000000007f35c000 00000000004a4000 00000004 00000001
0000000b 0000000100000000 0000000380000000 00000001 00000001
0000000c 0000000480000000 0000000100000000 0000000c 00000001
0000000d 0000000080000000 0000000010000000 00000002 00000001
0000000e 00000000fed1c000 0000000000024000 00000002 00000001
c 8GB
=~=~=~=~=~=~=~=~=~=~=~= PuTTY log 2015.02.11 09:42:31
=~=~=~=~=~=~=~=~=~=~=~=
Copyright (c) 2013-2015 Enmotus, Inc. All rights reserved.
[2015-02-11 09:31:25]
E820 Memory Map
Entry Nr Base Address Length Type Ext Attr
-------- ---------------- ---------------- -------- --------
00000001 0000000000000000 000000000009ac00 00000001 00000001
00000002 000000000009ac00 0000000000005400 00000002 00000001
00000003 00000000000e0000 0000000000020000 00000002 00000001
00000004 0000000000100000 00000000732fd000 00000001 00000001
00000005 00000000733fd000 000000000a857000 00000002 00000001
00000006 000000007dc54000 00000000002fa000 00000004 00000001
00000007 000000007df4e000 00000000007a0000 00000002 00000001
00000008 000000007e6ee000 0000000000030000 00000004 00000001
00000009 000000007e71e000 0000000000c3e000 00000002 00000001
0000000a 000000007f35c000 00000000004a4000 00000004 00000001
0000000b 0000000100000000 0000000380000000 00000001 00000001
0000000c 0000000480000000 0000000200000000 0000000c 00000001
0000000d 0000000080000000 0000000010000000 00000002 00000001
0000000e 00000000fed1c000 0000000000024000 00000002 00000001
3:01 p.m.
New subject: [Linux-nvdimm] modprobe: ERROR: could not insert 'pmem': Invalid argument
On Wed, Feb 11, 2015 at 8:32 AM, Boaz Harrosh <boaz(a)plexistor.com> wrote:
Because APCI spec, or not, There are out there, (And here in our lab)
type 12 DDR3 NvDIMMs and that is an hard cold fact. ACPI comity better
black list this type, yes I dare them to define a type-12 for something
else. I understand this is considered bad politics, but life is stronger
than any comity or politics and these chips are out there because users
wants them and already use them, and type 12 is good as any.
https://lists.01.org/mailman/listinfo/linux-nvdimm
I can confirm three NVDIMM vendors use Type 12. I had to send back the
fourth vendor's NVDIMMs due to reboot issues. Vendors' names redacted due
to NDAs.
a 4GB
=~=~=~=~=~=~=~=~=~=~=~= PuTTY log 2015.02.11 09:07:07
=~=~=~=~=~=~=~=~=~=~=~=
Copyright (c) 2013-2015 Enmotus, Inc. All rights reserved.
[2015-01-22 12:13:29]
E820 Memory Map
Entry Nr Base Address Length Type Ext Attr
-------- ---------------- ---------------- -------- --------
00000001 0000000000000000 000000000009ac00 00000001 00000001
00000002 000000000009ac00 0000000000005400 00000002 00000001
00000003 00000000000e0000 0000000000020000 00000002 00000001
00000004 0000000000100000 00000000732fd000 00000001 00000001
00000005 00000000733fd000 000000000a857000 00000002 00000001
00000006 000000007dc54000 00000000002fa000 00000004 00000001
00000007 000000007df4e000 00000000007a0000 00000002 00000001
00000008 000000007e6ee000 0000000000030000 00000004 00000001
00000009 000000007e71e000 0000000000c3e000 00000002 00000001
0000000a 000000007f35c000 00000000004a4000 00000004 00000001
0000000b 0000000180000000 0000000300000000 00000001 00000001
0000000c 0000000480000000 0000000100000000 0000000c 00000001
0000000d 0000000080000000 0000000010000000 00000002 00000001
0000000e 00000000fed1c000 0000000000024000 00000002 00000001
b 4GB
=~=~=~=~=~=~=~=~=~=~=~= PuTTY log 2015.02.11 09:19:02
=~=~=~=~=~=~=~=~=~=~=~=
Copyright (c) 2013-2015 Enmotus, Inc. All rights reserved.
[2015-01-22 12:13:29]
E820 Memory Map
Entry Nr Base Address Length Type Ext Attr
-------- ---------------- ---------------- -------- --------
00000001 0000000000000000 000000000009ac00 00000001 00000001
00000002 000000000009ac00 0000000000005400 00000002 00000001
00000003 00000000000e0000 0000000000020000 00000002 00000001
00000004 0000000000100000 00000000732fd000 00000001 00000001
00000005 00000000733fd000 000000000a857000 00000002 00000001
00000006 000000007dc54000 00000000002fa000 00000004 00000001
00000007 000000007df4e000 00000000007a0000 00000002 00000001
00000008 000000007e6ee000 0000000000030000 00000004 00000001
00000009 000000007e71e000 0000000000c3e000 00000002 00000001
0000000a 000000007f35c000 00000000004a4000 00000004 00000001
0000000b 0000000100000000 0000000380000000 00000001 00000001
0000000c 0000000480000000 0000000100000000 0000000c 00000001
0000000d 0000000080000000 0000000010000000 00000002 00000001
0000000e 00000000fed1c000 0000000000024000 00000002 00000001
c 8GB
=~=~=~=~=~=~=~=~=~=~=~= PuTTY log 2015.02.11 09:42:31
=~=~=~=~=~=~=~=~=~=~=~=
Copyright (c) 2013-2015 Enmotus, Inc. All rights reserved.
[2015-02-11 09:31:25]
E820 Memory Map
Entry Nr Base Address Length Type Ext Attr
-------- ---------------- ---------------- -------- --------
00000001 0000000000000000 000000000009ac00 00000001 00000001
00000002 000000000009ac00 0000000000005400 00000002 00000001
00000003 00000000000e0000 0000000000020000 00000002 00000001
00000004 0000000000100000 00000000732fd000 00000001 00000001
00000005 00000000733fd000 000000000a857000 00000002 00000001
00000006 000000007dc54000 00000000002fa000 00000004 00000001
00000007 000000007df4e000 00000000007a0000 00000002 00000001
00000008 000000007e6ee000 0000000000030000 00000004 00000001
00000009 000000007e71e000 0000000000c3e000 00000002 00000001
0000000a 000000007f35c000 00000000004a4000 00000004 00000001
0000000b 0000000100000000 0000000380000000 00000001 00000001
0000000c 0000000480000000 0000000200000000 0000000c 00000001
0000000d 0000000080000000 0000000010000000 00000002 00000001
0000000e 00000000fed1c000 0000000000024000 00000002 00000001
8:54 a.m.
New subject: [Linux-nvdimm] modprobe: ERROR: could not insert 'pmem': Invalid argument
adrbd is wrapped up in a quagmire of NDAs. But yes, I have built it and
tested it just fine. I am not allowed to give it out though. Sorry.
However, at a NVDIMM conference, I was directed to "PMEM.IO" (which is
completely useless for my purposes as it is only user libraries and had no
block device driver). It took me a lot of Googling before I came here and
found not one, but two "pmem" git trees.
So I tried both.
On Wed, Feb 11, 2015 at 12:54 AM, Boaz Harrosh <boaz(a)plexistor.com> wrote:
On 02/11/2015 10:52 AM, Boaz Harrosh wrote:
> On 02/11/2015 01:33 AM, Roger Pao wrote:
>> adrbd correctly found all NVDIMMs and presented a block device for
them. It was simply plug and play.
>>
>> Requiring the user to look at dmesg output for the location of the
NVDIMMs, creating a map= for pmem or the equivalent for prd is a bit
complex for an end user.
>>
>> It's worse if they have to hack the kernel's e820 handling of NVDIMMs.
I assume pmem/prd will have an easier to use solution in the future.
>>
>> On Tue, Feb 10, 2015 at 3:05 PM, Dan Williams <dan.j.williams(a)intel.com
<mailto:dan.j.williams@intel.com>> wrote:
>>
>> On Tue, Feb 10, 2015 at 2:56 PM, Roger C. Pao (Enmotus)
>> <rcpao.enmotus(a)gmail.com <mailto:rcpao.enmotus@gmail.com>>
wrote:
>> > Thanks Dan.
>> >
>> > Unfortunately, this tells me that NVDIMM support in Linux still
has a way to
>> > go before it can be used by our customers.
>> >
>> > At least these two implementations (pmem and prd) are publicly
available
>> > open source code unlike adrbd.
>> >
>> > I will continue using the prd driver for now as I have it working
in a md
>> > RAID0 with /dev/pmem0 and /dev/sdb (a disk). The md RAID0 is
only to verify
>> > the block device interface works for what we need it to do. We
will
>> > continue developing our products with the assumption that
/dev/pmem0 will
>> > behave similar to any other /dev/sdx in the system.
>> >
>> > Regardless of the implementation you finally decide to use, I
recommend
>> > adding a driver parameter to disable the call to
>> > request_mem_region_exclusive()
>>
>> That's dangerous. Especially when we have kernel command line and
>> kconfig options for defining ranges, we do not want a driver
>> scribbling on random locations that it may not own.
>>
>>
>
> I'm working on a patch that will ignore the failure of
request_mem_region_exclusive()
> and continue, in this case the driver tests the first and last word of
the region to
> see if it is actually accessible.
>
> In the same tree I will put a patch that fixes e820 handling of type-12
memory, the
> guys in the lab ran with this patch for a long time and did not tell me.
Note that
> the tree I have at pmem already has patches to mm. Though it could be
compiled out of
> tree for an STD Kernel if you set BLK_DEV_PMEM_USE_PAGES=n.
>
> It will take me a couple of days to actually test everything though
(mainly because
> I moved to a 3.20-rc base tree).
>
> Thanks
> Boaz
>
Ha, I forgot to ask do you have source code of adrbd?
Thanks
Boaz
9 a.m.
New subject: [Linux-nvdimm] modprobe: ERROR: could not insert 'pmem': Invalid argument
On Wed, 2015-02-11 at 08:54 -0800, Roger C. Pao wrote:
adrbd is wrapped up in a quagmire of NDAs. But yes, I have built it
and tested it just fine. I am not allowed to give it out though.
Sorry.
However, at a NVDIMM conference, I was directed to "PMEM.IO" (which is
completely useless for my purposes as it is only user libraries and
had no block device driver). It took me a lot of Googling before I
came here and found not one, but two "pmem" git trees.
pmem.io links to my tree on the base page, second paragraph? :)
http://pmem.io/
Link in this paragraph:
The Linux NVM Library builds on the Direct Access (DAX) changes under
development in Linux. Check out the PRD repo for the latest snapshot
of this work.
- Ross
9 a.m.
New subject: [Linux-nvdimm] modprobe: ERROR: could not insert 'pmem': Invalid argument
On 02/11/2015 06:54 PM, Roger C. Pao wrote:
adrbd is wrapped up in a quagmire of NDAs. But yes, I have built it
and tested it just fine. I am not allowed to give it out though. Sorry.
However, at a NVDIMM conference, I was directed to "PMEM.IO
<http://PMEM.IO>" (which is completely useless for my purposes as it
is only user libraries and had no block device driver). It took me a
lot of Googling before I came here and found not one, but two "pmem"
git trees.
So I tried both.
I intend to release a tree for you that you should be able to use.
You will have two choices:
1. Compile a Kernel with pmem built in + plus all advanced pmem fixtures
2. Compile just pmem for a foreign Kernel but with some stuff turned of
and an warn dmesg print on every load.
But only Next week, still in testing
Thanks
Boaz
<>
3:36 a.m.
New subject: [Linux-nvdimm] modprobe: ERROR: could not insert 'pmem': Invalid argument
On 02/11/2015 07:00 PM, Boaz Harrosh wrote:
On 02/11/2015 06:54 PM, Roger C. Pao wrote:
> adrbd is wrapped up in a quagmire of NDAs. But yes, I have built it and tested it
just fine. I am not allowed to give it out though. Sorry.
>
> However, at a NVDIMM conference, I was directed to "PMEM.IO
> <http://PMEM.IO>" (which is completely useless for my purposes as it
> is only user libraries and had no block device driver). It took me a
> lot of Googling before I came here and found not one, but two "pmem"
> git trees.
>
> So I tried both.
I intend to release a tree for you that you should be able to use.
You will have two choices:
1. Compile a Kernel with pmem built in + plus all advanced pmem fixtures
2. Compile just pmem for a foreign Kernel but with some stuff turned of
and an warn dmesg print on every load.
But only Next week, still in testing
Hi Roger
I have pushed my new tree. I will be very happy if you can test it with your
NvDIMM setup and report back.
You have two choices over that tree at git://git.open-osd.org/pmem.git
The Kernel there is based on morton's tree which includes DAX.
+ Plus a set of the pmem patches
+ Plus a few patches to mm + x86
You can run with this Kernel, or if you want to compile the pmem.c
driver out-of-tree against your Kernel. You can do that, all you
need is to set BLK_DEV_PMEM_IGNORE_REQUEST_MEM_RET=y.
Thanks
Boaz
2166
days inactive
2171
days old
10 comments
5 participants
participants (5)
-
Boaz Harrosh -
Jiang, Dave -
Roger C. Pao -
Roger C. Pao (Enmotus) -
Ross Zwisler