[SPDK] 回复:Re: spdk can't pass fio_test_if_4_clients_testing_with_4_split_partitions

Wodkowski, PawelX pawelx.wodkowski at intel.com
Tue Jun 27 02:27:17 PDT 2017


You can use isdct tool to change LBA of NVMe
https://downloadcenter.intel.com/product/87278/Intel-SSD-Data-Center-Tool

If this still fail please send us logs from testing tool you use.

PAwel

From: SPDK [mailto:spdk-bounces at lists.01.org] On Behalf Of nixun_992 at sina.com
Sent: Saturday, June 17, 2017 1:16 PM
To: spdk <spdk at lists.01.org>
Subject: [SPDK] 回复:Re: spdk can't pass fio_test_if_4_clients_testing_with_4_split_partitions

Hi, Pawel:

    This time, when i specify the bsrange to 4k-512k, it seems to pass the test.

    I run the ltp test, the test script is ltpstress, i run it in the following:   ./ltpstress.sh -npqS -t 5 -b /dev/sda1 -B ext4
    The result is FAIL, i guess it's also the 4k issue. because it has the direct-io with less than 4k io size.

    And if i want to do the direct-IO with less than 4k size, how do i set it in spdk vhost?  And i also very interested in the source code of spdk, can you give me the detail spdk function routines on what's spdk will do when the size is less than 4k?

Thanks,
Xun

-----------------------------------

Because 1 physical block is min unit you can read/write. If you need 1k direct IO you need to format undelaying NVMe with 512b sector size, then you can do direct IO with N x 512b size. In buffered mode (non-direct) you can do IO size whatever you want. Kernel will handle that for you.

Going back to bsrange: 4k-512k will force fio to produce IO that is N x 4k and this case must work with SPDK vhost.

Pawel

From: SPDK [mailto:spdk-bounces at lists.01.org] On Behalf Of nixun_992 at sina.com<mailto:nixun_992 at sina.com>
Sent: Friday, June 16, 2017 5:28 AM
To: spdk <spdk at lists.01.org<mailto:spdk at lists.01.org>>
Subject: Re: [SPDK] spdk can't pass fio_test_if_4_clients_testing_with_4_split_partitions

Why have such limitation(size>=4k)?   In my opinion, guest kernel should not have any limitation, spdk vhost should take over it.

Thanks,
Xun

---------------------------
Try bsrange from 4k (not 1k). For direct IO you should not send IO that is less than minimum_io_size/hw_sector_size. Also can you send  qemu and vhost launch command. Commit id of working version and not working also will also help us because we don’t know what is “old version”.

If resets are triggered from guest OS you should see some failures on guest dmesg.

Pawel

From: SPDK [mailto:spdk-bounces at lists.01.org] On Behalf Of nixun_992 at sina.com<mailto:nixun_992 at sina.com>
Sent: Tuesday, June 13, 2017 10:48 AM
To: spdk <spdk at lists.01.org<mailto:spdk at lists.01.org>>
Subject: [SPDK] 回复:Re: spdk can't pass fio test if 4 clients testing with 4 split partitions


Hi, Pawel & changpeng:

    No, not for 512 size, i just specify random size of IO to stress the spdk vhost program. and for /mnt/ssdtest1, i mount the target disk to /dev/sda, and mount /dev/sda1 /mnt.  the ssdtest1 is the test file for fio test.
    my guest os is using centos7u1, the dmesg seems to not have so much problem. The main problem is that the spdk is continuing reset controller, not sure why this happens. and i didn't see it in old version.

Thanks,
Xun

================================


I see bsrange 1k to 512k, is NVMe formatted as 512b block size here?

Which commit you use on this  test?
filename=/mnt/ssdtest1 – is this some FS mounted directory?
Can you send us dmesg from failure?

Paweł

From: SPDK [mailto:spdk-bounces at lists.01.org] On Behalf Of Liu, Changpeng
Sent: Tuesday, June 13, 2017 9:18 AM
To: Storage Performance Development Kit <spdk at lists.01.org<mailto:spdk at lists.01.org>>
Subject: Re: [SPDK] spdk can't pass fio test if 4 clients testing with 4 split partitions

Thanks Xun.

We’ll take a look at the issue.

From the error log message, there seems a SCSI task management command received by SPDK,
the reason why VM sent the task management command is most likely due to timeout for some
commands.


From: SPDK [mailto:spdk-bounces at lists.01.org] On Behalf Of nixun_992 at sina.com<mailto:nixun_992 at sina.com>
Sent: Monday, June 12, 2017 3:40 PM
To: spdk <spdk at lists.01.org<mailto:spdk at lists.01.org>>
Subject: [SPDK] spdk can't pass fio test if 4 clients testing with 4 split partitions

 Hi, All:

   spdk can't pass fio test after 2 hours testing. And it can pass the same test if we use the version before Mar 29

 the error message is the following

nvme_pcie.c:1133:nvme_pcie_qpair_abort_trackers: ***ERROR*** aborting outstanding command
READ sqid:1 cid:134 nsid:1 lba:1310481280 len:128
ABORTED - BY REQUEST (00/07) sqid:1 cid:134 cdw0:0 sqhd:0000 p:0 m:0 dnr:0
nvme_pcie.c:1133:nvme_pcie_qpair_abort_trackers: ***ERROR*** aborting outstanding command
READ sqid:1 cid:202 nsid:1 lba:1310481408 len:256
ABORTED - BY REQUEST (00/07) sqid:1 cid:202 cdw0:0 sqhd:0000 p:0 m:0 dnr:0
nvme_pcie.c:1133:nvme_pcie_qpair_abort_trackers: ***ERROR*** aborting outstanding command
READ sqid:1 cid:151 nsid:1 lba:1310481664 len:256
ABORTED - BY REQUEST (00/07) sqid:1 cid:151 cdw0:0 sqhd:0000 p:0 m:0 dnr:0
nvme_pcie.c:1133:nvme_pcie_qpair_abort_trackers: ***ERROR*** aborting outstanding command
READ sqid:1 cid:243 nsid:1 lba:1312030816 len:96
ABORTED - BY REQUEST (00/07) sqid:1 cid:243 cdw0:0 sqhd:0000 p:0 m:0 dnr:0
resetting controller
resetting controller
nvme_pcie.c:1133:nvme_pcie_qpair_abort_trackers: ***ERROR*** aborting outstanding command
READ sqid:1 cid:253 nsid:1 lba:998926248 len:88
ABORTED - BY REQUEST (00/07) sqid:1 cid:253 cdw0:0 sqhd:0000 p:0 m:0 dnr:0
nvme_pcie.c:1133:nvme_pcie_qpair_abort_trackers: ***ERROR*** aborting outstanding command
READ sqid:1 cid:243 nsid:1 lba:1049582336 len:176
ABORTED - BY REQUEST (00/07) sqid:1 cid:243 cdw0:0 sqhd:0000 p:0 m:0 dnr:0
nvme_pcie.c:1133:nvme_pcie_qpair_abort_trackers: ***ERROR*** aborting outstanding command
READ sqid:1 cid:169 nsid:1 lba:1109679488 len:128
ABORTED - BY REQUEST (00/07) sqid:1 cid:169 cdw0:0 sqhd:0000 p:0 m:0 dnr:0
nvme_pcie.c:1133:nvme_pcie_qpair_abort_trackers: ***ERROR*** aborting outstanding command
READ sqid:1 cid:134 nsid:1 lba:958884728 len:136
ABORTED - BY REQUEST (00/07) sqid:1 cid:134 cdw0:0 sqhd:0000 p:0 m:0 dnr:0
nvme_pcie.c:1133:nvme_pcie_qpair_abort_trackers: ***ERROR*** aborting outstanding command
READ sqid:1 cid:152 nsid:1 lba:1018345728 len:240
ABORTED - BY REQUEST (00/07) sqid:1 cid:152 cdw0:0 sqhd:0000 p:0 m:0 dnr:0
nvme_pcie.c:1133:nvme_pcie_qpair_abort_trackers: ***ERROR*** aborting outstanding command
READ sqid:1 cid:234 nsid:1 lba:898096896 len:8
ABORTED - BY REQUEST (00/07) sqid:1 cid:234 cdw0:0 sqhd:0000 p:0 m:0 dnr:0
nvme_pcie.c:1133:nvme_pcie_qpair_abort_trackers: ***ERROR*** aborting outstanding command
READ sqid:1 cid:130 nsid:1 lba:991125248 len:96
ABORTED - BY REQUEST (00/07) sqid:1 cid:130 cdw0:0 sqhd:0000 p:0 m:0 dnr:0
resetting controller
resetting controller
nvme_pcie.c:1133:nvme_pcie_qpair_abort_trackers: ***ERROR*** aborting outstanding command
READ sqid:1 cid:130 nsid:1 lba:609149952 len:64
ABORTED - BY REQUEST (00/07) sqid:1 cid:130 cdw0:0 sqhd:0000 p:0 m:0 dnr:0


=========================

our vhost conf is the following

# The Split virtual block device slices block devices into multiple smaller bdevs.
[Split]
  # Syntax:
  #   Split <bdev> <count> [<size_in_megabytes>]
  #
  # Split Nvme1n1 into two equally-sized portions, Nvme1n1p0 and Nvme1n1p1
  Split Nvme0n1 4 200000

  # Split Malloc2 into eight 1-megabyte portions, Malloc2p0 ... Malloc2p7,
  # leaving the rest of the device inaccessible
  #Split Malloc2 8 1
[VhostScsi0]
    Dev 0 Nvme0n1p0
[VhostScsi1]
    Dev 0 Nvme0n1p1
[VhostScsi2]
    Dev 0 Nvme0n1p2
[VhostScsi3]
    Dev 0 Nvme0n1p3
fio script is the following:

[global]
filename=/mnt/ssdtest1
size=100G
numjobs=8
iodepth=16
ioengine=libaio
group_reporting
do_verify=1
verify=md5

# direct rand read
[rand-read]
bsrange=1k-512k
#direct=1
rw=randread
runtime=10000
stonewall

# direct seq read
[seq-read]
bsrange=1k-512k
direct=1
rw=read
runtime=10000
stonewall

# direct rand write
[rand-write]
bsrange=1k-512k
direct=1
rw=randwrite
runtime=10000
stonewall

# direct seq write
[seq-write]
bsrange=1k-512k
direct=1
rw=write
runtime=10000
_______________________________________________
SPDK mailing list
SPDK at lists.01.org<mailto:SPDK at lists.01.org>
https://lists.01.org/mailman/listinfo/spdk
_______________________________________________
SPDK mailing list
SPDK at lists.01.org<mailto:SPDK at lists.01.org>
https://lists.01.org/mailman/listinfo/spdk
_______________________________________________
SPDK mailing list
SPDK at lists.01.org<mailto:SPDK at lists.01.org>
https://lists.01.org/mailman/listinfo/spdk
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.01.org/pipermail/spdk/attachments/20170627/1f68c26b/attachment.html>


More information about the SPDK mailing list