[SPDK] "Copy failed" errors with SPDK IOAT sample; SPDK IOAT performance

Rami Rosen ramirose at gmail.com
Fri Oct 5 13:58:43 PDT 2018


Hi all,
just a gentle reminder - does anybody have an idea about any of these
questions ?

Regards,
Rami

On Tue, Oct 2, 2018 at 8:03 PM Rami Rosen <ramirose at gmail.com> wrote:

> Hello, sdpk-dev,
>
> I have a question about IOAT SPDK example.
> When I am running the IOAT example (examples/ioat/verify) on Skylake host
> without any code changes for a single second, I get "copy failed" errors,
> over several tries. Also when I am running it for a full hour, there are
> no "copy failed" errors.
> For example:
> ...
> Queue depth:    32
> lcore = 0, copy success = 9132, copy failed = 0, fill success = 9132, fill
> failed = 0
> lcore = 1, copy success = 9435, copy failed = 0, fill success = 9435, fill
> failed = 0
> lcore = 2, copy success = 9128, copy failed = 0, fill success = 9129, fill
> failed = 0
> lcore = 3, copy success = 9259, copy failed = 0, fill success = 9259, fill
> failed = 0
> ...
>
> When I am trying to set a fixed size of 512 for the buffer and run the
> test for a
> single second, I **consistently** get several errors of "copy failed", and
> this
> occurs consistently.
>
> The only change I did in the code is setting the ioat->len to be of a
> fixed
> size of 512, as follows:
>
> diff --git a/examples/ioat/verify/verify.c b/examples/ioat/verify/verify.c
> index c344ba82..10628f51 100644
> --- a/examples/ioat/verify/verify.c
> +++ b/examples/ioat/verify/verify.c
> @@ -152,7 +152,9 @@ static void prepare_ioat_task(struct thread_entry
> *thread_entry, struct ioat_tas
>                 ioat_task->src = (void *)((uintptr_t)g_src + src_offset);
>                 ioat_task->dst = (void *)((uintptr_t)ioat_task->buffer +
> dst_offset);
>         }
> -       ioat_task->len = len;
> +       /* Test: use fixed len of 512 */
> +       //ioat_task->len = len;
> +       ioat_task->len = 512;
>         ioat_task->thread_entry = thread_entry;
> For example:
> ...
> lcore = 0, copy success = 13865, copy failed = 7, fill success = 13873,
> fill failed = 0
> lcore = 1, copy success = 14336, copy failed = 9, fill success = 14346,
> fill failed = 0
> lcore = 2, copy success = 14564, copy failed = 11, fill success = 14576,
> fill failed = 0
> lcore = 3, copy success = 14556, copy failed = 11, fill success = 14568,
> fill failed = 0
> ...
> Have you ever encountered such an issue ?
> Any ideas what can be the reason for it ?
>
>
> Running for an hour the IOAT example without any code changes again gives
> no errors:
>
> Run time:       3600 seconds
> Core mask:      0xf
> Queue depth:    32
> lcore = 0, copy success = 32604920, copy failed = 0, fill success =
> 32604920, fill failed = 0
> lcore = 1, copy success = 33510362, copy failed = 0, fill success =
> 33510362, fill failed = 0
> lcore = 2, copy success = 33471071, copy failed = 0, fill success =
> 33471071, fill failed = 0
> lcore = 3, copy success = 33443397, copy failed = 0, fill success =
> 33443397, fill failed = 0
>
> Another question is: unless I miss something, it seems that the amount of
> packets per second are very low when running the original IOAT example code
> (no code changes); it is about 13,000 copies per core + 13,000 fill copies.
> This is way less than when running a simple userspace app with memcpy which
> does something parallel. Is there any benefits for using SPDK IOAT with
> this performance ? Is this expected ? What is the use case
> for using SPDK with such performance ?
>
> Regards,
> Rami Rosen
>
>

-- 
regards,
Rami Rosen


More information about the SPDK mailing list