12:06 a.m.
On Fri, Dec 04, 2015 at 07:15:55PM +0100, Phil Sutter wrote:
> Only one should really do this, while others are waiting.
Sure, that was my previous understanding of how this thing works.
Yes that's clearly how it should be. Unfortunately while adding
the locking to do this, I found out that you can't actually call
__vmalloc with BH disabled so this is a no-go.
Unless we can make __vmalloc work with BH disabled, I guess we'll
have to go back to multi-level lookups unless someone has a better
suggestion.
Cheers,
--
Email: Herbert Xu <herbert(a)gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
8:35 a.m.
New subject: rhashtable: Use __vmalloc with GFP_ATOMIC for table allocation
On 12/05/15 at 03:06pm, Herbert Xu wrote:
On Fri, Dec 04, 2015 at 07:15:55PM +0100, Phil Sutter wrote:
>
> > Only one should really do this, while others are waiting.
>
> Sure, that was my previous understanding of how this thing works.
Yes that's clearly how it should be. Unfortunately while adding
the locking to do this, I found out that you can't actually call
__vmalloc with BH disabled so this is a no-go.
Unless we can make __vmalloc work with BH disabled, I guess we'll
have to go back to multi-level lookups unless someone has a better
suggestion.
Thanks for fixing the race.
As for the remaining problem, I think we'll have to find a way to
serve a hard pounding user if we want to convert TCP hashtables
later on.
Did you look into what __vmalloc prevents to work with BH disabled?
12:29 p.m.
New subject: rhashtable: Use __vmalloc with GFP_ATOMIC for table allocation
From: Thomas Graf <tgraf(a)suug.ch>
Date: Mon, 7 Dec 2015 16:35:24 +0100
Did you look into what __vmalloc prevents to work with BH disabled?
You can't issue the cross-cpu TLB flushes from atomic contexts.
It's the kernel page table updates that create the restriction.
7:18 p.m.
New subject: rhashtable: Use __vmalloc with GFP_ATOMIC for table allocation
On 12/05/15 at 03:06pm, Herbert Xu wrote:
Unless we can make __vmalloc work with BH disabled, I guess
we'll
have to go back to multi-level lookups unless someone has a better
suggestion.
Assuming that we only encounter this scenario with very large
table sizes, it might be OK to assume that deferring the actual
resize via the worker thread while continuing to insert above
100% utilization in atomic context is safe.
On 12/07/15 at 02:29pm, David Miller wrote:
You can't issue the cross-cpu TLB flushes from atomic contexts.
It's the kernel page table updates that create the restriction.
7:24 p.m.
New subject: rhashtable: Use __vmalloc with GFP_ATOMIC for table allocation
On Wed, Dec 09, 2015 at 03:18:26AM +0100, Thomas Graf wrote:
Assuming that we only encounter this scenario with very large
table sizes, it might be OK to assume that deferring the actual
resize via the worker thread while continuing to insert above
100% utilization in atomic context is safe.
As test_rhashtable has demonstrated already this approach doesn't
work. There is nothing in the kernel that will ensure that the
worker thread gets to run at all.
Cheers,
--
Email: Herbert Xu <herbert(a)gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
7:36 p.m.
New subject: rhashtable: Use __vmalloc with GFP_ATOMIC for table allocation
On 12/09/15 at 10:24am, Herbert Xu wrote:
On Wed, Dec 09, 2015 at 03:18:26AM +0100, Thomas Graf wrote:
>
> Assuming that we only encounter this scenario with very large
> table sizes, it might be OK to assume that deferring the actual
> resize via the worker thread while continuing to insert above
> 100% utilization in atomic context is safe.
As test_rhashtable has demonstrated already this approach doesn't
work. There is nothing in the kernel that will ensure that the
worker thread gets to run at all.
If we define work assuming that an insertion in atomic context
should never fail then yes. I'm not sure you can guarantee that
with a segmented table either though. I agree though that the
insertion behaviour is much better defined.
My argument is that if we are in a situation in which a worker
thread is never invoked and we've grown 2x from the original
table size, do we still need entries to be inserted into the
table or can we fail?
Without knowing your exact implementation plans: introducing an
additional reference indirection for every lookup will have a
huge performance penalty as well.
Is your plan to only introduce the master table after an
allocation has failed?
7:38 p.m.
New subject: rhashtable: Use __vmalloc with GFP_ATOMIC for table allocation
On Wed, Dec 09, 2015 at 03:36:32AM +0100, Thomas Graf wrote:
Without knowing your exact implementation plans: introducing an
additional reference indirection for every lookup will have a
huge performance penalty as well.
Is your plan to only introduce the master table after an
allocation has failed?
Right, obviously the extra indirections would only come into play
after a failed allocation. As soon as we can run the worker thread
it'll try to remove the extra indirections by doing vmalloc.
Cheers,
--
Email: Herbert Xu <herbert(a)gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
7:42 p.m.
New subject: rhashtable: Use __vmalloc with GFP_ATOMIC for table allocation
On 12/09/15 at 10:38am, Herbert Xu wrote:
On Wed, Dec 09, 2015 at 03:36:32AM +0100, Thomas Graf wrote:
>
> Without knowing your exact implementation plans: introducing an
> additional reference indirection for every lookup will have a
> huge performance penalty as well.
>
> Is your plan to only introduce the master table after an
> allocation has failed?
Right, obviously the extra indirections would only come into play
after a failed allocation. As soon as we can run the worker thread
it'll try to remove the extra indirections by doing vmalloc.
OK, this sounds like a good compromise. The penalty is isolated
for the duration of the atomic burst.
2394
days inactive
2398
days old
7 comments
3 participants
participants (3)
-
David Miller -
Herbert Xu -
Thomas Graf