page_pool: devmem support
authorMina Almasry <almasrymina@google.com>
Tue, 10 Sep 2024 17:14:49 +0000 (17:14 +0000)
committerJakub Kicinski <kuba@kernel.org>
Thu, 12 Sep 2024 03:44:31 +0000 (20:44 -0700)
commit8ab79ed50cf10f338465c296012500de1081646f
tree5cefb830ff8e266f2423b7ad6e35826c5da4e812
parent28c5c74eeaa0a2aad8b9cd9ede22a4c623f2a7fc
page_pool: devmem support

Convert netmem to be a union of struct page and struct netmem. Overload
the LSB of struct netmem* to indicate that it's a net_iov, otherwise
it's a page.

Currently these entries in struct page are rented by the page_pool and
used exclusively by the net stack:

struct {
unsigned long pp_magic;
struct page_pool *pp;
unsigned long _pp_mapping_pad;
unsigned long dma_addr;
atomic_long_t pp_ref_count;
};

Mirror these (and only these) entries into struct net_iov and implement
netmem helpers that can access these common fields regardless of
whether the underlying type is page or net_iov.

Implement checks for net_iov in netmem helpers which delegate to mm
APIs, to ensure net_iov are never passed to the mm stack.

Signed-off-by: Mina Almasry <almasrymina@google.com>
Reviewed-by: Pavel Begunkov <asml.silence@gmail.com>
Acked-by: Jakub Kicinski <kuba@kernel.org>
Link: https://patch.msgid.link/20240910171458.219195-6-almasrymina@google.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
include/net/netmem.h
include/net/page_pool/helpers.h
include/trace/events/page_pool.h
net/core/devmem.c
net/core/netmem_priv.h [new file with mode: 0644]
net/core/page_pool.c
net/core/page_pool_priv.h
net/core/skbuff.c