summaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2022-02-11Change all syscall function name prefix to __sysarchAmmar Faizi
Instead of using uring_mmap, uring_close, uring_madvise, etc. Let's use __sys_mmap, __sys_close, __sys_madvise, etc. That looks better convention for syscall function name like what we do with __sys_io_uring* functions. Cc: Nugra <richiisei@gmail.com> Signed-off-by: Ammar Faizi <ammarfaizi2@gnuweeb.org> Link: https://lore.kernel.org/r/20220211155753.143698-5-ammarfaizi2@gnuweeb.org Signed-off-by: Jens Axboe <axboe@kernel.dk>
2022-02-11lib.h: Split off lib header for arch specific and genericAmmar Faizi
1) Clean up #ifdef/#endif for get_page_size(). 2) Always use arch specific code for x86-64 to reduce libc usage. 3) For other archs, we will use src/arch/generic/lib.h header that contains libc wrapper. At this point, on x86-64, we only use libc for memset(), malloc() and free(). Cc: Nugra <richiisei@gmail.com> Signed-off-by: Ammar Faizi <ammarfaizi2@gnuweeb.org> Link: https://lore.kernel.org/r/20220211155753.143698-4-ammarfaizi2@gnuweeb.org Signed-off-by: Jens Axboe <axboe@kernel.dk>
2022-02-11arch/x86, syscall: Refactor arch specific and generic syscall wrappersAlviro Iskandar Setiawan
In the previous patch, we create a file src/arch/generic/syscall.h. Let's use it. There are 3 things in this patch (a, b, c): a) Remove all ____sys* and uring_* functions from src/syscall.h. We will define them in the src/arch/* files, so we can avoid many #ifdef/#endif. b) Rename all __arch_impl_* functions in src/arch/x86/syscall.h with ____sys* and uring_* to support point (1). c) Always use arch specific code for x86-64 syscalls, even with CONFIG_NOLIBC disabled. For other archs, currently, will still use the libc wrappers (we provided it in src/arch/generic*). Major changes happen in point (c). We will always use inline assembly for invoking syscall for x86-64. Reasoning: 1. It reduces function calls. ------------------------------ If we use libc, we need to call syscall(2) function and deal with a global state via `errno` macro (`errno` macro will expand to a function call too). If we use inline Assembly, we eliminate many functions calls, we don't need to use `errno` or any global state anymore as it will just directly return error code that we can check with a simple comparison. 2. Allow the compiler to reuse caller clobbered registers. ----------------------------------------------------------- By the rule of System V ABI x86-64, a function call clobbers %rax, %rdi, %rsi, %rdx, %rcx, %r8, %r9, %r10 and %r11. On Linux, syscall only clobbers %rax, %rcx and %r11. But since libc syscall(2) wrapper is a function call, the compiler will always miss the opportunity to reuse those clobbered registers. That means it has to preserve the life values on the stack if they happen to be in the clobbered registers (that's also extra memory access). By inlining the syscall instruction, the compiler has an opportunity to reuse all registers after invoking syscall, except %rax, %rcx and %r11. 3. Smaller binary size. ------------------------ Point (1) and (2) will also reduce the data movement, hence smaller Assembly code, smaller binary size. 4. Reduce %rip round trip to libc.so. -------------------------------------- Call to a libc function will make the %rip jump to libc.so memory area. This can have extra overhead and extra icache misses in some scenario. If we inline the syscall instruction, this overhead can be removed. No functional change should be visible to user. At this point, we may stil use libc for malloc(), free() and memset(), so CONFIG_NOLIBC is still meaningful after these changes. Cc: Nugra <richiisei@gmail.com> Signed-off-by: Alviro Iskandar Setiawan <alviro.iskandar@gnuweeb.org> Co-authored-by: Ammar Faizi <ammarfaizi2@gnuweeb.org> Signed-off-by: Ammar Faizi <ammarfaizi2@gnuweeb.org> Link: https://lore.kernel.org/r/20220211155753.143698-3-ammarfaizi2@gnuweeb.org Signed-off-by: Jens Axboe <axboe@kernel.dk>
2022-02-11arch/generic: Create arch generic syscall wrappersAlviro Iskandar Setiawan
This is a preparation for refactoring the syscall wrappers. This creates a new file src/arch/generic/syscall.h. This file contains libc syscall wrappers for architectures that don't have arch specific code. In the next patches, we will include this file from src/syscall.h. It aims to reduce the usage of #ifdef/#endif that occurs in every function in src/syscall.h file. Also, it will make the arch specific code structure cleaner and easier to manage. Cc: Nugra <richiisei@gmail.com> Signed-off-by: Alviro Iskandar Setiawan <alviro.iskandar@gnuweeb.org> Co-authored-by: Ammar Faizi <ammarfaizi2@gnuweeb.org> Signed-off-by: Ammar Faizi <ammarfaizi2@gnuweeb.org> Link: https://lore.kernel.org/r/20220211155753.143698-2-ammarfaizi2@gnuweeb.org Signed-off-by: Jens Axboe <axboe@kernel.dk>
2022-02-10test/read-write: update EFBIG to work with 256M filesJens Axboe
A previous commit changed the file size from 128M to 256M, and this caused the EFBIG test case to be buggy. Update it so it works again. Fixes: 42df1f30d8f6 ("test/read-write: work on 4kb blocksize devices") Signed-off-by: Jens Axboe <axboe@kernel.dk>
2022-02-09test/read-write: work on 4kb blocksize devicesJens Axboe
For some tests, we chop the iovec in two. Since the iovec is 4k in size, this fails on devices that are actually using 4k block sizes. Just bump the iovec size to 8k, then it'll work fine on 4k devices. Signed-off-by: Jens Axboe <axboe@kernel.dk>
2022-02-09test/double-poll-crash: only run on x86/x86-64Jens Axboe
Various syzbot oddities makes this x86 specific, so just disable the run on other archs. Also kill useless mmap() while in there. Signed-off-by: Jens Axboe <axboe@kernel.dk>
2022-02-04test/eventfd-reg.c: add basic test case to exercise reg+unregJens Axboe
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2022-02-03man/io_uring_enter.2: correct kernel version for IORING_OP_ACCEPTJens Axboe
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2022-01-28Merge branch 'fix-statx' of https://github.com/JonKohler/liburingJens Axboe
* 'fix-statx' of https://github.com/JonKohler/liburing: fix statx configure and build on !CONFIG_HAVE_STATX toolchains
2022-01-28fix statx configure and build on !CONFIG_HAVE_STATX toolchainsJon Kohler
Update configure and test/Makefile, such that if !CONFIG_HAVE_STATX will not generate statx.c test. Without this patch, liburing fails to compile on an older toolchain that does not have statx available. Fixes: #516 Signed-off-by: Jon Kohler <jon@nutanix.com>
2022-01-28Merge branch 'master' of https://github.com/DavidMusk93/liburingJens Axboe
* 'master' of https://github.com/DavidMusk93/liburing: examples/link-cp.c: fix the last file-block retry may overflow
2022-01-28examples/link-cp.c: fix the last file-block retry may overflowMingqiang Sun
Signed-off-by: Mingqiang Sun <mingqiang.sun@esgyn.cn>
2022-01-26man page typosJens Axboe
Fix a typo, and improve some of the documentation as well. Signed-off-by: Jens Axboe <axboe@kernel.dk>
2022-01-26liburing: add man pages for helper functions.Stefan Roesch
This adds man pages for the following functions: - io_uring_cq_advance - io_uring_cq_ready - io_uring_free_probe - io_uring_get_probe - io_uring_opcode_supported - io_uring_sq_ready - io_uring_sq_space_left - io_uring_sqe_set_flags - io_uring_sqring_wait Signed-off-by: Stefan Roesch <shr@fb.com> Signed-off-by: Jens Axboe <axboe@kernel.dk>
2022-01-23nolibc: Don't use `malloc()` and `free()` as the function nameAmmar Faizi
Miyasaki reports that liburing with CONFIG_NOLIBC breaks apps that use libc. The first spotted issue was a realloc() call that results in an invalid pointer, and then the program exits with SIGABRT. The cause is liburing nolibc overrides malloc() and free() from the libc (especially when we statically link the "liburing.a" to the apps that use libc). The malloc() and free() from liburing nolibc use hand-coded Assembly to invoke mmap() and munmap() syscall directly. That means any allocation from malloc() is not a valid object with respect to the libc, and free() will also break the app if we use it to free a pointer from a libc function (e.g., strdup()). liburing nolibc should not break any libc app. This renames the malloc() and free() to __uring_malloc() and __uring_free(). Also, add new inline functions to wrap the malloc() and free(), they are uring_malloc() and uring_free(). These wrappers will call the appropriate functions depending on CONFIG_NOLIBC. Fixes: f48ee3168cdc325233825603269f304d348d323c ("Add nolibc build support") Cc: Tea Inside Mailing List <timl@vger.teainside.org> Reported-by: Miyasaki Kohaku <kohaku.mski@gmail.com> Tested-by: Miyasaki Kohaku <kohaku.mski@gmail.com> Co-authored-by: Alviro Iskandar Setiawan <alviro.iskandar@gmail.com> Signed-off-by: Alviro Iskandar Setiawan <alviro.iskandar@gmail.com> Signed-off-by: Ammar Faizi <ammarfaizi2@gnuweeb.org> Link: https://lore.kernel.org/r/20220123074230.3353274-1-ammarfaizi2@gnuweeb.org Signed-off-by: Jens Axboe <axboe@kernel.dk>
2022-01-19man/io_uring_enter.2: make it clear that chains terminate at submitJens Axboe
The man page does state that: "This flag has no effect on previous SQE submissions [...]" but add some more verbiage to make it absolutely clear that chains cannot persist across submission boundaries. Link: https://github.com/axboe/liburing/issues/510 Signed-off-by: Jens Axboe <axboe@kernel.dk>
2022-01-14Add pollfree test caseJens Axboe
This is the syzbot test case, testing if POLLFREE is handled correctly. Signed-off-by: Jens Axboe <axboe@kernel.dk>
2022-01-09test/socket-rw-offset: Fix UB, accessing dead objectAmmar Faizi
Dereference to a local variable that has been out of its scope is undefined behavior, it may contain garbage or the compiler may reuse it for other local variables. Fix this by moving the struct iov variable declarations so their lifetime is extended. Cc: Jens Axboe <axboe@kernel.dk> Cc: Hrvoje Zeba <zeba.hrvoje@gmail.com> Fixes: 03be3e4fbddd491ef0426b6f9c9085a168acc1c4 ("Add test case for socket read with offset == -1") Signed-off-by: Ammar Faizi <ammarfaizi2@gnuweeb.org> Link: https://lore.kernel.org/r/20220107130218.1238910-4-ammarfaizi2@gnuweeb.org Signed-off-by: Jens Axboe <axboe@kernel.dk>
2022-01-09test/socket-rw: Fix UB, accessing dead objectAmmar Faizi
Dereference to a local variable that has been out of its scope is undefined behavior, it may contain garbage or the compiler may reuse it for other local variables. Fix this by moving the struct iov variable declarations so their lifetime is extended. Cc: Jens Axboe <axboe@kernel.dk> Cc: Hrvoje Zeba <zeba.hrvoje@gmail.com> Fixes: 79ba71a4881fb1cd300520553d7285b3c5ee1293 ("Add deadlock socket read/write test case") Signed-off-by: Ammar Faizi <ammarfaizi2@gnuweeb.org> Link: https://lore.kernel.org/r/20220107130218.1238910-3-ammarfaizi2@gnuweeb.org Signed-off-by: Jens Axboe <axboe@kernel.dk>
2022-01-09test/socket-rw-eagain: Fix UB, accessing dead objectAmmar Faizi
Dereference to a local variable that has been out of its scope is undefined behavior, it may contain garbage or the compiler may reuse it for other local variables. Fix this by moving the struct iov variable declarations so their lifetime is extended. Cc: Jens Axboe <axboe@kernel.dk> Fixes: 76e3b7921fee98a5627cd270628b6a5160d3857d ("Add nonblock empty socket read test") Signed-off-by: Ammar Faizi <ammarfaizi2@gnuweeb.org> Link: https://lore.kernel.org/r/20220107130218.1238910-2-ammarfaizi2@gnuweeb.org Signed-off-by: Jens Axboe <axboe@kernel.dk>
2022-01-05Remove getdents supportJens Axboe
There are still discussions about the API, and hence the feature will miss the 5.17 kernel release. As the API is likely to change, remove the getdents support for now and we can re-add it when it's finalized. Signed-off-by: Jens Axboe <axboe@kernel.dk>
2021-12-23test/socket-rw-offset: skip for kernels without cur position rwJens Axboe
This will fail on kernels that don't have IORING_FEAT_RW_CUR_POS, so just skip the test if we are running on one of those. Signed-off-by: Jens Axboe <axboe@kernel.dk>
2021-12-23Add test case for socket read with offset == -1Jens Axboe
This should just get ignored by the kernel, so check that it is. This is fixed by kernel commit: commit 7b9762a5e8837b92a027d58d396a9d27f6440c36 Author: Jens Axboe <axboe@kernel.dk> Date: Wed Dec 22 20:26:56 2021 -0700 io_uring: zero iocb->ki_pos for stream file types Signed-off-by: Jens Axboe <axboe@kernel.dk>
2021-12-23test: remove dead codeJens Axboe
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2021-12-23Merge branch 'man-page-offset-clarification' of ↵Jens Axboe
https://github.com/ioquatix/liburing * 'man-page-offset-clarification' of https://github.com/ioquatix/liburing: Clarify behaviour of offset in `prep_read`/`prep_write`.
2021-12-23Merge branch 'issue503' of https://github.com/aprilweet/liburingJens Axboe
* 'issue503' of https://github.com/aprilweet/liburing: examples/io_uring_cp: fix bug while re-queuing
2021-12-23Clarify behaviour of offset in `prep_read`/`prep_write`.Samuel Williams
Signed-off-by: Samuel Williams <samuel.williams@oriontransfer.co.nz>
2021-12-23examples/io_uring_cp: fix bug while re-queuingmonte
Add io_uring_submit after queue_prepped while re-queuing sqe. Signed-off-by: monte <wm.april@gmail.com>
2021-12-22Don't clear sqe->user_data as part of command prepJens Axboe
Reasoning: 1) If the app doesn't set it, it should not be reading it. 2) I've gotten more than one report on someone setting user_data before calling io_uring_prep_cmdtype() and it's a bit confusing. I think we'll be fine not clearing this, and it may actually help some cases that end up being a bit confusing, particular for people new to the interface. Signed-off-by: Jens Axboe <axboe@kernel.dk>
2021-12-22test/getdents: ignore run if argument specifiedJens Axboe
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2021-12-22test/getdents: don't fail on kernels not supporting the featureJens Axboe
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2021-12-22test/getdents: style fixupsJens Axboe
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2021-12-22liburing: Add man page for io_uring_prep_getdents callStefan Roesch
Adds the man page for the io_uring_prep_getdents call. Signed-off-by: Stefan Roesch <shr@fb.com> Link: https://lore.kernel.org/r/20211123180753.1598611-5-shr@fb.com Signed-off-by: Jens Axboe <axboe@kernel.dk>
2021-12-22liburing: Add test program for getdents callStefan Roesch
Add test program for getdents64 call. Signed-off-by: Stefan Roesch <shr@fb.com> Link: https://lore.kernel.org/r/20211123180753.1598611-4-shr@fb.com Signed-off-by: Jens Axboe <axboe@kernel.dk>
2021-12-22liburing: add prepare function for getdents64Stefan Roesch
Adds the io_uring_prep_getdents() function to setup a getdents64 call. Signed-off-by: Stefan Roesch <shr@fb.com> Link: https://lore.kernel.org/r/20211123180753.1598611-3-shr@fb.com Signed-off-by: Jens Axboe <axboe@kernel.dk>
2021-12-22liburing: update io_uring.h header fileStefan Roesch
Update io_uring.h header file with new op code for getdents64. Signed-off-by: Stefan Roesch <shr@fb.com> Link: https://lore.kernel.org/r/20211123180753.1598611-2-shr@fb.com Signed-off-by: Jens Axboe <axboe@kernel.dk>
2021-12-10test/exit-no-cleanup: skip file based runJens Axboe
This test doesn't do anything with the file passed in as an argument, hence it should just skip if that's the case. While in there, convert from MSDOS file format. Signed-off-by: Jens Axboe <axboe@kernel.dk>
2021-12-10test/skip-cqe: skip file based runJens Axboe
This test doesn't do anything with the file passed in as an argument, hence it should just skip if that's the case. Signed-off-by: Jens Axboe <axboe@kernel.dk>
2021-12-07man/io_uring_enter.2: notes about cqe-skip & drain interoperabilityPavel Begunkov
IOSQE_CQE_SKIP_SUCCESS can't be used together with draining in a single ring, add a paragraph explaining what are the restrictions. Signed-off-by: Pavel Begunkov <asml.silence@gmail.com> Link: https://lore.kernel.org/r/8c81bf9b01a54d1214bb65678c2ff1362a9f9328.1638364791.git.asml.silence@gmail.com [axboe: tweak the wording] Signed-off-by: Jens Axboe <axboe@kernel.dk>
2021-12-04Don't truncate addr fields to 32-bit on 32-bitJens Axboe
For the commands where we use it as a lookup key, then we don't want to rob 32-bit applications from the potential of using the full 64-bit type. This includes commands like: poll_remove, poll_update, prep_cancel Also provide 64-bit u64 types of the sqe set data and the cqe get data helpers, along with a define that allows applications to check for the presence of it. NOTE: this may trigger compile warnings in applications that currently use these helpers, which is also why a few test cases had to get adapted. The fixup is trivial, and the above define can help applications check if one or the other is the current one. Link: https://github.com/axboe/liburing/issues/490 Reported-by: Rémi Bernon <rbernon@codeweavers.com> Signed-off-by: Jens Axboe <axboe@kernel.dk>
2021-12-04liburing.h: don't truncate timeout_update/remove user data valueJens Axboe
If we're running on 32-bit, with the cast that io_uring_prep_rw() does, we lose the top 32 bits of the user_data for the timeout remove and update commands. If the application also uses a full 64-bit value for the original timeout user_data, then we cannot find the timeout to update. Link: https://github.com/axboe/liburing/issues/490 Signed-off-by: Rémi Bernon <rbernon@codeweavers.com> Signed-off-by: Jens Axboe <axboe@kernel.dk>
2021-12-02Add test case for hang condition on exit without cleanupJens Axboe
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2021-11-27man/io_uring_enter.2: document IOSQE_CQE_SKIP_SUCCESSPavel Begunkov
Add a section about IOSQE_CQE_SKIP_SUCCESS describing the behaviour and use cases. Signed-off-by: Pavel Begunkov <asml.silence@gmail.com> Link: https://lore.kernel.org/r/381237725f0f09a2668ea7f38b804d5733595b1f.1637977800.git.asml.silence@gmail.com [axboe: some rewording] Signed-off-by: Jens Axboe <axboe@kernel.dk>
2021-11-27test/skip-cqe: use correct index for write side of pipeJens Axboe
clang correctly complains about using out-of-array: skip-cqe.c:19:27: error: array index 2 is past the end of the array (which contains 2 elements) [-Werror,-Warray-bounds] io_uring_prep_write(sqe, fds[2], NULL, 100, 0); ^ ~ Fixes: 78f733007fd0 ("tests: add tests for IOSQE_CQE_SKIP_SUCCESS") Signed-off-by: Jens Axboe <axboe@kernel.dk>
2021-11-27test: skip-cqe with hardlinksPavel Begunkov
Test IOSQE_CQE_SKIP_SUCCESS together with IOSQE_IO_HARDLINK. Signed-off-by: Pavel Begunkov <asml.silence@gmail.com> Link: https://lore.kernel.org/r/fe5333e8832e6bacb5c9ca9ec8d0004a6fd2646d.1637958333.git.asml.silence@gmail.com Signed-off-by: Jens Axboe <axboe@kernel.dk>
2021-11-27test: poll cancellation with offset timeoutsPavel Begunkov
Test for a recent locking problem during poll cancellation with offset timeouts queued. Signed-off-by: Pavel Begunkov <asml.silence@gmail.com> Link: https://lore.kernel.org/r/21ed5cd19ba6cff23714dddb07358360a1ac6f91.1637955206.git.asml.silence@gmail.com Signed-off-by: Jens Axboe <axboe@kernel.dk>
2021-11-24tests: add tests for IOSQE_CQE_SKIP_SUCCESSPavel Begunkov
Signed-off-by: Pavel Begunkov <asml.silence@gmail.com> Link: https://lore.kernel.org/r/64a28950287c97b4d5bd11bf288ab6fab5d45b79.1637786880.git.asml.silence@gmail.com Signed-off-by: Jens Axboe <axboe@kernel.dk>
2021-11-24io_uring.h: update to reflect cqe-skip featurePavel Begunkov
Add IOSQE_CQE_SKIP_SUCCESS and friends Signed-off-by: Pavel Begunkov <asml.silence@gmail.com> Link: https://lore.kernel.org/r/665ef6b2f9440105f5826a63be5ac014b179ccbf.1637786880.git.asml.silence@gmail.com Signed-off-by: Jens Axboe <axboe@kernel.dk>
2021-11-18Rename man pageJens Axboe
This one refers to io_uring_cqe_get_data(), so rename it as such. Signed-off-by: Jens Axboe <axboe@kernel.dk>