summaryrefslogtreecommitdiff
path: root/os
AgeCommit message (Collapse)Author
2020-11-05Revert "Windows: update dobuild.cmd to run the configure/make"Jens Axboe
This reverts commit 38c2f9384db8dbd93f59d965d70ab0d3a53343fa. It's causing issues for CI, revert it for now. Signed-off-by: Jens Axboe <axboe@kernel.dk>
2020-11-04Windows: update dobuild.cmd to run the configure/makeRebecca Cran
Add a new shell script, _domake.sh which runs under Cygwin. This is invoked via dobuild.cmd to run the configure and make steps of the build. In this way, a single command, run from a Developer Command Prompt, can be used to go from a clean workspace to a final installer binary. Signed-off-by: Rebecca Cran <rebecca@bsdio.com> Signed-off-by: Jens Axboe <axboe@kernel.dk>
2020-09-12configure/Makefile: add option to generate pdb symbolsSitsofe Wheeler
Recent versions of Windows clang and LLVM's lld have the ability to build Windows PDB symbols for debugging so add an explicit option to test for support and use it so. Signed-off-by: Sitsofe Wheeler <sitsofe@yahoo.com>
2020-09-12windows: fix DWORD format string complaintsSitsofe Wheeler
Printing DWORD variables causes Windows clang to grumble about types: os/windows/cpu-affinity.c:263:5: error: format specifies type 'int' but the argument has type 'DWORD' (aka 'unsigned long') [-Werror,-Wformat] GetLastError()); ^~~~~~~~~~~~~~ Since a DWORD is defined to be an unsigned 32 bit integer and is declared to be an unsigned long (https://docs.microsoft.com/en-us/openspecs/windows_protocols/ms-dtyp/262627d8-3418-4627-9218-4ffe110850b2 ) just use long as the format specifier. When printing the cpu mask in hex on legacy Windows builds, cast to unsigned long long which should be safe because there won't be more than 64 CPUs with old versions of Windows. (As some format strings are longer than the wrapping width, make an exception and avoid splitting/preserve them to make greping easier) Suggested-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: Sitsofe Wheeler <sitsofe@yahoo.com>
2020-09-12windows: fix wrong format stringsSitsofe Wheeler
Found by Windows clang which mysteriously seemed able to ignore -Wno-format... (As some format strings are longer than the wrapping width, make an exception and avoid splitting/preserve them to make greping easier) Signed-off-by: Sitsofe Wheeler <sitsofe@yahoo.com>
2020-08-29Update the year to 2020 in os/windows/eula.rtfRebecca Cran
Signed-off-by: Rebecca Cran <rebecca@bsdio.com> Signed-off-by: Jens Axboe <axboe@kernel.dk>
2020-08-29Update os/windows/dobuild.cmd to support signing binaries/installerRebecca Cran
Since organizations are increasingly requiring software to be digitally signed, or have heuristics that penalize unsigned software, add support to do build.cmd to sign both the .exe binaries and the .msi installer. Signed-off-by: Rebecca Cran <rebecca@bsdio.com> Signed-off-by: Jens Axboe <axboe@kernel.dk>
2020-08-22engines/io_uring: use non-vectored read/write if availableJens Axboe
There's the nonvectored option to set this, by default to it if it wasn't set explicitly *and* we have it available. We issue a probe to detect this. Signed-off-by: Jens Axboe <axboe@kernel.dk>
2020-07-04os/windows/posix.c: Use INVALID_SOCKET instead of < 0Bart Van Assche
From https://docs.microsoft.com/en-us/windows/win32/winsock/socket-data-type-2: "Because the SOCKET type is unsigned, compiling existing source code from, for example, a UNIX environment may lead to compiler warnings about signed/unsigned data type mismatches. This means, for example, that checking for errors when the socket and accept functions return should not be done by comparing the return value with -1, or seeing if the value is negative (both common and legal approaches in UNIX). Instead, an application should use the manifest constant INVALID_SOCKET as defined in the Winsock2.h header file." This patch fixes the following build error for the latest version of the MinGW64 cross-compiler on Cygwin: os/windows/posix.c:886:17: error: comparison of unsigned expression < 0 is always false Signed-off-by: Bart Van Assche <bvanassche@acm.org>
2020-07-04os/windows/posix.c: Strip trailing whitespaceBart Van Assche
Signed-off-by: Bart Van Assche <bvanassche@acm.org>
2020-07-03configure: new --dynamic-libengines build optionYigal Korman
When enabled, some of the more dependency-heavy internal engines are converted to "plugin" engines, i.e. they are built into separate object files and are loaded by fio on demand. This helps downstream distros package these engines separately and not force a long list of package dependencies from the base fio package. Signed-off-by: Yigal Korman <ykorman@gmail.com> Signed-off-by: Jens Axboe <axboe@kernel.dk>
2020-05-31engines: pvsync2 libaio io_uring: add support for RWF_NOWAITKonstantin Khlebnikov
Add bool option "nowait" into engines which could support this feature. Add test for libaio >= 0.3.111 into configure script to be compatible with prior versions where field iocb.aio_rw_flags was declared as "__pad2". By default if a request cannot be executed immediately (e.g. resource starvation, waiting on locks) it is queued and the initiating process will be blocked until the required resource becomes free. This option sets the RWF_NOWAIT flag (supported from the 4.14 Linux kernel) and the call will return instantly with EAGAIN or a partial result rather than waiting. It is useful to also use ignore_error=EAGAIN when using this option. Note: glibc 2.27, 2.28 have a bug in syscall wrappers preadv2, pwritev2. They return EOPNOTSUP instead of EAGAIN. For cached I/O, using this option usually means a request operates only with cached data. Currently the RWF_NOWAIT flag does not supported for cached write. For direct I/O, requests will only succeed if cache invalidation isn't required, file blocks are fully allocated and the disk request could be issued immediately. Signed-off-by: Konstantin Khlebnikov <khlebnikov@yandex-team.ru> Link: https://lwn.net/Articles/724631/
2020-03-01filesetup: fix win raw disk access and improve dir creation failure msgSitsofe Wheeler
The commit df18600fd06258b96ae6f6b530ecdff541c2a82d ("filesetup: fix directory creation issues") broke Windows raw/physical disk access because Windows doesn't consider a path that only consists of a namespace (such as the device namespace "\\.\" - see https://docs.microsoft.com/en-gb/windows/win32/fileio/naming-a-file?redirectedfrom=MSDN#namespaces for information on Windows' namespaces) to exist as a directory(!). Workaround the issue for raw devices by explicitly considering the device namespace subcomponent to always be valid. Further, intermediate path components in UNC paths or paths starting with a namespace might also not "exist" so introduce backwards scanning for the longest pre-existing directory to sidestep this. The function doing this is made available for non-windows platforms so a similar code path is used everywhere. Tests done: Windows: > ./fio.exe --name=dtest --thread --size=16k --rw=write ` --filename 'fio.tmp' > ./fio.exe --name=dtest --thread --size=16k --rw=write ` --filename '\\?\C\:\Windows\Temp\fio\fio.tmp' > Clear-Disk 1 -RemoveData -Confirm:$false # Destroys partition data! > ./fio.exe --name=dtest --thread --size=16k --rw=write ` --filename '\\.\PhysicalDrive1' > ./fio.exe --name=dtest --thread --size=16k --rw=write ` --filename '\fio.tmp' > ./fio.exe --name=dtest --thread --size=16k --rw=write ` --filename '\\LOCALHOST\Users\User\fio\fio.tmp' macOS: $ rm -rf /tmp/fio $ ./fio --name=dtest --size=16k --filename /tmp/fio/fio.tmp \ --rw=write Finally, change the directory creation error message to give a human error message rather than just an errno. Fixes: https://github.com/axboe/fio/issues/916 Signed-off-by: Sitsofe Wheeler <sitsofe@yahoo.com>
2020-01-28Use swap16/32/64 on OpenBSD.Kurt Miller
2020-01-06Windows >= 7: Make fio_getaffinity() error reporting more detailedBart Van Assche
Report different error messages for GetProcessGroupAffinity() failures and also if a process is associated with multiple process groups. Signed-off-by: Bart Van Assche <bvanassche@acm.org>
2020-01-06Windows: Uninline CPU affinity functionsBart Van Assche
None of the CPU affinity functions is in the hot path. Reduce the build time by uninlining these functions. Signed-off-by: Bart Van Assche <bvanassche@acm.org>
2020-01-06Windows: Use snprintf() instead of StringCch*()Bart Van Assche
Use ANSI C functions instead of Windows-specific functions. Signed-off-by: Bart Van Assche <bvanassche@acm.org>
2020-01-06Windows: Remove more unused OS dependent codeBart Van Assche
Since commit 46bfd4e5170e ("Remove old OS dependent (unused) random code") rand_r() is no longer used. Hence remove its definition. Signed-off-by: Bart Van Assche <bvanassche@acm.org>
2020-01-06Change off64_t into uint64_tBart Van Assche
off64_t is Linux specific while uint64_t is defined by a C standard. This change makes fio easier to port. Signed-off-by: Bart Van Assche <bvanassche@acm.org>
2019-12-23io_uring: add option for non-vectored read/write commandsJens Axboe
Also syncs io_uring.h with the kernel 5.6 version. Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-12-18filesetup: fix directory creation issuesSitsofe Wheeler
- Fix double directory prefix concatenation that was happening after 95af8dd57a7a9a564f99492bb1e04bb6d13b95b1 ("filesetup: don't call create_work_dirs() for ioengine with FIO_DISKLESSIO") refactored some code - Stop always using '/' and switch to using FIO_OS_PATH_SEPARATOR when concatenating the directory value to the filename in set_name_idx() - Change the permissions directories are created with to have the execute bit so they can be entered(!) - Fix bug where filenames that were absolute unix paths would prevent the creation of intermediate directories - Remove the configure mkdir test and introduce fio_mkdir to hide mkdir platform differences - Make Windows mkdir emulation more robust against multiple slashes after the drive letter and when dealing dealing with with paths containing drive letters in general This commit makes the following work on Windows: ./fio --thread --name=doubleslashtest \ --filename='C\:\\Windows\Temp\fio.tmp' --size=4k --bs=4k --rw=read The next example now works when D:\1\ exists but D:\1\2\ doesn't: ./fio --thread --name=mkdirtest --directory='D\:\1' \ --filename='2\fio.tmp' --size=4k --bs=4k --rw=read The following now works on macOS/Linux when /tmp/1/ exists but /tmp/1/2/ doesn't: ./fio --name=mkdirtest --directory='/tmp/1' \ --filename='2/fio.tmp' --size=4k --bs=4k --rw=read Fixes https://github.com/axboe/fio/issues/784 , https://github.com/axboe/fio/issues/864 and https://github.com/axboe/fio/issues/870 Tested-by: Astolfo Rueda <astolfor@microsoft.com> Signed-off-by: Sitsofe Wheeler <sitsofe@yahoo.com>
2019-12-06Renumber RWF_UNCACHEDJens Axboe
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-12-06io_uring: add support for RWF_UNCACHEDJens Axboe
If this is set, and the kernel supports it, buffered IO will be uncached. This means that reads are dropped from the cache if we are the ones instantiating the pages, and writes are sync and dropped from the page cache on IO completion. Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-11-12filesetup: add fallocate=truncate option.Trip Volpe
Fixes #833. Provides the ability to initially "layout" the file by ftruncating it to the desired size before performing IO. This is mainly useful on Windows, which serializes all writes that extend the size of a file. Using this option with a suitable iodepth allows fio to emulate the behavior of Windows Explorer file copy, which always truncates to the expected size before issuing writes for performance reasons.
2019-10-06configure: stop enabling fdatasync on OSXSitsofe Wheeler
Change configure compile probes to fail on implicit declarations. This (correctly) stops us from enabling fdatasync on OSX which was problematic because we were actually calling a syscall stub with arguments different to what was declared in its prototype (see https://github.com/gbrault/picoc/issues/145#issuecomment-89734655 and https://gitlab.freedesktop.org/xdg/shared-mime-info/issues/7 ). Fixes: https://github.com/axboe/fio/issues/834 ("Drop macOS support for fdatasync") Signed-off-by: Sitsofe Wheeler <sitsofe@yahoo.com>
2019-08-05Add tests from t/ to the Windows installerRebecca Cran
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-05-22configure: add gettid() testJens Axboe
Apparently some newer glibcs now have it, decades after the fact. Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-05-20io_uring: sync with liburing/kernelJens Axboe
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-04-25io_uring: remove cachehit informationJens Axboe
This patch never made it into the upstream kernel, remove knowledge of it from fio. Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-04-17rand: fix truncated rand_seed on WindowsMing-Hung Tsai
The long data type is 32-bit on LLP64 platforms
2019-02-23sg: Clean up handling of big endian data fieldsDmitry Fomichev
Getting and setting values in SCSI commands and descriptors, which are big endian, in SG driver can use a bit of cleanup. This patch simplifies SG driver code by introducing a set of accessor functions for reading raw big endian values from SCSI buffers and another set for properly storing the local values as big endian byte sequences. The patch also adds some missing endianness conversion macros in os.h. Signed-off-by: Dmitry Fomichev <dmitry.fomichev@wdc.com> Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-02-10io_uring: sync header with the kernelJens Axboe
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-01-29io_uring: update to kernel struct io_uring_paramsJens Axboe
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-01-28io_uring: sync with kernelJens Axboe
- Update to newer kernel API header - Use IORING_ENTER_SQ_WAKEUP Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-01-16io_uring: sync with upstream APIJens Axboe
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-01-13io_uring: ensure that the io_uring_register() structs are 32-bit safeJens Axboe
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-01-13Move io_uring to os/linux/Jens Axboe
It's not a generic OS header, reflect the fact that it's Linux only by moving it to a linux/ directory. Also update io_uring_sqe to match current API. Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-01-12t/io_uring: add IORING_OP_NOP supportJens Axboe
Doesn't do anything on the kernel side, just a round trip through the SQ and CQ ring. Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-01-11io_uring: update to newer APIJens Axboe
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-01-10t/io_uring: add support for registered filesJens Axboe
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-01-10Update io_uring APIJens Axboe
- Fixed buffers are now available through io_uring_register() - Various thread/wq options are now dead and automatic instead - sqe->index is now sqe->buf_index - Fixed buffers require flag, not separate opcode Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-01-10Update to newer io_uring APIJens Axboe
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-01-09Update to newer io_uring APIJens Axboe
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-01-08io_uring: use kernel header directlyJens Axboe
The kernel header has been designed as such that it doesn't require a special userland version of it. Use it directly. Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-01-08io_uring.h should include <linux/fs.h>Jens Axboe
This ensures we have the __kernel_rwf_t definition. Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-01-07Rename t/aio-ring to t/io_uringJens Axboe
The new API is completely discoupled from the aio/libaio interface. Rename it while adopting the new API. Signed-off-by: Jens Axboe <axboe@kernel.dk>
2018-11-02Fix Windows CPU countJeff Furlong
Return the current activ CPUs rather than the maximum supported CPUs. Fixes: c479640d6208 ("Merge branch 'proc_group' of https://github.com/sitsofe/fio") Signed-off-by: Jens Axboe <axboe@kernel.dk>
2018-10-08Remove old OS dependent (unused) random codeJens Axboe
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2018-09-12windows: make win_to_posix_error() more resilientJens Axboe
If we pass in ERROR_SUCCESS, we should get a zero back. Signed-off-by: Jens Axboe <axboe@kernel.dk>
2018-09-11windows: fix the most egregious posix.c style errorsJens Axboe
Signed-off-by: Jens Axboe <axboe@kernel.dk>
generated by cgit v1.2.3 (git 2.25.1) at 2022-06-27 05:42:52 +0000