path: root/arch
AgeCommit message (Collapse)Author
2020-08-31flow: add ability for weight-based flow control on multiple jobsDavid, Bar
Fixes: #741 - fio how to run mix workload all 4 patterns in one job file Previously, 'flow' was used to regulate activity between 2 or more jobs with certain weight. However, the implementation was ineffective when regulating more than 2 jobs with different weights, since it relied on a single, shared counter, and was missing logic to track individual job's activity and regulate it according to it's requested weight in proportion to other jobs' requested weight and activity. This commit modifies the 'flow' functionality to track each job's activity and the total activity belonging to a 'flow_id' group. Using this new information, the 'flow' logic is then modified to regulate a job's activity in proportion to other jobs in the same group. Similar to previous behavior, the regulation is done by stalling a job's activity. New jobs joining an existing 'flow_id' group or jobs repead from said group will cause re-calibration of the weights and alter the activity as a result. For example, 3 jobs (j1,j2,j3) with weights (1:2:3) respectively will maintain activity ratio of 1:2:3. Thus, j1 will have activity of 1/6, j2 - 2/6 and j3 - 1/2. If j3 exits prematurely, the proportion is re-calibrated such that j1 will strive to have activity of 1/3 and j2 - 2/3. Re-calibration phase may cause a hiccup in the activity of one or more jobs since the new desired proportion might not be achieved quickly enough. Signed-off-by: David, Bar <>
2020-06-21arch/arch.h: Introduce atomic_{load_acquire,store_release}()Bart Van Assche
Implement atomic_load_acquire() and atomic_store_release() with C11 atomic operations. These two primitives will be used in later patches. This patch increases the minimum requirement for the compiler fio is built with from C90 to C11. Signed-off-by: Bart Van Assche <>
2020-02-06Unify architecture io_uring syscall numbersJens Axboe
Only Alpha is the odd one out, apart from that all architectures use the same system call number. So pull the code out arch headers, and into the generic arch header. Fixes: Signed-off-by: Jens Axboe <>
2019-12-25io_uring: Enable io_uring ioengine on aarch64 archZhenyu Ye
This patch enables io_uring ioengine on aarch64 arch. Signed-off-by: Zhenyu Ye <> Signed-off-by: Jens Axboe <>
2019-11-05arch-arm: Consider armv7ve arch as wellKhem Raj
Signed-off-by: Khem Raj <> Signed-off-by: Jens Axboe <>
2019-11-03Enable io_uring engine on powerpc archMikhail Malygin
Signed-off-by: Mikhail Malygin <>
2019-03-31arch: fix build breakage on armv6 againRosen Penev
6K was missing from the defines. Signed-off-by: Rosen Penev <>
2019-01-28Merge branch 'hygon-support' of Axboe
* 'hygon-support' of Add Hygon SoC support to enable tsc_reliable feature
2019-01-29Add Hygon SoC support to enable tsc_reliable featurehygonsoc
2019-01-23io_uring: system calls have been renumberedJens Axboe
Signed-off-by: Jens Axboe <>
2019-01-15x86-64: correct read/write barriersJens Axboe
Signed-off-by: Jens Axboe <>
2019-01-13io_uring: add 32-bit x86 supportJens Axboe
Signed-off-by: Jens Axboe <>
2019-01-10Update io_uring APIJens Axboe
- Fixed buffers are now available through io_uring_register() - Various thread/wq options are now dead and automatic instead - sqe->index is now sqe->buf_index - Fixed buffers require flag, not separate opcode Signed-off-by: Jens Axboe <>
2019-01-07Rename aioring engine to io_uringJens Axboe
The new API is completely discoupled from the aio/libaio interface. Rename it while adopting the new API. Signed-off-by: Jens Axboe <>
2019-01-04engines/aioring: update for newer mmap based APIJens Axboe
Signed-off-by: Jens Axboe <>
2018-12-12aioring: hide it if archs don't define syscallsJens Axboe
Signed-off-by: Jens Axboe <>
2018-12-12Add aioring engineJens Axboe
This is a new Linux aio engine, built on top of the new aio interfaces. It supports polled aio, regular aio, and buffered aio. Signed-off-by: Jens Axboe <>
2018-11-19engines/libaio: fix new aio poll APIJens Axboe
It'll be final. Some day. Fixes: ebec344dd336 ("engines/libaio: update to new io_setup2() system call") Signed-off-by: Jens Axboe <>
2018-11-19engines/libaio: update to new io_setup2() system callJens Axboe
We need that to enable polling. Signed-off-by: Jens Axboe <>
2018-08-12arch: fix build breakage on armv6Sitsofe Wheeler
The FreeBSD folks found fio didn't build on their armv6 systems ( ) but Mikaël Urankar fixed the issue in so copy the fix to mainline. Signed-off-by: Sitsofe Wheeler <>
2018-06-05x86: don't need 'level' passed to amd/intel init functionsJens Axboe
Signed-off-by: Jens Axboe <>
2018-03-30aarch64: refactor HW ARM CRC32c detectionSitsofe Wheeler
- Tweak how fio does the configure probe for hardware accelerated ARM CRC32c - Move the Linux specific runtime detection into os-linux.h making it cleaner to implement runtime probing for OSes other than Linux in the future - Make the ARCH_HAVE_CRC_CRYPTO define go into the config-host.* files the same way as most other configure defines - Set/use only the ARCH_HAVE_CRC_CRYPTO define rather than also creating/using ARCH_HAVE_ARM64_CRC_CRYPTO This commit should silence the "Fails to compile: error: sys/auxv.h file not found" compilation message that was seen on FreeBSD 12/arm64 (but note a future commit will be required to implement runtime probing and actually enable the accelerated path on that platform). Signed-off-by: Sitsofe Wheeler <>
2018-03-21Add include-what-you-use pragmasSitsofe Wheeler
Add IWYU pragmas to improve its analysis. Signed-off-by: Sitsofe Wheeler <>
2017-08-29arch/ppc: Fix get_cpu_clock asm clobber listOliver O'Halloran
Mark condition register 0 (cr0) as being clobbered by the inline asm in get_cpu_clock(). Not doing this results in strange behaviour due to GCC optimising away some checking. For example e.g: $ ./fio examples/null.fio --output-format=json time 5500 cycles_start=8085227422910 <--- bad! { "fio version" : "fio-2.21-89-gb034", <snip> The extra output is due to the flag checking in dprint being removed at higher optimisation levels. Signed-off-by: Oliver O'Halloran <>
2017-07-27arch: raise an error when compiling for an unknown ARM platformSitsofe Wheeler
The nop, read_barrier() and write_barrier() macros have to always be defined but due to the way os/arch-arm.h is written it is possible for them to be undefined when the ARM platform is not recognized. Change things so compiling for an unknown ARM platform raises an explicit compilation error at the point of the problem. Signed-off-by: Sitsofe Wheeler <>
2017-06-22arch: tsc_reliable can be a boolJens Axboe
Signed-off-by: Jens Axboe <>
2017-01-05Add arm64 hardware assisted crc32c supportwei xiao
HW assisted crc32c get ~ x39.6 speedups on ARM Cortex-A57@2GHz Auto-detect whether to use hw assisted crc32c. If the hardware assisted crypto is available, always use it. Otherwise, fallback to software.
2016-07-27Change arch_i386 to arch_x86Tomohiro Kusumi
fio_arch_strings[arch_i386] is "x86" (not "i386"), so change arch_i386 to arch_x86. Other than arch_i386, "i386" is only used in __i386__. Signed-off-by: Tomohiro Kusumi <> Signed-off-by: Jens Axboe <>
2016-07-19Add os/os-linux-syscall.h to separate syscall NR from arch headersTomohiro Kusumi
Linux syscall NR should be defined under os/ with ifdef(arch) switch rather than arch/ where headers are included not only by Linux but also all other supported OS that don't need these NR. (It'll get worse and worse if someone wants to add something similar under arch/ for other supported OS) Not sure if you like the way I separated these by adding os/os-linux-syscall.h, but doing something like #if FIO_OS is (Linux or Android) ... #endif in arch/arch-*.h doesn't work because arch headers are to be included prior to os/os-*.h. Signed-off-by: Tomohiro Kusumi <> Signed-off-by: Jens Axboe <>
2016-07-19Change ARCH_X86_64_h to ARCH_X86_64_HTomohiro Kusumi
Signed-off-by: Tomohiro Kusumi <> Signed-off-by: Jens Axboe <>
2016-06-02arch: wire up preadv2/pwritev2 for more architecturesJens Axboe
Signed-off-by: Jens Axboe <>
2016-06-02Enable preadv2/pwritev2 engines by default on LinuxJens Axboe
We don't have defines in libc yet, but let's wire them up so we can use them. Only enabled for x86 and x86-64, adding for other architectures would be as simple as adding the right defines for the two syscall numbers. Signed-off-by: Jens Axboe <>
2015-12-23arch: add ffs64 and ffz64Jens Axboe
These just use the normal C software version, extended to 64-bit even on 32-bit archs. The fast variant is still machine word native. Signed-off-by: Jens Axboe <>
2015-12-04powerpc: enable cpu clock for powerpc64Li Zhong
This patch tries to enable cpu clock for powerpc64, the code is copied from mftb() in kernel source. The two instructions after mfspr are added in the kernel to solve an erratum on Cell and fsl booke CPUs. On those CPUs, 64 bits mftb is not atomic, so it is possible that the low order 32 bits are already reset to 0x00000000 but the high order bits are not yet incremented by one. Don't know how to tell whether it will be running on those CPUs or not, I just keep the above fix for all ppc64 CPUs. Even if we have some method to check whether we are on those CPUs or not at run time, I think the check won't cost less than the two added instructions. Maybe we could use the similar fix up code kernel uses to dynamically patch the instructions with nops if not needed. But that would add much more complexity. Signed-off-by: Li Zhong <> Signed-off-by: Jens Axboe <>
2015-08-13arch-ppc.h: Add ilog2 implementation for ppc64Chandan Rajendra
On a ppc64 machine, when fio is invoked with the following configuration file (generated by fstests' generic/300 test), it loops indefinitely. [global] directory=/mnt/btrfs-xfstest-scratch filesize=536870912 size=999G continue_on_error=write ignore_error=,ENOSPC error_dump=0 create_on_open=1 fallocate=none exitall=1 [direct_aio_raicer] ioengine=libaio iodepth=128*1 bs=128k direct=1 numjobs=4 rw=randwrite runtime=100*1 time_based filename=racer [falloc_raicer] ioengine=falloc runtime=100*1 iodepth=1 bssplit=128k/80:512k/10:32k/10 rw=randwrite numjobs=1 filename=racer [punch_hole_raicer] ioengine=falloc runtime=100*1 bs=4k time_based=10 rw=randtrim numjobs=2 filename=racer time_based [aio-dio-verifier] ioengine=libaio iodepth=128*1 numjobs=1 verify=crc32c-intel verify_fatal=1 verify_dump=1 verify_backlog=1024 verify_async=4 verifysort=1 direct=1 bs=4k rw=randwrite filename=aio-dio-verifier This is because arch_ffz() ends up invoking the 32-bit version of __ilog2() which always returns a bit number in the range 0 - 31. This can cause "overlap" variable in axmap_set_fn() to never becomes zero. To fix this the commit adds a 64-bit version of __ilog2() (obtained from the Linux kernel's arch/powerpc/include/asm/bitops.h). Signed-off-by: Chandan Rajendra <> Signed-off-by: Jens Axboe <>
2015-05-29Add Intel rdrand supportJens Axboe
Not wired up. Signed-off-by: Jens Axboe <>
2015-05-05ARM: Use generic assembly nop and barrier code for armv8-aMilton Chiang
Signed-off-by: Milton Chiang <>
2015-02-27fio: fix cycles_start build issueChristian Ehrhardt
There is an issue introduced with "commit 73df3e07 gettime: offset CPU cycle counter by initial value". For architectures which define ARCH_CPU_CLOCK_CYCLES_PER_USEC (currently only s390) this causes a build error gettime.c:174:11: error: ‘cycles_start’ undeclared (first use in this function) if (t < cycles_start && !cycles_wrap) To make sure variables and code are only compiled in the same cases I added a clock definition called ARCH_CPU_CLOCK_WRAPS. We could merge ARCH_CPU_CLOCK_WRAPS and the existing ARCH_CPU_CLOCK_CYCLES_PER_USEC into one if you prefer, so far nobody else uses it. To avoid cluttering all architecture headers I enabled it by default in arch-generic.h, so any arch not needing the wrap handling can undef ARCH_CPU_CLOCK_WRAPS later in their headers. Signed-off-by: Christian Ehrhardt <> Signed-off-by: Jens Axboe <>
2015-02-25Add missing pre-defined ARM arch compiler macroRaphael Silva
Signed-off-by: Raphael Silva <> Signed-off-by: Jens Axboe <>
2014-06-16fio: Eliminate compilation warning in ppcSumanth K
Eliminate compilation warning : 'atb_clocktest' defined but not used. Function is declared as static in header file. So compiler will know if they used or not. Just eliminate the warning by #if 0 /unless we do use it. Signed-off-by: Sumanth K <> Signed-off-by: Jens Axboe <>
2014-06-08Add aarch64 support.Doug Kwan
This patch adds support for aarch64 for fio. This is tested by building FIO on an aarch64 Ubuntu system and running some samepls. Signed-off-by: Doug Kwan <> Signed-off-by: Jens Axboe <>
2014-04-10fio: fix s390 timing for older machines/binutilsChristian Ehrhardt
Older gcc/binutils or one with very low default march will refuse to use stckf. On top of that machines older than a z196 don not even have the store clock fast facility. To fix all of those issues, check for the facilities in configure step and make the usage of stckf and a matching march dependent to it. Signed-off-by: Christian Ehrhardt <> Signed-off-by: Jens Axboe <>
2014-04-08fio: fix s390 nopChristian Ehrhardt
When trying to run rate limited fio runs we encountered that the current definition of a nop uses a privileged instruction which is not even a nop, but a yield call to the hipervisor. That leads to a SIGILL as it is privileged. To solve that issue replace it with a nop, the assembler will take care of it (likely to become a BCR 0,0) Signed-off-by: Christian Ehrhardt <> Signed-off-by: Jens Axboe <>
2014-04-08fio: fix s390 time accountingChristian Ehrhardt
The current timer implementation could cause time warps on s390 which ends up as time bound jobs that would never end, because they always reset themself to the old time. When touching this code anyway, we also change it to use the faster stckf and avoid the calibration as we can control the result to be usecs. This also eliminates a few calculations cycle->usec in the hot path for the timer. In case other architectures have similar improved timers that might not be usec based, but nsec based or such a thing any architecture can set ARCH_CPU_CLOCK_CYCLES_PER_USEC to an appropriate per-arch value. This leaves the infrastructure open for others and the compiler will throw our division by 1 away anyway. Signed-off-by: Christian Ehrhardt <> Signed-off-by: Jens Axboe <>
2014-02-14Fix early termination of cpu id stringJens Axboe
Commit 2d60ad8e terminated the cpu id string at 12 chars, but the output from cpuid is 12 chars. So we need one more character, if we want it null terminated and strcmp() working. Signed-off-by: Jens Axboe <>
2014-02-13cpuid: Null terminate x86 CPU string.Sitsofe Wheeler
Signed-off-by: Sitsofe Wheeler <> Signed-off-by: Jens Axboe <>
2013-10-31Fix bad asm constrants for x86-64 cpuid()Jens Axboe
Bruce reports: It seems there's a bug in the x86_64 version of do_cpuid() that causes fio to segfault when built with clang at lower optimization levels: static inline void do_cpuid(unsigned int *eax, unsigned int *ebx, unsigned int *ecx, unsigned int *edx) { asm volatile("cpuid" : "=a" (*eax), "=b" (*ebx), "=r" (*ecx), "=d" (*edx) : "0" (*eax), "2" (*ecx) : "memory"); } via Tijl Coosemans: Should be: "=c" (*ecx) But you can also use the '+' modifier and remove the input operands: : "+a" (*eax), "=b" (*ebx), "+c" (*ecx), "=d" (*edx) : : "memory" Signed-off-by: Jens Axboe <>
2013-10-11ppc: disable CPU clock until we can detect whether we have it or notJens Axboe
The child segfault test should catch it, however it does not on AIX at least. Signed-off-by: Jens Axboe <>
2013-10-11arch-ppc.h: Fix typo in header guardAkash Verma
Signed-off-by: Akash Verma <> Signed-off-by: Jens Axboe <>
2013-08-09powerpc: disable use of ATBU clockJens Axboe
Signed-off-by: Jens Axboe <>