path: root/server.c
AgeCommit message (Collapse)Author
2019-06-04Fix string copy compilation warningsDamien Le Moal
Fix the many warnings that gcc 9 spits out. Signed-off-by: Damien Le Moal <> Signed-off-by: Jens Axboe <>
2018-12-14Add cache hit statsJens Axboe
With the aioring engine, we can get notified if a buffered read was a cache hit or if it hit media. Add that to the output stats for normal and json output. Signed-off-by: Jens Axboe <>
2018-12-14client/server: convert nr_zone_resets on the wireJens Axboe
Fixes: fd5d733fa34 ("Collect and show zone reset statistics") Signed-off-by: Jens Axboe <>
2018-10-04Bool conversionsJens Axboe
Just various low hanging fruits in terms of local or global variables that are only 0/1, switch them to boolean. Signed-off-by: Jens Axboe <>
2018-09-27server: be locally vocal about communication issuesJens Axboe
If we fail the CRC check or if we have a mismatched client/server, then we can't reliably transmit anything across the wire. For that case, in addition to attempting to relay this information across the wire, also print a local warning. This fixes a case where we silently exit when the client/server versions aren't properly matched. Fixes: Signed-off-by: Jens Axboe <>
2018-07-23Add support for >= 4G block sizesJeff Furlong
For trims, it's useful to be able to support larger than (or equal to) 4GB. This extends the block sizes support for that. Change from Jeff, various little fixups from me. Signed-off-by: Jens Axboe <>
2018-06-05Fix variable shadowingJens Axboe
Signed-off-by: Jens Axboe <>
2018-04-13Merge branch 'proc_group' of Axboe
* 'proc_group' of doc: add cpus_allowed reference to log_compression_cpus appveyor: make 32 bit build target XP + minor fixes configure/Makefile: make Cygwin force less doc: add Windows processor group behaviour and Windows target option windows: target Windows 7 and add support for more than 64 CPUs windows: prepare for Windows build split windows: update EULA
2018-04-08server: fix dead assignment of variableJens Axboe
Signed-off-by: Jens Axboe <>
2018-03-27windows: target Windows 7 and add support for more than 64 CPUsSitsofe Wheeler
Introduce support for targeting the build to either Windows XP or Windows 7 (the default). When targeting Windows 7 this allows us to use the process group APIs which are required are required to support more than 64 CPUs. These changes mean if you want a Windows binary that supports versions of Windows below Windows 7/Windows Server 2008 R2 you will need to explicitly use the --target-win-ver=xp configure option at build time. Such builds will lack features (such as the ability to access CPUs beyond those in fio's default process group) that are dependent on recent Windows APIs. Signed-off-by: Sitsofe Wheeler <>
2018-03-22server: use scalloc() for sk_out allocationJens Axboe
This informs valgrind that it's properly initialized. Signed-off-by: Jens Axboe <>
2018-03-21server: fix error handling for shared memory handlingJens Axboe
Signed-off-by: Jens Axboe <>
2018-03-21Merge branch 'include_refactor' of Axboe
* 'include_refactor' of Add include-what-you-use pragmas oslib: make str* compat functions more uniform Use POSIX path for poll.h and fcntl.h headers Refactor #includes and headers
2018-03-21server: handle shared mem pool allocation failuresJens Axboe
Signed-off-by: Jens Axboe <>
2018-03-21Use POSIX path for poll.h and fcntl.h headersSitsofe Wheeler
musl complains when using include paths like <sys/poll.h> rather than POSIX's <poll.h> so change the poll.h include path and update the windows poll.h compatibility shim to match. Allegedly sys/poll.h was needed for GLIBCs that were older than 2.3 according to but I can still see that path in an ancient Red Hat 5.0 glibc-devel-2.0.7-32.i386.rpm ... Signed-off-by: Sitsofe Wheeler <>
2018-03-21Refactor #includes and headersSitsofe Wheeler
- Try and remove unneeded #include lines - Try and add #include lines that would allow the files to be built in a more standalone manner Signed-off-by: Sitsofe Wheeler <>
2018-03-21server: process STOP/QUIT commands out-of-lineJens Axboe
At the end of a job, the stats entries are queued up for out-of-line transmission, but we immediately queue a client STOP+QUIT command right after that. If the command handling loop doesn't wakeup and process the stats commands before we process the STOP+QUIT inline, then we lose the stats output from those commands. The result is that the client won't see the job output from that job, or the "All clients" output. We should impose proper ordering for all commands, but this is a quick fix to ensure that we don't hit the above mentioned but. For other commands we don't care about ordering between inline and out-of-line commands, so it's likely good enough to carry us forward. Reported-by: Jeff Furlong <> Signed-off-by: Jens Axboe <>
2018-03-15Rename fio_mutex into fio_semBart Van Assche
Since fio_mutex implements a semaphore, rename fio_mutex into fio_sem. Rename those fio_sem instances that have 'mutex' embedded in their name. Rename the mutex.h and .c source files into fio_sem.h and .c respectively. Signed-off-by: Bart Van Assche <>
2018-03-01Fix overflow of counters incremented on each I/O operationAlexander Larin
- In the thread_stat struct: uint32_t counters are updated to uint64_t. - In the io_u_plat_entry struct: unsigned int counters are updated to uint64_t. It fixes overflow of these counters. Signed-off-by: Alexander Larin <>
2018-01-05Fix client/server "all clients" reportingJeff Furlong
In commit e883cb3501de0f99297282c18842d50aaa7fa531, the default significant figures was added. The default value was preserved for local jobs and when client/server was used if jobs=1, but not when client/server was used if jobs > 1. This patch restores some reporting outputs ("all clients" output for IOPS and BW) when client/server is used if > 1 jobs, for example: OLD: All clients: (groupid=0, jobs=2): err= 0: pid=0: Wed Jan 3 14:30:13 2018 read: IOPS=0, BW=0 (0)(0B/5001msec) <--ZERO VALUES slat (nsec): min=762, max=108579, avg=1058.97, stdev=221.30 clat (nsec): min=254, max=4544.2k, avg=11297.64, stdev=8284.14 lat (usec): min=9, max=4545, avg=12.41, stdev= 8.30 bw ( KiB/s): min=290192, max=347160, per=49.35%, avg=303613.33, stdev=11582.87, samples=18 iops : min=72548, max=86790, avg=75903.33, stdev=2895.72, samples=18 NEW: All clients: (groupid=0, jobs=2): err= 0: pid=0: Wed Jan 3 15:08:33 2018 read: IOPS=147k, BW=575Mi (603M)(2876MiB/5001msec) <--CORRECT VALUES slat (nsec): min=841, max=61945, avg=1304.83, stdev=232.75 clat (nsec): min=544, max=4487.2k, avg=11736.68, stdev=9123.64 lat (usec): min=10, max=4489, avg=13.10, stdev= 9.13 bw ( KiB/s): min=260945, max=305976, per=49.99%, avg=294350.22, stdev=9192.53, samples=18 iops : min=65236, max=76494, avg=73587.56, stdev=2298.13, samples=18 Regards, Jeff Signed-off-by: Jens Axboe <>
2017-12-14server: convert more memset to on-stack initializationJens Axboe
Signed-off-by: Jens Axboe <>
2017-12-14server: cleanup iolog pdu prepJens Axboe
We don't have to memset() the whole thing, we just need to ensure that the members we don't assign are cleared. Remove the memset and initialize the pdu on the stack. Signed-off-by: Jens Axboe <>
2017-12-13server: initialize first iolog header properlyJens Axboe
We're sending potential stack garbage over the wire, init the whole pdu to zeroes before filling it in. This shuts up valgrind about using uninitialized memory. Signed-off-by: Jens Axboe <>
2017-11-30steadystate: make flags conform to usual fio standardJens Axboe
__VAL are usually bit shift values, with VAL being the proper mask. For some reason, the steadystate code uses both as masks, fix that up. Signed-off-by: Jens Axboe <>
2017-11-30server: process connection list before executing triggerJens Axboe
This shrinks the window a bit for allowing the backend to notice that jobs have gone away, and notify the client. Signed-off-by: Jens Axboe <>
2017-11-22add significant_figures parameterJeff Furlong
Signed-off-by: Jens Axboe <>
2017-09-18fix strncpy(3) copy lengthTomohiro Kusumi
0 terminate the last byte, and copy at most size-1. (or 0 terminate the last byte after copying upto size bytes) Signed-off-by: Tomohiro Kusumi <> Signed-off-by: Jens Axboe <>
2017-09-13Add support for doing total latency percentilesJens Axboe
By default, fio does completion latency percentiles. Sometimes what you want is total IO latency percentiles, including the submission part as well. If that's your thing, then set lat_percentiles=1 and get that instead of the default completion latency percentiles. Signed-off-by: Jens Axboe <>
2017-08-26backend: verify-trigger fixesSitsofe Wheeler
Clear the trigger_timeout after we have found it has been exceeded in trigger_timedout() to stop us repeatedly firing the trigger again and again in the future. When --trigger-remote is unset, the server winds up with the remote trigger command represented by buf[] being zero length rather than buf being NULL in handle_trigger_cmd(). Cope with this by returning right away if we find the cmd string is NULL or empty in exec_trigger(). Make server mode fio terminate after sending the verify state back to the client so it goes on behave the same way as standalone fio. This stops the while loop in thread_main() restating and in turn prevents state being cleared away by clear_io_state(). Make keep_running() return false if td->terminate has been set Just In Case. The below would trigger the above problem before this patch: cat <<EOF > trigger.fio [global] ioengine=null time_based=1 runtime=10s size=1M bs=4k rate=4k [trigtest] EOF ./fio --server & ./fio --trigger-timeout=2s --trigger='hostname' --client=localhost \ --debug=net trigger.fio Fixes ("fio intermittently fails to perform IO when --trigger-timeout, --trigger is specified"). Reported-by: saurabhbpl <> Signed-off-by: Sitsofe Wheeler <>
2017-06-23stat: Print one-line iops statAndreas Herrmann
This adds basic iops statistics (likewise to what we already have for bandwidth) to normal and json fio output. Example for normal output: bw ( KiB/s): min=42192, max=162068, per=0.10%, avg=93236.21, stdev=22180.02 iops : min=21090, max=81020, avg=46606.84, stdev=11088.40 lat (usec) : 2=55.13%, 4=21.11%, 10=0.36%, 20=0.03%, 50=23.14% Added handling for iops_stat to client/server and bump server version. Signed-off-by: Andreas Herrmann <> Signed-off-by: Jens Axboe <>
2017-06-23server: don't use void* for pointer arithmetic (gcc)Tomohiro Kusumi
I think gcc extension should be avoided when it can be done by just changing a pointer type. Signed-off-by: Tomohiro Kusumi <> Signed-off-by: Jens Axboe <>
2017-06-22client/server: bool conversionJens Axboe
Signed-off-by: Jens Axboe <>
2017-06-21nanosecond: update completion latency recording and normal, json output to ↵Vincent Fu
use nanoseconds A new set of nsec bins (0-1, 2-3, 4-9, 10-19, ...) is created for tracking latencies. The number of io_u_plat[] latency bins is increased by about 50% to accommodate the greater resolution while retaining the same max latency value of ~9 seconds. Example output test: (g=0): rw=read, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=null, iodepth=1 fio-2.20-39-g83b3 Starting 1 process Jobs: 1 (f=1): [R(1)][100.0%][r=14.6GiB/s,w=0KiB/s][r=3832k,w=0 IOPS][eta 00m:00s] test: (groupid=0, jobs=1): err= 0: pid=17621: Thu Jun 1 15:09:38 2017 read: IOPS=3784k, BW=14.4GiB/s (15.5GB/s)(144GiB/10000msec) clat (nsec): min=15, max=1685.2k, avg=26.92, stdev=433.62 lat (nsec): min=46, max=5989.9k, avg=67.31, stdev=1659.58 clat percentiles (nsec): | 1.00th=[ 18], 5.00th=[ 18], 10.00th=[ 19], 20.00th=[ 19], | 30.00th=[ 22], 40.00th=[ 22], 50.00th=[ 25], 60.00th=[ 26], | 70.00th=[ 26], 80.00th=[ 27], 90.00th=[ 36], 95.00th=[ 41], | 99.00th=[ 46], 99.50th=[ 50], 99.90th=[ 104], 99.95th=[ 139], | 99.99th=[ 7840] bw ( MiB/s): min=13393, max=15647, per=0.10%, avg=14784.86, stdev=558.40 lat (nsec) : 20=20.87%, 50=78.57%, 100=0.43%, 250=0.10%, 500=0.01% lat (nsec) : 750=0.01%, 1000=0.01% lat (usec) : 2=0.01%, 4=0.01%, 10=0.01%, 20=0.01%, 50=0.01% lat (usec) : 100=0.01%, 250=0.01%, 500=0.01%, 1000=0.01% lat (msec) : 2=0.01% cpu : usr=99.72%, sys=0.12%, ctx=321, majf=0, minf=8 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwt: total=37840231,0,0, short=0,0,0, dropped=0,0,0 latency : target=0, window=0, percentile=100.00%, depth=1 Run status group 0 (all jobs): READ: bw=14.4GiB/s (15.5GB/s), 14.4GiB/s-14.4GiB/s (15.5GB/s-15.5GB/s), io=144GiB (155GB), run=10000-10000msec
2017-06-21nanosecond: initial commit changing timeval to timespecVincent Fu
struct timeval has usec resolution whereas struct timespec has nsec resolution. This commit changes several dozen instances of struct timeval to struct timespec. All of the output is unchanged. The ticks to nsec conversion in this patch is broken but fixed in a later patch.
2017-06-02use correct syscall name in log_err()Tomohiro Kusumi
Signed-off-by: Tomohiro Kusumi <> Signed-off-by: Jens Axboe <>
2017-05-03client/server: make sure that all elements in io_u_lat_m[] are transferred ↵Vincent Fu
and received FIO_IO_U_LAT_U_NR is 10 whereas FIO_IO_U_LAT_M_NR is 12. So we must iterate over io_u_lat_u and io_u_lat_m separately. Signed-off-by: Jens Axboe <>
2017-03-21Define struct sk_out in server.h (not server.c)Tomohiro Kusumi
struct sk_out isn't local to server.c, so it should be visible in its header (instead of having struct sk_out; in fio.h). Signed-off-by: Tomohiro Kusumi <> Signed-off-by: Jens Axboe <>
2017-01-03Add missing trailing \n in log_err/info()Tomohiro Kusumi
Callers need trailing \n in case of fwrite(3) (if not syslog(3)). These are the ones that seem to need trailing \n not followed by another log function call as consequence. Signed-off-by: Tomohiro Kusumi <> Signed-off-by: Jens Axboe <>
2017-01-02Convert group_run_stats to use bytes instead of KiB/KBRobert Elliott
In group_run_stats, convert these fields: * io_kb (bytes transferred) * agg (aggregate average bandwidth) * min_bw * max_bw from tracking either KiB or KB depending on kb_base to simply tracking bytes. Rename io_kb to iobytes to match this new meaning. This will simplify computing quantities and bandwidth rates with both IEC binary and SI decimal unit prefixes (e.g., GiB/s and GB/s) in future patches. Signed-off-by: Jens Axboe <>
2016-12-13Merge git:// into steady-stateVincent Fu
2016-12-07steadystate: instead of including ss_sum_y in thread_stat record whether ↵Vincent Fu
ss_sum_y is nonzero in ss_state via __FIO_SS_BUFFER_FULL
2016-12-07steadystate: implement transmission of steadystate data over the wire in ↵Vincent Fu
client/server mode 1) Pare down thread_stat steadystate members to only those that need to go over the wire 2) Plumb up sending/receiving of steadystate data including ring buffers
2016-12-01Fix conversion in fio_server_send_tsVincent Fu
Signed-off-by: Jens Axboe <>
2016-11-27stat: Change access to io_sample unionSitsofe Wheeler
Compiling on certain platforms produces warnings like stat.c: In function ‘add_clat_sample’: stat.c:2241:28: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] because the size of a pointer is different to that of a long. Work around this by explicitly naming the union used in io_sample, forcing the "val" in the union to be a uint64_t, changing function parameters to accept a union (rather than an unsigned long) and adding helper functions to wrap values into a union. Signed-off-by: Sitsofe Wheeler <>
2016-11-27fio: Fix (unsigned) integer overflow issuesSitsofe Wheeler
This patch tries to address issues found when compiling fio with clang's undefined behaviour sanitizer on 32 bit Linux. While this generated many false positives it did identify a number of subtle issues when rates/amounts/time calculations went above 32 bit values. The following job was used: [global] time_based runtime=180m bs=256k rw=randrw norandommap rate=512M ioengine=null direct=1 iodepth=8 thread numjobs=10 ramp_time=20m [null] size=1P It should fix because various 32 bit ETA variables were overflowing due to the microsecond arithmetic being done on them (even on 64 bit systems). This patch should also fix and because 64 bit Windows is a LLP64 platform so ints and longs still default to 32 bits. Signed-off-by: Sitsofe Wheeler <>
2016-09-25smalloc: OOM fixupsJens Axboe
Warn directly in smalloc() instead of having the callers do it. Add a hint on how to solve a situation where we run out of shared memory. Signed-off-by: Jens Axboe <>
2016-08-26Client / server code for handling histograms. The server:Karl Cronburg
- Deals with subtracting consecutive histograms so that client doesn't have to recreate linked list. - Uses existing IOLOG command code to create packets, checking for IO_LOG_TYPE_HIST when necessary. And the client: - Reconstructs the pointers to the histogram bins from the packet format, namely (sample_0, hist_0, sample_1, hist_1, ...) in lieu of the current (sample_0, sample_1, ...) format used for the (void *) cur_log->samples. - Flushes histograms to file with updated pointer calculation to get ith sample, disabling subtraction in hist_sum() using a null pointer. This does not cover plain-text transmission mode (i.e. when zlib is not present during fio compilation). Signed-off-by: Karl Cronburg <>
2016-08-15Fixup correct sparse warningsJens Axboe
Signed-off-by: Jens Axboe <>
2016-08-08Add missing FIO_NET_CMD entry to fio_server_ops[]Tomohiro Kusumi
Signed-off-by: Tomohiro Kusumi <> Signed-off-by: Jens Axboe <>
2016-05-27server: ensure that we flush compressed logs correctlyJens Axboe
Do chunkwise block compression, and flush at the end, adding more space as needed. Signed-off-by: Jens Axboe <>