Commit | Line | Data |
---|---|---|
b5cbc369 GB |
1 | |
2 | Kernel NFS Server Statistics | |
3 | ============================ | |
4 | ||
5 | This document describes the format and semantics of the statistics | |
6 | which the kernel NFS server makes available to userspace. These | |
7 | statistics are available in several text form pseudo files, each of | |
8 | which is described separately below. | |
9 | ||
10 | In most cases you don't need to know these formats, as the nfsstat(8) | |
11 | program from the nfs-utils distribution provides a helpful command-line | |
12 | interface for extracting and printing them. | |
13 | ||
14 | All the files described here are formatted as a sequence of text lines, | |
15 | separated by newline '\n' characters. Lines beginning with a hash | |
16 | '#' character are comments intended for humans and should be ignored | |
17 | by parsing routines. All other lines contain a sequence of fields | |
18 | separated by whitespace. | |
19 | ||
20 | /proc/fs/nfsd/pool_stats | |
21 | ------------------------ | |
22 | ||
23 | This file is available in kernels from 2.6.30 onwards, if the | |
24 | /proc/fs/nfsd filesystem is mounted (it almost always should be). | |
25 | ||
26 | The first line is a comment which describes the fields present in | |
27 | all the other lines. The other lines present the following data as | |
28 | a sequence of unsigned decimal numeric fields. One line is shown | |
29 | for each NFS thread pool. | |
30 | ||
31 | All counters are 64 bits wide and wrap naturally. There is no way | |
32 | to zero these counters, instead applications should do their own | |
33 | rate conversion. | |
34 | ||
35 | pool | |
36 | The id number of the NFS thread pool to which this line applies. | |
37 | This number does not change. | |
38 | ||
39 | Thread pool ids are a contiguous set of small integers starting | |
40 | at zero. The maximum value depends on the thread pool mode, but | |
41 | currently cannot be larger than the number of CPUs in the system. | |
42 | Note that in the default case there will be a single thread pool | |
43 | which contains all the nfsd threads and all the CPUs in the system, | |
44 | and thus this file will have a single line with a pool id of "0". | |
45 | ||
46 | packets-arrived | |
47 | Counts how many NFS packets have arrived. More precisely, this | |
48 | is the number of times that the network stack has notified the | |
49 | sunrpc server layer that new data may be available on a transport | |
50 | (e.g. an NFS or UDP socket or an NFS/RDMA endpoint). | |
51 | ||
52 | Depending on the NFS workload patterns and various network stack | |
53 | effects (such as Large Receive Offload) which can combine packets | |
54 | on the wire, this may be either more or less than the number | |
55 | of NFS calls received (which statistic is available elsewhere). | |
56 | However this is a more accurate and less workload-dependent measure | |
57 | of how much CPU load is being placed on the sunrpc server layer | |
58 | due to NFS network traffic. | |
59 | ||
60 | sockets-enqueued | |
61 | Counts how many times an NFS transport is enqueued to wait for | |
62 | an nfsd thread to service it, i.e. no nfsd thread was considered | |
63 | available. | |
64 | ||
65 | The circumstance this statistic tracks indicates that there was NFS | |
66 | network-facing work to be done but it couldn't be done immediately, | |
67 | thus introducing a small delay in servicing NFS calls. The ideal | |
68 | rate of change for this counter is zero; significantly non-zero | |
69 | values may indicate a performance limitation. | |
70 | ||
72faedae SM |
71 | This can happen because there are too few nfsd threads in the thread |
72 | pool for the NFS workload (the workload is thread-limited), in which | |
73 | case configuring more nfsd threads will probably improve the | |
74 | performance of the NFS workload. | |
b5cbc369 GB |
75 | |
76 | threads-woken | |
77 | Counts how many times an idle nfsd thread is woken to try to | |
78 | receive some data from an NFS transport. | |
79 | ||
80 | This statistic tracks the circumstance where incoming | |
81 | network-facing NFS work is being handled quickly, which is a good | |
82 | thing. The ideal rate of change for this counter will be close | |
83 | to but less than the rate of change of the packets-arrived counter. | |
84 | ||
b5cbc369 GB |
85 | threads-timedout |
86 | Counts how many times an nfsd thread triggered an idle timeout, | |
87 | i.e. was not woken to handle any incoming network packets for | |
88 | some time. | |
89 | ||
90 | This statistic counts a circumstance where there are more nfsd | |
91 | threads configured than can be used by the NFS workload. This is | |
92 | a clue that the number of nfsd threads can be reduced without | |
93 | affecting performance. Unfortunately, it's only a clue and not | |
94 | a strong indication, for a couple of reasons: | |
95 | ||
96 | - Currently the rate at which the counter is incremented is quite | |
97 | slow; the idle timeout is 60 minutes. Unless the NFS workload | |
98 | remains constant for hours at a time, this counter is unlikely | |
99 | to be providing information that is still useful. | |
100 | ||
101 | - It is usually a wise policy to provide some slack, | |
102 | i.e. configure a few more nfsds than are currently needed, | |
103 | to allow for future spikes in load. | |
104 | ||
105 | ||
106 | Note that incoming packets on NFS transports will be dealt with in | |
107 | one of three ways. An nfsd thread can be woken (threads-woken counts | |
108 | this case), or the transport can be enqueued for later attention | |
109 | (sockets-enqueued counts this case), or the packet can be temporarily | |
110 | deferred because the transport is currently being used by an nfsd | |
111 | thread. This last case is not very interesting and is not explicitly | |
112 | counted, but can be inferred from the other counters thus: | |
113 | ||
114 | packets-deferred = packets-arrived - ( sockets-enqueued + threads-woken ) | |
115 | ||
116 | ||
117 | More | |
118 | ---- | |
119 | Descriptions of the other statistics file should go here. | |
120 | ||
121 | ||
122 | Greg Banks <gnb@sgi.com> | |
123 | 26 Mar 2009 |