1 .. SPDX-License-Identifier: GPL-2.0
11 DAMON subsystem is configured with three layers including
13 - Operations Set: Implements fundamental operations for DAMON that depends on
14 the given monitoring target address-space and available set of
15 software/hardware primitives,
16 - Core: Implements core logics including monitoring overhead/accurach control
17 and access-aware system operations on top of the operations set layer, and
18 - Modules: Implements kernel modules for various purposes that provides
19 interfaces for the user space, on top of the core layer.
22 Configurable Operations Set
23 ---------------------------
25 For data access monitoring and additional low level work, DAMON needs a set of
26 implementations for specific operations that are dependent on and optimized for
27 the given target address space. On the other hand, the accuracy and overhead
28 tradeoff mechanism, which is the core logic of DAMON, is in the pure logic
29 space. DAMON separates the two parts in different layers, namely DAMON
30 Operations Set and DAMON Core Logics Layers, respectively. It further defines
31 the interface between the layers to allow various operations sets to be
32 configured with the core logic.
34 Due to this design, users can extend DAMON for any address space by configuring
35 the core logic to use the appropriate operations set. If any appropriate set
36 is unavailable, users can implement one on their own.
38 For example, physical memory, virtual memory, swap space, those for specific
39 processes, NUMA nodes, files, and backing memory devices would be supportable.
40 Also, if some architectures or devices supporting special optimized access
41 check primitives, those will be easily configurable.
47 Core layer of DAMON is implemented as a framework, and exposes its application
48 programming interface to all kernel space components such as subsystems and
49 modules. For common use cases of DAMON, DAMON subsystem provides kernel
50 modules that built on top of the core layer using the API, which can be easily
51 used by the user space end users.
57 The monitoring operations are defined in two parts:
59 1. Identification of the monitoring target address range for the address space.
60 2. Access check of specific address range in the target space.
62 DAMON currently provides the implementations of the operations for the physical
63 and virtual address spaces. Below two subsections describe how those work.
66 VMA-based Target Address Range Construction
67 -------------------------------------------
69 This is only for the virtual address space monitoring operations
70 implementation. That for the physical address space simply asks users to
71 manually set the monitoring target address ranges.
73 Only small parts in the super-huge virtual address space of the processes are
74 mapped to the physical memory and accessed. Thus, tracking the unmapped
75 address regions is just wasteful. However, because DAMON can deal with some
76 level of noise using the adaptive regions adjustment mechanism, tracking every
77 mapping is not strictly required but could even incur a high overhead in some
78 cases. That said, too huge unmapped areas inside the monitoring target should
79 be removed to not take the time for the adaptive mechanism.
81 For the reason, this implementation converts the complex mappings to three
82 distinct regions that cover every mapped area of the address space. The two
83 gaps between the three regions are the two biggest unmapped areas in the given
84 address space. The two biggest unmapped areas would be the gap between the
85 heap and the uppermost mmap()-ed region, and the gap between the lowermost
86 mmap()-ed region and the stack in most of the cases. Because these gaps are
87 exceptionally huge in usual address spaces, excluding these will be sufficient
88 to make a reasonable trade-off. Below shows this in detail::
91 <BIG UNMAPPED REGION 1>
92 <uppermost mmap()-ed region>
93 (small mmap()-ed regions and munmap()-ed regions)
94 <lowermost mmap()-ed region>
95 <BIG UNMAPPED REGION 2>
99 PTE Accessed-bit Based Access Check
100 -----------------------------------
102 Both of the implementations for physical and virtual address spaces use PTE
103 Accessed-bit for basic access checks. Only one difference is the way of
104 finding the relevant PTE Accessed bit(s) from the address. While the
105 implementation for the virtual address walks the page table for the target task
106 of the address, the implementation for the physical address walks every page
107 table having a mapping to the address. In this way, the implementations find
108 and clear the bit(s) for next sampling target address and checks whether the
109 bit(s) set again after one sampling period. This could disturb other kernel
110 subsystems using the Accessed bits, namely Idle page tracking and the reclaim
111 logic. DAMON does nothing to avoid disturbing Idle page tracking, so handling
112 the interference is the responsibility of sysadmins. However, it solves the
113 conflict with the reclaim logic using ``PG_idle`` and ``PG_young`` page flags,
114 as Idle page tracking does.
124 Below four sections describe each of the DAMON core mechanisms and the five
125 monitoring attributes, ``sampling interval``, ``aggregation interval``,
126 ``update interval``, ``minimum number of regions``, and ``maximum number of
130 Access Frequency Monitoring
131 ~~~~~~~~~~~~~~~~~~~~~~~~~~~
133 The output of DAMON says what pages are how frequently accessed for a given
134 duration. The resolution of the access frequency is controlled by setting
135 ``sampling interval`` and ``aggregation interval``. In detail, DAMON checks
136 access to each page per ``sampling interval`` and aggregates the results. In
137 other words, counts the number of the accesses to each page. After each
138 ``aggregation interval`` passes, DAMON calls callback functions that previously
139 registered by users so that users can read the aggregated results and then
140 clears the results. This can be described in below simple pseudo-code::
143 for page in monitoring_target:
145 nr_accesses[page] += 1
146 if time() % aggregation_interval == 0:
147 for callback in user_registered_callbacks:
148 callback(monitoring_target, nr_accesses)
149 for page in monitoring_target:
150 nr_accesses[page] = 0
151 sleep(sampling interval)
153 The monitoring overhead of this mechanism will arbitrarily increase as the
154 size of the target workload grows.
157 Region Based Sampling
158 ~~~~~~~~~~~~~~~~~~~~~
160 To avoid the unbounded increase of the overhead, DAMON groups adjacent pages
161 that assumed to have the same access frequencies into a region. As long as the
162 assumption (pages in a region have the same access frequencies) is kept, only
163 one page in the region is required to be checked. Thus, for each ``sampling
164 interval``, DAMON randomly picks one page in each region, waits for one
165 ``sampling interval``, checks whether the page is accessed meanwhile, and
166 increases the access frequency of the region if so. Therefore, the monitoring
167 overhead is controllable by setting the number of regions. DAMON allows users
168 to set the minimum and the maximum number of regions for the trade-off.
170 This scheme, however, cannot preserve the quality of the output if the
171 assumption is not guaranteed.
174 Adaptive Regions Adjustment
175 ~~~~~~~~~~~~~~~~~~~~~~~~~~~
177 Even somehow the initial monitoring target regions are well constructed to
178 fulfill the assumption (pages in same region have similar access frequencies),
179 the data access pattern can be dynamically changed. This will result in low
180 monitoring quality. To keep the assumption as much as possible, DAMON
181 adaptively merges and splits each region based on their access frequency.
183 For each ``aggregation interval``, it compares the access frequencies of
184 adjacent regions and merges those if the frequency difference is small. Then,
185 after it reports and clears the aggregated access frequency of each region, it
186 splits each region into two or three regions if the total number of regions
187 will not exceed the user-specified maximum number of regions after the split.
189 In this way, DAMON provides its best-effort quality and minimal overhead while
190 keeping the bounds users set for their trade-off.
196 By analyzing the monitoring results, users can also find how long the current
197 access pattern of a region has maintained. That could be used for good
198 understanding of the access pattern. For example, page placement algorithm
199 utilizing both the frequency and the recency could be implemented using that.
200 To make such access pattern maintained period analysis easier, DAMON maintains
201 yet another counter called ``age`` in each region. For each ``aggregation
202 interval``, DAMON checks if the region's size and access frequency
203 (``nr_accesses``) has significantly changed. If so, the counter is reset to
204 zero. Otherwise, the counter is increased.
207 Dynamic Target Space Updates Handling
208 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
210 The monitoring target address range could dynamically changed. For example,
211 virtual memory could be dynamically mapped and unmapped. Physical memory could
214 As the changes could be quite frequent in some cases, DAMON allows the
215 monitoring operations to check dynamic changes including memory mapping changes
216 and applies it to monitoring operations-related data structures such as the
217 abstracted monitoring target memory area only for each of a user-specified time
218 interval (``update interval``).
221 .. _damon_design_damos:
226 One common purpose of data access monitoring is access-aware system efficiency
227 optimizations. For example,
229 paging out memory regions that are not accessed for more than two minutes
233 using THP for memory regions that are larger than 2 MiB and showing a high
234 access frequency for more than one minute.
236 One straightforward approach for such schemes would be profile-guided
237 optimizations. That is, getting data access monitoring results of the
238 workloads or the system using DAMON, finding memory regions of special
239 characteristics by profiling the monitoring results, and making system
240 operation changes for the regions. The changes could be made by modifying or
241 providing advice to the software (the application and/or the kernel), or
242 reconfiguring the hardware. Both offline and online approaches could be
245 Among those, providing advice to the kernel at runtime would be flexible and
246 effective, and therefore widely be used. However, implementing such schemes
247 could impose unnecessary redundancy and inefficiency. The profiling could be
248 redundant if the type of interest is common. Exchanging the information
249 including monitoring results and operation advice between kernel and user
250 spaces could be inefficient.
252 To allow users to reduce such redundancy and inefficiencies by offloading the
253 works, DAMON provides a feature called Data Access Monitoring-based Operation
254 Schemes (DAMOS). It lets users specify their desired schemes at a high
255 level. For such specifications, DAMON starts monitoring, finds regions having
256 the access pattern of interest, and applies the user-desired operation actions
257 to the regions as soon as found.
260 .. _damon_design_damos_action:
265 The management action that the users desire to apply to the regions of their
266 interest. For example, paging out, prioritizing for next reclamation victim
267 selection, advising ``khugepaged`` to collapse or split, or doing nothing but
268 collecting statistics of the regions.
270 The list of supported actions is defined in DAMOS, but the implementation of
271 each action is in the DAMON operations set layer because the implementation
272 normally depends on the monitoring target address space. For example, the code
273 for paging specific virtual address ranges out would be different from that for
274 physical address ranges. And the monitoring operations implementation sets are
275 not mandated to support all actions of the list. Hence, the availability of
276 specific DAMOS action depends on what operations set is selected to be used
279 Applying an action to a region is considered as changing the region's
280 characteristics. Hence, DAMOS resets the age of regions when an action is
284 .. _damon_design_damos_access_pattern:
286 Target Access Pattern
287 ~~~~~~~~~~~~~~~~~~~~~
289 The access pattern of the schemes' interest. The patterns are constructed with
290 the properties that DAMON's monitoring results provide, specifically the size,
291 the access frequency, and the age. Users can describe their access pattern of
292 interest by setting minimum and maximum values of the three properties. If a
293 region's three properties are in the ranges, DAMOS classifies it as one of the
294 regions that the scheme is having an interest in.
297 .. _damon_design_damos_quotas:
302 DAMOS upper-bound overhead control feature. DAMOS could incur high overhead if
303 the target access pattern is not properly tuned. For example, if a huge memory
304 region having the access pattern of interest is found, applying the scheme's
305 action to all pages of the huge region could consume unacceptably large system
306 resources. Preventing such issues by tuning the access pattern could be
307 challenging, especially if the access patterns of the workloads are highly
310 To mitigate that situation, DAMOS provides an upper-bound overhead control
311 feature called quotas. It lets users specify an upper limit of time that DAMOS
312 can use for applying the action, and/or a maximum bytes of memory regions that
313 the action can be applied within a user-specified time duration.
316 .. _damon_design_damos_quotas_prioritization:
321 A mechanism for making a good decision under the quotas. When the action
322 cannot be applied to all regions of interest due to the quotas, DAMOS
323 prioritizes regions and applies the action to only regions having high enough
324 priorities so that it will not exceed the quotas.
326 The prioritization mechanism should be different for each action. For example,
327 rarely accessed (colder) memory regions would be prioritized for page-out
328 scheme action. In contrast, the colder regions would be deprioritized for huge
329 page collapse scheme action. Hence, the prioritization mechanisms for each
330 action are implemented in each DAMON operations set, together with the actions.
332 Though the implementation is up to the DAMON operations set, it would be common
333 to calculate the priority using the access pattern properties of the regions.
334 Some users would want the mechanisms to be personalized for their specific
335 case. For example, some users would want the mechanism to weigh the recency
336 (``age``) more than the access frequency (``nr_accesses``). DAMOS allows users
337 to specify the weight of each access pattern property and passes the
338 information to the underlying mechanism. Nevertheless, how and even whether
339 the weight will be respected are up to the underlying prioritization mechanism
343 .. _damon_design_damos_watermarks:
348 Conditional DAMOS (de)activation automation. Users might want DAMOS to run
349 only under certain situations. For example, when a sufficient amount of free
350 memory is guaranteed, running a scheme for proactive reclamation would only
351 consume unnecessary system resources. To avoid such consumption, the user would
352 need to manually monitor some metrics such as free memory ratio, and turn
353 DAMON/DAMOS on or off.
355 DAMOS allows users to offload such works using three watermarks. It allows the
356 users to configure the metric of their interest, and three watermark values,
357 namely high, middle, and low. If the value of the metric becomes above the
358 high watermark or below the low watermark, the scheme is deactivated. If the
359 metric becomes below the mid watermark but above the low watermark, the scheme
360 is activated. If all schemes are deactivated by the watermarks, the monitoring
361 is also deactivated. In this case, the DAMON worker thread only periodically
362 checks the watermarks and therefore incurs nearly zero overhead.
365 .. _damon_design_damos_filters:
370 Non-access pattern-based target memory regions filtering. If users run
371 self-written programs or have good profiling tools, they could know something
372 more than the kernel, such as future access patterns or some special
373 requirements for specific types of memory. For example, some users may know
374 only anonymous pages can impact their program's performance. They can also
375 have a list of latency-critical processes.
377 To let users optimize DAMOS schemes with such special knowledge, DAMOS provides
378 a feature called DAMOS filters. The feature allows users to set an arbitrary
379 number of filters for each scheme. Each filter specifies the type of target
380 memory, and whether it should exclude the memory of the type (filter-out), or
381 all except the memory of the type (filter-in).
383 Currently, anonymous page, memory cgroup, address range, and DAMON monitoring
384 target type filters are supported by the feature. Some filter target types
385 require additional arguments. The memory cgroup filter type asks users to
386 specify the file path of the memory cgroup for the filter. The address range
387 type asks the start and end addresses of the range. The DAMON monitoring
388 target type asks the index of the target from the context's monitoring targets
389 list. Hence, users can apply specific schemes to only anonymous pages,
390 non-anonymous pages, pages of specific cgroups, all pages excluding those of
391 specific cgroups, pages in specific address range, pages in specific DAMON
392 monitoring targets, and any combination of those.
394 To handle filters efficiently, the address range and DAMON monitoring target
395 type filters are handled by the core layer, while others are handled by
396 operations set. If a memory region is filtered by a core layer-handled filter,
397 it is not counted as the scheme has tried to the region. In contrast, if a
398 memory regions is filtered by an operations set layer-handled filter, it is
399 counted as the scheme has tried. The difference in accounting leads to changes
403 Application Programming Interface
404 ---------------------------------
406 The programming interface for kernel space data access-aware applications.
407 DAMON is a framework, so it does nothing by itself. Instead, it only helps
408 other kernel components such as subsystems and modules building their data
409 access-aware applications using DAMON's core features. For this, DAMON exposes
410 its all features to other kernel components via its application programming
411 interface, namely ``include/linux/damon.h``. Please refer to the API
412 :doc:`document </mm/damon/api>` for details of the interface.
418 Because the core of DAMON is a framework for kernel components, it doesn't
419 provide any direct interface for the user space. Such interfaces should be
420 implemented by each DAMON API user kernel components, instead. DAMON subsystem
421 itself implements such DAMON API user modules, which are supposed to be used
422 for general purpose DAMON control and special purpose data access-aware system
423 operations, and provides stable application binary interfaces (ABI) for the
424 user space. The user space can build their efficient data access-aware
425 applications using the interfaces.
428 General Purpose User Interface Modules
429 --------------------------------------
431 DAMON modules that provide user space ABIs for general purpose DAMON usage in
434 DAMON user interface modules, namely 'DAMON sysfs interface' and 'DAMON debugfs
435 interface' are DAMON API user kernel modules that provide ABIs to the
436 user-space. Please note that DAMON debugfs interface is currently deprecated.
438 Like many other ABIs, the modules create files on sysfs and debugfs, allow
439 users to specify their requests to and get the answers from DAMON by writing to
440 and reading from the files. As a response to such I/O, DAMON user interface
441 modules control DAMON and retrieve the results as user requested via the DAMON
442 API, and return the results to the user-space.
444 The ABIs are designed to be used for user space applications development,
445 rather than human beings' fingers. Human users are recommended to use such
446 user space tools. One such Python-written user space tool is available at
447 Github (https://github.com/awslabs/damo), Pypi
448 (https://pypistats.org/packages/damo), and Fedora
449 (https://packages.fedoraproject.org/pkgs/python-damo/damo/).
451 Please refer to the ABI :doc:`document </admin-guide/mm/damon/usage>` for
452 details of the interfaces.
455 Special-Purpose Access-aware Kernel Modules
456 -------------------------------------------
458 DAMON modules that provide user space ABI for specific purpose DAMON usage.
460 DAMON sysfs/debugfs user interfaces are for full control of all DAMON features
461 in runtime. For each special-purpose system-wide data access-aware system
462 operations such as proactive reclamation or LRU lists balancing, the interfaces
463 could be simplified by removing unnecessary knobs for the specific purpose, and
464 extended for boot-time and even compile time control. Default values of DAMON
465 control parameters for the usage would also need to be optimized for the
468 To support such cases, yet more DAMON API user kernel modules that provide more
469 simple and optimized user space interfaces are available. Currently, two
470 modules for proactive reclamation and LRU lists manipulation are provided. For
471 more detail, please read the usage documents for those
472 (:doc:`/admin-guide/mm/damon/reclaim` and
473 :doc:`/admin-guide/mm/damon/lru_sort`).