The Linux Kernel/Multitasking/CPU

Interrupts An interrupt is a signal to the processor emitted by hardware or software indicating an event that needs immediate attention. An interrupt alerts the processor to a high-priority condition requiring the interruption of the current code the processor is executing. The processor responds by suspending its current activities, saving its state, and executing a function called an interrupt handler (or an interrupt service routine, ISR) to deal with the event. This interruption is temporary, and, after the interrupt handler finishes, the processor resumes normal activities. There are two types of interrupts: hardware interrupts and software interrupts. Hardware interrupts are used by devices to communicate that they require attention from the operating system. For example, pressing a key on the keyboard or moving the mouse triggers hardware interrupts that cause the processor to read the keystroke or mouse position. Unlike the software type, hardware interrupts are asynchronous and can occur in the middle of instruction execution, requiring additional care in programming. The act of initiating a hardware interrupt is referred to as an interrupt request - IRQ ↪ Interrupt descriptor table IDT ↪ common_interrupt on x86. A software interrupt is caused either by an exceptional condition in the processor itself, or a special instruction in the instruction set which causes an interrupt when it is executed. The former is often called a Trap (computing) trap (⚙️ do_trap) or exception and is used for errors or events occurring during program execution that are exceptional enough that they cannot be handled within the program itself. For example, if the processor's arithmetic logic unit is commanded to divide a number by zero, this impossible demand will cause a divide-by-zero exception (⚙️ X86_TRAP_DE), perhaps causing the computer to abandon the calculation or display an error message. Software interrupt instructions function similarly to subroutine calls and are used for a variety of purposes, such as to request services from low-level system software such as device drivers. For example, computers often use software interrupt instructions to communicate with the disk controller to request data be read or written to the disk. Each interrupt has its own interrupt handler. The number of hardware interrupts is limited by the number of interrupt request (IRQ) lines to the processor, but there may be hundreds of different software interrupts. ⚲ API /proc/interrupts 1 irqtop – utility to display kernel interrupt information irqbalance – distribute hardware interrupts across processors on a multiprocessor system There are many ways to request ISR, two of them devm_request_threaded_irq – preferable, managed device with threaded ISR devm_request_irq – managed version request_irq request_threaded_irq free_irq linux/interrupt.h – main interrupt support header irqaction – contains handler functions rust/kernel/irq.rs – Rust bindings linux/irq.h irq_data linux/irqflags.h irqs_disabled local_irq_save ... local_irq_disable ... linux/irqdesc.h irq_desc linux/irqdomain.h irq_domain – hardware interrupt number translation object irq_domain_get_irq_data linux/msi.h – Message Signaled Interrupts msi_desc Structure of structures: irq_desc is container of irq_data irq – interrupt number irq_common_data list of irqaction ⚙️ Internals kernel/irq/settings.h – IRQ descriptor status flags kernel/irq – generic IRQ handling kernel/irq/internals.h – IRQ subsystem internal functions ls /sys/kernel/debug/irq/domains/ x86_vector_domain, x86_vector_domain_ops irq_chip load_idt – load Interrupt Descriptor Table arch/x86/include/asm/idtentry.h – interrupt entry/exit definitions arch/x86/include/asm/hw_irq.h – x86 hardware IRQ definitions irq_entries_start – hardware IRQ entry stubs arch/x86/kernel/irq.c – x86 interrupt handling common_interrupt – handles all normal device IRQs on x86 arch/x86/kernel/apic – Advanced Programmable Interrupt Controller APIC arch/x86/kernel/apic/apic.c – local APIC core (timer, calibration, spurious interrupt) arch/x86/kernel/apic/io_apic.c – IOAPIC I/O APIC, routes device IRQs to CPUs arch/x86/kernel/apic/x2apic_cluster.c – x2APIC (>256 CPUs) arch/x86/kernel/apic/ipi.c – Inter-processor interrupt IPI, see IPI arch/x86/kernel/apic/hw_nmi.c – hardware NMI watchdog via APIC arch/x86/kernel/apic/vector.c – IRQ vector allocation arch/x86/kernel/apic/msi.c – Message Signaled Interrupts MSI via APIC 📖 References IRQs core-api/irq The irq_domain interrupt number mapping library core-api/irq/irq-domain.html Linux generic IRQ handling core-api/genericirq.html Message Signaled Interrupts: The MSI Driver Guide PCI/msi-howto.html Lock types and their rules locking/locktypes.html Hard IRQ Context kernel-hacking/locking.html#hard-irq-context Interrupts 👁 Examples dummy_irq_chip – dummy interrupt chip implementation lib/locking-selftest.c – locking correctness self-tests IRQ affinity ⚲ API /proc/irq/default_smp_affinity /proc/irq/*/smp_affinity and /proc/irq/*/smp_affinity_list Common types and functions: struct irq_affinity – description for automatic irq affinity assignments, see devm_platform_get_irqs_affinity struct irq_affinity_desc – interrupt affinity descriptor, see irq_update_affinity_desc, irq_create_affinity_masks irq_set_affinity irq_get_affinity_mask irq_can_set_affinity irq_set_affinity_hint irqd_affinity_is_managed irq_data_get_affinity_mask irq_data_get_effective_affinity_mask irq_data_update_effective_affinity irq_set_affinity_notifier irq_affinity_notify irq_chip_set_affinity_parent irq_set_vcpu_affinity 🛠️ Utilities irqbalance – distributes hardware interrupts across CPUs 📖 References SMP IRQ affinity core-api/irq/irq-affinity.html IRQ affinity, LF managed_irq kernel parameter, @LKML irqaffinity kernel parameter, @LKML Non-maskable interrupts ⚲ API linux/nmi.h in_nmi touch_nmi_watchdog ... trace/events/nmi.h arch/x86/include/asm/nmi.h – x86 NMI handler registration register_nmi_handler unregister_nmi_handler ⚙️ Internals arch/x86/kernel/nmi.c – x86 NMI handler arch/x86/kernel/apic/hw_nmi.c – hardware NMI watchdog via APIC arch/x86/kernel/nmi_selftest.c – NMI IPI self-tests 📖 References NMI Trace Events trace/events-nmi.html 📚 Further reading Non-maskable interrupt handler (NMI) ... 📚 Further reading about interrupts IDT – Interrupt descriptor table LGDT/LIDT – Load Global/Interrupt Descriptor Table Register asm instruction Deferred works Scheduler context kthread work This framework simplifies the use of kernel kthreads. A kthread_work item can be queued with kthread_queue_work and flushed using kthread_flush_work. All queued kthread_work items are processed by a dedicated kernel thread executing the kthread_worker_fn function. ⚲ API kthread_work – contains kthread_work_func_t to execute kthread_init_work kthread_flush_work kthread_worker – links a kthread_work and a task kthread_run_worker – creates and wakes a kthread worker kthread_create_worker kthread_flush_worker kthread_destroy_worker kthread_queue_work – queues a kthread_work on a kthread_worker ⚙️ Internals __kthread_create_worker_on_node kthread_worker_fn – executes work's function 👁 Example usages watchdog_kworker, pwq_release_worker, pump_messages Threaded IRQ ⚲ API devm_request_threaded_irq, request_threaded_irq ISR should return IRQ_WAKE_THREAD to run thread function ⚙️ Internals setup_irq_thread, irq_thread kernel/irq/manage.c – IRQ request, free and affinity management 📖 References request_threaded_irq core-api/genericirq.html#c.request_threaded_irq Work and workqueue Generic async execution with shared worker pool. ⚲ API PF_WQ_WORKER – workqueue worker process flag linux/workqueue.h, linux/workqueue_types.h rust/kernel/workqueue.rs – Rust bindings work_struct, INIT_WORK delayed_work, INIT_DELAYED_WORK, cancel_delayed_work_sync workqueue_struct devm_alloc_workqueue alloc_workqueue destroy_workqueue schedule_work – queues work on the system workqueue ↯ queue_work – queues work on a workqueue queue_work_on – on a specific CPU show_all_workqueues show_one_workqueue system_power_efficient_wq ... 👁 Example usage samples/ftrace/sample-trace-array.c ⚙️ Internals kernel/workqueue.c – generic async execution with shared worker pool workqueue_init create_worker worker_thread process_one_work format_worker_id – names kworker kthreads pool_workqueue worker_pool 📖 References Concurrency Managed Workqueue core-api/workqueue.html LKMPG: Work queues Interrupt context linux/irq_work.h – run callbacks from hardirq or NMI context where normal work queues can't be used irq_work_queue – enqueue work on the current CPU, triggers IRQ_WORK_VECTOR IPI if needed irq_work_sync – wait for completion kernel/irq_work.c – irq_work implementation 👁 samples/trace_printk/trace-printk.c Timers Not to be confused with POSIX timer – userspace API, watchdog timer – hang detection, local APIC timer – per-CPU hardware, drives the tick. timer softirq Not to be confused with nanosecond-resolution high-resolution timer. This timer is a softirq for periodical tasks with jiffies resolution. Also known as kernel timer or timer wheel. ⚲ API linux/timer.h timer_list, DEFINE_TIMER, timer_setup mod_timer — sets expiration time in jiffies. timer_delete ⚙️ Internals kernel/time/timer.c – kernel internal timers timer_bases – per-CPU array of timer_base (timing wheel): BASE_LOCAL – pinned to this CPU (TIMER_PINNED) BASE_GLOBAL – can be run remotely by migrator via timer_expire_remote BASE_DEF – deferrable, can be delayed indefinitely run_timer_softirq – TIMER_SOFTIRQ handler __run_timer_base – iterate expired timers and call callbacks 👁 Examples input_enable_softrepeat and input_start_autorepeat 📚 References Time and timer routines driver-api/basics.html#time-and-timer-routines timer_init_key ... driver-api/basics.html#c.timer_init_key High-resolution timer Not to be confused with jiffies-resolution timer softirq. ⚲ API /proc/timer_list /proc/sys/kernel/timer_migration admin-guide/sysctl/kernel.html#timer-migration – enable/disable hrtimer migration linux/hrtimer_defs.h linux/hrtimer.h hrtimer, hrtimer.function — callback hrtimer_setup hrtimer_start — starts a timer with nanosecond resolution hrtimer_cancel 👁 Examples alarm_init, watchdog_enable ⚙️ Internals CONFIG_HIGH_RES_TIMERS kernel/time/tick-internal.h – tick and timer internal definitions hrtimer_bases – per-CPU hrtimer_cpu_base, contains 8 clock bases: hrtimer_clock_base – timerqueue per clock type per CPU hrtimer_base_type – MONOTONIC, REALTIME, BOOTTIME, TAI (× hardirq + softirq = 8) kernel/time/hrtimer.c – high-resolution timer implementation hrtimer_interrupt – called from clock event device, expires hardirq timers __hrtimer_run_queues – walk timerqueue, call expired callbacks hrtimer_run_softirq – HRTIMER_SOFTIRQ handler, expires HRTIMER_MODE_SOFT timers __hrtimer_run_queues kernel/time/itimer.c – interval timers kernel/time/timer_list.c – /proc/timer_list debugfs 📚 HR timers references High-resolution timers driver-api/basics.html#high-resolution-timers hrtimers - subsystem for high-resolution kernel timers timers/hrtimers.html high resolution timers and dynamic ticks design notes timers/highres.html Tick and NO_HZ The tick is the periodic timer interrupt that drives scheduling, timekeeping, and RCU. The NO_HZ subsystem reduces or eliminates tick interrupts to save power and reduce latency. ⚲ API /sys/devices/system/cpu/nohz_full – active nohz_full CPUs nohz_full= boot parameter – list of CPUs to run tickless linux/tick.h – tick control API tick_nohz_full_running – true if any CPU runs tickless tick_nohz_full_cpu – true if given CPU runs tickless enum tick_dep_bits – reasons a CPU/task/signal may need the tick (posix timer, perf events, sched, unstable clock, RCU, RCU expedited) tick_nohz_dep_set ... – modify a global tick dependency bit, used by perf for unstable clocks; kicks all full-nohz CPUs to restart their tick sched_tick_remote – remote CPU updates scheduler stats (load, timeslice) for nohz_full CPUs via delayed work /proc/sys/kernel/timer_migration admin-guide/sysctl/kernel.html#timer-migration – enable/disable hrtimer migration 🪛 Compile-time configurations (kernel/time/Kconfig) CONFIG_HZ_PERIODIC – constant-rate tick, always firing CONFIG_NO_HZ_IDLE – stop tick when CPU is idle (default on most distros) CONFIG_NO_HZ_FULL – stop tick even when running a single task, for RT and latency-sensitive workloads, see CPU isolation ⚙️ Internals kernel/time/tick-common.c – periodic tick setup kernel/time/tick-oneshot.c – oneshot mode for hrtimers and NO_HZ kernel/time/tick-sched.c – NO_HZ tick scheduling logic tick_nohz_handler – hrtimer callback driving the tick, rearmed at TICK_NSEC until idle/full-dynticks stops it tick_nohz_init – enable full nohz on boot if requested and supported kernel/time/tick-broadcast.c – broadcast tick for CPUs in deep idle where local timer stops (C3+) tick_dependency: sched_update_tick_dependency – modifies the TICK_DEP_BIT_SCHED scheduler tick dependency bit in tick_dep_mask sched_can_stop_tick – true if a runqueue has at most one runnable task and can go tickless can_stop_full_tick – decide if a full-dynticks CPU can stop its tick, checking CPU online state and the tick_dep_mask tick dependencies check_tick_dependency – test one tick dependency mask (global, CPU, task, signal) for a pending reason to keep the tick running kernel/time/timer_migration.h, kernel/time/timer_migration.c – migrate timer softirq (BASE_GLOBAL) from idle to busy CPUs via hierarchical groups tmigr_cpu_activate – CPU joins migration hierarchy on online tmigr_handle_remote – migrator CPU fires expired timers on behalf of idle CPUs timer_expire_remote → __run_timer_base tmigr_update_events – propagate timer events up/down the hierarchy /sys/kernel/debug/tracing/events/timer_migration/ – tracepoints /proc/sys/kernel/timer_migration admin-guide/sysctl/kernel.html#timer-migration – enable/disable hrtimer migration timers_migration_enabled Used in: switch_hrtimer_base – move hrtimer to target CPU base at enqueue time get_target_base – select hrtimer_cpu_base, migrates to busy CPU if not pinned get_nohz_timer_target – pick busy CPU hrtimer_prefer_local 📖 References NO_HZ: Reducing Scheduling-Clock Ticks timers/no_hz.html Tickless kernel, RT wiki high resolution timers and dynamic ticks design notes timers/highres.html Clock infrastructure Not to be confused with drivers/clk framework – SoC Clock signal clock tree management. Clocksources read current time (passive counters). Clock event devices fire interrupts at a programmed time (active). Together they are the foundation for timekeeping, timers, tick, and NO_HZ. ⚲ Clocksources – "what time is it?" /sys/devices/system/clocksource/clocksource0/current_clocksource /sys/devices/system/clocksource/clocksource0/available_clocksource Time Stamp Counter TSC – fastest, per-CPU, x86 High Precision Event Timer HPET – system-wide, MMIO ACPI PM timer – never stops in any C-state, slow linux/clocksource.h – clocksource, clocksource_register_hz ⚲ Clock event devices – "wake me at time X" linux/clockchips.h – clock_event_device, clockevents_config_and_register Local Advanced Programmable Interrupt Controller APIC timer – per-CPU, fastest, stops in C3+, local_apic_timer_interrupt High Precision Event Timer HPET – system-wide, used as broadcast fallback ⚙️ Internals kernel/time/clocksource.c – clocksource selection and watchdog kernel/time/clockevents.c – clock event device framework kernel/time/jiffies.c – jiffies clocksource (fallback) ... 📖 Timers references Timers timers Better CPU selection for timer expiration Tasklet tasklet is a softirq, for time critical operations ⚲ API is deprecated in favor of threaded IRQs: devm_request_threaded_irq tasklet_struct, tasklet_init, tasklet_schedule ⚙️ Internals: tasklet_action_common HI_SOFTIRQ, TASKLET_SOFTIRQ Softirq softirq is internal system facility and should not be used directly. Use tasklet or threaded IRQs ⚲ API linux/interrupt.h cat /proc/softirqs open_softirq registers softirq_action raise_softirq tasklet_schedule __tasklet_schedule ⚙️ Internals kernel/softirq.c – software interrupt handling __do_softirq – main softirq processing loop 📖 References Introduction to deferred interrupts (Softirq, Tasklets and Workqueues) Timers and time management Deferred work, linux-kernel-labs Chapter 7. Time, Delays, and Deferred Work CPU specific 🖱️ GUI tuna – program for tuning running processes ⚲ API cat /proc/cpuinfo /sys/devices/system/cpu/ /sys/devices/system/node/ /sys/cpu/ /sys/fs/cgroup/cpu/ grep -i cpu /proc/self/status rdmsr – tool for reading CPU machine specific registers (MSR) 1 lscpu – display information about the CPU architecture linux/arch_topology.h – arch specific cpu topology information linux/cpu.h – generic cpu definition linux/cpu_cooling.h linux/cpu_pm.h linux/cpufeature.h arch/x86/include/asm/cpufeature.h – x86 CPU feature detection linux/peci-cpu.h linux/sched/cputime.h – cputime accounting APIs linux/clk.h – interfaces for managing hardware clocks in device drivers ⚙️ Internals drivers/base/cpu.c – CPU driver model subsystem cpu_dev_init Cache linux/cacheflush.h arch/x86/include/asm/cacheflush.h: clflush_cache_range linux/cache.h – cache line alignment macros ____cacheline_aligned – align to cache line boundary to avoid false sharing __cacheline_aligned – same, plus place in .data..cacheline_aligned section ____cacheline_aligned_in_smp – same, but no-op on uniprocessor builds __read_mostly – place in a cache-friendly section for rarely written data __ro_after_init – writable during init, read-only after boot (security hardening) DEFINE_PER_CPU_CACHE_HOT – per-CPU variable in .data..hot section, used for current_task, __preempt_count L1_CACHE_BYTES, SMP_CACHE_BYTES – 64 bytes typically arch/x86/include/asm/cache.h – x86 cache line size definitions 🔨 stress-ng --cache ⚙️ Internals arch/x86/mm/pat/set_memory.c – page attribute table memory type control arch/x86/kernel/cpu/mtrr/ – memory type range register driver 📚 Further reading MTRR – Memory type range register CPU cache Symmetric_multiprocessing SMP This chapter is about multiprocessing and Multi-core processor multi-core aspects of Linux kernel. Key concepts and features of Linux SMP include: Symmetry: In an SMP system, all processors are considered the same without hardware hierarchy in contradiction to use of coprocessors. Load balancing: The Linux kernel employs load balancing mechanisms to distribute tasks evenly among available CPU cores. This prevents any one core from becoming overwhelmed while others remain underutilized. Parallelism: SMP enables parallel processing, where multiple threads or processes can execute simultaneously on different CPU cores. This can significantly improve the execution speed of applications that are designed to take advantage of multiple threads. Thread scheduling: The Linux kernel scheduler is responsible for determining which threads or processes run on which CPU cores and for how long. It aims to optimize performance by minimizing contention and maximizing CPU utilization. Shared memory: In an SMP system, all CPU cores typically share the same physical memory space. This allows processes and threads running on different cores to communicate and share data more efficiently. NUMA – Non-Uniform Memory Access: In larger SMP systems, memory access times might not be uniform due to the physical arrangement of memory banks and processors. Linux has mechanisms to handle NUMA architectures efficiently, allowing processes to be scheduled on CPUs closer to their associated memory. Cache coherency: SMP systems require mechanisms to ensure that all CPU cores have consistent views of memory. Cache coherency protocols ensure that changes made to shared memory locations are correctly propagated to all cores. Scalability: SMP systems can be scaled up to include more CPU cores, enhancing the overall computing power of the system. However, as the number of cores increases, challenges related to memory access, contention, and communication between cores may arise. Kernel and user space: Linux applications running in user space can take advantage of SMP without needing to be aware of the underlying hardware details. The kernel handles the management of CPU cores and resource allocation. ⚲ API ps -PLe – lists threads with processor that the thread last executed on (the third column PSR). 2 getcpu – determine CPU and NUMA node on which the calling thread is running 8 chcpu – configure CPUs 3 CPU_SET – macros for manipulating CPU sets linux/smp.h The most commonly used functions: crash_smp_send_stop – halts all CPUs except the calling one in a crash context get_cpu – disables preemption and returns the current processor ID panic_smp_self_stop – stops the local CPU during a panic while ensuring others halt put_cpu – re-enables preemption after a previous get_cpu call raw_smp_processor_id – returns the current CPU ID without preemption safety setup_max_cpus – sets up the maximum number of CPUs to be brought online smp_init – initializes core SMP structures and state during boot smp_prepare_boot_cpu – prepares the boot CPU during early SMP initialization smp_prepare_cpus – prepares all CPUs for booting before secondary CPUs are started smp_processor_id – returns the ID of the current CPU with preemption checks smp_send_stop – stops all other CPUs in response to critical events wake_up_all_idle_cpus – wakes all idle CPUs to ensure prompt task execution linux/cpu.h linux/group_cpus.h: group_cpus_evenly – groups all CPUs evenly per NUMA/CPU locality asm-generic/percpu.h linux/percpu-defs.h – basic definitions for percpu areas this_cpu_ptr linux/percpu.h linux/percpu-refcount.h linux/percpu-rwsem.h linux/preempt.h migrate_disable, migrate_enable /sys/bus/cpu per CPU local_lock linux/topology.h – cpu_to_node, numa_node_id arch/x86/include/asm/topology.h – x86 NUMA and CPU topology See also NUMA memory section ⚙️ Internals boot_cpu_init activates the first CPU smp_prepare_cpus initializes rest CPUs during boot cpu_number CONFIG_SMP CONFIG_NUMA trace/events/percpu.h drivers/base/cpu.c – CPU driver model subsystem support kernel/cpu.c – CPU hotplug state machine smpboot linux/smpboot.h kernel/smpboot.c – common SMP CPU bringup/teardown arch/x86/kernel/smpboot.c – x86 SMP booting lib/group_cpus.c – distribute CPUs evenly across groups 🛠️ Utilities irqbalance – distributes hardware interrupts across CPUs 8 numactl – controls NUMA policy for processes or shared memory 📖 References Per-CPU Data kernel-hacking/locking.html#per-cpu-data How CPU topology info is exported via sysfs admin-guide/cputopology.html 📚 Further reading Functionalities of the scheduler TuneD plugin tuned-adm – command line tool for switching between different tuning profiles Inter-processor interrupt IPI An inter-processor interrupt is a signal sent from one CPU to another to request an action such as rescheduling, TLB flush, or function execution. ⚲ API smp_send_reschedule – sends a reschedule IPI to a target CPU smp_call_function – invokes a function on all other CPUs via IPI smp_call_function_many – on a specified set of CPUs smp_call_function_single – on a single target CPU smp_call_function_single_async – asynchronous variant smp_call_on_cpu – execute on a specific CPU and wait for result on_each_cpu – run a function on all CPUs on_each_cpu_mask – on a specified set of CPUs on_each_cpu_cond_mask – conditionally on selected CPUs smp_call_func_t – callback typedef ⚙️ Internals trace/events/ipi.h kernel/irq/ipi.c – generic IPI helpers ipi_send_single, ipi_send_mask arch/x86/kernel/apic/ipi.c – x86 IPI via APIC arch/x86/include/asm/irq_vectors.h – x86 IPI vector types: RESCHEDULE_VECTOR – used by smp_send_reschedule CALL_FUNCTION_VECTOR – used by smp_call_function_many CALL_FUNCTION_SINGLE_VECTOR – used by smp_call_function_single IRQ_WORK_VECTOR – used by irq_work_queue, see Interrupt context NMI_VECTOR – used by watchdog, crash_smp_send_stop CPU affinity Affinity refers to assigning a process or thread to specific CPU cores. This helps control which CPUs execute tasks, potentially improving performance by reducing data movement between cores. It can be managed using system calls or commands. Affinity can be represented as CPU bitmask: cpumask_t or CPU affinity list: cpulist_parse. ⚲ API 1 taskset – set or retrieve a process's CPU affinity grep Cpus_allowed /proc/self/status 2 sched_setaffinity 2 sched_getaffinity – set and get a thread's CPU affinity mask ↪ sched_setaffinity set_cpus_allowed_ptr – common kernel function to change a task's affinity mask linux/cpu_rmap.h – CPU affinity reverse-map support linux/cpumask_types.h struct cpumask, cpumask_t – CPUs bitmap, can be very big cpumask_var_t – type for local cpumask variable, see alloc_cpumask_var, free_cpumask_var. linux/cpumask.h – Cpumasks provide a bitmap suitable for representing the set of CPU's in a system, one bit position per CPU number for_each_possible_cpu num_online_cpus cpumask_set_cpu cpumask_test_cpu for_each_cpu ⚙️ Internals cpus_mask – affinity of task_struct cpus_allowed – affinity of cpuset 📚 Further reading Processor affinity Affinity mask CPU hotplug CPU hotplugging in Linux refers to the ability to dynamically add or remove CPUs from the system without needing a reboot. This feature is crucial in environments requiring high availability and resource flexibility, such as data centers, virtualized systems, and systems that use power management aggressively. 🗝️ Acronyms BP – Bootstrap Processor AP – Application Processor ⚲ API /sys/devices/system/cpu/cpu*/online /sys/devices/system/cpu/cpu*/hotplug/ linux/cpu.h add_cpu ... linux/cpuhotplug.h cpuhp_state – CPU hotplug states cpuhp_setup_state ... – setups hotplug state callbacks cpuhp_setup_state_multi cpuhp_setup_state_nocalls linux/cpuhplock.h – CPU hotplug locking cpus_read_lock ... remove_cpu ... ⚙️ Internals kernel/cpu.c – CPU hotplug state machine cpuhp_state – from CPUHP_OFFLINE to CPUHP_AP_ACTIVE and CPUHP_ONLINE. cpuhp_hp_states boot_cpu_hotplug_init cpuhp_threads_init ... cpuhp_invoke_callback_range ... kernel/irq/cpuhotplug.c – IRQ migration on CPU hotunplug drivers/base/cpu.c – CPU subsystem support cpu_dev_init ... cpu_subsys_online trace/events/cpuhp.h linux/cpuhplock.h 👁️ Examples torture_onoff tools/testing/selftests/sched_ext/hotplug.c – sched_ext CPU hotplug test 📖 References CPU hotplug in the Kernel core-api/cpu_hotplug.html Introduction core-api/cpu_hotplug.html#introduction Command Line Switches core-api/cpu_hotplug.html#command-line-switches CPU maps core-api/cpu_hotplug.html#cpu-maps Using CPU hotplug core-api/cpu_hotplug.html#using-cpu-hotplug The CPU hotplug coordination core-api/cpu_hotplug.html#the-cpu-hotplug-coordination The CPU hotplug API core-api/cpu_hotplug.html#the-cpu-hotplug-api CPU hotplug state machine core-api/cpu_hotplug.html#cpu-hotplug-state-machine CPU online/offline operations core-api/cpu_hotplug.html#cpu-online-offline-operations Allocating a state core-api/cpu_hotplug.html#allocating-a-state Setup of a CPU hotplug state core-api/cpu_hotplug.html#setup-of-a-cpu-hotplug-state Removal of a CPU hotplug state core-api/cpu_hotplug.html#removal-of-a-cpu-hotplug-state Multi-Instance state instance management core-api/cpu_hotplug.html#multi-instance-state-instance-management Examples core-api/cpu_hotplug.html#examples Testing of hotplug states core-api/cpu_hotplug.html#testing-of-hotplug-states Architecture’s requirements core-api/cpu_hotplug.html#architecture-s-requirements User Space Notification core-api/cpu_hotplug.html#user-space-notification Kernel Inline Documentations Reference core-api/cpu_hotplug.html#kernel-inline-documentations-reference 🔨 stress-ng --cpu-online 📚 Further reading CONFIG_CPU_HOTPLUG_STATE_CONTROL – enables the ability to write incremental steps between "offline" and "online" states to the CPU's sysfs target file, allowing for more granular control of state transitions. target_store: cpu_up/cpu_down cpuhotplug, cpuhp @LKML CPU isolation CPU isolation ensures that specific tasks run on dedicated CPUs, reducing contention and latency. Housekeeping CPUs refer to the CPUs that are reserved for various system tasks. See hk_type. Isolated CPUs are dedicated to real-time applications, such as DPDK. ⚲ API /sys/devices/system/cpu/isolated /sys/devices/system/cpu/nohz_full /sys/fs/cgroup/cpuset.cpus.isolated /sys/fs/cgroup/.../cpuset.cpus.partition linux/sched/isolation.h hk_type – housekeeping type housekeeping_cpumask – returns CPUs available for housekeeping of a given type cpu_is_isolated linux/cpuset.h – cpuset interface 7 cpuset – confine processes to processor and memory node subsets ⚙️ Internals CONFIG_CPU_ISOLATION kernel/sched/isolation.c – housekeeping CPU management housekeeping_init, housekeeping_update CONFIG_CPUSETS kernel/cgroup/cpuset.c – cpuset cgroup controller cpuset_init cpuset_init_smp isolated_cpus partition_xcpus_add, partition_xcpus_del 📖 References CPU lists in command-line parameters admin-guide/kernel-parameters.html#cpu-lists nohz_full clears housekeeping.cpumasks for tick, wq, timer, rcu, misc, and kthread in housekeeping_nohz_full_setup isolcpus clears housekeeping.cpumasks for domain (by default), nohz, and managed_irq in housekeeping_isolcpus_setup Housekeeping core-api/housekeeping.html NO_HZ: Reducing Scheduling-Clock Ticks timers/no_hz.html CPUSETS of cgroup v2 admin-guide/cgroup-v2.html#cpuset CPUSETS of cgroup v1 admin-guide/cgroup-v1/cpusets.html 📚 Further reading CPU Isolation state of the art, LPC'23 CPU Isolation and IPI interference, LPC'25 CPU Isolation isolcpus @LKML housekeeping @LKML Explicitly Reserved CPU List, Kubernetes Documentation CPU Partitioning Scheduler Domains scheduler/sched-domains.html – the Scheduler balances CPUs (scheduling groups) within a sched domain nohz_full @LKML 💾 Historical since v7.0 cpuset_cpu_is_isolated – removed, use cpu_is_isolated instead Runtime code patching The kernel patches its own machine code at boot or runtime to optimize hot paths. Static keys patch a NOP_(code) NOP to a jump (or vice versa); static calls patch indirect calls to direct calls; alternatives replace instructions based on CPU features. ⚲ API linux/jump_label.h – static keys DEFINE_STATIC_KEY_FALSE – define a key, initially false static_branch_unlikely – zero-cost branch check (NOP when key is false) static_branch_enable – patch all sites to take the branch (expensive, rare use) linux/static_call.h – static calls DEFINE_STATIC_CALL – define a call site with initial target function static_call – invoke the patched direct call static_call_update – change target function, patch all sites arch/x86/include/asm/alternative.h – alternatives ALTERNATIVE – replace instructions at boot based on CPU features arch/x86/include/asm/cpufeatures.h – X86_FEATURE_* flags ⚙️ Internals CONFIG_JUMP_LABEL kernel/jump_label.c – static key patching core arch/x86/kernel/jump_label.c – x86 static key patching CONFIG_HAVE_STATIC_CALL kernel/static_call_inline.c – static call core arch/x86/kernel/static_call.c – x86 static call patching arch/x86/include/asm/alternative.h – runtime instruction patching arch/x86/kernel/alternative.c – x86 alternatives engine smp_text_poke_single – safely patch code on SMP using INT3 breakpoints; used by static keys, static calls, ftrace, kprobes smp_text_poke_batch_add smp_text_poke_batch_finish 👁 Examples housekeeping_overridden preempt_schedule tracepoints 📖 References Static Keys staging/static-keys.html x86 CPU feature flags arch/x86/cpuinfo.html Memory barriers Memory barriers (MB) are synchronization mechanisms used to ensure proper ordering of memory operations in a SMP environment. They play a crucial role in maintaining the consistency and correctness of data shared among different CPU cores or processors. MBs prevent unexpected and potentially harmful reordering of memory access instructions by the compiler or CPU, which can lead to data corruption and race conditions in a concurrent software system. ⚲ API 2 membarrier asm-generic/barrier.h mb, rmb, wmb smp_mb, smp_rmb, smp_wmb ⚙️ Internals arch/x86/include/asm/barrier.h – x86 memory barrier instructions kernel/sched/membarrier.c – membarrier system call 📖 References Memory barriers core-api/wrappers/memory-barriers.html States C-states and P-states are features in modern CPUs designed to improve energy efficiency. 🗝️ Acronyms C-states – CPU(?) states, aka idle states EPP – Energy Performance Preference admin-guide/pm/amd-pstate.html#energy-performance-preference-epp-rw HWP – hardware-managed P-states, see hwp_only P-states – Performance states, see CPU Power and frequency scaling ⚲ API cpupower 📖 References Working-State Power Management admin-guide/pm/working-state.html https://lwn.net/Kernel/Index/#Power_management Idle C-states, ACPI#Processor_states power states: C0 – operating, executing instructions. C1 (Halt) – internal Clock gating clocks stopped. C1E (Enhanced Halt) – clocks stopped, voltage and frequency reduced. C2 (Stop-Clock) – internal and external Clock gating clocks stopped. C3 (Deep Sleep) – clocks stopped, L1/L2 cache may flush, local APIC timer stops, requiring tick broadcast. C4 (Deeper Sleep) – voltage reduced. C6 (Deep Power Down) – state saved to SRAM, voltage can drop to 0V. C7 (Deeper Power Down) – C6 + L3 cache flushed. Modern Intel typically uses C1/C1E and C6, skipping C2–C5. ⚠️ ACPI state names (C1_ACPI, C2_ACPI, C3_ACPI) may not match actual hardware C-states. The MWAIT hint reveals the real state: e.g. MWAIT 0x60 = hardware C6 may appear as "C3_ACPI". ⚲ API turbostat --show CPU --quiet -n 1 --interval 0.1 --show sysfs idle= /dev/cpu_dma_latency – see set_cpu_dma_latency C-states interfaces: cpupower idle-info – show driver, governor, states with latency and residency /sys/devices/system/cpu/cpuidle/ /sys/devices/system/cpu/cpu*/cpuidle/ cat /sys/devices/system/cpu/cpu0/cpuidle/state*/name – list available C-states linux/cpuidle.h – generic framework for CPU idle power management linux/pm_qos.h ⚙️ Internals drivers/cpuidle – CPU idle governors and drivers drivers/idle/intel_idle.c – intel_idle admin-guide/pm/intel_idle.html, hardcoded C-states per CPU model, modern Intel (C1/C1E/C6) drivers/acpi/processor_idle.c – acpi_idle, reads C-states from ACPI tables, generic fallback (C1/C2/C3) kernel/power/qos.c – PM quality of service cpu_latency_qos_miscdev – implementation of /dev/cpu_dma_latency 📖 References CPU Idle Time Management admin-guide/pm/cpuidle.html Intel C-states reference HPE C-states reference 📚 Further reading https://lwn.net/Kernel/Index/#Power_management-cpuidle PM Quality Of Service Interface power/pm_qos_interface.html Power and frequency P-states, ACPI#Performance_state performance states: P0 – maximum power and frequency Pn – less power and frequency ... ⚲ API Reliable measurement of actual CPU frequency turbostat --quiet --show CPU,Bzy_MHz -n 1 --interval 0.1 Bzy_MHz – average clock rate while the CPU was not idle (ie. in "c0" state). intel_pstate= Kernel Command Line Options for intel_pstate admin-guide/pm/intel_pstate.html#kernel-command-line-options-for-intel-pstate cpufreq.default_governor= P-states interfaces: /sys/devices/system/cpu/cpufreq/ /sys/devices/system/cpu/cpu*/cpufreq/ scaling_cur_freq – unreliable assumption on CPU frequency /sys/devices/system/cpu/intel_pstate/ linux/cpufreq.h rust/kernel/cpufreq.rs – Rust bindings linux/sched/cpufreq.h – interface between cpufreq drivers and the scheduler ⚙️ Internals drivers/cpufreq – CPU frequency scaling drivers intel_pstate acpi_cpufreq_driver kernel/sched/cpufreq_schedutil.c – implementation of cpufreq.default_governor=schedutil arch/x86/kernel/cpu/intel_epb.c – Intel Performance and Energy Bias Hint support 📖 References CPU Performance Scaling admin-guide/pm/cpufreq.html Device Frequency Scaling driver-api/devfreq.html CPUFreq Governor CPUFreq - CPU frequency and voltage scaling cpu-freq Intel Performance and Energy Bias Hint admin-guide/pm/intel_epb.html intel_pstate CPU Performance Scaling Driver admin-guide/pm/intel_pstate.html General Information admin-guide/pm/intel_pstate.html#general-information Operation Modes admin-guide/pm/intel_pstate.html#operation-modes Active Mode admin-guide/pm/intel_pstate.html#active-mode Passive Mode admin-guide/pm/intel_pstate.html#passive-mode Turbo P-states Support admin-guide/pm/intel_pstate.html#turbo-p-states-support Processor Support admin-guide/pm/intel_pstate.html#processor-support Support for Hybrid Processors admin-guide/pm/intel_pstate.html#support-for-hybrid-processors Hybrid Processors with SMT admin-guide/pm/intel_pstate.html#hybrid-processors-with-smt Capacity-Aware Scheduling Support admin-guide/pm/intel_pstate.html#capacity-aware-scheduling-support Energy-Aware Scheduling Support admin-guide/pm/intel_pstate.html#energy-aware-scheduling-support User Space Interface in sysfs admin-guide/pm/intel_pstate.html#user-space-interface-in-sysfs Global Attributes admin-guide/pm/intel_pstate.html#global-attributes Interpretation of Policy Attributes admin-guide/pm/intel_pstate.html#interpretation-of-policy-attributes Coordination of P-State Limits admin-guide/pm/intel_pstate.html#coordination-of-p-state-limits Energy vs Performance Hints admin-guide/pm/intel_pstate.html#energy-vs-performance-hints intel_pstate vs acpi-cpufreq admin-guide/pm/intel_pstate.html#intel-pstate-vs-acpi-cpufreq Kernel Command Line Options for intel_pstate admin-guide/pm/intel_pstate.html#kernel-command-line-options-for-intel-pstate Diagnostics and Tuning admin-guide/pm/intel_pstate.html#diagnostics-and-tuning Trace Events admin-guide/pm/intel_pstate.html#trace-events ftrace admin-guide/pm/intel_pstate.html#ftrace 📚 Further reading https://lwn.net/Kernel/Index/#Power_management-Frequency_scaling CPU frequency scaling kernel/device-drivers cpufreq How to use cpufrequtils cpufreq-info cpufreq-set https://github.com/intel/power-optimization-library https://github.com/intel/kubernetes-power-manager