The Linux Kernel/System

systemsystem interfacesvirtualizationDriver Modelmodulesbuses, PCIhardware interfaces, [re]booting The System functionality is named after system calls and sysfs. It differs from other kernel functionalities. Its subsystems are not tightly coupled across layers but instead provide infrastructure to other parts of the kernel. For example, the System Calls subsystem offers a common interface layer for all functionalities exposed to user space. System interfaces There are several mechanisms available in Linux for user space system interfaces. One of the most common mechanisms is through system calls, which are functions that allow user space applications to request services from the kernel, such as opening files, creating processes, and accessing system resources. Another mechanism for user space communication is through device files, which are special files that represent physical or virtual devices, such as storage devices, network interfaces, and various peripheral devices. User space applications can communicate with these devices by reading from and writing to their corresponding device files. In summary, Linux kernel provides several mechanisms for user space communication, including system calls, device files, procfs, sysfs, and devtmpfs. These mechanisms enable user space applications to communicate with the kernel and access system resources in a safe and controlled manner. ⚲ APIs: kernel space API for user space uapi arch/x86/include/uapi – x86 user-space API headers 2 ioctl System calls Device files user space API for kernel space linux/uaccess.h: copy_to_user copy_from_user 📖 References User-space API guides userspace-api User space Linux kernel interfaces ULK3 Chapter 11. Signals System calls System calls are the fundamental interface between user space applications and the Linux kernel. They provide a way for programs to request services from the operating system, such as opening a file, allocating memory, or creating a new process. In the Linux kernel, system calls are implemented as functions that can be invoked by user space programs using a software interrupt mechanism. The Linux kernel provides hundreds of system calls, each with its own unique functionality. These system calls are organized into categories such as process management, file management, network communication, and memory management. User space applications can use these system calls to interact with the kernel and access the underlying system resources. ⚲ API Table of syscalls 2 syscalls ⚙️ Internals linux/syscalls.h kernel/entry – common syscall entry/exit and exception handling arch/x86/entry – x86 syscall and interrupt entry points syscall_init installs entry_SYSCALL_64 2 syscall ↪ entry_SYSCALL_64 ↯ call hierarchy: do_syscall_64 sys_call_table 📖 References System call Directory of system calls, man section 2 Anatomy of a system call, part 1 and part 2 kernel syscalls 💾 Historical ULK3 Chapter 10. System Calls Device files Classic UNIX devices are Char devices used as byte streams with 2 ioctl. ⚲ API ls /dev cat /proc/devices cat /proc/misc Examples: misc_fops usb_fops memory_fops Allocated devices admin-guide/devices.html drivers/char – miscellaneous byte stream devices Chapter 13. I/O Architecture and Device Drivers hiddev ⚠️ Warning: confusion. hiddev isn't real human interface device! It reuses USBHID infrastructure. hiddev is used for example for monitor controls and Uninterruptible Power Supplies. This module supports these devices separately using a separate event interface on /dev/usb/hiddevX (char 180:96 to 180:111) (⚙️ HIDDEV_MINOR_BASE) ⚲ API uapi/linux/hiddev.h HID_CONNECT_HIDDEV ⚙️ Internals CONFIG_USB_HIDDEV linux/hiddev.h hiddev_event drivers/hid/usbhid/hiddev.c, hiddev_fops 📖 References HIDDEV - Care and feeding of your Human Interface Devices hid/hiddev.html 📖 References Device file Administration 🔧 TODO 📖 References 7 netlink The Linux kernel user’s and administrator’s guide admin-guide procfs The proc filesystem (procfs) is a special filesystem that presents information about processes and other system information in a hierarchical file-like structure, providing a more convenient and standardized method for dynamically accessing process data held in the kernel than traditional tracing methods or direct access to kernel memory. Typically, it is mapped to a mount point named /proc at boot time. The proc file system acts as an interface to internal data structures in the kernel. It can be used to obtain information about the system and to change certain kernel parameters at runtime. /proc includes a directory for each running process —including kernel threads— in directories named /proc/PID, where PID is the process number. Each directory contains information about one process, including the command that originally started the process (/proc/PID/cmdline), the names and values of its environment variables (/proc/PID/environ), a symlink to its working directory (/proc/PID/cwd), another symlink to the original executable file —if it still exists— (/proc/PID/exe), a couple of directories with symlinks to each open file descriptor (/proc/PID/fd) and the status —position, flags, ...— of each of them (/proc/PID/fdinfo), information about mapped files and blocks like heap and stack (/proc/PID/maps), a binary image representing the process's virtual memory (/proc/PID/mem), a symlink to the root path as seen by the process (/proc/PID/root), a directory containing hard links to any child process or thread (/proc/PID/task), basic information about a process including its run state and memory usage (/proc/PID/status) and much more. 📖 References procfs 5 procfs 7 namespaces 7 capabilities linux/proc_fs.h linux/seq_file.h – sequential file interface for /proc entries fs/proc – procfs implementation LKMPG: The /proc Filesystem linux/sysctl.h – /proc/sys runtime kernel tuning: /etc/sysctl.conf, /etc/sysctl.d/, /lib/sysctl.d/ – persistent sysctl configuration loaded at boot by 8 sysctl fs/proc/proc_sysctl.c – filesystem interface kernel/sysctl.c – read/write handlers (proc_dointvec, proc_dostring, ...) sysfs sysfs is a pseudo-file system that exports information about various kernel subsystems, hardware devices, and associated device drivers from the kernel's device model to user space through virtual files. In addition to providing information about various devices and kernel subsystems, exported virtual files are also used for their configuring. Sysfs is designed to export the information present in the device tree, which would then no longer clutter up procfs. Sysfs is mounted under the /sys mount point. ⚲ API linux/sysfs.h 📖 References sysfs 5 sysfs sysfs - filesystem for exporting kernel objects filesystems/sysfs.html LKMPG: sysfs fs/sysfs – sysfs implementation fs/kernfs – kernfs, the backing filesystem for sysfs and cgroups devtmpfs devtmpfs is a hybrid User space and kernel space kernel/user space approach of a device filesystem to provide nodes before udev runs for the first time. 📖 References Device file drivers/base/devtmpfs.c – kernel-maintained tmpfs /dev configfs configfs filesystems/configfs.html is a userspace-driven kernel object configuration filesystem mounted at /sys/kernel/config/. Unlike sysfs (which exposes kernel-created objects), configfs lets userspace create and configure kernel objects via mkdir/rmdir. Used by USB gadget, NVMe target, ACPI, null_blk, GPIO simulator and other subsystems. ⚲ API: linux/configfs.h config_item – base object, created by userspace mkdir config_group – container for config_items configfs_attribute – per-item readable/writable attribute configfs_subsystem – top-level subsystem registration configfs_register_subsystem rust/kernel/configfs.rs – Rust bindings ⚙️ Internals: fs/configfs Virtualization 🔧 TODO See Kernel-based Virtual Machine, x86 virtualization ⚙️ Internals virt/kvm – KVM core arch/x86/kvm – x86 KVM hypervisor arch/x86/kvm/vmx – Intel VT-x VMX arch/x86/kvm/svm – AMD AMD-V SVM 📖 References Virtualization Support virt/ 📚 Further reading https://deepwiki.com/torvalds/linux/3-virtualization Containerization OS-level virtualization Containerization is a powerful technology that has revolutionized the way software applications are developed, deployed, and run. At its core, containerization provides an isolated environment for running applications, where the application has all the necessary dependencies and can be easily moved from one environment to another without worrying about any compatibility issues. Containerization technology has its roots in the chroot command, which was introduced in the Unix operating system in the 1979. Chroot provided a way to change the root directory of a process, effectively creating a new isolated environment with its own file system hierarchy. However, this early implementation of containerization had limited functionality, and it was difficult to manage and control the various processes running within the container. In the early 2000s, the Linux kernel introduced Linux namespaces namespaces and cgroups control groups to provide a more robust and scalable containerization solution. Namespaces allow processes to have their own isolated view of the system, including the file system, network, and process ID space, while control groups provide fine-grained control over the resources allocated to each container, such as CPU, memory, and I/O. Using these kernel features, containerization platforms such as Docker (software) Docker and Kubernetes have emerged as popular solutions for building and deploying containerized applications at scale. Containerization has become an essential tool for modern software development, allowing developers to easily package applications and deploy them in a consistent and predictable manner across different environments. Resources usage and limits ⚲ API 2 chroot – change root directory 2 sysinfo – return system information 2 getrusage – get resource usage get/set resource limits: 2 getrlimit 2 setrlimit 2 prlimit64 📖 References chroot Namespaces Linux namespaces provide the way to isolate and virtualize different aspects of the operating system. Namespaces allow multiple instances of an application to run in isolation from each other, without interfering with the host system or other instances. 🔧 TODO ⚲ API /proc/self/ns 8 lsns, 2 ioctl_ns ↪ ns_ioctl 1 unshare, 2 unshare 1 nsenter, 2 setns 2 clone3 ↪ clone_args linux/ns_common.h linux/proc_ns.h namespaces definition uts_namespace ipc_namespace mnt_namespace pid_namespace net/net_namespace.h - struct net user_namespace time_namespace cgroup_namespace ⚙️ Internals init_nsproxy - struct of namespaces kernel/nsproxy.c – namespace proxy management fs/namespace.c – mount namespace fs/proc/namespaces.c – /proc namespace entries net/core/net_namespace.c – network namespace kernel/time/namespace.c – time namespace kernel/user_namespace.c – user namespace kernel/pid_namespace.c – PID namespace kernel/utsname.c – UTS namespace kernel/cgroup/namespace.c – cgroup namespace ipc/namespace.c – IPC namespace 📖 References 7 namespaces 7 uts_namespaces 7 ipc_namespaces 7 mount_namespace 7 pid_namespaces 7 network_namespaces 7 user_namespaces 7 time_namespaces 7 cgroup_namespaces Control groups cgroups are used to limit and control the resource usage of groups of processes. They allow administrators to set limits on CPU usage, memory usage, disk I/O, network bandwidth, and other resources, which can be useful for managing system performance and preventing resource contention. There are two versions of cgroups. Unlike v1, cgroup v2 has only a single process hierarchy and discriminates between processes, not threads. Here are some of the key differences between cgroups v1 and v2: +cgroups v1cgroups v2Hierarchyeach subsystem had its own hierarchy, which could lead to complexity and confusionunified hierarchy, which simplifies management and enables better resource allocationControllershas several subsystems that are controlled by separate controllers, each with its own set of configuration files and parameterscontrollers are consolidated into a single "cgroup2" controller, which provides a unified interface for managing resourcesResource distributiondistributes resources among groups of processes based on proportional sharing, which can lead to unpredictable resultsresources are distributed based on a "weighted fair queuing" algorithm, which provides better predictability and fairness Cgroups v2 is not backward compatible with cgroups v1, which means that migrating from v1 to v2 can be challenging and requires careful planning. 🔧 TODO ⚲ Shell interfaces /proc/cgroups – available controllers with hierarchy and num_cgroups /proc/$pid/cgroup – cgroup membership of a process 1 systemd-cgls – show cgroup hierarchy as a tree sos cgroups plugin – diagnostics collection for cgroups ⚲ API linux/cgroup.h linux/cgroup-defs.h css_set – holds set of reference-counted pointers to cgroup_subsys_state objects cgroup_subsys linux/cgroup_subsys.h – list of cgroup subsystems ⚙️ Internals cg_list – list of css_set in task_struct kernel/cgroup – cgroup core cgroup_init cgroup2_fs_type tools/testing/selftests/cgroup – cgroup self-tests 📖 References Control Groups v2 Control Groups v1 admin-guide/cgroup-v1/ 1 systemd-cgtop 5 systemd.slice – slice unit configuration 7 cgroups 7 cgroup_namespaces CFS Bandwidth Control for cgroups scheduler/sched-bwc.html Real-Time group scheduling scheduler/sched-rt-group.html Understanding control groups, RHEL 📚 Further reading https://github.com/containers cgrc tool 💾 Historical https://github.com/mk-fg/cgroup-tools for cgroup v1 Driver Model The Linux driver model (or Device Model, or just DM) is a framework that provides a consistent and standardized way for device drivers to interface with the kernel. It defines a set of rules, interfaces, and data structures that enable device drivers to communicate with the kernel and perform various operations, such as managing resources, lifecycle and more. DM core structure consists of DM classes, DM buses, DM drivers and DM devices. kobject In the Linux kernel, a kobject is a fundamental data structure used to represent kernel objects and provide a standardized interface for interacting with them. A kobject is a generic object that can represent any type of kernel object, including devices, files, modules, and more. The kobject data structure contains several fields that describe the object, such as its name, type, parent, and operations. Each kobject has a unique name within its parent object, and the parent-child relationships form a hierarchy of kobjects. Kobjects are managed by the kernel's sysfs file system, which provides a virtual file system that exposes kernel objects as files and directories in the user space. Each kobject is associated with a sysfs directory, which contains files and attributes that can be read or written to interact with the kernel object. ⚲ Infrastructure API linux/kobject.h kobject Kernel objects manipulation driver-api/basics.html#kernel-objects-manipulation 🔧 TODO Classes A class is a higher-level view of a device that abstracts out low-level implementation details. Drivers may see a NVME storage or a SATA storage, but, at the class level, they are all simply block_class devices. Classes allow user space to work with devices based on what they do, rather than how they are connected or how they work. General DM classes structure match composite pattern. ⚲ API ls /sys/class/ class_register registers class linux/device/class.h 👁 Examples: input_class, block_class net_class Buses A peripheral bus is a channel between the processor and one or more peripheral devices. A DM bus is Proxy pattern proxy for a peripheral bus. General DM buses structure match composite pattern. For the purposes of the device model, all devices are connected via a bus, even if it is an internal, virtual, platform_bus_type. Buses can plug into each other. A USB controller is usually a PCI device, for example. The device model represents the actual connections between buses and the devices they control. A bus is represented by the bus_type structure. It contains the name, the default attributes, the bus' methods, PM operations, and the driver core's private data. ⚲ API ls /sys/bus/ bus_register registers bus_type linux/device/bus.h 👁 Examples: usb_bus_type, hid_bus_type, pci_bus_type, scsi_bus_type, platform_bus_type Peripheral buses Drivers ⚲ API ls /sys/bus/:/drivers/ module_driver - simple common driver initializer, 👁 for example used in module_pci_driver driver_register registers device_driver - basic device driver structure, one per all device instances. linux/device/driver.h 👁 Examples: hid_generic usb_register_device_driver Platform drivers module_platform_driver registers platform_driver (platform wrapper of device_driver) with platform_bus_type linux/platform_device.h 👁 Examples: gpio_mouse_device_driver Devices ⚲ API ls /sys/devices/ device_register registers device - the basic device structure, per each device instance linux/device.h – Device drivers infrastructure driver-api/infrastructure.html linux/dev_printk.h Device Resource Management driver-api/basics.html#device-resource-management, devres, devm ... 👁 Examples: platform_bus mousedev_create Platform devices platform_device - platform wrapper of struct device - the basic device structure driver-api/infrastructure.html#c.device, contains resources associated with the device it can be created dynamically by platform_device_register_simple or platform_device_alloc. Or registered with platform_device_register. platform_device_unregister - releases device and associated resources 👁 Examples: add_pcspkr ⚲ API 🔧 TODO platform_device_info platform_device_id platform_device_register_full platform_device_add platform_device_add_data platform_device_register_data platform_device_add_resources attribute_group dev_pm_ops ⚙️ Internals linux/dev_printk.h lib/kobject.c – kobject library drivers/base/platform.c – platform bus for legacy devices drivers/base/core.c – driver model core linux/notifier.h, kernel/notifier.c – notifier chains for event broadcasting 📖 References Device drivers infrastructure driver-api/infrastructure.html Everything you never wanted to know about kobjects, ksets, and ktypes core-api/kobject.html Driver Model driver-api/driver-model The Linux Kernel Device Model driver-api/driver-model/overview.html Platform Devices and Drivers driver-api/driver-model/platform.html Linux Device Model, by linux-kernel-labs LKMPG: The Device Model Peripheral buses Peripheral buses are the communication channels used to connect various peripheral devices to a computer system. These buses are used to transfer data between the peripheral devices and the system's processor or memory. In the Linux kernel, peripheral buses are implemented as drivers that enable communication between the operating system and the hardware. Peripheral buses in the Linux kernel include USB, PCI, SPI, I2C, and more. Each of these buses has its own unique characteristics, and the Linux kernel provides support for a wide range of peripheral devices. The PCI (Peripheral Component Interconnect) bus is used to connect internal hardware devices in a computer system. It is commonly used to connect graphics cards, network cards, and other expansion cards. The Linux kernel provides a PCI bus driver that enables communication between the operating system and the devices connected to the bus. The USB (Universal Serial Bus) is one of the most commonly used peripheral buses in modern computer systems. It allows devices to be hot-swapped and supports high-speed data transfer rates. 🔧 TODO: device enumeration ⚲ API Shell interface: ls /proc/bus/ /sys/bus/ See also Buses of Driver Model See Input: keyboard, mouse etc PCI ⚲ Shell API lspci -vv column -t /proc/bus/pci/devices Main article: PCI USB ⚲ API 8 lsusb – list USB devices ls /sys/bus/usb/ linux/usb.h – core USB API usb_device – represents a USB device usb_interface – represents a device function usb_driver – driver for USB interfaces usb_device_id, USB_DEVICE – device matching table and macro module_usb_driver – helper macro usb_register – register a USB driver usb_alloc_urb, usb_submit_urb, usb_free_urb – URB lifecycle usb_control_msg – send control transfer linux/usb/gadget.h – USB gadget (device-side) API usb_gadget – device-side USB representation rust/kernel/usb.rs – Rust bindings ⚙️ Internals drivers/usb – USB subsystem drivers/usb/core – USB core framework drivers/usb/core/hcd.c – Host controller host controller driver interface drivers/usb/core/hub.c – hub and port management drivers/usb/core/urb.c – USB Request Block submission drivers/usb/core/message.c – control/bulk transfer helpers drivers/usb/host – HCD implementations (xHCI, EHCI, OHCI) drivers/usb/gadget – USB gadget framework (device mode) drivers/usb/typec – USB Type-C and Power Delivery drivers/usb/storage – USB mass storage drivers/usb/serial – USB-to-serial converters 👁 Example drivers/usb/usb-skeleton.c – template for USB drivers 📖 References USB usb LDD3:USB Drivers Other buses drivers/bus – bus driver implementations Buses for 🤖 embedded devices: linux/gpio/driver.h linux/gpio.h drivers/gpio tools/gpio drivers/i2c – I2C bus framework, I2C i2c rust/kernel/i2c.rs – Rust bindings SPI ⚲ API linux/spi/spi.h tools/spi – SPI user-space tools ⚙️ Internals drivers/spi – SPI bus framework spi_register_controller spi_controller_list🚧 📖 References SPI spi Hardware interfaces Hardware interfaces are basic part of any operating, enabling communication between the processor and other HW components of a computer system: memory, peripheral devices and buses, various controllers. Interrupts I/O ports and registers I/O ports and registers are electronic components in computer systems that enable communication between CPU and other electronic controllers and devices. ⚲ API asm-generic/io.h — generic I/O port emulation. ioport_map ioread32 / iowrite32 ... readl/ writel ... The {in,out}[bwl] macros are for emulating x86-style PCI/ISA IO space: inl/ outl ... linux/ioport.h, kernel/resource.c — I/O and memory resource management. devm_request_mem_region request_mem_region arch/x86/include/asm/io.h – x86 I/O port and MMIO access Functions for memory mapped registers: devm_ioremap ... ioremap regmap The regmap subsystem provides a standardized abstraction layer for register access in device drivers. It simplifies interactions with hardware registers across various bus types, such as I2C, SPI, and MMIO, by offering a consistent API. ⚲ API linux/regmap.h — register map access API the most frequently used functions: regmap_update_bits regmap_write regmap_read regmap_reg_range regmap_bulk_read devm_regmap_init_i2c regmap_set_bits regmap_field_write regmap_bulk_write regmap_clear_bits regmap_write_bits regmap_config regmap_read_poll_timeout devm_regmap_init_mmio ⚙️ Internals drivers/base/regmap – register map abstraction layer Hardware Device Drivers Keywords: firmware, hotplug, clock, mux, pin ⚙️ Internals drivers/acpi – ACPI drivers drivers/base – driver model core drivers/sdio – Secure Digital#SDIO cards Secure Digital Input Output drivers/virtio – virtio guest drivers drivers/hwmon – hardware monitoring drivers/thermal – thermal management drivers/pinctrl – pin control and multiplexing drivers/clk – clock framework rust/kernel/clk.rs – Rust bindings 📖 References Pin control subsystem driver-api/pin-control.html Linux Hardware Monitoring hwmon Firmware guide firmware-guide Devicetree devicetree https://hwmon.wiki.kernel.org/ LDD3:The Linux Device Model http://www.tldp.org/LDP/tlk/dd/drivers.html http://www.xml.com/ldd/chapter/book/ http://examples.oreilly.com/linuxdrive2/ Live patching Kpatch Live patching (since 4.0) allows applying critical fixes to a running kernel without rebooting. ⚲ API: linux/livepatch.h klp_patch – patch structure klp_enable_patch ⚙️ Internals: kernel/livepatch – live patching core 📖 References: Livepatch livepatch Building and Updating ../Updating/