Linux Kernel Debugging: Leverage open source tools and advanced techniques to debug Linux kernel or module issues / Отладка ядра Linux: Используйте инструменты с открытым исходным кодом и передовые методы для отладки проблем с ядром Linux или модулями. Год издания: 2022 Автор: Billimoria Kaiwan / Биллимория Кайван Издательство: Packt Publishing ISBN: 978-1-80107-503-9 Язык: Английский Формат: PDF/EPUB Качество: Издательский макет или текст (eBook) Интерактивное оглавление: Да Количество страниц: 639 Описание: Debug Linux kernel modules and the kernel itself by mastering powerful open source tools and advanced kernel debugging techniques Key Features Learn how to use a variety of kernel and module debug tools and techniques with the help of examples Understand how to expertly interpret a kernel oops and identify the underlying defects Includes easy-to-look up tables and simplified explanations of complex kernel-level defects Book Description The Linux kernel is at the very core of arguably the world’s best production-quality OS. Debugging it, though, can be a complex endeavor. Linux Kernel Debugging is a thorough guide to advanced kernel debugging. This book covers a range of debugging techniques in detail, starting with instrumentation-based debugging techniques, the kernel’s powerful Kprobes framework, printk, and friends. The book features two whole chapters devoted to the tools and techniques for debugging memory-related bugs. It also prepares you to interpret the underlying issue when the Linux kernel throws an oops. As you advance through the chapters, you’ll be able to demystify concurrency by learning what data races are and how to handle them, including the use of modern features like Kernel Concurrency Sanitizer (KCSAN). You’ll also discover how to overcome some thorny issues related to debugging and performance through detailed kernel-level tracing and learn to wield the power of Ftrace and its frontends. What’s more? The book shows you how to deal with kernel hangs and panic, leverage the venerable GDB tool within the kernel, and much more. By the time you complete reading this debugging kernel book, you’ll have at your disposal, a whole range of powerful debug tools and techniques, along with a keen sense as to when to use which. What you will learn Instrumentation-based printk and the powerful dynamic debug framework Use Kprobes statically and dynamically to trap into kernel or module functions Catch kernel memory defects with KASAN, UBSAN, SLUB debug, and kmemleak Precisely identify the source location of kernel and module bugs Understand data races and use KCSAN to catch evasive concurrency defects Leverage Ftrace and trace-cmd to trace the kernel flow in great detail Write a custom kernel panic handler using the WD and hangcheck timer Learn to use KGDB to single-step and debug kernel or module source code Who This Book Is For This book is for Linux kernel developers, module or driver authors, and testers interested in debugging and enhancing their Linux systems at the level of the kernel. System administrators who want to understand and debug the internal infrastructure of their Linux kernels will also find this book useful. A good hold of C programming and the Linux command line is necessary. Some experience with module or kernel development will be beneficial too. Выполняйте отладку модулей ядра Linux и самого ядра, осваивая мощные инструменты с открытым исходным кодом и передовые методы отладки ядра Kлючевые функции На примерах вы узнаете, как использовать различные инструменты и методы отладки ядра и модулей Поймете, как квалифицированно интерпретировать ошибки ядра и выявлять лежащие в их основе дефекты Содержит удобные для просмотра таблицы и упрощенные объяснения сложных дефектов на уровне ядра Описание книги Ядро Linux - это, пожалуй, самая качественная операционная система в мире. Однако его отладка может быть сложной задачей. Отладка ядра Linux - это подробное руководство по расширенной отладке ядра. В этой книге подробно рассматривается ряд методов отладки, начиная с методов отладки на основе инструментария, мощной платформы Kprobes ядра, printk и других. В книге целых две главы посвящены инструментам и методам устранения ошибок, связанных с памятью. Это также подготавливает вас к интерпретации основной проблемы, когда ядро Linux выдает сообщение oops. По мере продвижения по главам вы сможете раскрыть тайну параллелизма, узнав, что такое "гонки данных" и как с ними справляться, включая использование современных функций, таких как Kernel Concurrency Sanitizer (KCSAN). Вы также узнаете, как преодолеть некоторые сложные проблемы, связанные с отладкой и производительностью, с помощью подробной трассировки на уровне ядра, и научитесь использовать возможности Ftrace и его интерфейсов. Что еще? В книге показано, как бороться с зависаниями ядра и паникой, использовать в ядре известный инструмент GDB и многое другое. К тому времени, когда вы закончите чтение этой книги по отладке ядра, в вашем распоряжении будет целый ряд мощных инструментов и методов отладки, а также четкое представление о том, когда какие из них использовать. Что вы узнаете Printk, основанный на инструментах, и мощная платформа динамической отладки Статически и динамически используют Kprobes для перехвата функций ядра или модуля Выявляют дефекты памяти ядра с помощью KASAN, UBSAN, SLUB debug и kmemleak Точно определяют местоположение источника ошибок ядра и модуля Разбирайтесь в "скачках" данных и используйте KCSAN для обнаружения дефектов параллелизма, вызывающих уклонение Используйте Ftrace и trace-cmd для детального отслеживания работы ядра Напишите пользовательский обработчик аварийных ситуаций ядра, используя WD и таймер зависания Научитесь использовать KGDB для пошаговой отладки исходного кода ядра или модуля Для кого предназначена эта книга Эта книга предназначена для разработчиков ядра Linux, авторов модулей или драйверов, а также тестировщиков, заинтересованных в отладке и улучшении своих Linux-систем на уровне ядра. Системным администраторам, которые хотят разобраться и отладить внутреннюю инфраструктуру своих ядер Linux, также будет полезна эта книга. Необходимо хорошо владеть программированием на C и командной строкой Linux. Также будет полезен некоторый опыт разработки модулей или ядра.
Примеры страниц (скриншоты)
Оглавление
Preface Part 1: A General Introduction and Approaches to Kernel Debugging 1 A General Introduction to Debugging Software Technical requirements 4 Cloning this book's code repository 4 Software debugging – what it is, origins, and myths 5 Software bugs – a few actual cases 7 Patriot missile failure 7 The ESA's unmanned Ariane 5 rocket 8 Mars Pathfinder reset issue 8 The Boeing 737 MAX aircraft – the MCAS and lack of training of the flight crew 9 Other cases 10 Setting up the workspace 10 Running Linux as a native or guest OS 11 Running Linux as a guest OS 12 Installing the Oracle VirtualBox guest additions 13 Installing required software packages 14 A tale of two kernels 15 A production and a debug kernel 16 Setting up our custom production kernel 19 Setting up our custom debug kernel 26 Seeing the difference – production and debug kernel config 35 Debugging – a few quick tips 38 A programmer's checklist – seven rules 41 Summary 41 Further reading 42 2 Approaches to Kernel Debugging Technical requirements 46 Classifying bug types 46 Types of bugs – the classic view 46 Types of bugs – the memory view 48 Types of bugs – the CVE/CWE security-related view 48 Types of bugs – the Linux kernel 49 Kernel debugging – why there are different approaches to it 50 Summarizing the different approaches to kernel debugging 51 The development phase 51 Unit testing and/or QA phases 52 Categorizing into different scenarios 54 Summary 58 Further reading 59 Part 2: Kernel and Driver Debugging Tools and Techniques 3 Debug via Instrumentation – printk and Friends Technical requirements 64 The ubiquitous kernel printk 64 Using the printk API's logging levels 66 Leveraging the pr_ convenience macros 68 Understanding where the printk output goes 72 Practically using the printk format specifiers – a few quick tips 74 Leveraging the printk for debug purposes 75 Writing debug messages to the kernel log 75 Debug printing – quick and useful tips 79 Device drivers – using the dev_dbg() macro 80 Trying our kernel module on the custom production kernel 81 Rate limiting the printk 82 Using the kernel's powerful dynamic debug feature 86 Dynamic debugging via module parameters 87 Specifying what and how to print debug messages 90 Exercising dynamic debugging on a kernel module on a production kernel 96 Remaining printk miscellany 104 Printing before console init – the early printk 105 Designating the printk to some known presets 109 Printing exactly once 110 Emitting a printk from userspace 112 Easily dumping buffer content 113 Remaining points – bootloader log peeking, LED flashing, and more 114 Summary 116 Further reading 116 4 Debug via Instrumentation – Kprobes Understanding kprobes basics 120 What we intend to do 122 Using static kprobes – traditional approaches to probing 123 Demo 1 – static kprobe – trapping into the file open the traditional static kprobes way – simplest case 123 Demo 2 – static kprobe – specifying the function to probe via a module parameter 130 Understanding the basics of the Application Binary Interface (ABI) 132 Using static kprobes – demo 3 and demo 4 134 Demo 3 – static kprobe – probing the file open syscall and retrieving the filename parameter 135 Demo 4 – semi-automated static kprobe via our helper script 141 Getting started with kretprobes 147 Kprobes miscellany 151 Kprobes – limitations and downsides 152 Interface stability 152 The easier way – dynamic kprobes or kprobe-based event tracing 153 Kprobe-based event tracing – minimal internal details 157 Setting up a dynamic kprobe (via kprobe events) on any function 160 Using dynamic kprobe event tracing on a kernel module 166 Setting up a return probe (kretprobe) with kprobe-perf 169 Trapping into the execve() API – via perf and eBPF tooling 171 System calls and where they land in the kernel 171 Observability with eBPF tools – an introduction 173 Summary 176 Further reading 176 5 Debugging Kernel Memory Issues – Part 1 Technical requirements 182 What's the problem with memory anyway? 183 Tools to catch kernel memory issues – a quick summary 184 Using KASAN and UBSAN to find memory bugs 186 Understanding KASAN – the basics 186 Requirements to use KASAN 187 Configuring the kernel for Generic KASAN mode 189 Bug hunting with KASAN 191 Using the UBSAN kernel checker to find Undefined Behaviour 214 Building your kernel and modules with Clang 223 Using Clang 13 on Ubuntu 21.10 223 Catching memory defects in the kernel – comparisons and notes (Part 1) 226 Miscellaneous notes 228 Summary 229 Further reading 230 6 Debugging Kernel Memory Issues – Part 2 Technical requirements 234 Detecting slab memory corruption via SLUB debug 234 Configuring the kernel for SLUB debug 235 Leveraging SLUB debug features via the slub_debug kernel parameter 236 Running and tabulating the SLUB debug test cases 239 Interpreting the kernel's SLUB debug error report 243 Learning how to use the slabinfo and related utilities 248 Finding memory leakage issues with kmemleak 260 Configuring the kernel for kmemleak 264 Using kmemleak 266 A few tips for developers regarding dynamic kernel memory allocation 281 Catching memory defects in the kernel – comparisons and notes (Part 2) 284 Miscellaneous notes 286 Summary 287 Further reading 287 7 Oops! Interpreting the Kernel Bug Diagnostic Technical requirements 290 Generating a simple kernel bug and Oops 290 The procmap utility 291 What's this NULL trap page anyway? 291 A simple Oops v1 – dereferencing the NULL pointer 294 Doing a bit more of an Oops – our buggy module v2 297 A kernel Oops and what it signifies 307 The devil is in the details – decoding the Oops 308 Line-by-line interpretation of an Oops 308 Tools and techniques to help determine the location of the Oops 325 Using objdump to help pinpoint the Oops code location 327 Using GDB to help debug the Oops 330 Using addr2line to help pinpoint the Oops code location 331 Taking advantage of kernel scripts to help debug kernel issues 332 Leveraging the console device to get the kernel log after Oopsing in IRQ context 340 An Oops on an ARM Linux system and using netconsole 348 Figuring out the actual buggy code location (on ARM) 352 A few actual Oopses 356 Summary 358 Further reading 358 8 Lock Debugging Technical requirements 363 Locking and lock debugging 363 Locking – a quick summarization of key points 364 Understanding data races – delving deeper 365 Catching concurrency bugs with KCSAN 367 What KCSAN does, in a nutshell 368 Configuring the kernel for KCSAN 369 Using KCSAN 374 Knee-jerk reactions to KCSAN reports – please don't! 380 A few actual use cases of kernel bugs due to locking defects 382 Defects identified by KCSAN 382 Identifying locking rules and bugs from the LDV project 382 Identifying locking bugs from the Linux kernel Bugzilla 385 Identifying some locking defects from various blog articles and the like 387 Summary 392 Further reading 393 Part 3: Additional Kernel Debugging Tools and Techniques 9 Tracing the Kernel Flow Technical requirements 400 Kernel tracing technology – an overview 400 Using the ftrace kernel tracer 403 Accessing ftrace via the filesystem 404 Configuring the kernel for ftrace 405 Using ftrace to trace the flow of the kernel 408 Useful ftrace filtering options 426 Case 1 – tracing a single ping with raw ftrace 433 Case 2 – tracing a single ping with raw ftrace via the set_event interface 439 Using trace_printk() for debugging 443 Ftrace – miscellaneous remaining points via FAQs 443 Ftrace use cases 447 Using the trace-cmd, KernelShark, and perf-tools ftrace frontends 451 An introduction to using trace-cmd 451 Using the KernelShark GUI 457 An introduction to using perf-tools 462 An introduction to kernel tracing with LTTng and Trace Compass 467 A quick introduction to recording a kernel tracing session with LTTng 467 Using the Trace Compass GUI to visualize the single ping LTTng trace 470 Summary 473 Further reading 474 10 Kernel Panic, Lockups, and Hangs Technical requirements 480 Panic! – what happens when a kernel panics 480 Let's panic 480 To the rescue with netconsole 483 Interpreting the panic output 484 Kernel parameters, tunables, and configs that affect kernel panic 489 Writing a custom kernel panic handler routine 492 Linux kernel panic notifier chains – the basics 492 Setting up our custom panic handler within a module 493 Detecting lockups and CPU stalls in the kernel 502 A short note on watchdogs 502 Employing the kernel's hard and soft lockup detector 504 Employing the kernel's hung task and workqueue stall detectors 516 Leveraging the kernel hung task detector 516 Detecting workqueue stalls 518 Summary 520 Further reading 520 11 Using Kernel GDB (KGDB) Technical requirements 524 Conceptually understanding how KGDB works 525 Setting up an ARM target system and kernel for KGDB 526 Building a minimal custom ARM Linux target system with SEALS 526 Configuring the kernel for KGDB 528 Testing the target system 533 Debugging the kernel with KGDB 536 Running our target (emulated) ARM32 system 537 Running and working with the remote GDB client on the host system 538 Debugging kernel modules with KGDB 542 Informing the GDB client about the target module's locations in memory 542 Step by step – debugging a buggy module with KGDB 543 [K]GDB – a few tips and tricks 561 Setting up and using GDB scripts with CONFIG_GDB_SCRIPTS 561 KGDB target remote :1234 command doesn't work on physical systems 563 Setting the system root with sysroot 564 Using GDB's TUI mode 564 What to do when the GDB response occurs 567 GDB convenience routines 567 GDB custom macros in its startup file 568 Fancy breakpoints and hardware watchpoints 568 Miscellaneous GDB tips 571 Summary 572 Further reading 572 12 A Few More Kernel Debugging Approaches An introduction to the kdump/ crash framework 576 Why use kdump/crash? 576 Understanding the kdump/crash basic framework 577 A mention on performing static analysis on kernel code 579 Examples using cppcheck and checkpatch.pl for static analysis 580 An introduction to kernel code coverage tools and testing frameworks 582 Why is code coverage important? 582 A brief note on kernel testing 584 Miscellaneous – using journalctl, assertions, and warnings 586 Looking up system logs with journalctl 586 Assertions, warnings, and BUG() macros 590 Summary 591 Further reading 592
Billimoria Kaiwan / Биллимория Кайван - Linux Kernel Debugging / Отладка ядра Linux [2022, PDF/EPUB, ENG] download torrent for free and without registration
You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum You cannot attach files in this forum You can download files in this forum
![[Quote]](./templates/default/images/lang/en/icon_quote.gif "Reply with quote"){kind=icon}
![[Profile]](./templates/default/images/lang/en/icon_profile.gif "View user's profile"){kind=icon}
![[PM]](./templates/default/images/lang/en/icon_pm.gif "Send private message"){kind=icon}
