Description
Book SynopsisReview topics ranging from Intel x64 assembly language instructions and writing programs in assembly language, to pointers, live debugging, and static binary analysis of compiled C and C++ code. This book is ideal for Linux desktop and cloud developers.Using the latest version of Debian, you'll focus on the foundations of the diagnostics of core memory dumps, live and postmortem debugging of Linux applications, services, and systems, memory forensics, malware, and vulnerability analysis. This requires an understanding of x64 Intel assembly language and how C and C++ compilers generate code, including memory layout and pointers. This book provides the background knowledge and practical foundations you'll need in order to master internal Linux program structure and behavior. It consists of practical step-by-step exercises of increasing complexity with explanations and ample diagrams.You'll also work with the GDB debugger and use it for disassembly and reversing.By the end of the book, yo
Table of ContentsChapter One - x64.1: Memory, Registers, and Simple Arithmetic 11Memory and Registers inside an Idealized Computer 11Memory and Registers inside Intel 64-bit PC 12“Arithmetic” Project: Memory Layout and Registers 13“Arithmetic” Project: A Computer Program 14“Arithmetic” Project: Assigning Numbers to Memory Locations 15Assigning Numbers to Registers 17“Arithmetic” Project: Adding Numbers to Memory Cells 18Incrementing/Decrementing Numbers in Memory and Registers 21Multiplying Numbers 24
Chapter Two - x64.2: Code Optimization 27“Arithmetic” Project: C/C++ Program 27Downloading GDB 28GDB Disassembly Output – No Optimization 29GDB Disassembly Output – Optimization 32
Chapter Three - x64.3: Number Representations 33Numbers and Their Representations 33Decimal Representation (Base Ten) 34Ternary Representation (Base Three) 35Binary Representation (Base Two) 36Hexadecimal Representation (Base Sixteen) 37Why are Hexadecimals Used? 38
Chapter Four - x64.4: Pointers 41A Definition 41“Pointers” Project: Memory Layout and Registers 42“Pointers” Project: Calculations 43Using Pointers to Assign Numbers to Memory Cells 44Adding Numbers Using Pointers 50Incrementing Numbers Using Pointers 53Multiplying Numbers Using Pointers 56
Chapter Five - x64.5: Bytes, Words, Double, and Quad Words 61Using Hexadecimal Numbers 61Byte Granularity 62Bit Granularity 63Memory Layout 64
Chapter Six - x64.6: Pointers to Memory 67Pointers Revisited 67Addressing Types 68Registers Revisited 73NULL Pointers 74Invalid Pointers 75Variables as Pointers 76Pointer Initialization 77Initialized and Uninitialized Data 78More Pseudo Notation 79“MemoryPointers” Project: Memory Layout 80
Chapter Seven - x64.7: Logical Instructions and RIP 89Instruction Format 89Logical Shift Instructions 90Logical Operations 91Zeroing Memory or Registers 92Instruction Pointer 93Code Section 95
Chapter Eight - x64.8: Reconstructing a Program with Pointers 97Example of Disassembly Output: No Optimization 97Reconstructing C/C++ Code: Part 1 99Reconstructing C/C++ Code: Part 2 101Reconstructing C/C++ Code: Part 3 103Reconstructing C/C++ Code: C/C++ program 104Example of Disassembly Output: Optimized Program 105
Chapter Nine - x64.9: Memory and Stacks 107Stack: A Definition 107Stack Implementation in Memory 108Things to Remember 110PUSH Instruction 111POP instruction 112Register Review 113Application Memory Simplified 115Stack Overflow 116Jumps 117Calls 119Call Stack 121Exploring Stack in GDB 123
Chapter Ten - x64.10: Frame Pointer and Local Variables 127Stack Usage 127Register Review 128Addressing Array Elements 129Stack Structure (No Function Parameters) 130Function Prolog 131Raw Stack (No Local Variables and Function Parameters) 132Function Epilog 134“Local Variables” Project 135Disassembly of Optimized Executable 138
Chapter Eleven - x64.11: Function Parameters 139“FunctionParameters” Project 139Stack Structure 140Function Prolog and Epilog 142Project Disassembled Code with Comments 144Parameter Mismatch Problem 147
Chapter Twelve - x64.12: More Instructions 149CPU Flags Register 149The Fast Way to Fill Memory 150Testing for 0 152TEST - Logical Compare 153CMP – Compare Two Operands 154TEST or CMP? 155Conditional Jumps 156The Structure of Registers 157Function Return Value 158Using Byte Registers 159
Chapter Thirteen - x64.13: Function Pointer Parameters 161“FunctionPointerParameters” Project 161Commented Disassembly 162
Chapter Fourteen - x64.14: Summary of Code Disassembly Patterns 169Function Prolog / Epilog 169LEA (Load Effective Address) 171Passing Parameters 172Accessing Parameters and Local Variables 173
