Kotlin 编译器将每个类分别编译成包含字节码的 .class 文件。随后,D8 编译器将所有 .class 文件整合并压缩成一个 .dex 文件(实际上可能生成多个,取决于方法数是否超过 65536)。
在编译过程中,D8 编译器会在文件开头放上一个 Header(文件头),记录了魔数(用于校验文件类型)、文件大小和校验和。
然后,D8 会把所有的字符串抽取出来放在一个 String IDs 大数组中。实际上,String IDs 只是存放了偏移量,真正的字符串内容存放在 Data 区。这包含了类名、类型描述符、方法名、方法签名、变量名、字符串常量、注解里的字符串以及路径。
接着,所有类型被抽取出来放入 Type IDs 大数组,其内容就是指向对应的 String IDs 的下标。
紧接着是 Proto IDs,记录了所有可能的方法输入参数和返回类型的组合。举例:
text
0A 00 00 00 // shorty_idx: 10 (指向 StringIds[#10] "LII")
02 00 00 00 // return_type_idx: 2 (指向 TypeIds[#2] String)
00 08 00 00 // parameters_off: 0x0800 (偏移量,指向 TypeList)
TypeList 实际存储在 Data 区。这里说明一点:Shorty 是输入参数和返回类型组合的一个极简表达,但所有的对象类型都用 L 表示,所以它只能用于初筛,不能用于精确匹配;TypeList 则是具体的输入参数类型组合,实际存放在 Data 区,Proto IDs 里只存放一个偏移量。
Field IDs(字段索引区):记录所有的成员变量。
java
public class User {
public String name;
public int age;
}
text
// 1. 第一个字段:name (占 8 字节)
05 00 // class_idx (2字节): 指向 TypeIds[#5],属于 User 类
0A 00 // type_idx (2字节): 指向 TypeIds[#10],类型是 String
14 00 00 00 // name_idx (4字节): 指向 StringIds[#20],变量名 "name"
// 2. 第二个字段:age (占 8 字节)
05 00 // class_idx (2字节): 指向 TypeIds[#5],属于 User 类
02 00 // type_idx (2字节): 指向 TypeIds[#2],类型是 int
15 00 00 00 // name_idx (4字节): 指向 StringIds[#21],变量名 "age"
Method IDs(方法索引区):记录所有的方法。
java
public class Calculator {
public int add(int a, int b) { return a + b; }
public void log(String msg) { ... }
}
text
// 1. 第一个方法:add (占 8 字节)
05 00 // class_idx (2字节): 指向 TypeIds[#5],属于 Calculator
08 00 // proto_idx (2字节): 指向 ProtoIds[#8],即 (Int,Int)->Int
14 00 00 00 // name_idx (4字节): 指向 StringIds[#20],即 "add"
// 2. 第二个方法:log (占 8 字节)
05 00 // class_idx (2字节): 指向 TypeIds[#5],属于 Calculator
09 00 // proto_idx (2字节): 指向 ProtoIds[#9],即 (String)->void
15 00 00 00 // name_idx (4字节): 指向 StringIds[#21],即 "log"
Class Defs(类定义区):这是核心!它把这些散落的 ID 串起来,告诉系统:“这个类叫什么、继承谁、有哪些方法、方法的代码在哪”。
java
public class User extends Object {
public int age;
public void sayHi() { ... }
}
text
// 1. 基本身份信息 (16 字节)
05 00 00 00 // class_idx (4字节): 指向 TypeIds[#5] -> "Lcom/example/User;"
01 00 00 00 // access_flags (4字节): 0x01 代表 public
02 00 00 00 // superclass_idx (4字节): 指向 TypeIds[#2] -> "Ljava/lang/Object;"
00 00 00 00 // interfaces_off (4字节): 0 (无接口)
// 2. 关联信息 (16 字节)
0A 00 00 00 // source_file_idx (4字节): 指向 StringIds[#10] -> "User.java"
00 00 00 00 // annotations_off (4字节): 0 (无注解)
00 08 00 00 // class_data_off (4字节): 0x0800 (🔥核心:指向 Data 区的具体数据)
00 00 00 00 // static_values_off (4字节): 0 (无静态初始值)
Data(数据区):真实存放代码指令(Dalvik bytecode)和常量值的地方。
当我们的代码中需要加载一个类的时候(在引入 DexCache 和哈希表优化之前),底层采用的是 mmap(内存映射+缺页中断)+ 二分查找的方法。从物理层面上具体来讲,所有的 String IDs 在 DEX 文件中都是排好序存放的,Data 区的字符串也是按同样的顺序紧密排放。因此,我们可以先读取最中间那个字符串,判断大小进行二分查找,一直到找到为止。查找过程中会多次触发缺页中断,按需读入对应的 4KB 数据页。
找到字符串后,用同样的逻辑在 Type IDs 中找到对应的类型,然后在 Class Defs 中找到对应的类定义。随后,将类的元数据读入内存的方法区。在实际执行某个方法时,才会再去 Data 区找到对应的指令字节码并读入内存执行。
🇬🇧 English Version
Title: Demystifying the Android DEX File Format: A Deep Dive into Compilation and Class Loading 🚀
The Kotlin compiler compiles each class into a .class file containing bytecode. Subsequently, the D8 compiler merges and compresses all these .class files into a single .dex file (or multiple files if the method count exceeds the 65,536 limit).
During compilation, D8 places a Header at the beginning of the file, recording the magic number (ensuring this is a valid dex file), file size, and checksum.
Then, all strings are extracted and placed into a large String IDs array. Actually, the String IDsarray only stores offsets; the real string contents are stored in the Data section. This includes class names, type descriptors, method names, method signatures, variable names, string constants, annotation strings, and paths.
Next, all types are extracted into the Type IDs array, which essentially contains indices pointing to the corresponding String IDs.
Following this is the Proto IDs array, which records all possible combinations of method input parameters and return types. For example:
text
0A 00 00 00 // shorty_idx: 10 (points to StringIds[#10] "LII")
02 00 00 00 // return_type_idx: 2 (points to TypeIds[#2] String)
00 08 00 00 // parameters_off: 0x0800 (offset pointing to TypeList)
The TypeList is actually stored in the Data section. Note: The Shorty is a simplified representation of input parameters and return types, where all object types are represented by L. Therefore, it can only be used for initial filtering, not exact matching. The TypeList contains the specific input parameter types, physically stored in the Data section, with only an offset stored in the Proto IDs.
Field IDs (Field Index Area): Records all member variables.
java
public class User {
public String name;
public int age;
}
text
// 1. First field: name (8 bytes)
05 00 // class_idx (2 bytes): points to TypeIds[#5] (User class)
0A 00 // type_idx (2 bytes): points to TypeIds[#10] (String)
14 00 00 00 // name_idx (4 bytes): points to StringIds[#20] ("name")
// 2. Second field: age (8 bytes)
05 00 // class_idx (2 bytes): points to TypeIds[#5] (User class)
02 00 // type_idx (2 bytes): points to TypeIds[#2] (int)
15 00 00 00 // name_idx (4 bytes): points to StringIds[#21] ("age")
Method IDs (Method Index Area): Records all methods.
java
public class Calculator {
public int add(int a, int b) { return a + b; }
public void log(String msg) { ... }
}
text
// 1. First method: add (8 bytes)
05 00 // class_idx (2 bytes): points to TypeIds[#5] (Calculator)
08 00 // proto_idx (2 bytes): points to ProtoIds[#8] (Int, Int) -> Int
14 00 00 00 // name_idx (4 bytes): points to StringIds[#20] ("add")
// 2. Second method: log (8 bytes)
05 00 // class_idx (2 bytes): points to TypeIds[#5] (Calculator)
09 00 // proto_idx (2 bytes): points to ProtoIds[#9] (String) -> void
15 00 00 00 // name_idx (4 bytes): points to StringIds[#21] ("log")
Class Defs (Class Definition Area): This is the core! It links all scattered IDs together, telling the system: "What is this class called, what does it inherit, what methods does it have, and where is the method code located."
java
public class User extends Object {
public int age;
public void sayHi() { ... }
}
text
// 1. Basic Identity Info (16 bytes)
05 00 00 00 // class_idx (4 bytes): points to TypeIds[#5] -> "Lcom/example/User;"
01 00 00 00 // access_flags (4 bytes): 0x01 indicates public
02 00 00 00 // superclass_idx (4 bytes): points to TypeIds[#2] -> "Ljava/lang/Object;"
00 00 00 00 // interfaces_off (4 bytes): 0 (No interfaces)
// 2. Associated Info (16 bytes)
0A 00 00 00 // source_file_idx (4 bytes): points to StringIds[#10] -> "User.java"
00 00 00 00 // annotations_off (4 bytes): 0 (No annotations)
00 08 00 00 // class_data_off (4 bytes): 0x0800 (🔥 Core: points to specific data in the Data section)
00 00 00 00 // static_values_off (4 bytes): 0 (No static initial values)
Data Section: The place where actual code instructions (Dalvik bytecode) and constant values are truly stored.
When our code needs to load a class (prior to the introduction of DexCache and hash table optimizations), the underlying mechanism relies on mmap (memory mapping + page faults) + binary search. Physically speaking, all String IDs are stored in sorted order in the DEX file, and the strings in the Data section are tightly packed in the same order. Thus, we can read the middle string first, compare for binary search until we find the target. During this process, multiple page faults will occur, loading the corresponding 4KB data pages on demand.
Once the string is found, we use the same logic to find the corresponding type in Type IDs, and then find the class definition in Class Defs. The class metadata is then loaded into the Method Area in memory. Only when a specific method is actually executed will the system jump to the Data section to load and execute the corresponding bytecode.