1. 简介
ARM64中断入口相关的代码主要在arch/arm64/kernel/entry.S，启动过程中，会将中断向量表vectors的起始虚拟地址写入到VBAR_EL1。
当发生中断或者异常时，硬件会保存一些寄存器，然后就是软件的工作：
- 进入中断后，根据中断原因，跳转到对应的中断处理函数，这部分是汇编实现。
- 在中断处理函数中，先通过
kernel_entry将寄存器压栈，然后将栈空间记录的struct pt_regs传递给C语言的对应中断处理函数。 - 完成中断处理后，根据中断发生的异常等级，决定是调用
ret_to_user还是ret_to_kernel退出中断，这两个hasn’t都是对汇编代码kernel_exit的封装。
1.1. 中断向量基地址寄存器配置
在kernel启动的一开始，会将中断向量表的起始虚拟地址写入到VBAR_EL1。
1/// arch/arm64/kernel/head.S
2 adr_l x8, vectors // load VBAR_EL1 with virtual
3 msr vbar_el1, x8 // vector table address
4 isb
2. 中断向量表
根据DEN0024A_v8_architecture_PG描述，后缀t和h表示使用不同的sp指针。意思是捕获异常时所在的异常等级使用的sp寄存器。
Arm Architecture Reference Manual for A-profile architecture
中断向量一共有四组表，每组有四个异常入口，分别对应同步异常、IRQ、FIQ、SError。
| 入口 | 异常等级切换 | 运行状态 | 异常前使用的sp |
|---|---|---|---|
| 1 | N | AArch64 | SP_EL0 |
| 2 | N | AArch64 | SP_EL1/2/3 |
| 3 | Y | AArch64 | SP_EL0 |
| 4 | Y | AArch32 | SP_EL0 |
对于第一组入口，Linux在内核态下不可能使用SP_EL0作为栈指针，所以内核将其handler设置为UNHANDLED。
内核运行在EL1，在中断处理函数中，使用SP_EL1，总是指向当前进程的栈顶。当前进程被调度运行时，SP_EL1会从内核栈中恢复。
3. Linux内核的中断入口
内核的vectors定义如下，通过捕获异常的等级、使用的sp寄存器、异常类型、处理器指令集(AArch32/AArch64)，可以唯一确定要跳转的中断入口。
1/// arch/arm64/kernel/entry.S
2/*
3 * Exception vectors.
4 */
5 .pushsection ".entry.text", "ax"
6
7 .align 11
8SYM_CODE_START(vectors)
9 /// UNHANDLED
10 kernel_ventry 1, t, 64, sync // Synchronous EL1t
11 kernel_ventry 1, t, 64, irq // IRQ EL1t
12 kernel_ventry 1, t, 64, fiq // FIQ EL1t
13 kernel_ventry 1, t, 64, error // Error EL1t
14
15 /// 中断或异常发生在EL1
16 kernel_ventry 1, h, 64, sync // Synchronous EL1h
17 kernel_ventry 1, h, 64, irq // IRQ EL1h
18 kernel_ventry 1, h, 64, fiq // FIQ EL1h
19 kernel_ventry 1, h, 64, error // Error EL1h
20
21 /// 中断或异常发生在EL0
22 kernel_ventry 0, t, 64, sync // Synchronous 64-bit EL0
23 kernel_ventry 0, t, 64, irq // IRQ 64-bit EL0
24 kernel_ventry 0, t, 64, fiq // FIQ 64-bit EL0
25 kernel_ventry 0, t, 64, error // Error 64-bit EL0
26
27 /// UNHANDLED或中断或异常发生在EL0(AArch32)
28 kernel_ventry 0, t, 32, sync // Synchronous 32-bit EL0
29 kernel_ventry 0, t, 32, irq // IRQ 32-bit EL0
30 kernel_ventry 0, t, 32, fiq
根据DEN0024A_v8_architecture_PG描述，不同的异常等级，使用的寄存器有所不同。
3.1. 硬件的工作
进入和退出中断时，硬件完成的工作。
进入时主要工作如下：
- 将处理器状态PSTATE寄存器保存到SPSR_ELx中，其中是捕获异常级别。
- 将异常返回地址保存到ELR_ELx，其中是捕获异常级别。
- 设置PSTATE的DAIF为1，关闭调试异常、SError、IRQ和FIQ。
- 如果是同步异常或SError，将异常信息写入ESR_ELx。
- 如果是地址访问相关的异常，将导致异常的虚拟地址写入FAR_ELx。
- 切换到指定异常等级的sp。
- 将处理器设置到对应的异常等级，然后跳转到异常向量表执行。
软件调用eret退出时： 使用ELR_ELx和SPSR_ELx恢复处理器状态，然后返回到异常发生的位置。
Learn the architecture - AArch64 Exception Model
1When an exception is taken, the current state must be preserved so that it can be returned to. The
2PE will automatically preserve the exception return address and the current PSTATE.
3The state stored in the general-purpose registers must be preserved by software. The PE will then
4update the current PSTATE to the one defined in the architecture for that exception type, and
5branch to the exception handler in the vector table.
6The PSTATE the exception was taken from is stored in the System register SPSR_ELx, where <x> is
7the number of the Exception level that the exception was taken to. The exception return address is
8stored in ELR_ELx, where <x> is the Exception level that the exception was taken to.
Arm Architecture Reference Manual for A-profile architecture
4. 中断入口定义：kernel_ventry
1/// arch/arm64/kernel/entry.S
2 .macro kernel_ventry, el:req, ht:req, regsize:req, label:req
3 .align 7 /// 128(2^7)字节对齐
4.Lventry_start\@:
5 .if \el == 0
6 /*
7 * This must be the first instruction of the EL0 vector entries. It is
8 * skipped by the trampoline vectors, to trigger the cleanup.
9 */
10 b .Lskip_tramp_vectors_cleanup\@ /// 是跳过清理跳板中断向量表
11 /// 还不清楚跳板中断向量表如何起作用，但是如下指令实际不会走到
12 .if \regsize == 64
13 /// Thread ID registers，TPIDRRO_EL0包含当前处理器的 CPU 编号
14 mrs x30, tpidrro_el0 /// 备份tpidrro_el0到x30
15 msr tpidrro_el0, xzr /// 清零tpidrro_el0
16 .else
17 mov x30, xzr /// 清零x30(w30)，对应AArch32 fiq下的r14(lr)
18 .endif
19.Lskip_tramp_vectors_cleanup\@:
20 .endif
21
22 sub sp, sp, #PT_REGS_SIZE /// 在栈中为struct pt_regs留出空间
23#ifdef CONFIG_VMAP_STACK
24 /// 一些栈溢出相关操作和sp的设置，不影响中断处理流程分析
25#endif
26 b el\el\ht\()_\regsize\()_\label /// 跳转到真正的处理函数
27.org .Lventry_start\@ + 128 // Did we overflow the ventry slot?
kernel_ventry是一个汇编宏，其主要作用如下：
- 在栈中为
struct pt_regs留出空间 - 跳转到真正的处理函数，比如代码中
b el\el\ht\()_\regsize\()_\label
以kernel_ventry 1, t, 64, sync // Synchronous EL1t为例来看一下效果，最后会调用el1t_64_sync
4.1. entry_handler
中断处理函数使用汇编宏entry_handler进行定义，其代码如下：
1/// arch/arm64/kernel/entry.S
2 .macro entry_handler el:req, ht:req, regsize:req, label:req
3 /// 定义符号
4SYM_CODE_START_LOCAL(el\el\ht\()_\regsize\()_\label)
5 /// 上下文保存
6 kernel_entry \el, \regsize
7 /// 此时sp指向struct pt_regs基地址，将其作为参数传给C语言handler
8 mov x0, sp
9 /// 调用C语言handler
10 bl el\el\ht\()_\regsize\()_\label\()_handler
11 .if \el == 0
12 b ret_to_user /// 退出中断，返回用户模式
13 .else
14 b ret_to_kernel /// 退出中断，返回kernel模式
15 .endif
16SYM_CODE_END(el\el\ht\()_\regsize\()_\label)
17 .endm
entry_handler宏的主要工作如下：
- 调用
kernel_entry，其主要工作是上下文保存 - 调用C语言的handler
- handler返回后，如果是在EL0发生的中断，调用
ret_to_user，否则调用ret_to_kernel。软件处理完成。
以entry_handler 1, t, 64, sync为例，这一行声明了el1t_64_sync符号，其内部会调用el1t_64_sync_handler。
1/// arch/arm64/kernel/entry.S
2/*
3 * Early exception handlers
4 */
5 entry_handler 1, t, 64, sync
6/// SYM_CODE_START_LOCAL(el1t_64_sync)
7/// ... ...
8/// bl el1t_64_sync_handler
9/// ... ...
10/// SYM_CODE_END(el1t_64_sync)
11 entry_handler 1, t, 64, irq
12 entry_handler 1, t, 64, fiq
13 entry_handler 1, t, 64, error
14
15 entry_handler 1, h, 64, sync
16 entry_handler 1, h, 64, irq
17 entry_handler 1, h, 64, fiq
18 entry_handler 1, h, 64, error
19
20 entry_handler 0, t, 64, sync
21 entry_handler 0, t, 64, irq
22 entry_handler 0, t, 64, fiq
23 entry_handler 0, t, 64, error
24
25 entry_handler 0, t, 32, sync
26 entry_handler 0, t, 32, irq
27 entry_handler 0, t, 32, fiq
28 entry_handler 0, t, 32, error
4.2. 调用C函数前的准备kernel_entry
前边提到，栈顶要存储的是struct pt_regs，这里先看一下其定义。
1/// arch/arm64/include/asm/ptrace.h
2/*
3 * User structures for general purpose, floating point and debug registers.
4 */
5struct user_pt_regs {
6 __u64 regs[31];
7 __u64 sp;
8 __u64 pc;
9 __u64 pstate;
10};
11
12/*
13 * This struct defines the way the registers are stored on the stack during an
14 * exception. Note that sizeof(struct pt_regs) has to be a multiple of 16 (for
15 * stack alignment). struct user_pt_regs must form a prefix of struct pt_regs.
16 */
17struct pt_regs {
18 union {
19 struct user_pt_regs user_regs;
20 struct {
21 u64 regs[31];
22 u64 sp;
23 u64 pc;
24 u64 pstate;
25 };
26 };
27 u64 orig_x0;
28#ifdef __AARCH64EB__
29 u32 unused2;
30 s32 syscallno;
31#else
32 s32 syscallno;
33 u32 unused2;
34#endif
35 u64 sdei_ttbr1;
36 /* Only valid when ARM64_HAS_GIC_PRIO_MASKING is enabled. */
37 u64 pmr_save;
38 u64 stackframe[2];
39
40 /* Only valid for some EL1 exceptions. */
41 u64 lockdep_hardirqs;
42 u64 exit_rcu;
43};
继续向下分析之前，先记住tsk，这个代表x28寄存器，用了记录当前进程的struct thread_info。
1/// arch/arm64/kernel/entry.S
2/* GPRs used by entry code */
3tsk .req x28 // current thread_info
4.2.1. 寄存器压栈
1/// arch/arm64/kernel/entry.S
2 .macro kernel_entry, el, regsize = 64
3 .if \el == 0
4 /// 用户模式中断，根据需要，修改PSTATE寄存器
5 alternative_insn nop, SET_PSTATE_DIT(1), ARM64_HAS_DIT
6 .endif
7 .if \regsize == 32
8 mov w0, w0 // zero upper 32 bits of x0
9 .endif
10 /// 下边这些是将寄存器x0~x29压栈，后续可以使用struct pt_regs指针访问
11 stp x0, x1, [sp, #16 * 0]
12 stp x2, x3, [sp, #16 * 1]
13 stp x4, x5, [sp, #16 * 2]
14 stp x6, x7, [sp, #16 * 3]
15 stp x8, x9, [sp, #16 * 4]
16 stp x10, x11, [sp, #16 * 5]
17 stp x12, x13, [sp, #16 * 6]
18 stp x14, x15, [sp, #16 * 7]
19 stp x16, x17, [sp, #16 * 8]
20 stp x18, x19, [sp, #16 * 9]
21 stp x20, x21, [sp, #16 * 10]
22 stp x22, x23, [sp, #16 * 11]
23 stp x24, x25, [sp, #16 * 12]
24 stp x26, x27, [sp, #16 * 13]
25 stp x28, x29, [sp, #16 * 14]
4.2.2. el0和el1的处理
4.2.2.1. EL0
1 .if \el == 0 /// 用户模式中断或异常
2 clear_gp_regs /// 清零x0~x29
3 mrs x21, sp_el0 /// x21记录用户模式sp_el0，后续会保存到pt_regs.sp
4 /// __entry_task是一个percpu变量，记录当前cpu的task_struct
5 /// x20作为临时变量
6 /// ldr_this_cpu是将当前cpu的当前task_struct读到tsk(x28)
7 ldr_this_cpu tsk, __entry_task, x20
8 msr sp_el0, tsk /// sp_el0记录当前进程
9 /*
10 * Ensure MDSCR_EL1.SS is clear, since we can unmask debug exceptions
11 * when scheduling.
12 */
13 ldr x19, [tsk, #TSK_TI_FLAGS] /// 加载thread_info.flags到x19
14 /// 如果进程允许单步调试，关闭MDSCR_EL1的软件单步控制
15 disable_step_tsk x19, x20
16
17 /* Check for asynchronous tag check faults in user space */
18 ldr x0, [tsk, THREAD_SCTLR_USER] /// 加载thread.sctlr_user到x0
19 check_mte_async_tcf x22, x23, x0 /// Check for MTE asynchronous tag check faults
20
21#ifdef CONFIG_ARM64_PTR_AUTH
22 /// ... ...
23#endif
24
25 apply_ssbd 1, x22, x23
26
27 mte_set_kernel_gcr x22, x23
28
29 /*
30 * Any non-self-synchronizing system register updates required for
31 * kernel entry should be placed before this point.
32 */
33alternative_if ARM64_MTE
34 isb
35 b 1f
36alternative_else_nop_endif
37alternative_if ARM64_HAS_ADDRESS_AUTH
38 isb
39alternative_else_nop_endif
401:
41
42 scs_load_current
__entry_task是一个percpu变量。用于记录当前CPU当前进程的struct task_struct。
1/// arch/arm64/kernel/process.c
2/*
3 * We store our current task in sp_el0, which is clobbered by userspace. Keep a
4 * shadow copy so that we can restore this upon entry from userspace.
5 *
6 * This is *only* for exception entry from EL0, and is not valid until we
7 * __switch_to() a user task.
8 */
9DEFINE_PER_CPU(struct task_struct *, __entry_task);
4.2.2.2. EL1
1 .else
2 /// sp + sizeof(struct pt_regs)，x21记录出现异常时的sp，后续会保存到pt_regs.sp
3 add x21, sp, #PT_REGS_SIZE
4 /// 将sp_el0读到tsk，后边汇编代码可以用tsk来访问当前task_struct
5 get_current_task tsk
6 .endif /* \el == 0 */
4.2.3. 寄存和栈配置
1 mrs x22, elr_el1 /// 保存elr_el1
2 mrs x23, spsr_el1 /// 保存spsr_el1
3 stp lr, x21, [sp, #S_LR] /// lr和sp保存到pt_regs.regs[30](lr)和pt_regs.sp
4
5 /*
6 * For exceptions from EL0, create a final frame record.
7 * For exceptions from EL1, create a synthetic frame record so the
8 * interrupted code shows up in the backtrace.
9 */
10 .if \el == 0
11 stp xzr, xzr, [sp, #S_STACKFRAME] /// 清零pt_regs.stackframe[2]，用于终止栈回溯
12 .else
13 stp x29, x22, [sp, #S_STACKFRAME] /// 保存x29和x22(elr_el1)到pt_regs.stackframe[2]
14 .endif
15 add x29, sp, #S_STACKFRAME /// x29指向pt_regs.stackframe[2]
16
17#ifdef CONFIG_ARM64_SW_TTBR0_PAN
18 /// ... ...
19#endif
20
21 stp x22, x23, [sp, #S_PC] /// 保存elr_el1和spsr_el1到pt_regs.pc和pt_regs.pstate
22
23 /* Not in a syscall by default (el0_svc overwrites for real syscall) */
24 .if \el == 0
25 mov w21, #NO_SYSCALL
26 str w21, [sp, #S_SYSCALLNO]
27 .endif
28
29#ifdef CONFIG_ARM64_PSEUDO_NMI
30 /// ... ...
31#endif
32
33 /*
34 * Registers that may be useful after this macro is invoked:
35 *
36 * x20 - ICC_PMR_EL1
37 * x21 - aborted SP
38 * x22 - aborted PC
39 * x23 - aborted PSTATE
40 */
41 .endm
| pt_regs.x | reg(el1) | reg(el0) |
|---|---|---|
| regs[0] | x0 | x0 |
| regs[1] | x1 | x1 |
| … | … | … |
| regs[29] | x29(fp) | x29(fp) |
| regs[30] | x30(lr) | x30(lr) |
| sp | sp + PT_REGS_SIZE | sp_el0 |
| pc | elr_el1 | elr_el1 |
| pstate | spsr_el1 | spsr_el1 |
| orig_x0 | ||
| unused2 | - | - |
| syscallno | - | NO_SYSCALL |
| sdei_ttbr1 | - | - |
| pmr_save | - | - |
| stackframe[0] | x29 | 0 |
| stackframe[1] | elr_el1 | 0 |
| lockdep_hardirqs | - | - |
| exit_rcu | - | - |
做完这些之后，就可以调用C语言的处理函数了。
5. 中断处理C语言函数
arch/arm64/kernel/entry.S中的汇编代码最终会调用到C语言的handler，这些函数在arch/arm64/kernel/entry-common.c中的定义，对应声明在arch/arm64/include/asm/exception.h中。
1/// arch/arm64/include/asm/exception.h
2asmlinkage void el1t_64_sync_handler(struct pt_regs *regs);
3asmlinkage void el1t_64_irq_handler(struct pt_regs *regs);
4asmlinkage void el1t_64_fiq_handler(struct pt_regs *regs);
5asmlinkage void el1t_64_error_handler(struct pt_regs *regs);
6
7asmlinkage void el1h_64_sync_handler(struct pt_regs *regs);
8asmlinkage void el1h_64_irq_handler(struct pt_regs *regs);
9asmlinkage void el1h_64_fiq_handler(struct pt_regs *regs);
10asmlinkage void el1h_64_error_handler(struct pt_regs *regs);
11
12asmlinkage void el0t_64_sync_handler(struct pt_regs *regs);
13asmlinkage void el0t_64_irq_handler(struct pt_regs *regs);
14asmlinkage void el0t_64_fiq_handler(struct pt_regs *regs);
15asmlinkage void el0t_64_error_handler(struct pt_regs *regs);
16
17asmlinkage void el0t_32_sync_handler(struct pt_regs *regs);
18asmlinkage void el0t_32_irq_handler(struct pt_regs *regs);
19asmlinkage void el0t_32_fiq_handler(struct pt_regs *regs);
20asmlinkage void el0t_32_error_handler(struct pt_regs *regs);
5.1. UNHANDLED
早期的Linux内核，在vectors直接调用不支持的的handler，比如ventry el1_sync_invalid。在linux-6.6中，汇编部分做了通用化，而将差异放到了C语言代码中。
1/// arch/arm64/kernel/entry-common.c
2#define UNHANDLED(el, regsize, vector) \
3asmlinkage void noinstr el##_##regsize##_##vector##_handler(struct pt_regs *regs) \
4{ \
5 const char *desc = #regsize "-bit " #el " " #vector; \
6 __panic_unhandled(regs, desc, read_sysreg(esr_el1)); \
7}
1UNHANDLED(el1t, 64, sync) /// el1t_64_sync_handler
2UNHANDLED(el1t, 64, irq) /// el1t_64_irq_handler
3UNHANDLED(el1t, 64, fiq) /// el1t_64_fiq_handler
4UNHANDLED(el1t, 64, error) /// el1t_64_error_handler
在未使能CONFIG_COMPAT的情况下，32位中断也是UNHANDLED。
1#ifdef CONFIG_COMPAT
2 /// ... ...
3#else /* CONFIG_COMPAT */
4UNHANDLED(el0t, 32, sync) /// el0t_32_sync_handler
5UNHANDLED(el0t, 32, irq) /// el0t_32_irq_handler
6UNHANDLED(el0t, 32, fiq) /// el0t_32_fiq_handler
7UNHANDLED(el0t, 32, error) /// el0t_32_error_handler
8#endif /* CONFIG_COMPAT */
6. 中断返回
进行一些中断处理的收尾工作，ret_to_kernel和ret_to_user都是直接使用kernel_exit实现的。
6.1. ret_to_kernel
1SYM_CODE_START_LOCAL(ret_to_kernel)
2 kernel_exit 1
3SYM_CODE_END(ret_to_kernel)
6.2. ret_to_user
1SYM_CODE_START_LOCAL(ret_to_user)
2 /// 加载thread_info.flags到x19
3 ldr x19, [tsk, #TSK_TI_FLAGS] // re-check for single-step
4 /// call with daif masked
5 /// 如果进程允许单步调试(TIF_SINGLESTEP)，开启MDSCR_EL1的软件单步控制(DBG_MDSCR_SS)
6 enable_step_tsk x19, x2
7#ifdef CONFIG_GCC_PLUGIN_STACKLEAK
8 bl stackleak_erase_on_task_stack
9#endif
10 kernel_exit 0
11SYM_CODE_END(ret_to_user)
7. kernel_exit
7.1. el0关闭daif
1/// arch/arm64/kernel/entry.S
2 .macro kernel_exit, el
3 .if \el != 0
4 disable_daif /// 屏蔽中断
5 .endif
7.2. 从ELR和SPSR恢复
1#ifdef CONFIG_ARM64_PSEUDO_NMI
2 /// ... ...
3#endif
4
5 ldp x21, x22, [sp, #S_PC] // load ELR, SPSR
6
7#ifdef CONFIG_ARM64_SW_TTBR0_PAN
8 /// ... ...
9#endif
7.3. el0栈空间处理
1 .if \el == 0
2 ldr x23, [sp, #S_SP] // load return stack pointer
3 msr sp_el0, x23 /// 从栈中恢复sp_el0
4 tst x22, #PSR_MODE32_BIT // native task?
5 b.eq 3f
6
7#ifdef CONFIG_ARM64_ERRATUM_845719
8/// ... ...
9#endif
103:
11 scs_save tsk
12
13 /* Ignore asynchronous tag check faults in the uaccess routines */
14 ldr x0, [tsk, THREAD_SCTLR_USER]
15 clear_mte_async_tcf x0
16
17#ifdef CONFIG_ARM64_PTR_AUTH
18/// ... ...
19#endif
20
21 mte_set_user_gcr tsk, x0, x1
22
23 apply_ssbd 0, x0, x1
24 .endif
7.4. 寄存器出栈
1 /// 前边从栈中恢复了ELR和SPSR到x21和x22
2 msr elr_el1, x21 // set up the return data
3 msr spsr_el1, x22
4 /// 恢复通用寄存器x0~x29
5 ldp x0, x1, [sp, #16 * 0]
6 ldp x2, x3, [sp, #16 * 1]
7 ldp x4, x5, [sp, #16 * 2]
8 ldp x6, x7, [sp, #16 * 3]
9 ldp x8, x9, [sp, #16 * 4]
10 ldp x10, x11, [sp, #16 * 5]
11 ldp x12, x13, [sp, #16 * 6]
12 ldp x14, x15, [sp, #16 * 7]
13 ldp x16, x17, [sp, #16 * 8]
14 ldp x18, x19, [sp, #16 * 9]
15 ldp x20, x21, [sp, #16 * 10]
16 ldp x22, x23, [sp, #16 * 11]
17 ldp x24, x25, [sp, #16 * 12]
18 ldp x26, x27, [sp, #16 * 13]
19 ldp x28, x29, [sp, #16 * 14]
7.5. el0和el1恢复
7.5.1. el0
1 .if \el == 0
2alternative_if ARM64_WORKAROUND_2966298
3 tlbi vale1, xzr
4 dsb nsh
5alternative_else_nop_endif
6alternative_if_not ARM64_UNMAP_KERNEL_AT_EL0
7 /// 不使能KPTI(Kernel page table isolation) patch
8 ldr lr, [sp, #S_LR] /// 中断处理过程中修改了lr，这时恢复用户态lr
9 add sp, sp, #PT_REGS_SIZE // restore sp
10 eret /// 从异常返回，继续从中断处执行
11alternative_else_nop_endif
12#ifdef CONFIG_UNMAP_KERNEL_AT_EL0
13 msr far_el1, x29
14
15 ldr_this_cpu x30, this_cpu_vector, x29
16 tramp_alias x29, tramp_exit
17 msr vbar_el1, x30 // install vector table
18 ldr lr, [sp, #S_LR] // restore x30
19 add sp, sp, #PT_REGS_SIZE // restore sp
20 br x29
21#endif
7.5.2. el1
1 .else
2 ldr lr, [sp, #S_LR] /// 中断处理过程中修改了lr，从栈中恢复lr
3 add sp, sp, #PT_REGS_SIZE // restore sp
4
5 /* Ensure any device/NC reads complete */
6 alternative_insn nop, "dmb sy", ARM64_WORKAROUND_1508412
7
8 eret /// 从异常返回
9 .endif
7.6. Speculation barrier
1 sb
2 .endm
1/// arch/arm64/include/asm/assembler.h
2/*
3 * Speculation barrier
4 */
5 .macro sb
6alternative_if_not ARM64_HAS_SB
7 dsb nsh
8 isb
9alternative_else
10 SB_BARRIER_INSN
11 nop
12alternative_endif
13 .endm
8. 总结
8.1. daif状态切换
- 发生异常时，硬件将
pstate.daif保存到spsr_el1，屏蔽daif。 - 中断C函数中，关闭或部分/全部打开。
- 在kernel_exit中，如果是返回用户态中断，将daif关闭。
- 硬件从SPSR中恢复原daif状态。
8.2. SP和current
Linux内核运行在EL1，sp_el0压栈后，就可以用sp_el0来存放当前task_struct。
- 对于
el0t_64和el0t_32，是用户态中断，在陷入内核时，通过msr sp_el0, tsk将sp_el0指向了当前task_struct 。 - 对于
el1t_64和el1h_64，是内核态中断，在发生中断前，要么是初始化阶段经设置sp_el0指向init_task，要么是从用户态陷入内核态，sp_el0也指向了当前task_struct 。
ARM上中断栈和内核栈是共享的。每个进程/线程都有自己的内核栈，除init_task外，其他进程/线程在fork时会申请内存作为内核栈，并将sp_el1指向该内核栈，见copy_thread。而在进程第一次运行或被调度后，栈会向下增长。在发生中断时，就会使当前进程/线程的栈向下增长，在中断处理完成后，又逐步将栈还原。
| el1t_64 | el1h_64 | el0t_64 | el0t_32 | |
|---|---|---|---|---|
| 异常前使用的sp | sp_el0 | sp_el1 | sp_el0 | sp_el0 |
| kernel_ventry | sp_el1 -= PT_REGS_SIZE | sp_el1 -= PT_REGS_SIZE | sp_el1 -= PT_REGS_SIZE | sp_el1 -= PT_REGS_SIZE |
| kernel_entry | add x21, sp, #PT_REGS_SIZE | add x21, sp, #PT_REGS_SIZE | mrs x21, sp_el0 | mrs x21, sp_el0 |
| kernel_entry | get_current_task tsk | get_current_task tsk | msr sp_el0, tsk | msr sp_el0, tsk |
| kernel_entry | stp lr, x21, [sp, #S_LR] | stp lr, x21, [sp, #S_LR] | stp lr, x21, [sp, #S_LR] | stp lr, x21, [sp, #S_LR] |
| C语言handler | current | current | current | |
| kernel_exit | ldr x23, [sp, #S_SP] | ldr x23, [sp, #S_SP] | ||
| kernel_exit | msr sp_el0, x23 | msr sp_el0, x23 | ||
| kernel_exit | sp_el1 += PT_REGS_SIZE | sp_el1 += PT_REGS_SIZE | sp_el1 += PT_REGS_SIZE | sp_el1 += PT_REGS_SIZE |
9. 反汇编
这里给出ARM64 vmlinux的反汇编，方便对比源码和最终的指令。
9.1. vectors
1ffff800080011000 <vectors>:
2vectors():
3linux-6.6/arch/arm64/kernel/entry.S:517
4/// 由kernel_ventry宏生成
5ffff800080011000: d10543ff sub sp, sp, #0x150
6ffff800080011004: 8b2063ff add sp, sp, x0
7ffff800080011008: cb2063e0 sub x0, sp, x0
8ffff80008001100c: 37700080 tbnz w0, #14, ffff80008001101c <vectors+0x1c>
9ffff800080011010: cb2063e0 sub x0, sp, x0
10ffff800080011014: cb2063ff sub sp, sp, x0
11/// el1t_64_sync由entry_handler声明
12ffff800080011018: 1400021e b ffff800080011890 <el1t_64_sync>
13ffff80008001101c: d51bd040 msr tpidr_el0, x0
14ffff800080011020: cb2063e0 sub x0, sp, x0
15ffff800080011024: d51bd060 msr tpidrro_el0, x0
16ffff800080011028: f000aec0 adrp x0, ffff8000815ec000 <overflow_stack+0xcf0>
17ffff80008001102c: 910c401f add sp, x0, #0x310
18ffff800080011030: d538d080 mrs x0, tpidr_el1
19ffff800080011034: 8b2063ff add sp, sp, x0
20ffff800080011038: d53bd040 mrs x0, tpidr_el0
21ffff80008001103c: cb2063e0 sub x0, sp, x0
22ffff800080011040: f274cc1f tst x0, #0xfffffffffffff000
23ffff800080011044: 54003de1 b.ne ffff800080011800 <__bad_stack> // b.any
24ffff800080011048: cb2063ff sub sp, sp, x0
25ffff80008001104c: d53bd060 mrs x0, tpidrro_el0
26ffff800080011050: 14000210 b ffff800080011890 <el1t_64_sync>
27 ...
28linux-6.6/arch/arm64/kernel/entry.S:518
29ffff800080011080: d10543ff sub sp, sp, #0x150
30ffff800080011084: 8b2063ff add sp, sp, x0
31ffff800080011088: cb2063e0 sub x0, sp, x0
32ffff80008001108c: 37700080 tbnz w0, #14, ffff80008001109c <vectors+0x9c>
33ffff800080011090: cb2063e0 sub x0, sp, x0
34ffff800080011094: cb2063ff sub sp, sp, x0
35ffff800080011098: 1400021e b ffff800080011910 <el1t_64_irq>
36ffff80008001109c: d51bd040 msr tpidr_el0, x0
37ffff8000800110a0: cb2063e0 sub x0, sp, x0
38ffff8000800110a4: d51bd060 msr tpidrro_el0, x0
39ffff8000800110a8: f000aec0 adrp x0, ffff8000815ec000 <overflow_stack+0xcf0>
40ffff8000800110ac: 910c401f add sp, x0, #0x310
41ffff8000800110b0: d538d080 mrs x0, tpidr_el1
42ffff8000800110b4: 8b2063ff add sp, sp, x0
43ffff8000800110b8: d53bd040 mrs x0, tpidr_el0
44ffff8000800110bc: cb2063e0 sub x0, sp, x0
45ffff8000800110c0: f274cc1f tst x0, #0xfffffffffffff000
46ffff8000800110c4: 540039e1 b.ne ffff800080011800 <__bad_stack> // b.any
47ffff8000800110c8: cb2063ff sub sp, sp, x0
48ffff8000800110cc: d53bd060 mrs x0, tpidrro_el0
49ffff8000800110d0: 14000210 b ffff800080011910 <el1t_64_irq>
50
51/// ... ...
52/data/eel/source/kernel/linux-6.6/arch/arm64/kernel/entry.S:527
53ffff800080011400: 14000003 b ffff80008001140c <vectors+0x40c>
54ffff800080011404: d53bd07e mrs x30, tpidrro_el0
55ffff800080011408: d51bd07f msr tpidrro_el0, xzr
56ffff80008001140c: d10543ff sub sp, sp, #0x150
57ffff800080011410: 8b2063ff add sp, sp, x0
58ffff800080011414: cb2063e0 sub x0, sp, x0
59ffff800080011418: 37700080 tbnz w0, #14, ffff800080011428 <vectors+0x428>
60ffff80008001141c: cb2063e0 sub x0, sp, x0
61ffff800080011420: cb2063ff sub sp, sp, x0
62ffff800080011424: 1400021b b ffff800080011c90 <el0t_64_sync>
63ffff800080011428: d51bd040 msr tpidr_el0, x0
64ffff80008001142c: cb2063e0 sub x0, sp, x0
65ffff800080011430: d51bd060 msr tpidrro_el0, x0
66ffff800080011434: f000aec0 adrp x0, ffff8000815ec000 <overflow_stack+0xcf0>
67ffff800080011438: 910c401f add sp, x0, #0x310
68ffff80008001143c: d538d080 mrs x0, tpidr_el1
69ffff800080011440: 8b2063ff add sp, sp, x0
70ffff800080011444: d53bd040 mrs x0, tpidr_el0
71ffff800080011448: cb2063e0 sub x0, sp, x0
72ffff80008001144c: f274cc1f tst x0, #0xfffffffffffff000
73ffff800080011450: 54001d81 b.ne ffff800080011800 <__bad_stack> // b.any
74ffff800080011454: cb2063ff sub sp, sp, x0
75ffff800080011458: d53bd060 mrs x0, tpidrro_el0
76ffff80008001145c: 1400020d b ffff800080011c90 <el0t_64_sync>
77 ...
9.2. el1t_64_sync和el1t_64_irq
1/// el1t_64_sync由entry_handler声明
2ffff800080011890 <el1t_64_sync>:
3el1t_64_sync():
4linux-6.6/arch/arm64/kernel/entry.S:585
5/// kernel_entry生成
6ffff800080011890: a90007e0 stp x0, x1, [sp]
7ffff800080011894: a9010fe2 stp x2, x3, [sp, #16]
8ffff800080011898: a90217e4 stp x4, x5, [sp, #32]
9ffff80008001189c: a9031fe6 stp x6, x7, [sp, #48]
10ffff8000800118a0: a90427e8 stp x8, x9, [sp, #64]
11ffff8000800118a4: a9052fea stp x10, x11, [sp, #80]
12ffff8000800118a8: a90637ec stp x12, x13, [sp, #96]
13ffff8000800118ac: a9073fee stp x14, x15, [sp, #112]
14ffff8000800118b0: a90847f0 stp x16, x17, [sp, #128]
15ffff8000800118b4: a9094ff2 stp x18, x19, [sp, #144]
16ffff8000800118b8: a90a57f4 stp x20, x21, [sp, #160]
17ffff8000800118bc: a90b5ff6 stp x22, x23, [sp, #176]
18ffff8000800118c0: a90c67f8 stp x24, x25, [sp, #192]
19ffff8000800118c4: a90d6ffa stp x26, x27, [sp, #208]
20ffff8000800118c8: a90e77fc stp x28, x29, [sp, #224]
21ffff8000800118cc: 910543f5 add x21, sp, #0x150
22ffff8000800118d0: d538411c mrs x28, sp_el0
23ffff8000800118d4: d5384036 mrs x22, elr_el1
24ffff8000800118d8: d5384017 mrs x23, spsr_el1
25ffff8000800118dc: a90f57fe stp x30, x21, [sp, #240]
26ffff8000800118e0: a9135bfd stp x29, x22, [sp, #304]
27ffff8000800118e4: 9104c3fd add x29, sp, #0x130
28ffff8000800118e8: a9105ff6 stp x22, x23, [sp, #256]
29ffff8000800118ec: 14000005 b ffff800080011900 <el1t_64_sync+0x70>
30ffff8000800118f0: d5384614 mrs x20, s3_0_c4_c6_0
31ffff8000800118f4: f90097f4 str x20, [sp, #296]
32ffff8000800118f8: d2801e14 mov x20, #0xf0 // #240
33ffff8000800118fc: d5184614 msr s3_0_c4_c6_0, x20
34ffff800080011900: 910003e0 mov x0, sp
35/// 调用C语言处理函数
36ffff800080011904: 94364225 bl ffff800080da2198 <el1t_64_sync_handler>
37/// 中断返回
38ffff800080011908: 1400043a b ffff8000800129f0 <ret_to_kernel>
39ffff80008001190c: d503201f nop
40
41ffff800080011910 <el1t_64_irq>:
42el1t_64_irq():
43linux-6.6/arch/arm64/kernel/entry.S:586
44ffff800080011910: a90007e0 stp x0, x1, [sp]
45ffff800080011914: a9010fe2 stp x2, x3, [sp, #16]
46ffff800080011918: a90217e4 stp x4, x5, [sp, #32]
47ffff80008001191c: a9031fe6 stp x6, x7, [sp, #48]
48ffff800080011920: a90427e8 stp x8, x9, [sp, #64]
49ffff800080011924: a9052fea stp x10, x11, [sp, #80]
50ffff800080011928: a90637ec stp x12, x13, [sp, #96]
51ffff80008001192c: a9073fee stp x14, x15, [sp, #112]
52ffff800080011930: a90847f0 stp x16, x17, [sp, #128]
53ffff800080011934: a9094ff2 stp x18, x19, [sp, #144]
54ffff800080011938: a90a57f4 stp x20, x21, [sp, #160]
55ffff80008001193c: a90b5ff6 stp x22, x23, [sp, #176]
56ffff800080011940: a90c67f8 stp x24, x25, [sp, #192]
57ffff800080011944: a90d6ffa stp x26, x27, [sp, #208]
58ffff800080011948: a90e77fc stp x28, x29, [sp, #224]
59ffff80008001194c: 910543f5 add x21, sp, #0x150
60ffff800080011950: d538411c mrs x28, sp_el0
61ffff800080011954: d5384036 mrs x22, elr_el1
62ffff800080011958: d5384017 mrs x23, spsr_el1
63ffff80008001195c: a90f57fe stp x30, x21, [sp, #240]
64ffff800080011960: a9135bfd stp x29, x22, [sp, #304]
65ffff800080011964: 9104c3fd add x29, sp, #0x130
66ffff800080011968: a9105ff6 stp x22, x23, [sp, #256]
67ffff80008001196c: 14000005 b ffff800080011980 <el1t_64_irq+0x70>
68ffff800080011970: d5384614 mrs x20, s3_0_c4_c6_0
69ffff800080011974: f90097f4 str x20, [sp, #296]
70ffff800080011978: d2801e14 mov x20, #0xf0 // #240
71ffff80008001197c: d5184614 msr s3_0_c4_c6_0, x20
72ffff800080011980: 910003e0 mov x0, sp
73/// 调用C语言处理函数
74ffff800080011984: 9436420d bl ffff800080da21b8 <el1t_64_irq_handler>
75/// 中断返回
76ffff800080011988: 1400041a b ffff8000800129f0 <ret_to_kernel>
77ffff80008001198c: d503201f nop
9.3. el0t_64_sync
1ffff800080011c90 <el0t_64_sync>:
2el0t_64_sync():
3linux-6.6/arch/arm64/kernel/entry.S:595
4ffff800080011c90: d503201f nop
5ffff800080011c94: a90007e0 stp x0, x1, [sp]
6ffff800080011c98: a9010fe2 stp x2, x3, [sp, #16]
7ffff800080011c9c: a90217e4 stp x4, x5, [sp, #32]
8ffff800080011ca0: a9031fe6 stp x6, x7, [sp, #48]
9ffff800080011ca4: a90427e8 stp x8, x9, [sp, #64]
10ffff800080011ca8: a9052fea stp x10, x11, [sp, #80]
11ffff800080011cac: a90637ec stp x12, x13, [sp, #96]
12ffff800080011cb0: a9073fee stp x14, x15, [sp, #112]
13ffff800080011cb4: a90847f0 stp x16, x17, [sp, #128]
14ffff800080011cb8: a9094ff2 stp x18, x19, [sp, #144]
15ffff800080011cbc: a90a57f4 stp x20, x21, [sp, #160]
16ffff800080011cc0: a90b5ff6 stp x22, x23, [sp, #176]
17ffff800080011cc4: a90c67f8 stp x24, x25, [sp, #192]
18ffff800080011cc8: a90d6ffa stp x26, x27, [sp, #208]
19ffff800080011ccc: a90e77fc stp x28, x29, [sp, #224]
20/// 用户态中断，重置部分通用寄存器
21ffff800080011cd0: aa1f03e0 mov x0, xzr
22ffff800080011cd4: aa1f03e1 mov x1, xzr
23ffff800080011cd8: aa1f03e2 mov x2, xzr
24ffff800080011cdc: aa1f03e3 mov x3, xzr
25ffff800080011ce0: aa1f03e4 mov x4, xzr
26ffff800080011ce4: aa1f03e5 mov x5, xzr
27ffff800080011ce8: aa1f03e6 mov x6, xzr
28ffff800080011cec: aa1f03e7 mov x7, xzr
29ffff800080011cf0: aa1f03e8 mov x8, xzr
30ffff800080011cf4: aa1f03e9 mov x9, xzr
31ffff800080011cf8: aa1f03ea mov x10, xzr
32ffff800080011cfc: aa1f03eb mov x11, xzr
33ffff800080011d00: aa1f03ec mov x12, xzr
34ffff800080011d04: aa1f03ed mov x13, xzr
35ffff800080011d08: aa1f03ee mov x14, xzr
36ffff800080011d0c: aa1f03ef mov x15, xzr
37ffff800080011d10: aa1f03f0 mov x16, xzr
38ffff800080011d14: aa1f03f1 mov x17, xzr
39ffff800080011d18: aa1f03f2 mov x18, xzr
40ffff800080011d1c: aa1f03f3 mov x19, xzr
41ffff800080011d20: aa1f03f4 mov x20, xzr
42ffff800080011d24: aa1f03f5 mov x21, xzr
43ffff800080011d28: aa1f03f6 mov x22, xzr
44ffff800080011d2c: aa1f03f7 mov x23, xzr
45ffff800080011d30: aa1f03f8 mov x24, xzr
46ffff800080011d34: aa1f03f9 mov x25, xzr
47ffff800080011d38: aa1f03fa mov x26, xzr
48ffff800080011d3c: aa1f03fb mov x27, xzr
49ffff800080011d40: aa1f03fc mov x28, xzr
50ffff800080011d44: aa1f03fd mov x29, xzr
51ffff800080011d48: d5384115 mrs x21, sp_el0
52ffff800080011d4c: d000aedc adrp x28, ffff8000815eb000 <this_cpu_vector>
53ffff800080011d50: 910c239c add x28, x28, #0x308
54ffff800080011d54: d538d094 mrs x20, tpidr_el1
55ffff800080011d58: f8746b9c ldr x28, [x28, x20]
56ffff800080011d5c: d518411c msr sp_el0, x28
57ffff800080011d60: f9400393 ldr x19, [x28]
58ffff800080011d64: 36a800b3 tbz w19, #21, ffff800080011d78 <el0t_64_sync+0xe8>
59ffff800080011d68: d5300254 mrs x20, mdscr_el1
60ffff800080011d6c: 927ffa94 and x20, x20, #0xfffffffffffffffe
61ffff800080011d70: d5100254 msr mdscr_el1, x20
62ffff800080011d74: d5033fdf isb
63ffff800080011d78: f9476b80 ldr x0, [x28, #3792]
64ffff800080011d7c: 14000007 b ffff800080011d98 <el0t_64_sync+0x108>
65ffff800080011d80: b63800c0 tbz x0, #39, ffff800080011d98 <el0t_64_sync+0x108>
66ffff800080011d84: d5385636 mrs x22, tfsre0_el1
67ffff800080011d88: 36000096 tbz w22, #0, ffff800080011d98 <el0t_64_sync+0x108>
68ffff800080011d8c: d2800416 mov x22, #0x20 // #32
69ffff800080011d90: 91000397 add x23, x28, #0x0
70ffff800080011d94: f83632ff stset x22, [x23]
71ffff800080011d98: d503201f nop
72ffff800080011d9c: d503201f nop
73ffff800080011da0: d503201f nop
74ffff800080011da4: d503201f nop
75ffff800080011da8: d503201f nop
76ffff800080011dac: d503201f nop
77ffff800080011db0: d503201f nop
78ffff800080011db4: d503201f nop
79ffff800080011db8: d503201f nop
80ffff800080011dbc: d503201f nop
81ffff800080011dc0: 1400000b b ffff800080011dec <el0t_64_sync+0x15c>
82ffff800080011dc4: d000aed7 adrp x23, ffff8000815eb000 <this_cpu_vector>
83ffff800080011dc8: 9100a2f7 add x23, x23, #0x28
84ffff800080011dcc: d538d096 mrs x22, tpidr_el1
85ffff800080011dd0: f8766af7 ldr x23, [x23, x22]
86ffff800080011dd4: b40000d7 cbz x23, ffff800080011dec <el0t_64_sync+0x15c>
87ffff800080011dd8: f9400397 ldr x23, [x28]
88ffff800080011ddc: 37c80097 tbnz w23, #25, ffff800080011dec <el0t_64_sync+0x15c>
89ffff800080011de0: 32013fe0 mov w0, #0x80007fff // #-2147450881
90ffff800080011de4: 52800021 mov w1, #0x1 // #1
91ffff800080011de8: d503201f nop
92ffff800080011dec: d503201f nop
93ffff800080011df0: d503201f nop
94ffff800080011df4: d503201f nop
95ffff800080011df8: d5384036 mrs x22, elr_el1
96ffff800080011dfc: d5384017 mrs x23, spsr_el1
97ffff800080011e00: a90f57fe stp x30, x21, [sp, #240]
98ffff800080011e04: a9137fff stp xzr, xzr, [sp, #304]
99ffff800080011e08: 9104c3fd add x29, sp, #0x130
100ffff800080011e0c: a9105ff6 stp x22, x23, [sp, #256]
101ffff800080011e10: 12800015 mov w21, #0xffffffff // #-1
102ffff800080011e14: b9011bf5 str w21, [sp, #280]
103ffff800080011e18: 14000005 b ffff800080011e2c <el0t_64_sync+0x19c>
104ffff800080011e1c: d5384614 mrs x20, s3_0_c4_c6_0
105ffff800080011e20: f90097f4 str x20, [sp, #296]
106ffff800080011e24: d2801e14 mov x20, #0xf0 // #240
107ffff800080011e28: d5184614 msr s3_0_c4_c6_0, x20
108ffff800080011e2c: 910003e0 mov x0, sp
109/// 调用C语言处理函数
110ffff800080011e30: 94364160 bl ffff800080da23b0 <el0t_64_sync_handler>
111/// 中断返回
112ffff800080011e34: 1400030d b ffff800080012a68 <ret_to_user>
9.4. ret_to_kernel
1ffff8000800129f0 <ret_to_kernel>:
2ret_to_kernel():
3linux-6.6/arch/arm64/kernel/entry.S:606
4ffff8000800129f0: d5034fdf msr daifset, #0xf
5ffff8000800129f4: 14000004 b ffff800080012a04 <ret_to_kernel+0x14>
6ffff8000800129f8: f94097f4 ldr x20, [sp, #296]
7ffff8000800129fc: d5184614 msr s3_0_c4_c6_0, x20
8ffff800080012a00: d5033f9f dsb sy
9ffff800080012a04: a9505bf5 ldp x21, x22, [sp, #256]
10ffff800080012a08: d5184035 msr elr_el1, x21
11ffff800080012a0c: d5184016 msr spsr_el1, x22
12ffff800080012a10: a94007e0 ldp x0, x1, [sp]
13ffff800080012a14: a9410fe2 ldp x2, x3, [sp, #16]
14ffff800080012a18: a94217e4 ldp x4, x5, [sp, #32]
15ffff800080012a1c: a9431fe6 ldp x6, x7, [sp, #48]
16ffff800080012a20: a94427e8 ldp x8, x9, [sp, #64]
17ffff800080012a24: a9452fea ldp x10, x11, [sp, #80]
18ffff800080012a28: a94637ec ldp x12, x13, [sp, #96]
19ffff800080012a2c: a9473fee ldp x14, x15, [sp, #112]
20ffff800080012a30: a94847f0 ldp x16, x17, [sp, #128]
21ffff800080012a34: a9494ff2 ldp x18, x19, [sp, #144]
22ffff800080012a38: a94a57f4 ldp x20, x21, [sp, #160]
23ffff800080012a3c: a94b5ff6 ldp x22, x23, [sp, #176]
24ffff800080012a40: a94c67f8 ldp x24, x25, [sp, #192]
25ffff800080012a44: a94d6ffa ldp x26, x27, [sp, #208]
26ffff800080012a48: a94e77fc ldp x28, x29, [sp, #224]
27ffff800080012a4c: f9407bfe ldr x30, [sp, #240]
28ffff800080012a50: 910543ff add sp, sp, #0x150
29ffff800080012a54: d503201f nop
30/// 从异常返回
31ffff800080012a58: d69f03e0 eret
32ffff800080012a5c: d503379f dsb nsh
33ffff800080012a60: d5033fdf isb
34ffff800080012a64: d503201f nop
9.5. ret_to_user
1ffff800080012a68 <ret_to_user>:
2ret_to_user():
3linux-6.6/arch/arm64/kernel/entry.S:610
4ffff800080012a68: f9400393 ldr x19, [x28]
5linux-6.6/arch/arm64/kernel/entry.S:611
6ffff800080012a6c: 36a80093 tbz w19, #21, ffff800080012a7c <ret_to_user+0x14>
7ffff800080012a70: d5300242 mrs x2, mdscr_el1
8ffff800080012a74: b2400042 orr x2, x2, #0x1
9ffff800080012a78: d5100242 msr mdscr_el1, x2
10linux-6.6/arch/arm64/kernel/entry.S:615
11ffff800080012a7c: 14000004 b ffff800080012a8c <ret_to_user+0x24>
12ffff800080012a80: f94097f4 ldr x20, [sp, #296]
13ffff800080012a84: d5184614 msr s3_0_c4_c6_0, x20
14ffff800080012a88: d5033f9f dsb sy
15ffff800080012a8c: a9505bf5 ldp x21, x22, [sp, #256]
16ffff800080012a90: f9407ff7 ldr x23, [sp, #248]
17ffff800080012a94: d5184117 msr sp_el0, x23
18ffff800080012a98: f27c02df tst x22, #0x10
19ffff800080012a9c: 54000020 b.eq ffff800080012aa0 <ret_to_user+0x38> // b.none
20ffff800080012aa0: f9476b80 ldr x0, [x28, #3792]
21ffff800080012aa4: d503201f nop
22ffff800080012aa8: d503201f nop
23ffff800080012aac: d503201f nop
24ffff800080012ab0: d503201f nop
25ffff800080012ab4: d503201f nop
26ffff800080012ab8: d503201f nop
27ffff800080012abc: d503201f nop
28ffff800080012ac0: d503201f nop
29ffff800080012ac4: d503201f nop
30ffff800080012ac8: d503201f nop
31ffff800080012acc: d503201f nop
32ffff800080012ad0: d503201f nop
33ffff800080012ad4: d503201f nop
34ffff800080012ad8: 1400000b b ffff800080012b04 <ret_to_user+0x9c>
35ffff800080012adc: b000aec1 adrp x1, ffff8000815eb000 <this_cpu_vector>
36ffff800080012ae0: 9100a021 add x1, x1, #0x28
37ffff800080012ae4: d538d080 mrs x0, tpidr_el1
38ffff800080012ae8: f8606821 ldr x1, [x1, x0]
39ffff800080012aec: b40000c1 cbz x1, ffff800080012b04 <ret_to_user+0x9c>
40ffff800080012af0: f9400381 ldr x1, [x28]
41ffff800080012af4: 37c80081 tbnz w1, #25, ffff800080012b04 <ret_to_user+0x9c>
42ffff800080012af8: 32013fe0 mov w0, #0x80007fff // #-2147450881
43ffff800080012afc: 52800001 mov w1, #0x0 // #0
44ffff800080012b00: d503201f nop
45ffff800080012b04: d5184035 msr elr_el1, x21
46ffff800080012b08: d5184016 msr spsr_el1, x22
47ffff800080012b0c: a94007e0 ldp x0, x1, [sp]
48ffff800080012b10: a9410fe2 ldp x2, x3, [sp, #16]
49ffff800080012b14: a94217e4 ldp x4, x5, [sp, #32]
50ffff800080012b18: a9431fe6 ldp x6, x7, [sp, #48]
51ffff800080012b1c: a94427e8 ldp x8, x9, [sp, #64]
52ffff800080012b20: a9452fea ldp x10, x11, [sp, #80]
53ffff800080012b24: a94637ec ldp x12, x13, [sp, #96]
54ffff800080012b28: a9473fee ldp x14, x15, [sp, #112]
55ffff800080012b2c: a94847f0 ldp x16, x17, [sp, #128]
56ffff800080012b30: a9494ff2 ldp x18, x19, [sp, #144]
57ffff800080012b34: a94a57f4 ldp x20, x21, [sp, #160]
58ffff800080012b38: a94b5ff6 ldp x22, x23, [sp, #176]
59ffff800080012b3c: a94c67f8 ldp x24, x25, [sp, #192]
60ffff800080012b40: a94d6ffa ldp x26, x27, [sp, #208]
61ffff800080012b44: a94e77fc ldp x28, x29, [sp, #224]
62ffff800080012b48: d503201f nop
63ffff800080012b4c: d503201f nop
64ffff800080012b50: f9407bfe ldr x30, [sp, #240]
65ffff800080012b54: 910543ff add sp, sp, #0x150
66/// 从异常返回
67ffff800080012b58: d69f03e0 eret
68#ifdef CONFIG_UNMAP_KERNEL_AT_EL0
69ffff800080012b5c: d518601d msr far_el1, x29
70ffff800080012b60: b000aede adrp x30, ffff8000815eb000 <this_cpu_vector>
71ffff800080012b64: 910003de add x30, x30, #0x0
72ffff800080012b68: d538d09d mrs x29, tpidr_el1
73ffff800080012b6c: f87d6bde ldr x30, [x30, x29]
74ffff800080012b70: 92c0801d mov x29, #0xfffffbffffffffff // #-4398046511105
75ffff800080012b74: f2bfbbfd movk x29, #0xfddf, lsl #16
76ffff800080012b78: f290001d movk x29, #0x8000
77ffff800080012b7c: d518c01e msr vbar_el1, x30
78ffff800080012b80: f9407bfe ldr x30, [sp, #240]
79ffff800080012b84: 910543ff add sp, sp, #0x150
80ffff800080012b88: d61f03a0 br x29
81#endif
82ffff800080012b8c: d503379f dsb nsh
83ffff800080012b90: d5033fdf isb
84ffff800080012b94: d503201f nop
85ffff800080012b98: d503201f nop
