1. * - pointer-to short-cut
1.1. SYNOPSIS
1* (struct or union command arguments)
1.2. DESCRIPTION
1 This command is a short-cut command that replaces the requirement to enter
2 "struct" or "union" command names. For details on the arguments to
3 those commands, enter "help struct" or "help union".
1.3. EXAMPLES
1 Dump the page structure at address c02943c0:
2
3 crash> *page c02943c0
4 struct page {
5 next = 0xc0fae740,
6 prev = 0xc0018fb0,
7 inode = 0x0,
8 offset = 0x3f000,
9 next_hash = 0xc02d6310,
10 count = {
11 counter = 0x1
12 },
13 flags = 0x310,
14 wait = 0xc02943d8,
15 pprev_hash = 0x0,
16 buffers = 0x0
17 }
2. alias - command aliases
2.1. SYNOPSIS
1alias [alias] [command string]
2.2. DESCRIPTION
1 This command creates an alias for a given command string. If no arguments
2 are entered, the current list of aliases are displayed. If one argument is
3 entered, the command string for that alias, if any, is displayed.
4
5 alias the single word to be used as an alias
6 command string the word(s) that will be substituted for the alias
7
8 Aliases may be created in four manners:
9
10 1. entering the alias in $HOME/.crashrc.
11 2. entering the alias in .crashrc in the current directory.
12 3. executing an input file containing the alias command.
13 4. during runtime with this command.
14
15 During initialization, $HOME/.crashrc is read first, followed by the
16 .crashrc file in the current directory. Aliases in the .crashrc file
17 in the current directory override those in $HOME/.crashrc. Aliases
18 entered with this command or by runtime input file override those
19 defined in either .crashrc file. Aliases may be deleted by entering an
20 empty string for the second argument. If redirection characters are to
21 be part of the command string, the command string must be enclosed by
22 quotation marks.
23
24 Note that there are a number of helpful built-in aliases -- see the
25 first example below.
2.3. EXAMPLES
1 Display the currently-defined aliases, which in this example, only
2 consist of the built-in aliases:
3
4 crash> alias
5 ORIGIN ALIAS COMMAND
6 builtin man help
7 builtin ? help
8 builtin quit q
9 builtin sf set scroll off
10 builtin sn set scroll on
11 builtin hex set radix 16
12 builtin dec set radix 10
13 builtin g gdb
14 builtin px p -x
15 builtin pd p -d
16 builtin for foreach
17 builtin size *
18 builtin dmesg log
19 builtin lsmod mod
20 builtin last ps -l
21
22 Create a new alias to be added to the list:
23
24 crash> alias kp kmem -p
25 ORIGIN ALIAS COMMAND
26 runtime kp kmem -p
27
28 Create an alias with redirection characters:
29
30 crash> alias ksd "kmem -p | grep slab | grep DMA"
31 ORIGIN ALIAS COMMAND
32 runtime ksd kmem -p | grep slab | grep DMA
33
34 Remove an alias:
35
36 crash> alias kp ""
37 alias deleted: kp
3. ascii - translate a hexadecimal string to ASCII
3.1. SYNOPSIS
1ascii value ...
3.2. DESCRIPTION
1 Translates 32-bit or 64-bit hexadecimal values to ASCII. If no argument
2 is entered, an ASCII chart is displayed.
3.3. EXAMPLES
1 Translate the hexadecimal value of 0x62696c2f7273752f to ASCII:
2
3 crash> ascii 62696c2f7273752f
4 62696c2f7273752f: /usr/lib
5
6 Display an ASCII chart:
7
8 crash> ascii
9
10 0 1 2 3 4 5 6 7
11 +-------------------------------
12 0 | NUL DLE SP 0 @ P ' p
13 1 | SOH DC1 ! 1 A Q a q
14 2 | STX DC2 " 2 B R b r
15 3 | ETX DC3 # 3 C S c s
16 4 | EOT DC4 $ 4 D T d t
17 5 | ENQ NAK % 5 E U e u
18 6 | ACK SYN & 6 F V f v
19 7 | BEL ETB ` 7 G W g w
20 8 | BS CAN ( 8 H X h x
21 9 | HT EM ) 9 I Y i y
22 A | LF SUB * : J Z j z
23 B | VT ESC + ; K [ k {
24 C | FF FS , < L \ l |
25 D | CR GS _ = M ] m }
26 E | SO RS . > N ^ n ~
27 F | SI US / ? O - o DEL
4. bpf - extended Berkeley Packet Filter (eBPF)
4.1. SYNOPSIS
1bpf [[-p ID | -P] [-tTj]] [[-m ID] | -M] [-s] [-xd]
4.2. DESCRIPTION
1
2 This command provides information on currently-loaded eBPF programs and maps.
3 With no arguments, basic information about each loaded eBPF program and map
4 is displayed. For each eBPF program, its ID number, the addresses of its
5 bpf_prog and bpf_prog_aux data structures, its type, tag, and the IDs of the
6 eBPF maps that it uses are displayed. For each eBPF map, its ID number, the
7 address of its bpf_map data structure, its type, and the hexadecimal value of
8 its map_flags are displayed.
9
10 -p ID displays the basic information specific to the program ID, plus the
11 size in bytes of its translated bytecode, the size in bytes of its
12 jited code, the number of bytes locked into memory, the time that
13 the program was loaded, whether it is GPL compatible, and its UID.
14 -P same as -p, but displays the basic and extra data for all programs.
15 -m ID displays the basic information specific to the map ID, plus the
16 size in bytes of its key and value, the maximum number of key-value
17 pairs that can be stored within the map, the number of bytes locked
18 into memory, its name string, and its UID.
19 -M same as -m, but displays the basic and extra data for all maps.
20 -t translate the bytecode of the specified program ID.
21 -T same as -t, but also dump the bytecode of each instruction.
22 -j disassemble the jited code of the specified program ID.
23 -s with -p or -P, dump the bpf_prog and bpf_prog_aux data structures.
24 with -m or -M, dump the bpf_map structure.
25 -x with -s, override default output format with hexadecimal format.
26 -d with -s, override default output format with decimal format.
4.3. EXAMPLES
1 Display all loaded eBPF programs and maps:
2
3 crash> bpf
4 ID BPF_PROG BPF_PROG_AUX BPF_PROG_TYPE TAG USED_MAPS
5 13 ffffbc00c06d1000 ffff9ff260f0c400 CGROUP_SKB 7be49e3934a125ba 13,14
6 14 ffffbc00c0761000 ffff9ff260f0f600 CGROUP_SKB 2a142ef67aaad174 13,14
7 15 ffffbc00c001d000 ffff9ff2618f9e00 CGROUP_SKB 7be49e3934a125ba 15,16
8 16 ffffbc00c06c9000 ffff9ff2618f9400 CGROUP_SKB 2a142ef67aaad174 15,16
9 19 ffffbc00c0d39000 ffff9ff2610fa000 CGROUP_SKB 7be49e3934a125ba 19,20
10 20 ffffbc00c0d41000 ffff9ff2610f8e00 CGROUP_SKB 2a142ef67aaad174 19,20
11 30 ffffbc00c065f000 ffff9ff1b64de200 KPROBE 69fed6de18629d7a 32
12 31 ffffbc00c065b000 ffff9ff1b64df200 KPROBE 69fed6de18629d7a 37
13 32 ffffbc00c0733000 ffff9ff1b64dc600 KPROBE 69fed6de18629d7a 38
14 33 ffffbc00c0735000 ffff9ff1b64dca00 KPROBE 69fed6de18629d7a 39
15 34 ffffbc00c0737000 ffff9ff1b64dfc00 KPROBE 4abbddae72a6ee17 33,36,34
16 36 ffffbc00c0839000 ffff9ff1b64dd000 KPROBE da4fc6a3f41761a2 32
17 41 ffffbc00c07ec000 ffff9ff207b70400 TRACEPOINT e2094f9f46284bf6 55,54
18 44 ffffbc00c07ee000 ffff9ff1b64dc800 PERF_EVENT 19578a12836c4115 62
19 46 ffffbc00c07f0000 ffff9ff207b70400 SOCKET_FILTER 1fcfc04afd689133 64
20
21 ID BPF_MAP BPF_MAP_TYPE MAP_FLAGS
22 13 ffff9ff260f0ec00 LPM_TRIE 00000001
23 14 ffff9ff260f0de00 LPM_TRIE 00000001
24 15 ffff9ff2618fbe00 LPM_TRIE 00000001
25 16 ffff9ff2618fb800 LPM_TRIE 00000001
26 19 ffff9ff2610faa00 LPM_TRIE 00000001
27 20 ffff9ff2610fb800 LPM_TRIE 00000001
28 32 ffff9ff260d74000 HASH 00000000
29 33 ffff9ff260d76400 LRU_HASH 00000000
30 34 ffff9ff260d70000 LRU_HASH 00000002
31 35 ffff9ff260d73800 LRU_HASH 00000004
32 36 ffff9ff1b4f44000 ARRAY_OF_MAPS 00000000
33 37 ffff9ff260d77c00 PERCPU_HASH 00000000
34 38 ffff9ff260d70800 HASH 00000001
35 39 ffff9ff260d76c00 PERCPU_HASH 00000001
36 54 ffff9ff260dd2c00 HASH 00000000
37 55 ffff9ff260dd1400 HASH 00000000
38 62 ffff9ff1ae784000 HASH 00000000
39 64 ffff9ff1aea15000 ARRAY 00000000
40
41 Display additional data about program ID 20:
42
43 crash> bpf -p 20
44 ID BPF_PROG BPF_PROG_AUX BPF_PROG_TYPE TAG USED_MAPS
45 20 ffffbc00c0d41000 ffff9ff2610f8e00 CGROUP_SKB 2a142ef67aaad174 19,20
46 XLATED: 296 JITED: 229 MEMLOCK: 4096
47 LOAD_TIME: Fri Apr 20 19:39:21 2018
48 GPL_COMPATIBLE: yes UID: 0
49
50 Display additional data about map ID 34:
51
52 crash> bpf -m 34
53 ID BPF_MAP BPF_MAP_TYPE MAP_FLAGS
54 34 ffff9ff260d70000 LRU_HASH 00000000
55 KEY_SIZE: 4 VALUE_SIZE: 8 MAX_ENTRIES: 10000 MEMLOCK: 1953792
56 NAME: "lru_hash_map" UID: 0
57
58 Disassemble the jited program of program ID 20:
59
60 crash> bpf -p 20 -j
61 ID BPF_PROG BPF_PROG_AUX BPF_PROG_TYPE TAG USED_MAPS
62 20 ffffbc00c0d41000 ffff9ff2610f8e00 CGROUP_SKB 2a142ef67aaad174 19,20
63 XLATED: 296 JITED: 229 MEMLOCK: 4096
64 LOAD_TIME: Fri Apr 20 19:39:21 2018
65 GPL_COMPATIBLE: yes UID: 0
66
67 0xffffffffc06887a2: push %rbp
68 0xffffffffc06887a3: mov %rsp,%rbp
69 0xffffffffc06887a6: sub $0x40,%rsp
70 0xffffffffc06887ad: sub $0x28,%rbp
71 0xffffffffc06887b1: mov %rbx,0x0(%rbp)
72 0xffffffffc06887b5: mov %r13,0x8(%rbp)
73 0xffffffffc06887b9: mov %r14,0x10(%rbp)
74 0xffffffffc06887bd: mov %r15,0x18(%rbp)
75 0xffffffffc06887c1: xor %eax,%eax
76 0xffffffffc06887c3: mov %rax,0x20(%rbp)
77 0xffffffffc06887c7: mov %rdi,%rbx
78 0xffffffffc06887ca: movzwq 0xc0(%rbx),%r13
79 0xffffffffc06887d2: xor %r14d,%r14d
80 0xffffffffc06887d5: cmp $0x8,%r13
81 0xffffffffc06887d9: jne 0xffffffffc068881b
82 0xffffffffc06887db: mov %rbx,%rdi
83 0xffffffffc06887de: mov $0xc,%esi
84 0xffffffffc06887e3: mov %rbp,%rdx
85 0xffffffffc06887e6: add $0xfffffffffffffffc,%rdx
86 0xffffffffc06887ea: mov $0x4,%ecx
87 0xffffffffc06887ef: callq 0xffffffffb0865340 <bpf_skb_load_bytes>
88 0xffffffffc06887f4: movabs $0xffff9ff2610faa00,%rdi
89 0xffffffffc06887fe: mov %rbp,%rsi
90 0xffffffffc0688801: add $0xfffffffffffffff8,%rsi
91 0xffffffffc0688805: movl $0x20,0x0(%rsi)
92 0xffffffffc068880c: callq 0xffffffffb01fcba0 <bpf_map_lookup_elem>
93 0xffffffffc0688811: cmp $0x0,%rax
94 0xffffffffc0688815: je 0xffffffffc068881b
95 0xffffffffc0688817: or $0x2,%r14d
96 0xffffffffc068881b: cmp $0xdd86,%r13
97 0xffffffffc0688822: jne 0xffffffffc0688864
98 0xffffffffc0688824: mov %rbx,%rdi
99 0xffffffffc0688827: mov $0x8,%esi
100 0xffffffffc068882c: mov %rbp,%rdx
101 0xffffffffc068882f: add $0xfffffffffffffff0,%rdx
102 0xffffffffc0688833: mov $0x10,%ecx
103 0xffffffffc0688838: callq 0xffffffffb0865340 <bpf_skb_load_bytes>
104 0xffffffffc068883d: movabs $0xffff9ff2610fb800,%rdi
105 0xffffffffc0688847: mov %rbp,%rsi
106 0xffffffffc068884a: add $0xffffffffffffffec,%rsi
107 0xffffffffc068884e: movl $0x80,0x0(%rsi)
108 0xffffffffc0688855: callq 0xffffffffb01fcba0 <bpf_map_lookup_elem>
109 0xffffffffc068885a: cmp $0x0,%rax
110 0xffffffffc068885e: je 0xffffffffc0688864
111 0xffffffffc0688860: or $0x2,%r14d
112 0xffffffffc0688864: mov $0x1,%eax
113 0xffffffffc0688869: cmp $0x2,%r14
114 0xffffffffc068886d: jne 0xffffffffc0688871
115 0xffffffffc068886f: xor %eax,%eax
116 0xffffffffc0688871: mov 0x0(%rbp),%rbx
117 0xffffffffc0688875: mov 0x8(%rbp),%r13
118 0xffffffffc0688879: mov 0x10(%rbp),%r14
119 0xffffffffc068887d: mov 0x18(%rbp),%r15
120 0xffffffffc0688881: add $0x28,%rbp
121 0xffffffffc0688885: leaveq
122 0xffffffffc0688886: retq
123
124 Translate each bytecode instruction of program ID 13:
125
126 crash> bpf -p 13 -t
127 ID BPF_PROG BPF_PROG_AUX BPF_PROG_TYPE TAG USED_MAPS
128 13 ffffbc00c06d1000 ffff9ff260f0c400 CGROUP_SKB 7be49e3934a125ba 13,14
129 XLATED: 296 JITED: 229 MEMLOCK: 4096
130 LOAD_TIME: Fri Apr 20 19:39:11 2018
131 GPL_COMPATIBLE: yes UID: 0
132
133 0: (bf) r6 = r1
134 1: (69) r7 = *(u16 *)(r6 +192)
135 2: (b4) (u32) r8 = (u32) 0
136 3: (55) if r7 != 0x8 goto pc+14
137 4: (bf) r1 = r6
138 5: (b4) (u32) r2 = (u32) 16
139 6: (bf) r3 = r10
140 7: (07) r3 += -4
141 8: (b4) (u32) r4 = (u32) 4
142 9: (85) call bpf_skb_load_bytes#6793152
143 10: (18) r1 = map[id:13]
144 12: (bf) r2 = r10
145 13: (07) r2 += -8
146 14: (62) *(u32 *)(r2 +0) = 32
147 15: (85) call bpf_map_lookup_elem#73760
148 16: (15) if r0 == 0x0 goto pc+1
149 17: (44) (u32) r8 |= (u32) 2
150 18: (55) if r7 != 0xdd86 goto pc+14
151 19: (bf) r1 = r6
152 20: (b4) (u32) r2 = (u32) 24
153 21: (bf) r3 = r10
154 22: (07) r3 += -16
155 23: (b4) (u32) r4 = (u32) 16
156 24: (85) call bpf_skb_load_bytes#6793152
157 25: (18) r1 = map[id:14]
158 27: (bf) r2 = r10
159 28: (07) r2 += -20
160 29: (62) *(u32 *)(r2 +0) = 128
161 30: (85) call bpf_map_lookup_elem#73760
162 31: (15) if r0 == 0x0 goto pc+1
163 32: (44) (u32) r8 |= (u32) 2
164 33: (b7) r0 = 1
165 34: (55) if r8 != 0x2 goto pc+1
166 35: (b7) r0 = 0
167 36: (95) exit
168
169 Translate, and then dump each bytecode instruction of program ID 13:
170
171 crash> bpf -p 13 -T
172 ID BPF_PROG BPF_PROG_AUX BPF_PROG_TYPE TAG USED_MAPS
173 13 ffffbc00c06d1000 ffff9ff260f0c400 CGROUP_SKB 7be49e3934a125ba 13,14
174 XLATED: 296 JITED: 229 MEMLOCK: 4096
175 LOAD_TIME: Fri Apr 20 19:39:11 2018
176 GPL_COMPATIBLE: yes UID: 0
177
178 0: (bf) r6 = r1
179 bf 16 00 00 00 00 00 00
180 1: (69) r7 = *(u16 *)(r6 +192)
181 69 67 c0 00 00 00 00 00
182 2: (b4) (u32) r8 = (u32) 0
183 b4 08 00 00 00 00 00 00
184 3: (55) if r7 != 0x8 goto pc+14
185 55 07 0e 00 08 00 00 00
186 4: (bf) r1 = r6
187 bf 61 00 00 00 00 00 00
188 5: (b4) (u32) r2 = (u32) 16
189 b4 02 00 00 10 00 00 00
190 6: (bf) r3 = r10
191 bf a3 00 00 00 00 00 00
192 7: (07) r3 += -4
193 07 03 00 00 fc ff ff ff
194 8: (b4) (u32) r4 = (u32) 4
195 b4 04 00 00 04 00 00 00
196 9: (85) call bpf_skb_load_bytes#6793152
197 85 00 00 00 c0 a7 67 00
198 10: (18) r1 = map[id:13]
199 18 01 00 00 00 7a 96 61 00 00 00 00 b2 9d ff ff
200 12: (bf) r2 = r10
201 bf a2 00 00 00 00 00 00
202 13: (07) r2 += -8
203 07 02 00 00 f8 ff ff ff
204 14: (62) *(u32 *)(r2 +0) = 32
205 62 02 00 00 20 00 00 00
206 15: (85) call bpf_map_lookup_elem#73760
207 85 00 00 00 20 20 01 00
208 16: (15) if r0 == 0x0 goto pc+1
209 15 00 01 00 00 00 00 00
210 17: (44) (u32) r8 |= (u32) 2
211 44 08 00 00 02 00 00 00
212 18: (55) if r7 != 0xdd86 goto pc+14
213 55 07 0e 00 86 dd 00 00
214 19: (bf) r1 = r6
215 bf 61 00 00 00 00 00 00
216 20: (b4) (u32) r2 = (u32) 24
217 b4 02 00 00 18 00 00 00
218 21: (bf) r3 = r10
219 bf a3 00 00 00 00 00 00
220 22: (07) r3 += -16
221 07 03 00 00 f0 ff ff ff
222 23: (b4) (u32) r4 = (u32) 16
223 b4 04 00 00 10 00 00 00
224 24: (85) call bpf_skb_load_bytes#6793152
225 85 00 00 00 c0 a7 67 00
226 25: (18) r1 = map[id:14]
227 18 01 00 00 00 68 96 61 00 00 00 00 b2 9d ff ff
228 27: (bf) r2 = r10
229 bf a2 00 00 00 00 00 00
230 28: (07) r2 += -20
231 07 02 00 00 ec ff ff ff
232 29: (62) *(u32 *)(r2 +0) = 128
233 62 02 00 00 80 00 00 00
234 30: (85) call bpf_map_lookup_elem#73760
235 85 00 00 00 20 20 01 00
236 31: (15) if r0 == 0x0 goto pc+1
237 15 00 01 00 00 00 00 00
238 32: (44) (u32) r8 |= (u32) 2
239 44 08 00 00 02 00 00 00
240 33: (b7) r0 = 1
241 b7 00 00 00 01 00 00 00
242 34: (55) if r8 != 0x2 goto pc+1
243 55 08 01 00 02 00 00 00
244 35: (b7) r0 = 0
245 b7 00 00 00 00 00 00 00
246 36: (95) exit
247 95 00 00 00 00 00 00 00
248
249 Display the bpf_map data structure for map ID 13:
250
251 crash> bpf -m 13 -s
252 ID BPF_MAP BPF_MAP_TYPE MAP_FLAGS
253 13 ffff9ff260f0ec00 LPM_TRIE 00000001
254 KEY_SIZE: 8 VALUE_SIZE: 8 MAX_ENTRIES: 1 MEMLOCK: 4096
255 NAME: (unused) UID: 0
256
257 struct bpf_map {
258 ops = 0xffffffffb0e36720,
259 inner_map_meta = 0x0,
260 security = 0xffff9ff26873a158,
261 map_type = BPF_MAP_TYPE_LPM_TRIE,
262 key_size = 8,
263 value_size = 8,
264 max_entries = 1,
265 map_flags = 1,
266 pages = 1,
267 id = 13,
268 numa_node = -1,
269 unpriv_array = false,
270 user = 0xffffffffb14578a0,
271 refcnt = {
272 counter = 3
273 },
274 usercnt = {
275 counter = 1
276 },
277 work = {
278 data = {
279 counter = 0
280 },
281 entry = {
282 next = 0x0,
283 prev = 0x0
284 },
285 func = 0x0,
286 lockdep_map = {
287 key = 0x0,
288 class_cache = {0x0, 0x0},
289 name = 0x0,
290 cpu = 0,
291 ip = 0
292 }
293 },
294 name = "
295 }
296
297 Display the bpf_prog and bpf_prog_aux structures for program ID 13:
298
299 crash> bpf -p 13 -s
300 ID BPF_PROG BPF_PROG_AUX BPF_PROG_TYPE TAG USED_MAPS
301 13 ffffbc00c06d1000 ffff9ff260f0c400 CGROUP_SKB 7be49e3934a125ba 13,14
302 XLATED: 296 JITED: 229 MEMLOCK: 4096
303 LOAD_TIME: Fri Apr 20 19:39:10 2018
304 GPL_COMPATIBLE: yes UID: 0
305
306 struct bpf_prog {
307 pages = 1,
308 jited = 1,
309 jit_requested = 1,
310 locked = 1,
311 gpl_compatible = 1,
312 cb_access = 0,
313 dst_needed = 0,
314 blinded = 0,
315 is_func = 0,
316 kprobe_override = 0,
317 type = BPF_PROG_TYPE_CGROUP_SKB,
318 len = 37,
319 jited_len = 229,
320 tag = "{\344\236\071\064\241%\272",
321 aux = ffff9ff260f0c400,
322 orig_prog = 0x0,
323 bpf_func = 0xffffffffc0218a59,
324 {
325 insns = 0xffffb0cf406d1030,
326 insnsi = 0xffffb0cf406d1030
327 }
328 }
329
330 struct bpf_prog_aux {
331 refcnt = {
332 counter = 2
333 },
334 used_map_cnt = 2,
335 max_ctx_offset = 20,
336 stack_depth = 20,
337 id = 13,
338 func_cnt = 0,
339 offload_requested = false,
340 func = 0x0,
341 jit_data = 0x0,
342 ksym_tnode = {
343 node = {{
344 __rb_parent_color = 18446635988194065457,
345 rb_right = 0x0,
346 rb_left = 0x0
347 }, {
348 __rb_parent_color = 18446635988194065481,
349 rb_right = 0x0,
350 rb_left = 0x0
351 }}
352 },
353 ksym_lnode = {
354 next = 0xffff9db261966460,
355 prev = 0xffffffffb85d1150
356 },
357 ops = 0xffffffffb7f09060,
358 used_maps = 0xffff9db261e03600,
359 prog = 0xffffb0cf406d1000,
360 user = 0xffffffffb84578a0,
361 load_time = 23962237943,
362 name = "
363 security = 0xffff9db266f9cf50,
364 offload = 0x0,
365 {
366 work = {
367 data = {
368 counter = 0
369 },
370 entry = {
371 next = 0x0,
372 prev = 0x0
373 },
374 func = 0x0,
375 lockdep_map = {
376 key = 0x0,
377 class_cache = {0x0, 0x0},
378 name = 0x0,
379 cpu = 0,
380 ip = 0
381 }
382 },
383 rcu = {
384 next = 0x0,
385 func = 0x0
386 }
387 }
388 }
389
390 Display the extra data about all programs:
391
392 crash> bpf -P
393 ID BPF_PROG BPF_PROG_AUX BPF_PROG_TYPE TAG USED_MAPS
394 13 ffffbc00c06d1000 ffff9ff260f0c400 CGROUP_SKB 7be49e3934a125ba 13,14
395 XLATED: 296 JITED: 229 MEMLOCK: 4096
396 LOAD_TIME: Fri Apr 20 19:39:10 2018
397 GPL_COMPATIBLE: yes UID: 0
398
399 ID BPF_PROG BPF_PROG_AUX BPF_PROG_TYPE TAG USED_MAPS
400 14 ffffbc00c0761000 ffff9ff260f0f600 CGROUP_SKB 2a142ef67aaad174 13,14
401 XLATED: 296 JITED: 229 MEMLOCK: 4096
402 LOAD_TIME: Fri Apr 20 19:39:10 2018
403 GPL_COMPATIBLE: yes UID: 0
404
405 ID BPF_PROG BPF_PROG_AUX BPF_PROG_TYPE TAG USED_MAPS
406 15 ffffbc00c001d000 ffff9ff2618f9e00 CGROUP_SKB 7be49e3934a125ba 15,16
407 XLATED: 296 JITED: 229 MEMLOCK: 4096
408 LOAD_TIME: Fri Apr 20 19:39:11 2018
409 GPL_COMPATIBLE: yes UID: 0
410
411 ...
412
413 ID BPF_PROG BPF_PROG_AUX BPF_PROG_TYPE TAG USED_MAPS
414 75 ffffbc00c0ed1000 ffff9ff2429c6400 KPROBE da4fc6a3f41761a2 107
415 XLATED: 5168 JITED: 2828 MEMLOCK: 8192
416 LOAD_TIME: Fri Apr 27 14:54:40 2018
417 GPL_COMPATIBLE: yes UID: 0
418
419 Display the extra data for all maps:
420
421 crash> bpf -M
422 ID BPF_MAP BPF_MAP_TYPE MAP_FLAGS
423 13 ffff9ff260f0ec00 LPM_TRIE 00000001
424 KEY_SIZE: 8 VALUE_SIZE: 8 MAX_ENTRIES: 1 MEMLOCK: 4096
425 NAME: (unused) UID: 0
426
427 ID BPF_MAP BPF_MAP_TYPE MAP_FLAGS
428 14 ffff9ff260f0de00 LPM_TRIE 00000001
429 KEY_SIZE: 20 VALUE_SIZE: 8 MAX_ENTRIES: 1 MEMLOCK: 4096
430 NAME: (unused) UID: 0
431
432 ...
433
434 ID BPF_MAP BPF_MAP_TYPE MAP_FLAGS
435 108 ffff9ff1aeab9400 LRU_HASH 00000000
436 KEY_SIZE: 4 VALUE_SIZE: 8 MAX_ENTRIES: 1000 MEMLOCK: 147456
437 NAME: "lru_hash_lookup" UID: 0
438
439 To display all possible information that this command offers about
440 all programs and maps, enter:
441
442 crash> bpf -PM -jTs
5. bt - backtrace
5.1. SYNOPSIS
1bt [-a|-c cpu(s)|-g|-r|-t|-T|-l|-e|-E|-f|-F|-o|-O|-v|-p] [-R ref] [-s [-x|d]]
2 [-I ip] [-S sp] [pid | task]
5.2. DESCRIPTION
1 Display a kernel stack backtrace. If no arguments are given, the stack
2 trace of the current context will be displayed.
3
4 -a displays the stack traces of the active task on each CPU.
5 (only applicable to crash dumps)
6 -A same as -a, but also displays vector registers (S390X only).
7 -p display the stack trace of the panic task only.
8 (only applicable to crash dumps)
9 -c cpu display the stack trace of the active task on one or more CPUs,
10 which can be specified using the format "3", "1,8,9", "1-23",
11 or "1,8,9-14". (only applicable to crash dumps)
12 -g displays the stack traces of all threads in the thread group of
13 the target task; the thread group leader will be displayed first.
14 -r display raw stack data, consisting of a memory dump of the two
15 pages of memory containing the task_union structure.
16 -t display all text symbols found from the last known stack location
17 to the top of the stack. (helpful if the back trace fails)
18 -T display all text symbols found from just above the task_struct or
19 thread_info to the top of the stack. (helpful if the back trace
20 fails or the -t option starts too high in the process stack).
21 -l show file and line number of each stack trace text location.
22 -e search the stack for possible kernel and user mode exception frames.
23 -E search the IRQ stacks (x86, x86_64, arm64, and ppc64), and the
24 exception stacks (x86_64) for possible exception frames; all other
25 arguments except for -c will be ignored since this is not a context-
26 sensitive operation.
27 -f display all stack data contained in a frame; this option can be
28 used to determine the arguments passed to each function; on ia64,
29 the argument register contents are dumped.
30 -F[F] similar to -f, except that the stack data is displayed symbolically
31 when appropriate; if the stack data references a slab cache object,
32 the name of the slab cache will be displayed in brackets; on ia64,
33 the substitution is done to the argument register contents. If -F
34 is entered twice, and the stack data references a slab cache object,
35 both the address and the name of the slab cache will be displayed
36 in brackets.
37 -v check the kernel stack of all tasks for evidence of stack overflows.
38 It does so by verifying the thread_info.task pointer, ensuring that
39 the thread_info.cpu is a valid cpu number, and checking the end of
40 the stack for the STACK_END_MAGIC value.
41 -o arm64: use optional backtrace method; not supported on Linux 4.14 or
42 later kernels.
43 x86: use old backtrace method, permissible only on kernels that were
44 compiled without the -fomit-frame_pointer.
45 x86_64: use old backtrace method, which dumps potentially stale
46 kernel text return addresses found on the stack.
47 -O arm64: use optional backtrace method by default; subsequent usage
48 of this option toggles the backtrace method.
49 x86: use old backtrace method by default, permissible only on kernels
50 that were compiled without the -fomit-frame_pointer; subsequent usage
51 of this option toggles the backtrace method.
52 x86_64: use old backtrace method by default; subsequent usage of this
53 option toggles the backtrace method.
54 -R ref display stack trace only if there is a reference to this symbol
55 or text address.
56 -s display the symbol name plus its offset.
57 -x when displaying a symbol offset with the -s option, override the
58 default output format with hexadecimal format.
59 -d when displaying a symbol offset with the -s option, override the
60 default output format with decimal format.
61 -I ip use ip as the starting text location.
62 -S sp use sp as the starting stack frame address.
63 pid displays the stack trace(s) of this pid.
64 taskp displays the stack trace the the task referenced by this hexadecimal
65 task_struct pointer.
66
67 Multiple pid and taskp arguments may be specified.
68
69 Note that all examples below are for x86 only. The output format will differ
70 for other architectures. x86 backtraces from kernels that were compiled
71 with the --fomit-frame-pointer CFLAG occasionally will drop stack frames,
72 or display a stale frame reference. When in doubt as to the accuracy of a
73 backtrace, the -t or -T options may help fill in the blanks.
5.3. EXAMPLES
1 Display the stack trace of the active task(s) when the kernel panicked:
2
3 crash> bt -a
4 PID: 286 TASK: c0b3a000 CPU: 0 COMMAND: "in.rlogind"
5 #0 [c0b3be90] crash_save_current_state at c011aed0
6 #1 [c0b3bea4] panic at c011367c
7 #2 [c0b3bee8] tulip_interrupt at c01bc820
8 #3 [c0b3bf08] handle_IRQ_event at c010a551
9 #4 [c0b3bf2c] do_8259A_IRQ at c010a319
10 #5 [c0b3bf3c] do_IRQ at c010a653
11 #6 [c0b3bfbc] ret_from_intr at c0109634
12 EAX: 00000000 EBX: c0e68280 ECX: 00000000 EDX: 00000004 EBP: c0b3bfbc
13 DS: 0018 ESI: 00000004 ES: 0018 EDI: c0e68284
14 CS: 0010 EIP: c012f803 ERR: ffffff09 EFLAGS: 00000246
15 #7 [c0b3bfbc] sys_select at c012f803
16 #8 [c0b3bfc0] system_call at c0109598
17 EAX: 0000008e EBX: 00000004 ECX: bfffc9a0 EDX: 00000000
18 DS: 002b ESI: bfffc8a0 ES: 002b EDI: 00000000
19 SS: 002b ESP: bfffc82c EBP: bfffd224
20 CS: 0023 EIP: 400d032e ERR: 0000008e EFLAGS: 00000246
21
22 Display the stack trace of the active task on CPU 0 and 1:
23
24 crash> bt -c 0,1
25 PID: 0 TASK: ffffffff81a8d020 CPU: 0 COMMAND: "swapper"
26 #0 [ffff880002207e90] crash_nmi_callback at ffffffff8102fee6
27 #1 [ffff880002207ea0] notifier_call_chain at ffffffff8152d525
28 #2 [ffff880002207ee0] atomic_notifier_call_chain at ffffffff8152d58a
29 #3 [ffff880002207ef0] notify_die at ffffffff810a155e
30 #4 [ffff880002207f20] do_nmi at ffffffff8152b1eb
31 #5 [ffff880002207f50] nmi at ffffffff8152aab0
32 [exception RIP: native_safe_halt+0xb]
33 RIP: ffffffff8103eacb RSP: ffffffff81a01ea8 RFLAGS: 00000296
34 RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000000000
35 RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffffffff81de5228
36 RBP: ffffffff81a01ea8 R8: 0000000000000000 R9: 0000000000000000
37 R10: 0012099429a6bea3 R11: 0000000000000000 R12: ffffffff81c066c0
38 R13: 0000000000000000 R14: ffffffffffffffff R15: ffffffff81de1000
39 ORIG_RAX: ffffffffffffffff CS: 0010 SS: 0018
40 --- <NMI exception stack> ---
41 #6 [ffffffff81a01ea8] native_safe_halt at ffffffff8103eacb
42 #7 [ffffffff81a01eb0] default_idle at ffffffff810167bd
43 #8 [ffffffff81a01ed0] cpu_idle at ffffffff81009fc6
44
45 PID: 38 TASK: ffff88003eaae040 CPU: 1 COMMAND: "khungtaskd"
46 #0 [ffff88003ad97ce8] machine_kexec at ffffffff81038f3b
47 #1 [ffff88003ad97d48] crash_kexec at ffffffff810c5da2
48 #2 [ffff88003ad97e18] panic at ffffffff8152721a
49 #3 [ffff88003ad97e98] watchdog at ffffffff810e6346
50 #4 [ffff88003ad97ee8] kthread at ffffffff8109af06
51 #5 [ffff88003ad97f48] kernel_thread at ffffffff8100c20a
52
53 Display the stack traces of task f2814000 and PID 1592:
54
55 crash> bt f2814000 1592
56 PID: 1018 TASK: f2814000 CPU: 1 COMMAND: "java"
57 #0 [f2815db4] schedule at c011af85
58 #1 [f2815de4] __down at c010600f
59 #2 [f2815e14] __down_failed at c01061b3
60 #3 [f2815e24] stext_lock (via drain_cpu_caches) at c025fa55
61 #4 [f2815ec8] kmem_cache_shrink_nr at c013a53e
62 #5 [f2815ed8] do_try_to_free_pages at c013f402
63 #6 [f2815f04] try_to_free_pages at c013f8d2
64 #7 [f2815f1c] _wrapped_alloc_pages at c01406bd
65 #8 [f2815f40] __alloc_pages at c014079d
66 #9 [f2815f60] __get_free_pages at c014083e
67 #10 [f2815f68] do_fork at c011cebb
68 #11 [f2815fa4] sys_clone at c0105ceb
69 #12 [f2815fc0] system_call at c010740c
70 EAX: 00000078 EBX: 00000f21 ECX: bc1ffbd8 EDX: bc1ffbe0
71 DS: 002b ESI: 00000000 ES: 002b EDI: bc1ffd04
72 SS: 002b ESP: 0807316c EBP: 080731bc
73 CS: 0023 EIP: 4012881e ERR: 00000078 EFLAGS: 00000296
74
75 PID: 1592 TASK: c0cec000 CPU: 3 COMMAND: "httpd"
76 #0 [c0ceded4] schedule at c011af85
77 #1 [c0cedf04] pipe_wait at c0153083
78 #2 [c0cedf58] pipe_read at c015317f
79 #3 [c0cedf7c] sys_read at c0148be6
80 #4 [c0cedfc0] system_call at c010740c
81 EAX: 00000003 EBX: 00000004 ECX: bffed4a3 EDX: 00000001
82 DS: 002b ESI: 00000001 ES: 002b EDI: bffed4a3
83 SS: 002b ESP: bffed458 EBP: bffed488
84 CS: 0023 EIP: 4024f1d4 ERR: 00000003 EFLAGS: 00000286
85
86 In order to examine each stack frame's contents use the bt -f option.
87 From the extra frame data that is displayed, the arguments passed to each
88 function can be determined. Re-examining the PID 1592 trace above:
89
90 crash> bt -f 1592
91 PID: 1592 TASK: c0cec000 CPU: 3 COMMAND: "httpd"
92 #0 [c0ceded4] schedule at c011af85
93 [RA: c0153088 SP: c0ceded4 FP: c0cedf04 SIZE: 52]
94 c0ceded4: c0cedf00 c0cec000 ce1a6000 00000003
95 c0cedee4: c0cec000 f26152c0 cfafc8c0 c0cec000
96 c0cedef4: ef70a0a0 c0cec000 c0cedf28 c0cedf54
97 c0cedf04: c0153088
98 #1 [c0cedf04] pipe_wait at c0153083
99 [RA: c0153184 SP: c0cedf08 FP: c0cedf58 SIZE: 84]
100 c0cedf08: 00000000 c0cec000 00000000 00000000
101 c0cedf18: 00000000 c0a41fa0 c011d38b c0394120
102 c0cedf28: 00000000 c0cec000 ceeebf30 ce4adf30
103 c0cedf38: 00000000 d4b60ce0 00000000 c0cedf58
104 c0cedf48: e204f820 ef70a040 00000001 c0cedf78
105 c0cedf58: c0153184
106 #2 [c0cedf58] pipe_read at c015317f
107 [RA: c0148be8 SP: c0cedf5c FP: c0cedf7c SIZE: 36]
108 c0cedf5c: ef70a040 c0cec000 00000000 00000000
109 c0cedf6c: 00000001 f27ae680 ffffffea c0cedfbc
110 c0cedf7c: c0148be8
111 #3 [c0cedf7c] sys_read at c0148be6
112 [RA: c0107413 SP: c0cedf80 FP: c0cedfc0 SIZE: 68]
113 c0cedf80: f27ae680 bffed4a3 00000001 f27ae6a0
114 c0cedf90: 40160370 24000000 4019ba28 00000000
115 c0cedfa0: 00000000 fffffffe bffba207 fffffffe
116 c0cedfb0: c0cec000 00000001 bffed4a3 bffed488
117 c0cedfc0: c0107413
118 #4 [c0cedfc0] system_call at c010740c
119 EAX: 00000003 EBX: 00000004 ECX: bffed4a3 EDX: 00000001
120 DS: 002b ESI: 00000001 ES: 002b EDI: bffed4a3
121 SS: 002b ESP: bffed458 EBP: bffed488
122 CS: 0023 EIP: 4024f1d4 ERR: 00000003 EFLAGS: 00000286
123 [RA: 4024f1d4 SP: c0cedfc4 FP: c0cedffc SIZE: 60]
124 c0cedfc4: 00000004 bffed4a3 00000001 00000001
125 c0cedfd4: bffed4a3 bffed488 00000003 0000002b
126 c0cedfe4: 0000002b 00000003 4024f1d4 00000023
127 c0cedff4: 00000286 bffed458 0000002b
128
129 Typically the arguments passed to a function will be the last values
130 that were pushed onto the stack by the next higher-numbered function, i.e.,
131 the lowest stack addresses in the frame above the called function's
132 stack frame. That can be verified by disassembling the calling function.
133 For example, the arguments passed from sys_read() to pipe_read() above
134 are the file pointer, the user buffer address, the count, and a pointer
135 to the file structure's f_pos field. Looking at the frame #3 data for
136 sys_read(), the last four items pushed onto the stack (lowest addresses)
137 are f27ae680, bffed4a3, 00000001, and f27ae6a0 -- which are the 4 arguments
138 above, in that order. Note that the first (highest address) stack content
139 in frame #2 data for pipe_read() is c0148be8, which is the return address
140 back to sys_read().
141
142 Dump the text symbols found in the current context's stack:
143
144 crash> bt -t
145 PID: 1357 TASK: c1aa0000 CPU: 0 COMMAND: "lockd"
146 START: schedule at c01190e0
147 [c1aa1f28] dput at c0157dbc
148 [c1aa1f4c] schedule_timeout at c0124cd4
149 [c1aa1f78] svc_recv at cb22c4d8 [sunrpc]
150 [c1aa1f98] put_files_struct at c011eb21
151 [c1aa1fcc] nlmclnt_proc at cb237bef [lockd]
152 [c1aa1ff0] kernel_thread at c0105826
153 [c1aa1ff8] nlmclnt_proc at cb237a60 [lockd]
154
155 Search the current stack for possible exception frames:
156
157 crash> bt -e
158 PID: 286 TASK: c0b3a000 CPU: 0 COMMAND: "in.rlogind"
159
160 KERNEL-MODE EXCEPTION FRAME AT c0b3bf44:
161 EAX: 00000000 EBX: c0e68280 ECX: 00000000 EDX: 00000004 EBP: c0b3bfbc
162 DS: 0018 ESI: 00000004 ES: 0018 EDI: c0e68284
163 CS: 0010 EIP: c012f803 ERR: ffffff09 EFLAGS: 00000246
164
165 USER-MODE EXCEPTION FRAME AT c0b3bfc4:
166 EAX: 0000008e EBX: 00000004 ECX: bfffc9a0 EDX: 00000000
167 DS: 002b ESI: bfffc8a0 ES: 002b EDI: 00000000
168 SS: 002b ESP: bfffc82c EBP: bfffd224
169 CS: 0023 EIP: 400d032e ERR: 0000008e EFLAGS: 00000246
170
171 Display the back trace from a dumpfile that resulted from the execution
172 of the crash utility's "sys -panic" command:
173
174 crash> bt
175 PID: 12523 TASK: c610c000 CPU: 0 COMMAND: "crash"
176 #0 [c610de64] die at c01076ec
177 #1 [c610de74] do_invalid_op at c01079bc
178 #2 [c610df2c] error_code (via invalid_op) at c0107256
179 EAX: 0000001d EBX: c024a4c0 ECX: c02f13c4 EDX: 000026f6 EBP: c610c000
180 DS: 0018 ESI: 401de2e0 ES: 0018 EDI: c610c000
181 CS: 0010 EIP: c011bbb4 ERR: ffffffff EFLAGS: 00010296
182 #3 [c610df68] panic at c011bbb4
183 #4 [c610df78] do_exit at c011f1fe
184 #5 [c610dfc0] system_call at c0107154
185 EAX: 00000001 EBX: 00000000 ECX: 00001000 EDX: 401df154
186 DS: 002b ESI: 401de2e0 ES: 002b EDI: 00000000
187 SS: 002b ESP: bffebf0c EBP: bffebf38
188 CS: 0023 EIP: 40163afd ERR: 00000001 EFLAGS: 00000246
189
190 Display the back trace from a dumpfile that resulted from an attempt to
191 insmod the sample "crash.c" kernel module that comes as part of the
192 Red Hat netdump package:
193
194 crash> bt
195 PID: 1696 TASK: c74de000 CPU: 0 COMMAND: "insmod"
196 #0 [c74dfdcc] die at c01076ec
197 #1 [c74dfddc] do_page_fault at c0117bbc
198 #2 [c74dfee0] error_code (via page_fault) at c0107256
199 EAX: 00000013 EBX: cb297000 ECX: 00000000 EDX: c5962000 EBP: c74dff28
200 DS: 0018 ESI: 00000000 ES: 0018 EDI: 00000000
201 CS: 0010 EIP: cb297076 ERR: ffffffff EFLAGS: 00010282
202 #3 [c74dff1c] crash_init at cb297076 [crash]
203 #4 [c74dff2c] sys_init_module at c011d233
204 #5 [c74dffc0] system_call at c0107154
205 EAX: 00000080 EBX: 08060528 ECX: 08076450 EDX: 0000000a
206 DS: 002b ESI: 0804b305 ES: 002b EDI: 08074ed0
207 SS: 002b ESP: bffe9a90 EBP: bffe9ac8
208 CS: 0023 EIP: 4012066e ERR: 00000080 EFLAGS: 00000246
209
210 Display the symbol name plus its offset in each frame, overriding
211 the current output format with hexadecimal:
212
213 crash> bt -sx
214 PID: 1499 TASK: ffff88006af43cc0 CPU: 2 COMMAND: "su"
215 #0 [ffff8800664a1c90] machine_kexec+0x167 at ffffffff810327b7
216 #1 [ffff8800664a1ce0] crash_kexec+0x60 at ffffffff810a9ec0
217 #2 [ffff8800664a1db0] oops_end+0xb0 at ffffffff81504160
218 #3 [ffff8800664a1dd0] general_protection+0x25 at ffffffff81503435
219 [exception RIP: kmem_cache_alloc+120]
220 RIP: ffffffff8113cf88 RSP: ffff8800664a1e88 RFLAGS: 00010086
221 RAX: 0000000000000000 RBX: ff88006ef56840ff RCX: ffffffff8114e9e4
222 RDX: 0000000000000000 RSI: 00000000000080d0 RDI: ffffffff81796020
223 RBP: ffffffff81796020 R8: ffff88000a3137a0 R9: 0000000000000000
224 R10: ffff88007ac97300 R11: 0000000000000400 R12: 00000000000080d0
225 R13: 0000000000000292 R14: 00000000000080d0 R15: 00000000000000c0
226 ORIG_RAX: ffffffffffffffff CS: 0010 SS: 0018
227 #4 [ffff8800664a1ed0] get_empty_filp+0x74 at ffffffff8114e9e4
228 #5 [ffff8800664a1ef0] sock_alloc_fd+0x23 at ffffffff8142f553
229 #6 [ffff8800664a1f10] sock_map_fd+0x23 at ffffffff8142f693
230 #7 [ffff8800664a1f50] sys_socket+0x43 at ffffffff814302a3
231 #8 [ffff8800664a1f80] system_call_fastpath+0x16 at ffffffff81013042
232 RIP: 00007f5720b368e7 RSP: 00007fff52b629a8 RFLAGS: 00010206
233 RAX: 0000000000000029 RBX: ffffffff81013042 RCX: 0000000000000000
234 RDX: 0000000000000009 RSI: 0000000000000003 RDI: 0000000000000010
235 RBP: 000000000066f320 R8: 0000000000000001 R9: 0000000000000000
236 R10: 0000000000000000 R11: 0000000000000202 R12: ffff88007ac97300
237 R13: 0000000000000000 R14: 00007f571e104a80 R15: 00007f571e305048
238 ORIG_RAX: 0000000000000029 CS: 0033 SS: 002b
239
240 The following three examples show the difference in the display of
241 the same stack frame's contents using -f, -F, and -FF:
242
243 crash> bt -f
244 ...
245 #4 [ffff810072b47f10] vfs_write at ffffffff800789d8
246 ffff810072b47f18: ffff81007e020380 ffff81007e2c2880
247 ffff810072b47f28: 0000000000000002 fffffffffffffff7
248 ffff810072b47f38: 00002b141825d000 ffffffff80078f75
249 #5 [ffff810072b47f40] sys_write at ffffffff80078f75
250 ...
251 crash> bt -F
252 ...
253 #4 [ffff810072b47f10] vfs_write at ffffffff800789d8
254 ffff810072b47f18: [files_cache] [filp]
255 ffff810072b47f28: 0000000000000002 fffffffffffffff7
256 ffff810072b47f38: 00002b141825d000 sys_write+69
257 #5 [ffff810072b47f40] sys_write at ffffffff80078f75
258 ...
259 crash> bt -FF
260 ...
261 #4 [ffff810072b47f10] vfs_write at ffffffff800789d8
262 ffff810072b47f18: [ffff81007e020380:files_cache] [ffff81007e2c2880:filp]
263 ffff810072b47f28: 0000000000000002 fffffffffffffff7
264 ffff810072b47f38: 00002b141825d000 sys_write+69
265 #5 [ffff810072b47f40] sys_write at ffffffff80078f75
266 ...
267
268 Check the kernel stack of all tasks for evidence of a stack overflow:
269
270 crash> bt -v
271 PID: 5823 TASK: ffff88102aae0040 CPU: 1 COMMAND: "flush-253:0"
272 possible stack overflow: thread_info.task: 102efb5adc0 != ffff88102aae0040
273 possible stack overflow: 40ffffffff != STACK_END_MAGIC
6. btop - bytes to page
6.1. SYNOPSIS
1btop address ...
6.2. DESCRIPTION
1 This command translates a hexadecimal address to its page number.
6.3. EXAMPLES
1 crash> btop 512a000
2 512a000: 512a
7. dev - device data
7.1. SYNOPSIS
1dev [-i | -p | -d | -D ] [-V | -v index [file]]
7.2. DESCRIPTION
1 If no argument is entered, this command dumps character and block
2 device data.
3
4 -i display I/O port usage; on 2.4 kernels, also display I/O memory usage.
5 -p display PCI device data.
6 -d display disk I/O statistics:
7 TOTAL: total number of allocated in-progress I/O requests
8 SYNC: I/O requests that are synchronous
9 ASYNC: I/O requests that are asynchronous
10 READ: I/O requests that are reads (older kernels)
11 WRITE: I/O requests that are writes (older kernels)
12 DRV: I/O requests that are in-flight in the device driver.
13 If the device driver uses blk-mq interface, this field
14 shows N/A(MQ). If not available, this column is not shown.
15 -D same as -d, but filter out disks with no in-progress I/O requests.
16
17 If the dumpfile contains device dumps:
18 -V display an indexed list of all device dumps present in the vmcore,
19 showing their file offset, size and name.
20 -v index select and display one device dump based upon an index value
21 shown by the -V option, shown in a default human-readable format;
22 alternatively, the "rd -f" option along with its various format
23 options may be used to further tailor the output.
24 file only used with -v, copy the device dump data to a file.
7.3. EXAMPLES
1 Display character and block device data:
2
3 crash> dev
4 CHRDEV NAME CDEV OPERATIONS
5 1 mem f79b83c0 memory_fops
6 4 /dev/vc/0 c07bc560 console_fops
7 4 tty f7af5004 tty_fops
8 4 ttyS f7b02204 tty_fops
9 5 /dev/tty c07bc440 tty_fops
10 5 /dev/console c07bc4a0 console_fops
11 5 /dev/ptmx c07bc500 ptmx_fops
12 6 lp c5797e40 lp_fops
13 7 vcs f7b03d40 vcs_fops
14 10 misc f7f68640 misc_fops
15 13 input f79b8840 input_fops
16 21 sg f7f12840 sg_fops
17 29 fb f7f8c640 fb_fops
18 128 ptm f7b02604 tty_fops
19 136 pts f7b02404 tty_fops
20 162 raw c0693e40 raw_fops
21 180 usb f79b8bc0 usb_fops
22 189 usb_device c06a0300 usbfs_device_file_operations
23 216 rfcomm f5961a04 tty_fops
24 254 pcmcia f79b82c0 ds_fops
25
26 BLKDEV NAME GENDISK OPERATIONS
27 1 ramdisk f7b23480 rd_bd_op
28 8 sd f7cab280 sd_fops
29 9 md f7829b80 md_fops
30 11 sr f75c24c0 sr_bdops
31 65 sd (none)
32 66 sd (none)
33 67 sd (none)
34 68 sd (none)
35 69 sd (none)
36 70 sd (none)
37 71 sd (none)
38 128 sd (none)
39 129 sd (none)
40 130 sd (none)
41 131 sd (none)
42 132 sd (none)
43 133 sd (none)
44 134 sd (none)
45 135 sd (none)
46 253 device-mapper c57a0ac0 dm_blk_dops
47 254 mdp (none)
48
49 Display PCI data:
50
51 crash> dev -p
52 PCI_DEV BU:SL.FN CLASS: VENDOR-DEVICE
53 c00051c0 00:00.0 Host bridge: Intel 440BX - 82443BX Host
54 c0005250 00:01.0 PCI bridge: Intel 440BX - 82443BX AGP
55 c00052e0 00:07.0 ISA bridge: Intel 82371AB PIIX4 ISA
56 c0005370 00:07.1 IDE interface: Intel 82371AB PIIX4 IDE
57 c0005400 00:07.2 USB Controller: Intel 82371AB PIIX4 USB
58 c0005490 00:07.3 Bridge: Intel 82371AB PIIX4 ACPI
59 c0005520 00:11.0 Ethernet controller: 3Com 3C905B 100bTX
60 c00055b0 00:13.0 PCI bridge: DEC DC21152
61 c0005640 01:00.0 VGA compatible controller: NVidia [PCI_DEVICE 28]
62 c00056d0 02:0a.0 SCSI storage controller: Adaptec AIC-7890/1
63 c0005760 02:0e.0 SCSI storage controller: Adaptec AIC-7880U
64
65 Display I/O port and I/O memory usage:
66
67 crash> dev -i
68 RESOURCE RANGE NAME
69 c03036d4 0000-ffff PCI IO
70 c0302594 0000-001f dma1
71 c03025b0 0020-003f pic1
72 c03025cc 0040-005f timer
73 c03025e8 0060-006f keyboard
74 c0302604 0080-008f dma page reg
75 c0302620 00a0-00bf pic2
76 c030263c 00c0-00df dma2
77 c0302658 00f0-00ff fpu
78 c122ff20 0170-0177 ide1
79 c122f240 0213-0213 isapnp read
80 c122ff40 02f8-02ff serial(auto)
81 c122ff00 0376-0376 ide1
82 c03186e8 03c0-03df vga+
83 c122ff60 03f8-03ff serial(auto)
84 c123851c 0800-083f Intel Corporation 82371AB PIIX4 ACPI
85 c1238538 0840-085f Intel Corporation 82371AB PIIX4 ACPI
86 c122f220 0a79-0a79 isapnp write
87 c122f200 0cf8-0cff PCI conf1
88 c1238858 dc00-dc7f 3Com Corporation 3c905B 100BaseTX [Cyclone]
89 c122fc00 dc00-dc7f 00:11.0
90 c12380c8 dce0-dcff Intel Corporation 82371AB PIIX4 USB
91 c1238d1c e000-efff PCI Bus #02
92 c1237858 e800-e8ff Adaptec AIC-7880U
93 c1237458 ec00-ecff Adaptec AHA-2940U2/W / 7890
94 c1239cc8 ffa0-ffaf Intel Corporation 82371AB PIIX4 IDE
95
96 RESOURCE RANGE NAME
97 c03036f0 00000000-ffffffff PCI mem
98 c0004000 00000000-0009ffff System RAM
99 c03026ac 000a0000-000bffff Video RAM area
100 c03026fc 000c0000-000c7fff Video ROM
101 c0302718 000c9800-000cdfff Extension ROM
102 c0302734 000ce000-000ce7ff Extension ROM
103 c0302750 000ce800-000cffff Extension ROM
104 c03026e0 000f0000-000fffff System ROM
105 c0004040 00100000-07ffdfff System RAM
106 c0302674 00100000-0028682b Kernel code
107 c0302690 0028682c-0031c63f Kernel data
108 c0004060 07ffe000-07ffffff reserved
109 c1239058 ec000000-efffffff Intel Corporation 440BX/ZX - 82443BX/ZX Host
110 bridge
111 c1238d54 f1000000-f1ffffff PCI Bus #02
112 c1239554 f2000000-f5ffffff PCI Bus #01
113 c1237074 f4000000-f5ffffff nVidia Corporation Riva TnT2 [NV5]
114 c1238d38 fa000000-fbffffff PCI Bus #02
115 c1237874 faffe000-faffefff Adaptec AIC-7880U
116 c127ec40 faffe000-faffefff aic7xxx
117 c1237474 fafff000-faffffff Adaptec AHA-2940U2/W / 7890
118 c127eec0 fafff000-faffffff aic7xxx
119 c1239538 fc000000-fdffffff PCI Bus #01
120 c1237058 fc000000-fcffffff nVidia Corporation Riva TnT2 [NV5]
121 c1238874 fe000000-fe00007f 3Com Corporation 3c905B 100BaseTX [Cyclone]
122 c0004080 fec00000-fec0ffff reserved
123 c00040a0 fee00000-fee0ffff reserved
124 c00040c0 ffe00000-ffffffff reserved
125
126 Display disk I/O statistics:
127
128 crash> dev -d
129 MAJOR GENDISK NAME REQUEST_QUEUE TOTAL READ WRITE DRV
130 2 ffff81012d8a5000 fd0 ffff81012dc053c0 12 0 12 0
131 22 ffff81012dc6b000 hdc ffff81012d8ae340 2 2 0 0
132 8 ffff81012dd71000 sda ffff81012d8af040 6 0 6 6
133 8 ffff81012dc77000 sdb ffff81012d8b5740 0 0 0 0
134 8 ffff81012d8d0c00 sdc ffff81012d8ae9c0 0 0 0 0
135
136 Display the available device dumps:
137
138 crash> dev -V
139 INDEX OFFSET SIZE NAME
140 0 0x240 33558464 cxgb4_0000:02:00.4
141 1 0x2001240 33558464 cxgb4_0000:03:00.4
142
143 Extract a specified device dump to file:
144
145 crash> dev -v 0 device_dump_0.bin
146 DEVICE: cxgb4_0000:02:00.4
147 33558464 bytes copied from 0x240 to device_dump_0.bin
148
149 Format and display a device's dump data to the screen using the "rd" command:
150
151 crash> rd -f 0x240 -32 8
152 240: 040b69e2 00000038 000e0001 00675fd4 .i..8........_g.
153 250: 00000000 21600047 00000000 00000000 ....G.`!........
154
155 Display a device's dump data to the screen using the default format:
156
157 crash> dev -v 1
158 DEVICE: cxgb4_0000:03:00.4
159 2001240: 00000038040b69e2 00af985c000e0001 .i..8.......\...
160 2001250: 2150004700000000 0000000000000000 ....G.P!........
161 2001260: 0000000000000000 0000000000000000 ................
162 2001270: 0000000000000000 0002fccc00000001 ................
163 2001280: 00000000000027b0 0000000000000000 .'..............
164 ...
8. dis - disassemble
8.1. SYNOPSIS
1dis [-rfludxs][-b [num]] [address | symbol | (expression)] [count]
8.2. DESCRIPTION
1 This command disassembles source code instructions starting (or ending) at
2 a text address that may be expressed by value, symbol or expression:
3
4 -r (reverse) displays all instructions from the start of the
5 routine up to and including the designated address.
6 -f (forward) displays all instructions from the given address
7 to the end of the routine.
8 -l displays source code line number data in addition to the
9 disassembly output.
10 -u address is a user virtual address in the current context;
11 otherwise the address is assumed to be a kernel virtual address.
12 If this option is used, then -r and -l are ignored.
13 -x override default output format with hexadecimal format.
14 -d override default output format with decimal format.
15 -s displays the filename and line number of the source code that
16 is associated with the specified text location, followed by a
17 source code listing if it is available on the host machine.
18 The line associated with the text location will be marked with
19 an asterisk; depending upon gdb's internal "listsize" variable,
20 several lines will precede the marked location. If a "count"
21 argument is entered, it specifies the number of source code
22 lines to be displayed after the marked location; otherwise
23 the remaining source code of the containing function will be
24 displayed.
25 -b [num] modify the pre-calculated number of encoded bytes to skip after
26 a kernel BUG ("ud2a") instruction; with no argument, displays
27 the current number of bytes being skipped. (x86 and x86_64 only)
28 address starting hexadecimal text address.
29 symbol symbol of starting text address. On ppc64, the symbol
30 preceded by '.' is used.
31 (expression) expression evaluating to a starting text address.
32 count the number of instructions to be disassembled (default is 1).
33 If no count argument is entered, and the starting address
34 is entered as a text symbol, then the whole routine will be
35 disassembled. The count argument is supported when used with
36 the -r and -f options.
8.3. EXAMPLES
1 Disassemble the sys_signal() routine without, and then with, line numbers:
2
3 crash> dis sys_signal
4 0xc0112c88 <sys_signal>: push %ebp
5 0xc0112c89 <sys_signal+1>: mov %esp,%ebp
6 0xc0112c8b <sys_signal+3>: sub $0x28,%esp
7 0xc0112c8e <sys_signal+6>: mov 0xc(%ebp),%eax
8 0xc0112c91 <sys_signal+9>: mov %eax,0xffffffec(%ebp)
9 0xc0112c94 <sys_signal+12>: movl $0xc0000000,0xfffffff0(%ebp)
10 0xc0112c9b <sys_signal+19>: lea 0xffffffd8(%ebp),%eax
11 0xc0112c9e <sys_signal+22>: push %eax
12 0xc0112c9f <sys_signal+23>: lea 0xffffffec(%ebp),%eax
13 0xc0112ca2 <sys_signal+26>: push %eax
14 0xc0112ca3 <sys_signal+27>: pushl 0x8(%ebp)
15 0xc0112ca6 <sys_signal+30>: call 0xc01124b8 <do_sigaction>
16 0xc0112cab <sys_signal+35>: test %eax,%eax
17 0xc0112cad <sys_signal+37>: jne 0xc0112cb2 <sys_signal+42>
18 0xc0112caf <sys_signal+39>: mov 0xffffffd8(%ebp),%eax
19 0xc0112cb2 <sys_signal+42>: leave
20 0xc0112cb3 <sys_signal+43>: ret
21
22 crash> dis -l sys_signal
23 /usr/src/linux-2.2.5/kernel/signal.c: 1074
24 0xc0112c88 <sys_signal>: push %ebp
25 0xc0112c89 <sys_signal+1>: mov %esp,%ebp
26 0xc0112c8b <sys_signal+3>: sub $0x28,%esp
27 0xc0112c8e <sys_signal+6>: mov 0xc(%ebp),%eax
28 /usr/src/linux-2.2.5/kernel/signal.c: 1078
29 0xc0112c91 <sys_signal+9>: mov %eax,0xffffffec(%ebp)
30 /usr/src/linux-2.2.5/kernel/signal.c: 1079
31 0xc0112c94 <sys_signal+12>: movl $0xc0000000,0xfffffff0(%ebp)
32 /usr/src/linux-2.2.5/kernel/signal.c: 1081
33 0xc0112c9b <sys_signal+19>: lea 0xffffffd8(%ebp),%eax
34 0xc0112c9e <sys_signal+22>: push %eax
35 0xc0112c9f <sys_signal+23>: lea 0xffffffec(%ebp),%eax
36 0xc0112ca2 <sys_signal+26>: push %eax
37 0xc0112ca3 <sys_signal+27>: pushl 0x8(%ebp)
38 0xc0112ca6 <sys_signal+30>: call 0xc01124b8 <do_sigaction>
39 /usr/src/linux-2.2.5/kernel/signal.c: 1083
40 0xc0112cab <sys_signal+35>: test %eax,%eax
41 0xc0112cad <sys_signal+37>: jne 0xc0112cb2 <sys_signal+42>
42 0xc0112caf <sys_signal+39>: mov 0xffffffd8(%ebp),%eax
43 /usr/src/linux-2.2.5/kernel/signal.c: 1084
44 0xc0112cb2 <sys_signal+42>: leave
45 0xc0112cb3 <sys_signal+43>: ret
46
47 Given a return address expression of "do_no_page+65", find out the
48 function that do_no_page() calls by using the reverse flag:
49
50 crash> dis -r (do_no_page+65)
51 0xc011ea68 <do_no_page>: push %ebp
52 0xc011ea69 <do_no_page+1>: mov %esp,%ebp
53 0xc011ea6b <do_no_page+3>: push %edi
54 0xc011ea6c <do_no_page+4>: push %esi
55 0xc011ea6d <do_no_page+5>: push %ebx
56 0xc011ea6e <do_no_page+6>: mov 0xc(%ebp),%ebx
57 0xc011ea71 <do_no_page+9>: mov 0x10(%ebp),%edx
58 0xc011ea74 <do_no_page+12>: mov 0x14(%ebp),%edi
59 0xc011ea77 <do_no_page+15>: mov 0x28(%ebx),%eax
60 0xc011ea7a <do_no_page+18>: test %eax,%eax
61 0xc011ea7c <do_no_page+20>: je 0xc011ea85 <do_no_page+29>
62 0xc011ea7e <do_no_page+22>: mov 0x18(%eax),%ecx
63 0xc011ea81 <do_no_page+25>: test %ecx,%ecx
64 0xc011ea83 <do_no_page+27>: jne 0xc011eab0 <do_no_page+72>
65 0xc011ea85 <do_no_page+29>: mov $0xffffe000,%eax
66 0xc011ea8a <do_no_page+34>: and %esp,%eax
67 0xc011ea8c <do_no_page+36>: decl 0x30(%eax)
68 0xc011ea8f <do_no_page+39>: jns 0xc011ea9a <do_no_page+50>
69 0xc011ea91 <do_no_page+41>: lock btrl $0x0,0xc022fb60
70 0xc011ea9a <do_no_page+50>: push %edi
71 0xc011ea9b <do_no_page+51>: mov 0x18(%ebp),%esi
72 0xc011ea9e <do_no_page+54>: push %esi
73 0xc011ea9f <do_no_page+55>: push %ebx
74 0xc011eaa0 <do_no_page+56>: mov 0x8(%ebp),%esi
75 0xc011eaa3 <do_no_page+59>: push %esi
76 0xc011eaa4 <do_no_page+60>: call 0xc011e9e4 <do_anonymous_page>
77 0xc011eaa9 <do_no_page+65>: jmp 0xc011eb47 <do_no_page+223>
78
79 Disassemble 10 instructions starting at user virtual address 0x81ec624:
80
81 crash> dis -u 81ec624 10
82 0x81ec624: push %ebp
83 0x81ec625: mov %esp,%ebp
84 0x81ec627: sub $0x18,%esp
85 0x81ec62a: movl $0x1,0x8(%ebp)
86 0x81ec631: mov 0x82f9040,%eax
87 0x81ec636: mov 0x10(%eax),%edx
88 0x81ec639: and $0x100,%edx
89 0x81ec63f: mov 0x14(%eax),%ecx
90 0x81ec642: and $0x0,%ecx
91 0x81ec645: mov %ecx,%eax
92
93 Override the current decimal output radix format:
94
95 crash> dis sys_read 10 -x
96 0xffffffff8001178f <sys_read>: push %r13
97 0xffffffff80011791 <sys_read+0x2>: mov %rsi,%r13
98 0xffffffff80011794 <sys_read+0x5>: push %r12
99 0xffffffff80011796 <sys_read+0x7>: mov $0xfffffffffffffff7,%r12
100 0xffffffff8001179d <sys_read+0xe>: push %rbp
101 0xffffffff8001179e <sys_read+0xf>: mov %rdx,%rbp
102 0xffffffff800117a1 <sys_read+0x12>: push %rbx
103 0xffffffff800117a2 <sys_read+0x13>: sub $0x18,%rsp
104 0xffffffff800117a6 <sys_read+0x17>: lea 0x14(%rsp),%rsi
105 0xffffffff800117ab <sys_read+0x1c>: callq 0xffffffff8000b5b4 <fget_light>
106
107 Disassemble from vfs_read+320 until the end of the function:
108
109 crash> dis -f vfs_read+320
110 0xffffffff8119d4e0 <vfs_read+320>: cmpq $0x0,0x20(%rax)
111 0xffffffff8119d4e5 <vfs_read+325>: jne 0xffffffff8119d3e8 <vfs_read+72>
112 0xffffffff8119d4eb <vfs_read+331>: mov $0xffffffffffffffea,%r12
113 0xffffffff8119d4f2 <vfs_read+338>: jmp 0xffffffff8119d4c3 <vfs_read+291>
114 0xffffffff8119d4f4 <vfs_read+340>: nopl 0x0(%rax)
115 0xffffffff8119d4f8 <vfs_read+344>: callq 0xffffffff8119cc40 <do_sync_read>
116 0xffffffff8119d4fd <vfs_read+349>: mov %rax,%r12
117 0xffffffff8119d500 <vfs_read+352>: jmpq 0xffffffff8119d44c <vfs_read+172>
118 0xffffffff8119d505 <vfs_read+357>: nopl (%rax)
119 0xffffffff8119d508 <vfs_read+360>: mov $0xfffffffffffffff7,%r12
120 0xffffffff8119d50f <vfs_read+367>: jmp 0xffffffff8119d4c3 <vfs_read+291>
121 0xffffffff8119d511 <vfs_read+369>: mov $0xfffffffffffffff2,%r12
122 0xffffffff8119d518 <vfs_read+376>: jmp 0xffffffff8119d4c3 <vfs_read+291>
123 0xffffffff8119d51a <vfs_read+378>: nopw 0x0(%rax,%rax,1)
124
125 Display the source code listing of the mmput() function:
126
127 crash> dis -s mmput
128 FILE: kernel/fork.c
129 LINE: 617
130
131 612
132 613 /*
133 614 * Decrement the use count and release all resources for an mm.
134 615 */
135 616 void mmput(struct mm_struct *mm)
136 * 617 {
137 618 might_sleep();
138 619
139 620 if (atomic_dec_and_test(&mm->mm_users)) {
140 621 uprobe_clear_state(mm);
141 622 exit_aio(mm);
142 623 ksm_exit(mm);
143 624 khugepaged_exit(mm); /* must run before exit_mmap */
144 625 exit_mmap(mm);
145 626 set_mm_exe_file(mm, NULL);
146 627 if (!list_empty(&mm->mmlist)) {
147 628 spin_lock(&mmlist_lock);
148 629 list_del(&mm->mmlist);
149 630 spin_unlock(&mmlist_lock);
150 631 }
151 632 if (mm->binfmt)
152 633 module_put(mm->binfmt->module);
153 634 mmdrop(mm);
154 635 }
155 636 }
156
157 The disassembly of dentry_kill() shows an indirect call to a function
158 whose address is contained within a register. Display the source code
159 associated with the indirect function call:
160
161 crash> dis dentry_kill
162 ...
163 0xffffffff811dcfb4 <dentry_kill+324>: callq *%rax
164 ...
165 crash> dis -s 0xffffffff811dcfb4
166 FILE: fs/dcache.c
167 LINE: 276
168
169 271 spin_unlock(&dentry->d_lock);
170 272 spin_unlock(&inode->i_lock);
171 273 if (!inode->i_nlink)
172 274 fsnotify_inoderemove(inode);
173 275 if (dentry->d_op && dentry->d_op->d_iput)
174 * 276 dentry->d_op->d_iput(dentry, inode);
175 277 else
176 278 iput(inode);
177 279 } else {
178 280 spin_unlock(&dentry->d_lock);
179 281 }
180 282 }
9. eval - evaluate
9.1. SYNOPSIS
1eval [-b][-l] (expression) | value
9.2. DESCRIPTION
1 This command evaluates an expression or numeric value, and displays its
2 result in hexadecimal, decimal, octal and binary. If the resultant value
3 is an integral number of gigabytes, megabytes, or kilobytes, a short-hand
4 translation of the number will also be shown next to the hexadecimal
5 value. If the most significant bit is set, the decimal display will show
6 both unsigned and signed (negative) values. Expressions must of the format
7 (x operator y), where "x" and "y" may be either numeric values or
8 symbols. The list of operators are:
9
10 + - & | ^ * % / << >>
11
12 Enclosing the expression within parentheses is optional except when the
13 "|", "<<" or ">>" operators are used. The single "value" argument may
14 be a number or symbol. Number arguments must be hexadecimal or decimal.
15 A leading "0x" identifies a number as hexadecimal, but is not required
16 when obvious. Numbers may be followed by the letters "k" or "K", "m"
17 or "M", and "g" or "G", which multiplies the value by a factor of 1024,
18 1 megabyte or 1 gigabyte, respectively. Numeric arguments may be preceded
19 by the one's complement operator ~.
20
21 -b Indicate which bit positions in the resultant value are set.
22 -l Numeric arguments are presumed to be 64-bit values, and the result
23 will be expressed as a 64-bit value. (ignored on 64-bit processors)
24 However, if either operand or the resultant value are 64-bit values,
25 then the result will be also be expressed as a 64-bit value.
26
27 The -b and -l options must precede the expression or value arguments.
9.3. EXAMPLES
1 crash> eval 128m
2 hexadecimal: 8000000 (128MB)
3 decimal: 134217728
4 octal: 1000000000
5 binary: 00001000000000000000000000000000
6
7 crash> eval 128 * 1m
8 hexadecimal: 8000000 (128MB)
9 decimal: 134217728
10 octal: 1000000000
11 binary: 00001000000000000000000000000000
12
13 crash> eval (1 << 27)
14 hexadecimal: 8000000 (128MB)
15 decimal: 134217728
16 octal: 1000000000
17 binary: 00001000000000000000000000000000
18
19 crash> eval (1 << 32)
20 hexadecimal: 100000000 (4GB)
21 decimal: 4294967296
22 octal: 40000000000
23 binary: 0000000000000000000000000000000100000000000000000000000000000000
24
25 crash> eval -b 41dc065
26 hexadecimal: 41dc065
27 decimal: 69058661
28 octal: 407340145
29 binary: 00000100000111011100000001100101
30 bits set: 26 20 19 18 16 15 14 6 5 2 0
31
32 crash> eval -lb 64g
33 hexadecimal: 1000000000 (64GB)
34 decimal: 68719476736
35 octal: 1000000000000
36 binary: 0000000000000000000000000001000000000000000000000000000000000000
37 bits set: 36
10. exit - exit this session
10.1. SYNOPSIS
1exit
10.2. DESCRIPTION
1 Bail out of the current crash session.
10.3. NOTE
1 This command is equivalent to the "q" command.
11. extend - extend the crash command set
11.1. SYNOPSIS
1extend [shared-object ...] | [-u [shared-object ...]]
11.2. DESCRIPTION
1 This command dynamically loads or unloads crash extension shared object
2 libraries:
3
4 shared-object load the specified shared object file; more than one
5 one object file may be entered.
6 -u shared-object unload the specified shared object file; if no file
7 arguments are specified, unload all objects.
8
9 If the shared-object filename is not expressed with a fully-qualified
10 pathname, the following directories will be searched in the order shown,
11 and the first instance of the file that is found will be selected:
12
13 1. the current working directory
14 2. the directory specified in the CRASH_EXTENSIONS environment variable
15 3. /usr/lib64/crash/extensions (64-bit architectures)
16 4. /usr/lib/crash/extensions
17
18 If no arguments are entered, the current set of shared object files and
19 a list of their commands will be displayed. The registered commands
20 contained in each shared object file will appear automatically in the
21 "help" command screen.
22
23 An example of a shared object prototype file, and how to compile it
24 into a shared object, is appended below.
11.3. EXAMPLES
1 Load two shared object files:
2
3 crash> extend extlib1.so extlib2.so
4 ./extlib1.so: shared object loaded
5 ./extlib2.so: shared object loaded
6
7 Display the current set of shared object files and their commands:
8
9 crash> extend
10 SHARED OBJECT COMMANDS
11 ./extlib1.so echo util bin
12 ./extlib2.so smp show
13
14 Unload one of the shared object files:
15
16 crash> extend -u extlib1.so
17 ./extlib1.so: shared object unloaded
18
19 Unload all currently-loaded object files:
20
21 crash> extend -u
22 ./extlib2.so: shared object unloaded
23
24CREATING A SHARED OBJECT
25 The extend command loads shared object files using dlopen(3), which in
26 turn calls the shared object's constructor function. The shared object's
27 constructor function should register its command set by calling
28 register_extension(), passing it a pointer to an array of one or more
29 structures of the following type:
30
31 struct command_table_entry {
32 char *name;
33 cmd_func_t func;
34 char **help_data,
35 ulong flags;
36 };
37
38 Each command_table_entry structure contains the ASCII name of a command,
39 the command's function address, a pointer to an array of help data strings,
40 and a flags field. The help_data field is optional; if it is non-NULL, it
41 should point to an array of character strings used by the "help"
42 command, and during command failures. The flags field currently has two
43 available bit settings, REFRESH_TASK_TABLE, which should be set if it is
44 preferable to reload the current set of running processes just prior to
45 executing the command (on a live system) and MINIMAL, which should be
46 set if the command should be available in minimal mode. Terminate the array
47 of command_table_entry structures with an entry with a NULL command name.
48
49 Below is an example shared object file consisting of just one command,
50 called "echo", which simply echoes back all arguments passed to it.
51 Note the comments contained within it for further details. Cut and paste
52 the following output into a file, and call it, for example, "echo.c".
53 Then compiled in either of two manners. Either manually like so:
54
55 gcc -shared -rdynamic -o echo.so echo.c -fPIC -D<machine-type> $(TARGET_CFLAGS)
56
57 where <machine-type> must be one of the MACHINE_TYPE #define's in defs.h,
58 and where $(TARGET_CFLAGS) is the same as it is declared in the top-level
59 Makefile after a build is completed. Or alternatively, the "echo.c" file
60 can be copied into the "extensions" subdirectory, and compiled automatically
61 like so:
62
63 make extensions
64
65 The echo.so file may be dynamically linked into crash during runtime, or
66 during initialization by putting "extend echo.so" into a .crashrc file
67 located in the current directory, or in the user's $HOME directory.
68
69---------------------------------- cut here ----------------------------------
70
71#include "defs.h" /* From the crash source top-level directory */
72
73void echo_init(void); /* constructor function */
74void echo_fini(void); /* destructor function (optional) */
75
76void cmd_echo(void); /* Declare the commands and their help data. */
77char *help_echo[];
78
79static struct command_table_entry command_table[] = {
80 { "echo", cmd_echo, help_echo, 0}, /* One or more commands, */
81 { NULL }, /* terminated by NULL, */
82};
83
84
85void __attribute__((constructor))
86echo_init(void) /* Register the command set. */
87{
88 register_extension(command_table);
89}
90
91/*
92 * This function is called if the shared object is unloaded.
93 * If desired, perform any cleanups here.
94 */
95void __attribute__((destructor))
96echo_fini(void) { }
97
98
99/*
100 * Arguments are passed to the command functions in the global args[argcnt]
101 * array. See getopt(3) for info on dash arguments. Check out defs.h and
102 * other crash commands for usage of the myriad of utility routines available
103 * to accomplish what your task.
104 */
105void
106cmd_echo(void)
107{
108 int c;
109
110 while ((c = getopt(argcnt, args, "")) != EOF) {
111 switch(c)
112 {
113 default:
114 argerrs++;
115 break;
116 }
117 }
118
119 if (argerrs)
120 cmd_usage(pc->curcmd, SYNOPSIS);
121
122 while (args[optind])
123 fprintf(fp, "%s ", args[optind++]);
124
125 fprintf(fp, "\n");
126}
127
128/*
129 * The optional help data is simply an array of strings in a defined format.
130 * For example, the "help echo" command will use the help_echo[] string
131 * array below to create a help page that looks like this:
132 *
133 * NAME
134 * echo - echoes back its arguments
135 *
136 * SYNOPSIS
137 * echo arg ...
138 *
139 * DESCRIPTION
140 * This command simply echoes back its arguments.
141 *
142 * EXAMPLE
143 * Echo back all command arguments:
144 *
145 * crash> echo hello, world
146 * hello, world
147 *
148 */
149
150char *help_echo[] = {
151 "echo", /* command name */
152 "echoes back its arguments", /* short description */
153 "arg ...", /* argument synopsis, or " " if none */
154
155 " This command simply echoes back its arguments.",
156 "\nEXAMPLE",
157 " Echo back all command arguments:\n",
158 " crash> echo hello, world",
159 " hello, world",
160 NULL
161};
12. files - open files
12.1. SYNOPSIS
1files [-d dentry] | [-p inode] | [-c] [-R reference] [pid | taskp] ...
12.2. DESCRIPTION
1 This command displays information about open files of a context.
2 It prints the context's current root directory and current working
3 directory, and then for each open file descriptor it prints a pointer
4 to its file struct, a pointer to its dentry struct, a pointer to the
5 inode, the file type, and the pathname. If no arguments are entered,
6 the current context is used. The -R option, typically invoked from
7 "foreach files", searches for references to a supplied number, address,
8 or filename argument, and prints only the essential information leading
9 up to and including the reference. The -d option is not context
10 specific, and only shows the data requested.
11
12 -d dentry given a hexadecimal dentry address, display its inode,
13 super block, file type, and full pathname.
14 -p inode given a hexadecimal inode address, dump all of its pages
15 that are in the page cache.
16 -c for each open file descriptor, prints a pointer to its
17 inode, a pointer to the inode's i_mapping address_space
18 structure, the number of pages of the inode that are in
19 the page cache, the file type, and the pathname.
20 -R reference search for references to this file descriptor number,
21 filename, dentry, inode, address_space, or file structure
22 address.
23 pid a process PID.
24 taskp a hexadecimal task_struct pointer.
12.3. EXAMPLES
1 Display the open files of the current context:
2
3 crash> files
4 PID: 720 TASK: c67f2000 CPU: 1 COMMAND: "innd"
5 ROOT: / CWD: /var/spool/news/articles
6 FD FILE DENTRY INODE TYPE PATH
7 0 c6b9c740 c7cc45a0 c7c939e0 CHR /dev/null
8 1 c6b9c800 c537bb20 c54d0000 REG /var/log/news/news
9 2 c6df9600 c537b420 c5c36360 REG /var/log/news/errlog
10 3 c74182c0 c6ede260 c6da3d40 PIPE
11 4 c6df9720 c696c620 c69398c0 SOCK
12 5 c6b9cc20 c68e7000 c6938d80 SOCK
13 6 c6b9c920 c7cc45a0 c7c939e0 CHR /dev/null
14 7 c6b9c680 c58fa5c0 c58a1200 REG /var/lib/news/history
15 8 c6df9f00 c6ede760 c6da3200 PIPE
16 9 c6b9c6e0 c58fa140 c5929560 REG /var/lib/news/history.dir
17 10 c7fa9320 c7fab160 c7fafd40 CHR /dev/console
18 11 c6b9c7a0 c58fa5c0 c58a1200 REG /var/lib/news/history
19 12 c377ec60 c58fa5c0 c58a1200 REG /var/lib/news/history
20 13 c4528aa0 c58fa6c0 c52fbb00 REG /var/lib/news/history.pag
21 14 c6df9420 c68e7700 c6938360 SOCK
22 15 c6df9360 c68e7780 c6938120 SOCK
23 16 c6b9c0e0 c68e7800 c6772000 SOCK
24 17 c6b9c200 c6b5f9c0 c6b5cea0 REG /var/lib/news/active
25 21 c6b9c080 c6ede760 c6da3200 PIPE
26
27 Display the files opened by the "crond" daemon, which is PID 462:
28
29 crash> files 462
30 PID: 462 TASK: f7220000 CPU: 2 COMMAND: "crond"
31 ROOT: / CWD: /var/spool
32 FD FILE DENTRY INODE TYPE PATH
33 0 f7534ae0 f7538de0 f7518dc0 CHR /dev/console
34 1 f7368f80 f72c7a40 f72f27e0 FIFO pipe:/[1456]
35 2 f74f3c80 f72c79c0 f72f2600 FIFO pipe:/[1457]
36 3 f7368b60 f72a5be0 f74300c0 REG /var/run/crond.pid
37 4 f7534360 f73408c0 f72c2840 REG /var/log/cron
38 7 f7368ce0 f72c7940 f72f2420 FIFO pipe:/[1458]
39 8 f7295de0 f72c7940 f72f2420 FIFO pipe:/[1458]
40 21 f74f36e0 f747cdc0 f747e840 CHR /dev/null
41
42 The -R option is typically invoked from "foreach files". This example
43 shows all tasks that have "/dev/pts/4" open:
44
45 crash> foreach files -R pts/4
46 PID: 18633 TASK: c310a000 CPU: 0 COMMAND: "crash"
47 ROOT: / CWD: /home/CVS_pool/crash
48 FD FILE DENTRY INODE TYPE PATH
49 0 c1412850 c2cb96d0 c2cad430 CHR /dev/pts/4
50 1 c1412850 c2cb96d0 c2cad430 CHR /dev/pts/4
51 2 c1412850 c2cb96d0 c2cad430 CHR /dev/pts/4
52
53 PID: 18664 TASK: c2392000 CPU: 1 COMMAND: "less"
54 ROOT: / CWD: /home/CVS_pool/crash
55 FD FILE DENTRY INODE TYPE PATH
56 1 c1412850 c2cb96d0 c2cad430 CHR /dev/pts/4
57 2 c1412850 c2cb96d0 c2cad430 CHR /dev/pts/4
58
59 PID: 23162 TASK: c5088000 CPU: 1 COMMAND: "bash"
60 ROOT: / CWD: /home/CVS_pool/crash
61 FD FILE DENTRY INODE TYPE PATH
62 0 c1412850 c2cb96d0 c2cad430 CHR /dev/pts/4
63 1 c1412850 c2cb96d0 c2cad430 CHR /dev/pts/4
64 2 c1412850 c2cb96d0 c2cad430 CHR /dev/pts/4
65 255 c1412850 c2cb96d0 c2cad430 CHR /dev/pts/4
66
67 PID: 23159 TASK: c10fc000 CPU: 1 COMMAND: "xterm"
68 ROOT: / CWD: /homes/anderson/
69 FD FILE DENTRY INODE TYPE PATH
70 5 c1560da0 c2cb96d0 c2cad430 CHR /dev/pts/4
71
72 Display information about the dentry at address f745fd60:
73
74 crash> files -d f745fd60
75 DENTRY INODE SUPERBLK TYPE PATH
76 f745fd60 f7284640 f73a3e00 REG /var/spool/lpd/lpd.lock
77
78 For each open file, display the number of pages that are in the page cache:
79
80 crash> files -c 1954
81 PID: 1954 TASK: f7a28000 CPU: 1 COMMAND: "syslogd"
82 ROOT: / CWD: /
83 FD INODE I_MAPPING NRPAGES TYPE PATH
84 0 cb3ae868 cb3ae910 0 SOCK socket:/[4690]
85 2 f2721c5c f2721d04 461 REG /var/log/messages
86 3 cbda4884 cbda492c 47 REG /var/log/secure
87 4 e48092c0 e4809368 58 REG /var/log/maillog
88 5 f65192c0 f6519368 48 REG /var/log/cron
89 6 e4809e48 e4809ef0 0 REG /var/log/spooler
90 7 d9c43884 d9c4392c 0 REG /var/log/boot.log
91
92 For the inode at address f59b90fc, display all of its pages that are in
93 the page cache:
94
95 crash> files -p f59b90fc
96 INODE NRPAGES
97 f59b90fc 6
98
99 PAGE PHYSICAL MAPPING INDEX CNT FLAGS
100 ca3353e0 39a9f000 f59b91ac 0 2 82c referenced,uptodate,lru,private
101 ca22cb20 31659000 f59b91ac 1 2 82c referenced,uptodate,lru,private
102 ca220160 3100b000 f59b91ac 2 2 82c referenced,uptodate,lru,private
103 ca1ddde0 2eeef000 f59b91ac 3 2 82c referenced,uptodate,lru,private
104 ca36b300 3b598000 f59b91ac 4 2 82c referenced,uptodate,lru,private
105 ca202680 30134000 f59b91ac 5 2 82c referenced,uptodate,lru,private
13. foreach - display command data for multiple tasks in the system
13.1. SYNOPSIS
1foreach [[pid | taskp | name | state | [kernel | user | gleader]] ...]
2 command [flag] [argument]
13.2. DESCRIPTION
1 This command allows for an examination of various kernel data associated
2 with any, or all, tasks in the system, without having to set the context
3 to each targeted task.
4
5 pid perform the command(s) on this PID.
6 taskp perform the command(s) on task referenced by this hexadecimal
7 task_struct pointer.
8 name perform the command(s) on all tasks with this name. If the
9 task name can be confused with a foreach command name, then
10 precede the name string with a "\". If the name string is
11 enclosed within "'" characters, then the encompassed string
12 must be a POSIX extended regular expression that will be used
13 to match task names.
14 user perform the command(s) on all user (non-kernel) threads.
15 gleader perform the command(s) on all user (non-kernel) thread group leaders.
16 kernel perform the command(s) on all kernel threads.
17 active perform the command(s) on the active thread on each CPU.
18 state perform the command(s) on all tasks in the specified state, which
19 may be one of: RU, IN, UN, ST, ZO, TR, SW, DE, WA, PA, ID or NE.
20
21 If none of the task-identifying arguments above are entered, the command
22 will be performed on all tasks.
23
24 command select one or more of the following commands to be run on the tasks
25 selected, or on all tasks:
26
27 bt run the "bt" command (optional flags: -r -t -l -e -R -f -F
28 -o -s -x -d)
29 vm run the "vm" command (optional flags: -p -v -m -R -d -x)
30 task run the "task" command (optional flags: -R -d -x)
31 files run the "files" command (optional flag: -c -R)
32 net run the "net" command (optional flags: -s -S -R -d -x)
33 set run the "set" command
34 ps run the "ps" command (optional flags: -G -s -p -c -t -l -a
35 -g -r -y)
36 sig run the "sig" command (optional flag: -g)
37 vtop run the "vtop" command (optional flags: -c -u -k)
38
39 flag Pass this optional flag to the command selected.
40 argument Pass this argument to the command selected.
41
42 A header containing the PID, task address, cpu and command name will be
43 pre-pended before the command output for each selected task. Consult the
44 help page of each of the command types above for details.
13.3. EXAMPLES
1 Display the stack traces for all tasks:
2
3 crash> foreach bt
4 PID: 4752 TASK: c7680000 CPU: 1 COMMAND: "xterm"
5 #0 [c7681edc] schedule at c01135f6
6 (void)
7 #1 [c7681f34] schedule_timeout at c01131ff
8 (24)
9 #2 [c7681f64] do_select at c0132838
10 (5, c7681fa4, c7681fa0)
11 #3 [c7681fbc] sys_select at c0132dad
12 (5, 8070300, 8070380, 0, 0)
13 #4 [bffffb0c] system_call at c0109944
14 EAX: 0000008e EBX: 00000005 ECX: 08070300 EDX: 08070380
15 DS: 002b ESI: 00000000 ES: 002b EDI: 00000000
16 SS: 002b ESP: bffffadc EBP: bffffb0c
17 CS: 0023 EIP: 402259ee ERR: 0000008e EFLAGS: 00000246
18
19 PID: 557 TASK: c5600000 CPU: 0 COMMAND: "nfsd"
20 #0 [c5601f38] schedule at c01135f6
21 (void)
22 #1 [c5601f90] schedule_timeout at c01131ff
23 (c5600000)
24 #2 [c5601fb8] svc_recv at c805363a
25 (c0096f40, c5602800, 7fffffff, 100, c65c9f1c)
26 #3 [c5601fec] (nfsd module) at c806e303
27 (c5602800, c5602800, c0096f40, 6c6e0002, 50)
28 #4 [c65c9f24] kernel_thread at c010834f
29 (0, 0, ext2_file_inode_operations)
30
31 PID: 824 TASK: c7c84000 CPU: 0 COMMAND: "mingetty"
32 ...
33
34 Display the task_struct structure for each "bash" command:
35
36 crash> foreach bash task
37 ...
38
39 Display the open files for all tasks:
40
41 crash> foreach files
42 ...
43
44 Display the state of tasks whose name contains a match to "event.*":
45
46 crash> foreach 'event.*' task -R state
47 PID: 99 TASK: ffff8804750d5500 CPU: 0 COMMAND: "events/0"
48 state = 1,
49
50 PID: 100 TASK: ffff8804750d4ac0 CPU: 1 COMMAND: "events/1"
51 state = 1,
52
53 PID: 101 TASK: ffff8804750d4080 CPU: 2 COMMAND: "events/2"
54 state = 1,
55 ...
56
57 Display the stack traces for all blocked (TASK_UNINTERRUPTIBLE) tasks:
58
59 crash> foreach UN bt
60 PID: 428 TASK: ffff880036b6c560 CPU: 1 COMMAND: "jbd2/dm-1-8"
61 #0 [ffff880035779a70] __schedule at ffffffff815df272
62 #1 [ffff880035779b08] schedule at ffffffff815dfacf
63 #2 [ffff880035779b18] io_schedule at ffffffff815dfb7f
64 #3 [ffff880035779b38] sleep_on_page at ffffffff81119a4e
65 #4 [ffff880035779b48] __wait_on_bit at ffffffff815e039f
66 #5 [ffff880035779b98] wait_on_page_bit at ffffffff81119bb8
67 #6 [ffff880035779be8] filemap_fdatawait_range at ffffffff81119ccc
68 #7 [ffff880035779cd8] filemap_fdatawait at ffffffff81119d8b
69 #8 [ffff880035779ce8] jbd2_journal_commit_transaction at ffffffff8123a99c
70 #9 [ffff880035779e58] kjournald2 at ffffffff8123ee7b
71 #10 [ffff880035779ee8] kthread at ffffffff8108fb9c
72 #11 [ffff880035779f48] kernel_thread_helper at ffffffff815ebaf4
73 ...
14. fuser - file users
14.1. SYNOPSIS
1fuser [pathname | inode]
14.2. DESCRIPTION
1 This command displays the tasks using specified files or sockets.
2 Tasks will be listed that reference the file as the current working
3 directory, root directory, an open file descriptor, or that mmap the
4 file. If the file is held open in the kernel by the lockd server on
5 behalf of a client discretionary file lock, the client hostname is
6 listed.
7
8 pathname the full pathname of the file.
9 inode the hexadecimal inode address for the file.
14.3. EXAMPLES
1 Display the tasks using file /usr/lib/libkfm.so.2.0.0
2
3 crash> fuser /usr/lib/libkfm.so.2.0.0
4 PID TASK COMM USAGE
5 779 c5e82000 "kwm" mmap
6 808 c5a8e000 "krootwm" mmap
7 806 c5b42000 "kfm" mmap
8 809 c5dde000 "kpanel" mmap
15. gdb - gdb command
15.1. SYNOPSIS
1gdb command ...
15.2. DESCRIPTION
1 This command passes its arguments directly to gdb for processing.
2 This is typically not necessary, but where ambiguities between crash and
3 gdb command names exist, this will force the command to be executed by gdb.
4
5 Alternatively, if "set gdb on" is entered, the session will be run in a
6 mode where all commands are passed directly to gdb. When running in that
7 mode, native crash commands may be executed by preceding them with the
8 "crash" directive. To restore native crash mode, enter "set gdb off".
15.3. EXAMPLES
1 crash> gdb help
2 List of classes of commands:
3
4 aliases -- Aliases of other commands
5 breakpoints -- Making program stop at certain points
6 data -- Examining data
7 files -- Specifying and examining files
8 internals -- Maintenance commands
9 obscure -- Obscure features
10 running -- Running the program
11 stack -- Examining the stack
12 status -- Status inquiries
13 support -- Support facilities
14 tracepoints -- Tracing of program execution without stopping the program
15 user-defined -- User-defined commands
16
17 Type "help" followed by a class name for a list of commands in that class.
18 Type "help" followed by command name for full documentation.
19 Command name abbreviations are allowed if unambiguous.
16. help - get help
16.1. SYNOPSIS
1help [command | all] [-<option>]
16.2. DESCRIPTION
1 When entered with no argument, a list of all currently available crash
2 commands is listed. If a name of a crash command is entered, a man-like
3 page for the command is displayed. If "all" is entered, help pages
4 for all commands will be displayed. If neither of the above is entered,
5 the argument string will be passed on to the gdb help command.
6
7 A number of internal debug, statistical, and other dumpfile related
8 data is available with the following options:
9
10 -a - alias data
11 -b - shared buffer data
12 -B - build data
13 -c - numargs cache
14 -d - device table
15 -D - dumpfile contents/statistics
16 -e - extension table data
17 -f - filesys table
18 -g - gdb data
19 -h - hash_table data
20 -H - hash_table data (verbose)
21 -k - kernel_table
22 -K - kernel_table (verbose)
23 -L - LKCD page cache environment
24 -M <num> machine specific
25 -m - machdep_table
26 -N - net_table
27 -n - dumpfile contents/statistics
28 -o - offset_table and size_table
29 -p - program_context
30 -r - dump registers from dumpfile header
31 -s - symbol table data
32 -t - task_table
33 -T - task_table plus context_array
34 -v - vm_table
35 -V - vm_table (verbose)
36 -x - text cache
37 -z - help options
17. ipcs - System V IPC facilities
17.1. SYNOPSIS
1ipcs [-smMq] [-n pid|task] [id | addr]
17.2. DESCRIPTION
1 This command provides information on the System V IPC facilities. With no
2 arguments, the command will display kernel usage of all three factilities.
3
4 -s show semaphore arrays.
5 -m show shared memory segments.
6 -M show shared memory segments with additional details.
7 -q show message queues.
8 id show the data associated with this resource ID.
9 addr show the data associated with this virtual address of a
10 shmid_kernel, sem_array or msq_queue.
11
12 For kernels supporting namespaces, the -n option may be used to
13 display the IPC facilities with respect to the namespace of a
14 specified task:
15
16 -n pid a process PID.
17 -n task a hexadecimal task_struct pointer.
17.3. EXAMPLES
1 Display all IPC facilities:
2
3 crash> ipcs
4 SHMID_KERNEL KEY SHMID UID PERMS BYTES NATTCH STATUS
5 ffff880473a28310 00000000 0 0 666 90000 1
6 ffff880473a28490 00000001 32769 0 666 90000 1
7 ffff880473a28250 00000002 65538 0 666 90000 1
8
9 SEM_ARRAY KEY SEMID UID PERMS NSEMS
10 ffff88047200f9d0 00000000 0 0 600 1
11 ffff88046f826910 00000000 32769 0 600 1
12
13 MSG_QUEUE KEY MSQID UID PERMS USED-BYTES MESSAGES
14 ffff8100036bb8d0 000079d7 0 3369 666 16640 104
15 ffff8100036bb3d0 000079d8 32769 3369 666 12960 81
16 ffff810026d751d0 000079d9 65538 3369 666 10880 68
17
18 Display shared memory usage with detailed information:
19
20 crash> ipcs -M
21 SHMID_KERNEL KEY SHMID UID PERMS BYTES NATTCH STATUS
22 ffff880473a28310 00000000 0 0 666 90000 1
23 PAGES ALLOCATED/RESIDENT/SWAPPED: 22/1/0
24 INODE: ffff88047239cd98
25
26 SHMID_KERNEL KEY SHMID UID PERMS BYTES NATTCH STATUS
27 ffff880473a28490 00000001 32769 0 666 90000 1
28 PAGES ALLOCATED/RESIDENT/SWAPPED: 22/1/0
29 INODE: ffff88047239c118
30
31 SHMID_KERNEL KEY SHMID UID PERMS BYTES NATTCH STATUS
32 ffff880473a28250 00000002 65538 0 666 90000 1
33 PAGES ALLOCATED/RESIDENT/SWAPPED: 22/1/0
34 INODE: ffff880470503758
35
36 Display the shared memory data associated with shmid_kernel ffff880473a28250:
37
38 crash> ipcs -M ffff880473a28250
39 SHMID_KERNEL KEY SHMID UID PERMS BYTES NATTCH STATUS
40 ffff880473a28250 00000002 65538 0 666 90000 1
41 PAGES ALLOCATED/RESIDENT/SWAPPED: 22/1/0
42 INODE: ffff880470503758
18. irq - IRQ data
18.1. SYNOPSIS
1irq [[[index ...] | -u ] | -d | -b | -a | -s [-c cpu]]
18.2. DESCRIPTION
1 This command collaborates the data in an irq_desc_t, along with its
2 associated hw_interrupt_type and irqaction structure data, into a
3 consolidated per-IRQ display. For kernel versions 2.6.37 and later
4 the display consists of the irq_desc/irq_data address, its irqaction
5 address(es), and the irqaction name strings. Alternatively, the
6 intel interrupt descriptor table, bottom half data, cpu affinity for
7 in-use irqs, or kernel irq stats may be displayed. If no index value
8 argument(s) nor any options are entered, the IRQ data for all IRQs will
9 be displayed.
10
11 index a valid IRQ index.
12 -u dump data for in-use IRQs only.
13 -d dump the intel interrupt descriptor table.
14 -b dump bottom half data.
15 -a dump cpu affinity for in-use IRQs.
16 -s dump the kernel irq stats; if no cpu specified with -c, the
17 irq stats of all cpus will be displayed.
18 -c cpu only usable with the -s option, dump the irq stats of the
19 specified cpu[s]; cpu can be specified as "1,3,5", "1-3",
20 "1,3,5-7,10", "all", or "a" (shortcut for "all").
18.3. EXAMPLES
1 Display the relevant data for IRQ 18 from a pre-2.6.37 kernel:
2
3 crash> irq 18
4 IRQ: 18
5 STATUS: 0
6 HANDLER: c02301e0 <ioapic_level_irq_type>
7 typename: c01f9e0c "IO-APIC-level"
8 startup: c0110234 <unmask_IO_APIC_irq>
9 shutdown: c01101cc <mask_IO_APIC_irq>
10 handle: c0110518 <do_level_ioapic_IRQ>
11 enable: c0110234 <unmask_IO_APIC_irq>
12 disable: c01101cc <mask_IO_APIC_irq>
13 ACTION: c009c6b0
14 handler: c01ce818 <do_aic7xxx_isr>
15 flags: 4000000 (SA_SHIRQ)
16 mask: 0
17 name: c0217780 "aic7xxx"
18 dev_id: c0090078
19 next: c009c770
20 ACTION: c009c770
21 handler: c01ce818 <do_aic7xxx_isr>
22 flags: 4000000 (SA_SHIRQ)
23 mask: 0
24 name: c0217780 "aic7xxx"
25 dev_id: c0091078
26 next: 0
27 DEPTH: 0
28
29 Display the relevant data for IRQ 21 from a 2.6.37 kernel:
30
31 crash> irq 21
32 IRQ IRQ_DESC/_DATA IRQACTION NAME
33 21 ffff88003787f780 ffff8800379a8b40 "ehci_hcd:usb2"
34 ffff8800379cbac0 "uhci_hcd:usb5"
35 ffff8800379cb140 "uhci_hcd:usb7"
36
37 Display the intel interrupt descriptor table entries:
38
39 crash> irq -d
40 [0] divide_error
41 [1] debug
42 [2] nmi
43 [3] int3
44 [4] overflow
45 [5] bounds
46 [6] invalid_op
47 [7] device_not_available
48 [8] double_fault
49 [9] coprocessor_segment_overrun
50 [10] invalid_TSS
51 [11] segment_not_present
52 [12] stack_segment
53 [13] general_protection
54 [14] page_fault
55 [15] spurious_interrupt_bug
56 [16] coprocessor_error
57 [17] alignment_check
58 [18] ignore_int
59 [19] ignore_int
60 [20] ignore_int
61 [21] ignore_int
62 ...
63
64 [250] IRQ0xda_interrupt
65 [251] IRQ0xdb_interrupt
66 [252] IRQ0xdc_interrupt
67 [253] IRQ0xdd_interrupt
68 [254] IRQ0xde_interrupt
69 [255] spurious_interrupt
70
71 Display the bottom half data:
72
73 crash> irq -b
74 SOFTIRQ_VEC ACTION
75 [0] ffffffff81068f60 <tasklet_hi_action>
76 [1] ffffffff81071b80 <run_timer_softirq>
77 [2] ffffffff813e6f30 <net_tx_action>
78 [3] ffffffff813ee370 <net_rx_action>
79 [4] ffffffff81211a60 <blk_done_softirq>
80 [5] ffffffff812122f0 <blk_iopoll_softirq>
81 [6] ffffffff81069090 <tasklet_action>
82 [7] ffffffff81058830 <run_rebalance_domains>
83 [8] ffffffff81087f00 <run_hrtimer_softirq>
84 [9] ffffffff810ca7a0 <rcu_process_callbacks>
85
86 Display the cpu affinity for in-use IRQs:
87
88 crash> irq -a
89 IRQ NAME AFFINITY
90 0 timer 0-23
91 1 i8042 0-23
92 8 rtc0 0-23
93 9 acpi 0-23
94 16 ehci_hcd:usb2,uhci_hcd:usb3,uhci_hcd:usb6 0,6,18
95 17 uhci_hcd:usb4,uhci_hcd:usb7 0-23
96 18 ehci_hcd:usb1,uhci_hcd:usb5,uhci_hcd:usb8,ioc0 0,11,23
97 24 dmar0 0
98 35 pciehp 0-23
99 36 pciehp 0-23
100 37 pciehp 0-23
101 38 pciehp 0-23
102 39 megasas 0-5,12-17
103 40 lpfc:sp 0-5,12-17
104 41 lpfc:fp 0,6-11,18-23
105 42 lpfc:sp 0,6-11,18-23
106 43 lpfc:fp 0,6-11,18-23
107 ...
108
109 80 ioat-msix 0-23
110 81 ioat-msix 0-23
111 82 ioat-msix 0-23
112 83 ioat-msix 0-23
113 84 ioat-msix 0-23
114 85 ioat-msix 0-23
115 86 ioat-msix 0-23
116 87 ioat-msix 0-23
117 88 eth4 0,17
118
119 Display the kernel irq stats:
120
121 crash>irq -c 0,2 -s
122 CPU0 CPU2
123 0: 2068161471 0 IR-IO-APIC-edge timer
124 1: 9 0 IR-IO-APIC-edge i8042
125 8: 1 0 IR-IO-APIC-edge rtc0
126 9: 0 0 IR-IO-APIC-fasteoi acpi
127 16: 36 0 IR-IO-APIC-fasteoi ehci_hcd:usb2
128 ...
129
130 85: 3 0 IR-PCI-MSI-edge ioat-msix
131 86: 3 0 IR-PCI-MSI-edge ioat-msix
132 87: 3 0 IR-PCI-MSI-edge ioat-msix
133 88: 24 295 IR-PCI-MSI-edge eth4
19. kmem - kernel memory
19.1. SYNOPSIS
1kmem [-f|-F|-c|-C|-i|-v|-V|-n|-z|-o|-h] [-p | -m member[,member]]
2 [[-s|-S|-r] [slab] [-I slab[,slab]]] [-g [flags]] [[-P] address]]
19.2. DESCRIPTION
1 This command displays information about the use of kernel memory.
2
3 -f displays the contents of the system free memory headers.
4 also verifies that the page count equals nr_free_pages.
5 -F same as -f, but also dumps all pages linked to that header.
6 -c walks through the page_hash_table and verifies page_cache_size.
7 -C same as -c, but also dumps all pages in the page_hash_table.
8 -i displays general memory usage information
9 -v displays the mapped virtual memory regions allocated by vmalloc().
10 -V displays the kernel vm_stat table if it exists, or in more recent
11 kernels, the vm_zone_stat, vm_node_stat and vm_numa_stat tables,
12 the cumulative page_states counter values if they exist, and/or
13 the cumulative, vm_event_states counter values if they exist.
14 -n display memory node, memory section, and memory block data
15 and state; the state of each memory section state is encoded
16 as "P", "M", "O" and/or "E", meaning SECTION_MARKED_PRESENT,
17 SECTION_HAS_MEM_MAP, SECTION_IS_ONLINE and SECTION_IS_EARLY.
18 -z displays per-zone memory statistics.
19 -o displays each cpu's offset value that is added to per-cpu symbol
20 values to translate them into kernel virtual addresses.
21 -h display the address of hugepage hstate array entries, along with
22 their hugepage size, total and free counts, and name.
23 -p displays basic information about each page structure in the system
24 mem_map[] array, made up of the page struct address, its associated
25 physical address, the page.mapping, page.index, page._count and
26 page.flags fields.
27 -m member similar to -p, but displays page structure contents specified by
28 a comma-separated list of one or more struct page members. The
29 "flags" member will always be expressed in hexadecimal format, and
30 the "_count" and "_mapcount" members will always be expressed
31 in decimal format. Otherwise, all other members will be displayed
32 in hexadecimal format unless the output radix is 10 and the member
33 is a signed/unsigned integer. Members that are data structures may
34 be specified either by the data structure's member name, or expanded
35 to specify a member of the data structure. For example, "-m lru"
36 refers to a list_head data structure, and both the list_head.next
37 and list_head.prev pointer values will be displayed, whereas if
38 "-m lru.next" is specified, just the list_head.next value will
39 be displayed.
40 -s displays basic kmalloc() slab data.
41 -S displays all kmalloc() slab data, including all slab objects,
42 and whether each object is in use or is free. If CONFIG_SLUB,
43 slab data for each per-cpu slab is displayed, along with the
44 address of each kmem_cache_node, its count of full and partial
45 slabs, and a list of all tracked slabs.
46 -r displays the accumulated basic kmalloc() slab data of each
47 root slab cache and its children. The kernel must contain the
48 "slab_root_caches" list_head. (currently only available if
49 CONFIG_SLUB)
50 slab when used with -s, -S or -r, limits the command to only the slab
51 cache of name "slab". If the slab argument is "list", then
52 all slab cache names and addresses are listed.
53 -I slab when used with -s, -S or -r, one or more slab cache names in a
54 comma-separated list may be specified as slab caches to ignore.
55 -g displays the enumerator value of all bits in the page structure's
56 "flags" field.
57 flags when used with -g, translates all bits in this hexadecimal page
58 structure flags value into its enumerator values.
59 -P declares that the following address argument is a physical address.
60 address when used without any flag, the address can be a kernel virtual,
61 or physical address; a search is made through the symbol table,
62 the kmalloc() slab subsystem, the free list, the page_hash_table,
63 the vmalloc() region subsystem, the current set of task_structs
64 and kernel stacks, and the mem_map array. If found in any of
65 those areas, the information will be dumped in the same manner as
66 if the location-specific flags were used; if contained within a
67 curent task_struct or kernel stack, that task's context will be
68 displayed.
69 address when used with -s or -S, searches the kmalloc() slab subsystem
70 for the slab containing of this virtual address, showing whether
71 it is in use or free.
72 address when used with -f, the address can be either a page pointer,
73 a physical address, or a kernel virtual address; the free_area
74 header containing the page (if any) is displayed.
75 address when used with -p, the address can be either a page pointer, a
76 physical address, or a kernel virtual address; its basic mem_map
77 page information is displayed.
78 address when used with -m, the address can be either a page pointer, a
79 physical address, or a kernel virtual address; the specified
80 members of the associated page struct are displayed.
81 address when used with -c, the address must be a page pointer address;
82 the page_hash_table entry containing the page is displayed.
83 address when used with -l, the address must be a page pointer address;
84 the page address is displayed if it is contained with the list.
85 address when used with -v, the address can be a mapped kernel virtual
86 address or physical address; the mapped region containing the
87 address is displayed.
88
89 All address arguments above must be expressed in hexadecimal format.
19.3. EXAMPLES
1 Display memory usage information:
2
3 crash> kmem -i
4 PAGES TOTAL PERCENTAGE
5 TOTAL MEM 1974231 7.5 GB ----
6 FREE 208962 816.3 MB 10% of TOTAL MEM
7 USED 1765269 6.7 GB 89% of TOTAL MEM
8 SHARED 365066 1.4 GB 18% of TOTAL MEM
9 BUFFERS 111376 435.1 MB 5% of TOTAL MEM
10 CACHED 1276196 4.9 GB 64% of TOTAL MEM
11 SLAB 120410 470.4 MB 6% of TOTAL MEM
12
13 TOTAL HUGE 524288 2 GB ----
14 HUGE FREE 524288 2 GB 100% of TOTAL HUGE
15
16 TOTAL SWAP 2498559 9.5 GB ----
17 SWAP USED 81978 320.2 MB 3% of TOTAL SWAP
18 SWAP FREE 2416581 9.2 GB 96% of TOTAL SWAP
19
20 COMMIT LIMIT 3485674 13.3 GB ----
21 COMMITTED 850651 3.2 GB 24% of TOTAL LIMIT
22
23 Display and verify free memory data:
24
25 crash> kmem -f
26 NODE
27 0
28 ZONE NAME SIZE FREE MEM_MAP START_PADDR START_MAPNR
29 0 DMA 4096 3372 c4000040 0 0
30 AREA SIZE FREE_AREA_STRUCT BLOCKS PAGES
31 0 4k c02eb004 2 2
32 1 8k c02eb010 3 6
33 2 16k c02eb01c 5 20
34 3 32k c02eb028 4 32
35 4 64k c02eb034 5 80
36 5 128k c02eb040 3 96
37 6 256k c02eb04c 3 192
38 7 512k c02eb058 1 128
39 8 1024k c02eb064 1 256
40 9 2048k c02eb070 5 2560
41
42 ZONE NAME SIZE FREE MEM_MAP START_PADDR START_MAPNR
43 1 Normal 225280 202269 c4044040 1000000 4096
44 AREA SIZE FREE_AREA_STRUCT BLOCKS PAGES
45 0 4k c02eb0b8 1 1
46 1 8k c02eb0c4 2 4
47 2 16k c02eb0d0 0 0
48 3 32k c02eb0dc 1 8
49 4 64k c02eb0e8 1 16
50 5 128k c02eb0f4 0 0
51 6 256k c02eb100 0 0
52 7 512k c02eb10c 0 0
53 8 1024k c02eb118 0 0
54 9 2048k c02eb124 395 202240
55
56 ZONE NAME SIZE FREE MEM_MAP START_PADDR START_MAPNR
57 2 HighMem 819200 748686 c4ee0040 38000000 229376
58 AREA SIZE FREE_AREA_STRUCT BLOCKS PAGES
59 0 4k c02eb16c 10 10
60 1 8k c02eb178 2 4
61 2 16k c02eb184 0 0
62 3 32k c02eb190 2 16
63 4 64k c02eb19c 1 16
64 5 128k c02eb1a8 1 32
65 6 256k c02eb1b4 1 64
66 7 512k c02eb1c0 0 0
67 8 1024k c02eb1cc 0 0
68 9 2048k c02eb1d8 1462 748544
69
70 nr_free_pages: 954327 (verified)
71
72 Dump all the base addresses of each free memory area from above:
73
74 crash> kmem -F
75 NODE
76 0
77 ZONE NAME SIZE FREE MEM_MAP START_PADDR START_MAPNR
78 0 DMA 4096 3372 c4000040 0 0
79 AREA SIZE FREE_AREA_STRUCT
80 0 4k c02eb004
81 c400ded8
82 c4042528
83 AREA SIZE FREE_AREA_STRUCT
84 1 8k c02eb010
85 c400de50
86 c400cee8
87 c40424a0
88 AREA SIZE FREE_AREA_STRUCT
89 2 16k c02eb01c
90 c400dd40
91 c400cf70
92 c40425b0
93 c400f7d0
94 c40028a0
95 AREA SIZE FREE_AREA_STRUCT
96 3 32k c02eb028
97 c4042280
98 c400f8e0
99 c4002680
100 c4000260
101 AREA SIZE FREE_AREA_STRUCT
102 4 64k c02eb034
103 c400d080
104 c4041e40
105 ...
106
107 Dump the mem_map[] array:
108
109 crash> kmem -p
110 PAGE PHYSICAL MAPPING INDEX CNT FLAGS
111 f5c51200 10000 0 0 1 80 slab
112 f5c51220 11000 0 0 1 80 slab
113 f5c51240 12000 0 0 1 80 slab
114 f5c51260 13000 0 0 1 80 slab
115 f5c51280 14000 0 0 1 80 slab
116 f5c512a0 15000 0 0 1 80 slab
117 f5c512c0 16000 0 0 1 80 slab
118 f5c512e0 17000 0 0 1 80 slab
119 f5c51300 18000 0 0 1 80 slab
120 f5c51320 19000 0 0 1 80 slab
121 f5c51340 1a000 0 0 1 80 slab
122 f5c51360 1b000 0 0 1 80 slab
123 f5c51380 1c000 e6c6a754 13b67 2 868 uptodate,lru,active,private
124 f5c513a0 1d000 0 0 1 80 slab
125 f5c513c0 1e000 0 0 1 80 slab
126 f5c513e0 1f000 0 0 1 80 slab
127 f5c51400 20000 e6c6a754 13bbb 2 868 uptodate,lru,active,private
128 f5c51420 21000 0 0 1 80 slab
129 f5c51440 22000 0 0 1 80 slab
130 ...
131
132 Display the "page.lru" list_head structure member in each page:
133
134 crash> kmem -m lru
135 PAGE lru
136 ffffea0000000000 0000000000000000,0000000000000000
137 ffffea0000000040 ffffea0000000060,ffffea0000000060
138 ffffea0000000080 ffffea00000000a0,ffffea00000000a0
139 ffffea00000000c0 ffffea00000000e0,ffffea00000000e0
140 ffffea0000000100 ffffea0000000120,ffffea0000000120
141 ffffea0000000140 ffffea0000000160,ffffea0000000160
142 ffffea0000000180 ffffea00000001a0,ffffea00000001a0
143 ffffea00000001c0 ffffea00000001e0,ffffea00000001e0
144 ffffea0000000200 ffffea0000000220,ffffea0000000220
145 ffffea0000000240 ffffea0000000260,ffffea0000000260
146 ffffea0000000280 ffffea00000002a0,ffffea00000002a0
147 ffffea00000002c0 ffffea00000002e0,ffffea00000002e0
148 ffffea0000000300 ffffea0000000320,ffffea0000000320
149 ffffea0000000340 ffffea0000000360,ffffea0000000360
150 ffffea0000000380 ffffea00000003a0,ffffea00000003a0
151 ffffea00000003c0 ffffea00000003e0,ffffea00000003e0
152 ffffea0000000400 ffff88021e5e41e8,ffffea0000002020
153 ffffea0000000440 dead000000100100,dead000000200200
154 ffffea0000000480 dead000000100100,dead000000200200
155 ffffea00000004c0 dead000000100100,dead000000200200
156 ...
157
158 Find the two pages that link to the page at ffffea0001dafb20
159 via their page.lru list_head's next and prev pointers:
160
161 crash> kmem -m lru | grep ffffea0001dafb20
162 ffffea000006b500 ffffea0001dafb20,ffffea0001eb4520
163 ffffea0000127d80 ffffea000152b620,ffffea0001dafb20
164
165 Find all of the combined slab/page structures that are used by
166 the kmalloc-8192 slab cache:
167
168 crash> kmem -s kmalloc-8192
169 CACHE OBJSIZE ALLOCATED TOTAL SLABS SSIZE NAME
170 ffff880215802e00 8192 65 80 20 32k kmalloc-8192
171 crash> kmem -m slab_cache | grep ffff880215802e00
172 ffffea0004117800 ffff880215802e00
173 ffffea00041ca600 ffff880215802e00
174 ffffea00044ab200 ffff880215802e00
175 ffffea0004524000 ffff880215802e00
176 ffffea0004591600 ffff880215802e00
177 ffffea00047eac00 ffff880215802e00
178 ffffea0004875800 ffff880215802e00
179 ffffea0008357a00 ffff880215802e00
180 ffffea0008362a00 ffff880215802e00
181 ffffea00083b9400 ffff880215802e00
182 ffffea00083c1000 ffff880215802e00
183 ffffea00083c1e00 ffff880215802e00
184 ffffea00083c2000 ffff880215802e00
185 ffffea00083c2a00 ffff880215802e00
186 ffffea00083d2000 ffff880215802e00
187 ffffea00083d3e00 ffff880215802e00
188 ffffea0008407c00 ffff880215802e00
189 ffffea000848ce00 ffff880215802e00
190 ffffea0008491800 ffff880215802e00
191 ffffea00084bf800 ffff880215802e00
192
193 Use the commands above with a page pointer or a physical address argument:
194
195 crash> kmem -f c40425b0
196 NODE
197 0
198 ZONE NAME SIZE FREE MEM_MAP START_PADDR START_MAPNR
199 0 DMA 4096 3372 c4000040 0 0
200 AREA SIZE FREE_AREA_STRUCT
201 2 16k c02eb01c
202 c40425b0 (c40425b0 is 1st of 4 pages)
203
204 crash> kmem -p c25a9c00
205 PAGE PHYSICAL MAPPING INDEX CNT FLAGS
206 c25a9c00 1fe0000 f429d2e4 21fe3eb 2 800828 uptodate,lru,private
207
208 crash> kmem -p 1fe0000
209 PAGE PHYSICAL MAPPING INDEX CNT FLAGS
210 c25a9c00 1fe0000 f429d2e4 21fe3eb 2 800828 uptodate,lru,private
211
212 Display the mapped memory regions allocated by vmalloc():
213
214 crash> kmem -v
215 VMAP_AREA VM_STRUCT ADDRESS RANGE SIZE
216 f7048e00 f7048e40 f7dfe000 - f7e00000 8192
217 f7048ec0 f7048f00 f7e00000 - f7e05000 20480
218 f7151fc0 f7159540 f7e06000 - f7e08000 8192
219 f704da80 f704dac0 f7e0a000 - f7e0c000 8192
220 f704d980 f704d9c0 f7e0e000 - f7e10000 8192
221 f724f1c0 f724f200 f7e12000 - f7e14000 8192
222 f704d840 f704d880 f7e14000 - f7e17000 12288
223 f704d400 f704d440 f7e18000 - f7e1d000 20480
224 f73f5840 f73f5880 f7e1e000 - f7e2a000 49152
225 f6334480 f63344c0 f7e2c000 - f7e2e000 8192
226 f635d600 f635d640 f7e4a000 - f7e5b000 69632
227 f41b4700 f5771a40 f7e6e000 - f7e70000 8192
228 f622f6c0 f622f700 f7e71000 - f7e79000 32768
229 f63a9f00 f63a9f40 f7e84000 - f7e87000 12288
230 f63a9d00 f63a9d40 f7e8f000 - f7e91000 8192
231 f5546480 f39db800 f7eb8000 - f7ec2000 40960
232 f5ce9640 f5777e80 f7ec6000 - f7ed1000 45056
233 f63a9b00 f63a9b40 f7ed1000 - f7efd000 180224
234 f63a9800 f63a9840 f7f1d000 - f7f26000 36864
235 f63a9640 f63a9880 f7f43000 - f7f52000 61440
236 f5771f00 f4183840 f7f53000 - f7f64000 69632
237 f5ce9a00 f30c4a00 f7fcf000 - f801e000 323584
238 f63a93c0 f63a9400 f805d000 - f8132000 872448
239 f63a91c0 f63a95c0 f814b000 - f8150000 20480
240 f63a9140 f63a9180 f8151000 - f8352000 2101248
241 f624eb00 f624eb40 f8353000 - f8355000 8192
242 f563eb40 f563eb80 f8356000 - f835e000 32768
243 f63d5ec0 f63d5f00 f8360000 - f8371000 69632
244 f63d5cc0 f6287b80 f83c2000 - f84c3000 1052672
245 ...
246
247 Dump the virtual memory statistics:
248
249 crash> kmem -V
250 VM_ZONE_STAT:
251 NR_FREE_PAGES: 30085
252 NR_ZONE_INACTIVE_ANON: 1985
253 NR_ZONE_ACTIVE_ANON: 338275
254 NR_ZONE_INACTIVE_FILE: 19760
255 NR_ZONE_ACTIVE_FILE: 12018
256 NR_ZONE_UNEVICTABLE: 0
257 NR_ZONE_WRITE_PENDING: 4
258 NR_MLOCK: 0
259 NR_PAGETABLE: 1562
260 NR_KERNEL_STACK_KB: 1728
261 NR_BOUNCE: 0
262 NR_FREE_CMA_PAGES: 0
263
264 VM_NODE_STAT:
265 NR_INACTIVE_ANON: 1985
266 NR_ACTIVE_ANON: 338275
267 NR_INACTIVE_FILE: 19760
268 NR_ACTIVE_FILE: 12018
269 NR_UNEVICTABLE: 0
270 NR_SLAB_RECLAIMABLE: 3111
271 NR_SLAB_UNRECLAIMABLE: 3039
272 NR_ISOLATED_ANON: 0
273 NR_ISOLATED_FILE: 0
274 WORKINGSET_REFAULT: 0
275 WORKINGSET_ACTIVATE: 0
276 WORKINGSET_NODERECLAIM: 0
277 NR_ANON_MAPPED: 338089
278 NR_FILE_MAPPED: 8102
279 NR_FILE_PAGES: 33949
280 NR_FILE_DIRTY: 4
281 NR_WRITEBACK: 0
282 NR_WRITEBACK_TEMP: 0
283 NR_SHMEM: 2171
284 NR_SHMEM_THPS: 0
285 NR_SHMEM_PMDMAPPED: 0
286 NR_ANON_THPS: 86
287 NR_UNSTABLE_NFS: 0
288 NR_VMSCAN_WRITE: 0
289 NR_VMSCAN_IMMEDIATE: 0
290 NR_DIRTIED: 155
291 NR_WRITTEN: 75
292
293 VM_NUMA_STAT:
294 NUMA_HIT: 575409
295 NUMA_MISS: 0
296 NUMA_FOREIGN: 0
297 NUMA_INTERLEAVE_HIT: 12930
298 NUMA_LOCAL: 575409
299 NUMA_OTHER: 0
300
301 VM_EVENT_STATES:
302 PGPGIN: 282492
303 PGPGOUT: 6773
304 PSWPIN: 0
305 PSWPOUT: 0
306 PGALLOC_DMA: 0
307 PGALLOC_DMA32: 693092
308 PGALLOC_NORMAL: 0
309 ...
310
311 Display hugepage hstate information:
312
313 crash> kmem -h
314 HSTATE SIZE FREE TOTAL NAME
315 ffffffff81f7a800 2MB 10 64 hugepages-2048kB
316
317 Determine (and verify) the page cache size:
318
319 crash> kmem -c
320 page_cache_size: 18431 (verified)
321
322 Dump all pages in the page_hash_table:
323
324 crash> kmem -C
325 page_hash_table[0]
326 c0325b40
327 c03a0598
328 c03b4070
329 c0364c28
330 c0357690
331 c02ef338
332 c02d7c60
333 c02c11e0
334 c02a3d70
335 page_hash_table[1]
336 c0394ce8
337 c03c4218
338 c03b4048
339 c0364c00
340 c0357668
341 c02d6e50
342 c02d7dc8
343 c02c0cb8
344 c02db630
345 c02ebad0
346 page_hash_table[2]
347 c037e808
348 c034e248
349 c03b4020
350 c02ec868
351 c03baa60
352 ...
353 page_hash_table[2047]
354 c033a798
355 c0390b48
356 c03b4098
357 c0364890
358 c03576b8
359 c02d2c38
360 c02d7c88
361 c02de5d8
362
363 page_cache_size: 18437 (verified)
364
365 Find the page_hash_table entry containing page c03576b8:
366
367 crash> kmem -c c03576b8
368 page_hash_table[2047]
369 c03576b8
370
371 Display kmalloc() slab data:
372
373 crash> kmem -s
374 CACHE OBJSIZE ALLOCATED TOTAL SLABS SSIZE NAME
375 c02eadc0 232 58 68 4 4k kmem_cache
376 f79c2888 128 0 0 0 4k ip_vs_conn
377 f79c2970 96 0 0 0 4k tcp_tw_bucket
378 f79c2a58 32 12 565 5 4k tcp_bind_bucket
379 f79c2b40 64 0 59 1 4k tcp_open_request
380 f79c2c28 64 1 59 1 4k inet_peer_cache
381 f79c2d10 32 11 339 3 4k ip_fib_hash
382 f79c2df8 160 8 120 5 4k ip_dst_cache
383 f79c2ee0 128 1 30 1 4k arp_cache
384 c8402970 96 30208 37800 945 4k blkdev_requests
385 c8402a58 384 0 0 0 4k nfs_read_data
386 c8402b40 384 0 0 0 4k nfs_write_data
387 c8402c28 96 0 0 0 4k nfs_page
388 c8402d10 20 0 0 0 4k dnotify cache
389 c8402df8 92 3 336 8 4k file lock cache
390 c8402ee0 16 0 0 0 4k fasync cache
391 c84027a0 32 3 339 3 4k uid_cache
392 c84026b8 160 320 624 26 4k skbuff_head_cache
393 c84025d0 832 32 180 20 8k sock
394 c84024e8 132 0 203 7 4k sigqueue
395 c8402400 64 19 472 8 4k cdev_cache
396 c8402318 64 8 236 4 4k bdev_cache
397 c8402230 96 11 120 3 4k mnt_cache
398 c8402148 480 817 848 106 4k inode_cache
399 c8402060 128 1352 1470 49 4k dentry_cache
400 c8403ee0 96 244 440 11 4k filp
401 c8403df8 4096 0 12 12 4k names_cache
402 c8403d10 96 14936 16000 400 4k buffer_head
403 c8403c28 128 25 240 8 4k mm_struct
404 c8403b40 64 393 1298 22 4k vm_area_struct
405 c8403a58 64 30 472 8 4k fs_cache
406 c8403970 416 30 135 15 4k files_cache
407 c8403888 1312 32 99 33 4k signal_act
408 c84037a0 131072 0 0 0 128k size-131072(DMA)
409 c84036b8 131072 1 1 1 128k size-131072
410 c84035d0 65536 0 0 0 64k size-65536(DMA)
411 c84034e8 65536 0 0 0 64k size-65536
412 c8403400 32768 0 0 0 32k size-32768(DMA)
413 c8403318 32768 0 1 1 32k size-32768
414 c8403230 16384 0 0 0 16k size-16384(DMA)
415 c8403148 16384 0 0 0 16k size-16384
416 c8403060 8192 0 0 0 8k size-8192(DMA)
417 c8401ee0 8192 1 2 2 8k size-8192
418 c8401df8 4096 0 0 0 4k size-4096(DMA)
419 c8401d10 4096 30 30 30 4k size-4096
420 c8401c28 2048 0 0 0 4k size-2048(DMA)
421 c8401b40 2048 37 132 66 4k size-2048
422 c8401a58 1024 0 0 0 4k size-1024(DMA)
423 c8401970 1024 301 328 82 4k size-1024
424 c8401888 512 0 0 0 4k size-512(DMA)
425 c84017a0 512 141 168 21 4k size-512
426 c84016b8 256 0 0 0 4k size-256(DMA)
427 c84015d0 256 80 435 29 4k size-256
428 c84014e8 128 0 0 0 4k size-128(DMA)
429 c8401400 128 508 840 28 4k size-128
430 c8401318 64 0 0 0 4k size-64(DMA)
431 c8401230 64 978 1357 23 4k size-64
432 c8401148 32 0 0 0 4k size-32(DMA)
433 c8401060 32 1244 1808 16 4k size-32
434
435 Display all slab data in the "arp_cache" cache:
436
437 crash> kmem -S arp_cache
438 CACHE OBJSIZE ALLOCATED TOTAL SLABS SSIZE NAME
439 f79c2ee0 128 1 30 1 4k arp_cache
440 SLAB MEMORY TOTAL ALLOCATED FREE
441 f729d000 f729d0a0 30 1 29
442 FREE / [ALLOCATED]
443 f729d0a0 (cpu 7 cache)
444 f729d120 (cpu 7 cache)
445 f729d1a0 (cpu 7 cache)
446 f729d220 (cpu 7 cache)
447 f729d2a0 (cpu 7 cache)
448 f729d320 (cpu 7 cache)
449 f729d3a0 (cpu 7 cache)
450 f729d420 (cpu 7 cache)
451 f729d4a0 (cpu 7 cache)
452 f729d520 (cpu 7 cache)
453 f729d5a0 (cpu 7 cache)
454 f729d620 (cpu 7 cache)
455 f729d6a0 (cpu 7 cache)
456 f729d720 (cpu 7 cache)
457 f729d7a0 (cpu 7 cache)
458 f729d820 (cpu 7 cache)
459 f729d8a0 (cpu 7 cache)
460 f729d920 (cpu 7 cache)
461 f729d9a0 (cpu 7 cache)
462 f729da20 (cpu 7 cache)
463 f729daa0 (cpu 7 cache)
464 f729db20 (cpu 7 cache)
465 f729dba0 (cpu 7 cache)
466 f729dc20 (cpu 7 cache)
467 f729dca0 (cpu 7 cache)
468 f729dd20 (cpu 7 cache)
469 f729dda0 (cpu 7 cache)
470 f729de20 (cpu 7 cache)
471 f729dea0 (cpu 3 cache)
472 [f729df20]
473
474 Search the kmalloc() slab subsystem for address c3fbdb60:
475
476 crash> kmem -s c3fbdb60
477 CACHE OBJSIZE ALLOCATED TOTAL SLABS SSIZE NAME
478 c8402970 96 30208 37800 945 4k blkdev_requests
479 SLAB MEMORY TOTAL ALLOCATED FREE
480 c3fbd020 c3fbd0e0 40 40 0
481 FREE / [ALLOCATED]
482 [c3fbdb60]
483
484 Make a generic search (no flags) for the same address c3fbdb60:
485
486 crash> kmem c3fbdb60
487 CACHE OBJSIZE ALLOCATED TOTAL SLABS SSIZE NAME
488 c8402970 96 30208 37800 945 4k blkdev_requests
489 SLAB MEMORY TOTAL ALLOCATED FREE
490 c3fbd020 c3fbd0e0 40 40 0
491 FREE / [ALLOCATED]
492 [c3fbdb60]
493
494 PAGE PHYSICAL MAPPING INDEX CNT FLAGS
495 c410ee74 3fbd000 0 0 1 slab
496
497 Display memory node data (if supported):
498
499 crash> kmem -n
500 NODE SIZE PGLIST_DATA BOOTMEM_DATA NODE_ZONES
501 0 262095 ffff88003d52a000 ---- ffff88003d52a000
502 ffff88003d52a740
503 ffff88003d52ae80
504 ffff88003d52b5c0
505 MEM_MAP START_PADDR START_MAPNR
506 ffffea0000000040 1000 1
507
508 ZONE NAME SIZE MEM_MAP START_PADDR START_MAPNR
509 0 DMA 4095 ffffea0000000040 1000 1
510 1 DMA32 258000 ffffea0000040000 1000000 4096
511 2 Normal 0 0 0 0
512 3 Movable 0 0 0 0
513
514 -------------------------------------------------------------------
515
516 NR SECTION CODED_MEM_MAP MEM_MAP STATE PFN
517 0 ffff88003d4d9000 ffffea0000000000 ffffea0000000000 PM 0
518 1 ffff88003d4d9020 ffffea0000000000 ffffea0000200000 PM 32768
519 2 ffff88003d4d9040 ffffea0000000000 ffffea0000400000 PM 65536
520 3 ffff88003d4d9060 ffffea0000000000 ffffea0000600000 PM 98304
521 4 ffff88003d4d9080 ffffea0000000000 ffffea0000800000 PM 131072
522 5 ffff88003d4d90a0 ffffea0000000000 ffffea0000a00000 PM 163840
523 6 ffff88003d4d90c0 ffffea0000000000 ffffea0000c00000 PM 196608
524 7 ffff88003d4d90e0 ffffea0000000000 ffffea0000e00000 PM 229376
525
526 MEM_BLOCK NAME PHYSICAL RANGE STATE START_SECTION_NO
527 ffff88003a707c00 memory0 0 - 7ffffff ONLINE 0
528 ffff88003a6e0000 memory1 8000000 - fffffff ONLINE 1
529 ffff88003a6e1000 memory2 10000000 - 17ffffff ONLINE 2
530 ffff88003a6e1400 memory3 18000000 - 1fffffff ONLINE 3
531 ffff88003a6e1800 memory4 20000000 - 27ffffff ONLINE 4
532 ffff88003a6e0400 memory5 28000000 - 2fffffff ONLINE 5
533 ffff88003a6e0800 memory6 30000000 - 37ffffff ONLINE 6
534 ffff88003a6e0c00 memory7 38000000 - 3fffffff ONLINE 7
535
536 Translate a page structure's flags field contents:
537
538 crash> kmem -g 4080
539 FLAGS: 4080
540 PAGE-FLAG BIT VALUE
541 PG_slab 7 0000080
542 PG_head 14 0004000
543 crash>
20. list - linked list
20.1. SYNOPSIS
1list [[-o] offset][-e end][-[s|S] struct[.member[,member] [-l offset]] -[x|d]]
2 [-r|-B] [-h|-H] start
20.2. DESCRIPTION
1
2 This command dumps the contents of a linked list. The entries in a linked
3 list are typically data structures that are tied together in one of two
4 formats:
5
6 1. A starting address points to a data structure; that structure contains
7 a member that is a pointer to the next structure, and so on. This type
8 of a singly-linked list typically ends when a "next" pointer value
9 contains one of the following:
10
11 (a) a NULL pointer.
12 (b) a pointer to the start address.
13 (c) a pointer to the first item pointed to by the start address.
14 (d) a pointer to its containing structure.
15
16 2. Most Linux lists of data structures are doubly-linked using "list_head"
17 structures that are embedded members of the data structures in the list:
18
19 struct list_head {
20 struct list_head *next, *prev;
21 };
22
23 The linked list is typically headed by an external, standalone list_head,
24 which is simply initialized to point to itself, signifying that the list
25 is empty:
26
27 #define LIST_HEAD_INIT(name) { &(name), &(name) }
28 #define LIST_HEAD(name) struct list_head name = LIST_HEAD_INIT(name)
29
30 In the case of list_head-linked lists, the "list_head.next" pointer is
31 the address of a list_head structure that is embedded in the next data
32 structure in the list, and not the address of the next data structure
33 itself. The starting point of the list may be:
34
35 (a) an external, standalone, LIST_HEAD().
36 (b) a list_head that is embedded within a data structure of the same
37 type as the whole linked list.
38 (c) a list_head that is embedded within a data structure that is
39 different than the type of structures in the the linked list.
40
41 The list typically ends when the embedded "list_head.next" pointer of
42 a data structure in the linked list points back to the LIST_HEAD()
43 address. However, some list_head-linked lists have no defined starting
44 point, but just link back onto themselves in a circular manner.
45
46 This command can handle both types of linked list; in both cases the list
47 of addresses that are dumped are the addresses of the data structures
48 themselves.
49
50 Alternatively, the address of a list_head, or other similar list linkage
51 structure whose first member points to the next linkage structure, may be
52 used as the starting address. The caveat with this type of usage is that
53 the list may pass through, and display the address of, an external standalone
54 list head which is not an address of a list linkage structure that is embedded
55 within the data structure of interest.
56
57 The arguments are as follows:
58
59 [-o] offset The offset within the structure to the "next" pointer
60 (default is 0). If non-zero, the offset may be entered
61 in either of two manners:
62
63 1. In "structure.member" format; the "-o" is not necessary.
64 2. A number of bytes; the "-o" is only necessary on processors
65 where the offset value could be misconstrued as a kernel
66 virtual address.
67
68 -e end If the list ends in a manner unlike the typical manners that
69 are described above, an explicit ending address value may be
70 entered.
71 -s struct For each address in list, format and print as this type of
72 structure; use the "struct.member" format in order to display
73 a particular member of the structure. To display multiple
74 members of a structure, use a comma-separated list of members.
75 If any structure member contains an embedded structure or is an
76 array, the output may be restricted to the embedded structure
77 or an array element by expressing the struct argument as
78 "struct.member.member" or "struct.member[index]"; embedded
79 member specifications may extend beyond one level deep by
80 expressing the argument as "struct.member.member.member...".
81 -S struct Similar to -s, but instead of parsing gdb output, member values
82 are read directly from memory, so the command works much faster
83 for 1-, 2-, 4-, and 8-byte members.
84 -l offset Only used in conjunction with -s, if the start address argument
85 is a pointer to an embedded list head (or any other similar list
86 linkage structure whose first member points to the next linkage
87 structure), the offset to the embedded member may be entered
88 in either of the following manners:
89
90 1. in "structure.member" format.
91 2. a number of bytes.
92
93 -x Override the default output format with hexadecimal format.
94 -d Override the default output format with decimal format.
95 -r For a list linked with list_head structures, traverse the list
96 in the reverse order by using the "prev" pointer instead
97 of "next".
98 -B Use the algorithm from R. P. Brent to detect loops instead of
99 using a hash table. This algorithm uses a tiny fixed amount of
100 memory and so is especially helpful for longer lists. The output
101 is slightly different than the normal list output as it will
102 print the length of the loop, the start of the loop, and the
103 first duplicate in the list.
104
105 The meaning of the "start" argument, which can be expressed symbolically,
106 in hexadecimal format, or an expression evaluating to an address, depends
107 upon whether the -h or -H option is pre-pended:
108
109 start The address of the first data structure in the list.
110 start When both the -s and -l options are used, the address of an
111 embedded list_head or similar linkage structure whose first
112 member points to the next linkage structure.
113 -H start The address of a list_head structure, typically that of an
114 external, standalone LIST_HEAD(). The list typically ends
115 when the embedded "list_head.next" of a data structure in
116 the linked list points back to this "start" address.
117 -h start The address of a data structure which contains an embedded
118 list_head. The list typically ends when the embedded
119 "list_head.next" of a data structure in the linked list
120 points back to the embedded list_head contained in the data
121 structure whose address is this "start" argument.
122
123WARNING
124 When the "-h start" option is used, it is possible that the list_head-linked
125 list will:
126
127 1. pass through an external standalone LIST_HEAD(), or
128 2. pass through a list_head that is the actual starting list_head, but is
129 contained within a data structure that is not the same type as all of
130 the other data structures in the list.
131
132 When that occurs, the data structure address displayed for that list_head
133 will be incorrect, because the "-h start" option presumes that all
134 list_head structures in the list are contained within the same type of
135 data structure. Furthermore, if the "-s struct[.member[,member]" option
136 is used, it will display bogus data for that particular list_head.
137
138 A similar issue may be encountered when the "start" address is an embedded
139 list_head or similar linkage structure whose first member points to the next
140 linkage structure. When that occurs, the address of any external list head
141 will not be distinguishable from the addresses that are embedded in the data
142 structure of interest. Furthermore, if the "-s" and "-l" options are used,
143 it will display bogus structure data when passing through any external list
144 head structure that is not embedded in the specified data structure type.
20.3. EXAMPLES
1 Note that each task_struct is linked to its parent's task_struct via the
2 p_pptr member:
3
4 crash> struct task_struct.p_pptr
5 struct task_struct {
6 [136] struct task_struct *p_pptr;
7 }
8
9 That being the case, given a task_struct pointer of c169a000, show its
10 parental hierarchy back to the "init_task" (the "swapper" task):
11
12 crash> list task_struct.p_pptr c169a000
13 c169a000
14 c0440000
15 c50d0000
16 c0562000
17 c0d28000
18 c7894000
19 c6a98000
20 c009a000
21 c0252000
22
23 Given that the "task_struct.p_pptr" offset is 136 bytes, the same
24 result could be accomplished like so:
25
26 crash> list 136 c169a000
27 c169a000
28 c0440000
29 c50d0000
30 c0562000
31 c0d28000
32 c7894000
33 c6a98000
34 c009a000
35 c0252000
36
37 The list of currently-registered file system types are headed up by a
38 struct file_system_type pointer named "file_systems", and linked by
39 the "next" field in each file_system_type structure. The following
40 sequence displays the structure address followed by the name and
41 fs_flags members of each registered file system type:
42
43 crash> p file_systems
44 file_systems = $1 = (struct file_system_type *) 0xc03adc90
45 crash> list file_system_type.next -s file_system_type.name,fs_flags c03adc90
46 c03adc90
47 name = 0xc02c05c8 "rootfs",
48 fs_flags = 0x30,
49 c03abf94
50 name = 0xc02c0319 "bdev",
51 fs_flags = 0x10,
52 c03acb40
53 name = 0xc02c07c4 "proc",
54 fs_flags = 0x8,
55 c03e9834
56 name = 0xc02cfc83 "sockfs",
57 fs_flags = 0x10,
58 c03ab8e4
59 name = 0xc02bf512 "tmpfs",
60 fs_flags = 0x20,
61 c03ab8c8
62 name = 0xc02c3d6b "shm",
63 fs_flags = 0x20,
64 c03ac394
65 name = 0xc02c03cf "pipefs",
66 fs_flags = 0x10,
67 c03ada74
68 name = 0xc02c0e6b "ext2",
69 fs_flags = 0x1,
70 c03adc74
71 name = 0xc02c0e70 "ramfs",
72 fs_flags = 0x20,
73 c03ade74
74 name = 0xc02c0e76 "hugetlbfs",
75 fs_flags = 0x20,
76 c03adf8c
77 name = 0xc02c0f84 "iso9660",
78 fs_flags = 0x1,
79 c03aec14
80 name = 0xc02c0ffd "devpts",
81 fs_flags = 0x8,
82 c03e93f4
83 name = 0xc02cf1b9 "pcihpfs",
84 fs_flags = 0x28,
85 e0831a14
86 name = 0xe082f89f "ext3",
87 fs_flags = 0x1,
88 e0846af4
89 name = 0xe0841ac6 "usbdevfs",
90 fs_flags = 0x8,
91 e0846b10
92 name = 0xe0841acf "usbfs",
93 fs_flags = 0x8,
94 e0992370
95 name = 0xe099176c "autofs",
96 fs_flags = 0x0,
97 e2dcc030
98 name = 0xe2dc8849 "nfs",
99 fs_flags = 0x48000,
100
101 In some kernels, the system run queue is a linked list headed up by the
102 "runqueue_head", which is defined like so:
103
104 static LIST_HEAD(runqueue_head);
105
106 The run queue linking is done with the "run_list" member of the task_struct:
107
108 crash> struct task_struct.run_list
109 struct task_struct {
110 [60] struct list_head run_list;
111 }
112
113 Therefore, to view the list of task_struct addresses in the run queue,
114 either of the following commands will work:
115
116 crash> list task_struct.run_list -H runqueue_head
117 f79ac000
118 f7254000
119 f7004000
120 crash> list 60 -H runqueue_head
121 f79ac000
122 f7254000
123 f7004000
124
125 In some kernel versions, the vfsmount structures of the mounted
126 filesystems are linked by the LIST_HEAD "vfsmntlist", which uses the
127 mnt_list list_head of each vfsmount structure in the list. To dump each
128 vfsmount structure in the list, append the -s option:
129
130 crash> list -H vfsmntlist vfsmount.mnt_list -s vfsmount
131 c3fc9e60
132 struct vfsmount {
133 mnt_hash = {
134 next = 0xc3fc9e60,
135 prev = 0xc3fc9e60
136 },
137 mnt_parent = 0xc3fc9e60,
138 mnt_mountpoint = 0xc3fc5dc0,
139 mnt_root = 0xc3fc5dc0,
140 mnt_instances = {
141 next = 0xc3f60a74,
142 prev = 0xc3f60a74
143 },
144 mnt_sb = 0xc3f60a00,
145 mnt_mounts = {
146 next = 0xf7445e08,
147 prev = 0xf7445f88
148 },
149 mnt_child = {
150 next = 0xc3fc9e88,
151 prev = 0xc3fc9e88
152 },
153 mnt_count = {
154 counter = 209
155 },
156 mnt_flags = 0,
157 mnt_devname = 0xc8465b20 "/dev/root",
158 mnt_list = {
159 next = 0xf7445f9c,
160 prev = 0xc02eb828
161 },
162 mnt_owner = 0
163 }
164 f7445f60
165 struct vfsmount {
166 ...
167
168 The task_struct of every task in the system is linked into a circular list
169 by its embedded "tasks" list_head. Show the task_struct addresses and the
170 pids of all tasks in the system using "-h" option, starting with the
171 task_struct at ffff88012b98e040:
172
173 crash> list task_struct.tasks -s task_struct.pid -h ffff88012b98e040
174 ffff88012b98e040
175 pid = 14187
176 ffff8801277be0c0
177 pid = 14248
178 ffffffff81a2d020
179 pid = 0
180 ffff88012d7dd4c0
181 pid = 1
182 ffff88012d7dca80
183 pid = 2
184 ffff88012d7dc040
185 pid = 3
186 ffff88012d7e9500
187 pid = 4
188 ...
189 ffff88012961a100
190 pid = 14101
191 ffff880129017580
192 pid = 14134
193 ffff8801269ed540
194 pid = 14135
195 ffff880128256080
196 pid = 14138
197 ffff88012b8f4100
198 pid = 14183
199
200 Similar to the above, display the embedded sched_entity structure's on_rq
201 member from each task_struct in the system:
202
203 crash> list task_struct.tasks -s task_struct.se.on_rq -h ffff8800b66a0000
204 ffff8800b66a0000
205 se.on_rq = 1,
206 ffff8800b66a0ad0
207 se.on_rq = 0,
208 ffff8800b66a15a0
209 se.on_rq = 0,
210 ffff8800b66a2070
211 se.on_rq = 0,
212 ffff8800b66a2b40
213 se.on_rq = 0,
214 ffff8800b67315a0
215 se.on_rq = 0,
216 ffff8800b6732b40
217 se.on_rq = 0,
218 ...
219
220 The task_struct.tasks example above requires that the -h option be given
221 the address of a task_struct. Alternatively, the -l option can be given
222 the address of a list_head or similar linkage structure whose first member
223 points to the next linkage structure. Again using the task_struct.tasks
224 embedded list_head, dump the "comm" member of all tasks by using -l in
225 conjunction with -s option:
226
227 crash> task -R tasks.next
228 PID: 7044 TASK: ffff88005ac10000 CPU: 2 COMMAND: "crash"
229 tasks.next = 0xffff880109b8e3d0,
230 crash> list 0xffff880109b8e3d0 -l task_struct.tasks -s task_struct.comm
231 ffff880109b8e3d0
232 comm = "kworker/1:2"
233 ffff880109b8be00
234 comm = "bash"
235 ffff88019d26c590
236 comm = "cscope"
237 ffff880109b8b670
238 comm = "kworker/0:1"
239 ffff880109b8cd20
240 comm = "kworker/1:0"
241 ffff88005ac15c40
242 comm = "vi"
243 ffff88005ac11fc0
244 comm = "sleep"
245 ffffffff81c135c0
246 comm = "swapper/0"
247 ffff880212828180
248 comm = "systemd"
249 ...
250 ffff8801288d1830
251 comm = "chrome"
252 ffff8801534dd4b0
253 comm = "kworker/0:0"
254 ffff8801534d8180
255 comm = "kworker/1:1"
256 ffff88010902b670
257 comm = "kworker/2:2"
258 ffff880109b8a750
259 comm = "sudo"
260 ffff88005ac10180
261 comm = "crash"
21. log - dump system message buffer
21.1. SYNOPSIS
1log [-tdma]
21.2. DESCRIPTION
1 This command dumps the kernel log_buf contents in chronological order. The
2 command supports the older log_buf formats, which may or may not contain a
3 timestamp inserted prior to each message, as well as the newer variable-length
4 record format, where the timestamp is contained in each log entry's header.
5
6 -t Display the message text without the timestamp; only applicable to the
7 variable-length record format.
8 -d Display the dictionary of key/value pair properties that are optionally
9 appended to a message by the kernel's dev_printk() function; only
10 applicable to the variable-length record format.
11 -m Display the message log level in brackets preceding each message. For
12 the variable-length record format, the level will be displayed in
13 hexadecimal, and depending upon the kernel version, also contains the
14 facility or flags bits.
15 -a Dump the audit logs remaining in kernel audit buffers that have not
16 been copied out to the user-space audit daemon.
21.3. EXAMPLES
1 Dump the kernel message buffer:
2
3 crash> log
4 Linux version 2.2.5-15smp (root@mclinux1) (gcc version egcs-2.91.66 19990
5 314/Linux (egcs-1.1.2 release)) #1 SMP Thu Aug 26 11:04:37 EDT 1999
6 Intel MultiProcessor Specification v1.4
7 Virtual Wire compatibility mode.
8 OEM ID: DELL Product ID: WS 410 APIC at: 0xFEE00000
9 Processor #0 Pentium(tm) Pro APIC version 17
10 Processor #1 Pentium(tm) Pro APIC version 17
11 I/O APIC #2 Version 17 at 0xFEC00000.
12 Processors: 2
13 mapped APIC to ffffe000 (fee00000)
14 mapped IOAPIC to ffffd000 (fec00000)
15 Detected 447696347 Hz processor.
16 Console: colour VGA+ 80x25
17 Calibrating delay loop... 445.64 BogoMIPS
18 ...
19 8K byte-wide RAM 5:3 Rx:Tx split, autoselect/Autonegotiate interface.
20 MII transceiver found at address 24, status 782d.
21 Enabling bus-master transmits and whole-frame receives.
22 Installing knfsd (copyright (C) 1996 okir@monad.swb.de).
23 nfsd_init: initialized fhcache, entries=256
24 ...
25
26 Do the same thing, but also show the log level preceding each message:
27
28 crash> log -m
29 <4>Linux version 2.2.5-15smp (root@mclinux1) (gcc version egcs-2.91.66 19990
30 314/Linux (egcs-1.1.2 release)) #1 SMP Thu Aug 26 11:04:37 EDT 1999
31 <4>Intel MultiProcessor Specification v1.4
32 <4> Virtual Wire compatibility mode.
33 <4>OEM ID: DELL Product ID: WS 410 APIC at: 0xFEE00000
34 <4>Processor #0 Pentium(tm) Pro APIC version 17
35 <4>Processor #1 Pentium(tm) Pro APIC version 17
36 <4>I/O APIC #2 Version 17 at 0xFEC00000.
37 <4>Processors: 2
38 <4>mapped APIC to ffffe000 (fee00000)
39 <4>mapped IOAPIC to ffffd000 (fec00000)
40 <4>Detected 447696347 Hz processor.
41 <4>Console: colour VGA+ 80x25
42 <4>Calibrating delay loop... 445.64 BogoMIPS
43 ...
44 <6> 8K byte-wide RAM 5:3 Rx:Tx split, autoselect/Autonegotiate interface.
45 <6> MII transceiver found at address 24, status 782d.
46 <6> Enabling bus-master transmits and whole-frame receives.
47 <6>Installing knfsd (copyright (C) 1996 okir@monad.swb.de).
48 <7>nfsd_init: initialized fhcache, entries=256
49 ...
50
51 On a system with the variable-length record format, and whose log_buf has been
52 filled and wrapped around, display the log with timestamp data:
53
54 crash> log
55 [ 0.467730] pci 0000:ff:02.0: [8086:2c10] type 00 class 0x060000
56 [ 0.467749] pci 0000:ff:02.1: [8086:2c11] type 00 class 0x060000
57 [ 0.467769] pci 0000:ff:02.4: [8086:2c14] type 00 class 0x060000
58 [ 0.467788] pci 0000:ff:02.5: [8086:2c15] type 00 class 0x060000
59 [ 0.467809] pci 0000:ff:03.0: [8086:2c18] type 00 class 0x060000
60 [ 0.467828] pci 0000:ff:03.1: [8086:2c19] type 00 class 0x060000
61 ...
62
63 Display the same message text as above, without the timestamp data:
64
65 crash> log -t
66 pci 0000:ff:02.0: [8086:2c10] type 00 class 0x060000
67 pci 0000:ff:02.1: [8086:2c11] type 00 class 0x060000
68 pci 0000:ff:02.4: [8086:2c14] type 00 class 0x060000
69 pci 0000:ff:02.5: [8086:2c15] type 00 class 0x060000
70 pci 0000:ff:03.0: [8086:2c18] type 00 class 0x060000
71 pci 0000:ff:03.1: [8086:2c19] type 00 class 0x060000
72 ...
73
74 Display the same message text as above, with appended dictionary data:
75
76 crash> log -td
77 pci 0000:ff:02.0: [8086:2c10] type 00 class 0x060000
78 SUBSYSTEM=pci
79 DEVICE=+pci:0000:ff:02.0
80 pci 0000:ff:02.1: [8086:2c11] type 00 class 0x060000
81 SUBSYSTEM=pci
82 DEVICE=+pci:0000:ff:02.1
83 pci 0000:ff:02.4: [8086:2c14] type 00 class 0x060000
84 SUBSYSTEM=pci
85 DEVICE=+pci:0000:ff:02.4
86 pci 0000:ff:02.5: [8086:2c15] type 00 class 0x060000
87 SUBSYSTEM=pci
88 DEVICE=+pci:0000:ff:02.5
89 pci 0000:ff:03.0: [8086:2c18] type 00 class 0x060000
90 SUBSYSTEM=pci
91 DEVICE=+pci:0000:ff:03.0
92 pci 0000:ff:03.1: [8086:2c19] type 00 class 0x060000
93 SUBSYSTEM=pci
94 DEVICE=+pci:0000:ff:03.1
95 ...
96
97 Dump the kernel audit logs:
98
99 crash> log -a
100 type=1320 audit(1489384479.809:4342):
101 type=1300 audit(1489384479.809:4343): arch=c000003e syscall=0 success=yes
102 exit=0 a0=4 a1=7f84154a2000 a2=400 a3=22 items=0 ppid=2560 pid=2591 auid=0
103 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=ttyS0 ses=1
104 comm="pidof" exe="/usr/sbin/killall5"
105 subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 key=(null)
106 type=1320 audit(1489384479.809:4343):
107 type=1300 audit(1489384479.809:4344): arch=c000003e syscall=3 success=yes
108 exit=0 a0=4 a1=1 a2=8 a3=0 items=0 ppid=2560 pid=2591 auid=0 uid=0 gid=0
109 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=ttyS0 ses=1 comm="pidof"
110 exe="/usr/sbin/killall5"
111 subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 key=(null)
112 type=1320 audit(1489384479.809:4344):
113 type=1300 audit(1489384479.809:4345): arch=c000003e syscall=11
114 success=yes exit=0 a0=7f84154a2000 a1=1000 a2=0 a3=0 items=0 ppid=2560
115 pid=2591 auid=0 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0
116 tty=ttyS0 ses=1 comm="pidof" exe="/usr/sbin/killall5"
117 subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 key=(null)
118 type=1320 audit(1489384479.809:4345):
119 type=1300 audit(1489384479.809:4346): arch=c000003e syscall=2 success=yes
120 exit=4 a0=7ffcfd20f5a0 a1=0 a2=1b6 a3=24 items=1 ppid=2560 pid=2591 auid=0
121 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=ttyS0 ses=1
122 comm="pidof" exe="/usr/sbin/killall5"
123 subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 key=(null)
124 type=1307 audit(1489384479.809:4346): cwd="/proc"
125 ...
22. mach - machine specific data
22.1. SYNOPSIS
1mach [-m | -c -[xd] | -o]
22.2. DESCRIPTION
1 This command displays data specific to a machine type.
2
3 -m Display the physical memory map (x86, x86_64 and ia64 only).
4 -c Display each cpu's cpuinfo structure (x86, x86_64 and ia64 only).
5 Display each cpu's x8664_pda structure (x86_64 only),
6 Display the hwrpb_struct, and each cpu's percpu_struct (alpha only).
7 -x override default output format with hexadecimal format.
8 -d override default output format with decimal format.
9 -o Display the OPAL console log (ppc64 only).
22.3. EXAMPLES
1 crash> mach
2 MACHINE TYPE: i686
3 MEMORY SIZE: 512 MB
4 CPUS: 2
5 HYPERVISOR: KVM
6 PROCESSOR SPEED: 1993 Mhz
7 HZ: 100
8 PAGE SIZE: 4096
9 KERNEL VIRTUAL BASE: c0000000
10 KERNEL VMALLOC BASE: e0800000
11 KERNEL STACK SIZE: 8192
12
13 Display the system physical memory map:
14
15 crash> mach -m
16 PHYSICAL ADDRESS RANGE TYPE
17 0000000000000000 - 00000000000a0000 E820_RAM
18 00000000000f0000 - 0000000000100000 E820_RESERVED
19 0000000000100000 - 000000001ff75000 E820_RAM
20 000000001ff75000 - 000000001ff77000 E820_NVS
21 000000001ff77000 - 000000001ff98000 E820_ACPI
22 000000001ff98000 - 0000000020000000 E820_RESERVED
23 00000000fec00000 - 00000000fec90000 E820_RESERVED
24 00000000fee00000 - 00000000fee10000 E820_RESERVED
25 00000000ffb00000 - 0000000100000000 E820_RESERVED
26
27 Display the OPAL console log:
28
29 crash> mach -o
30 [ 65.219056911,5] SkiBoot skiboot-5.4.0-218-ge0225cc-df9a248 starting...
31 [ 65.219065872,5] initial console log level: memory 7, driver 5
32 [ 65.219068917,6] CPU: P8 generation processor(max 8 threads/core)
33 [ 65.219071681,7] CPU: Boot CPU PIR is 0x0060 PVR is 0x004d0200
34 [ 65.219074685,7] CPU: Initial max PIR set to 0x1fff
35 [ 65.219607955,5] FDT: Parsing fdt @0xff00000
36 [ 494.026291523,7] BT: seq 0x25 netfn 0x0a cmd 0x48: Message sent to host
37 [ 494.027636927,7] BT: seq 0x25 netfn 0x0a cmd 0x48: IPMI MSG done
23. mod - module information and loading of symbols and debugging data
23.1. SYNOPSIS
1mod -s module [objfile] | -d module | -S [directory] [-D|-t|-r|-R|-o|-g]
23.2. DESCRIPTION
1 With no arguments, this command displays basic information of the currently
2 installed modules, consisting of the module address, name, size, the
3 object file name (if known), and whether the module was compiled with
4 CONFIG_KALLSYMS.
5
6 The arguments are concerned with with the loading or deleting of symbolic
7 and debugging data from a module's object file. A modules's object file
8 always contains symbolic data (symbol names and addresses), but contains
9 debugging data only if the module was compiled with the -g CFLAG. In
10 addition, the module may have compiled with CONFIG_KALLSYMS, which means
11 that the module's symbolic data will have been loaded into the kernel's
12 address space when it was installed. If the module was not compiled with
13 CONFIG_KALLSYMS, then only the module's exported symbols will be loaded
14 into the kernel's address space. Therefore, for the purpose of this
15 command, it should noted that a kernel module may have been compiled in
16 one of following manners:
17
18 1. If the module was built without CONFIG_KALLSYMS and without the -g CFLAG,
19 then the loading of the module's additional non-exported symbols can
20 be accomplished with this command.
21 2. If the module was built with CONFIG_KALLSYMS, but without the -g CFLAG,
22 then there is no benefit in loading the symbols from the module object
23 file, because all of the module's symbols will have been loaded into the
24 kernel's address space when it was installed.
25 3. If the module was built with CONFIG_KALLSYMS and with the the -g CFLAG,
26 then the loading of the module's debugging data can be accomplished
27 with this command.
28 4. If the module was built without CONFIG_KALLSYMS but with the -g CFLAG,
29 then the loading of the both module's symbolic and debugging data can
30 be accomplished with this command.
31
32 -s module [objfile] Loads symbolic and debugging data from the object file
33 for the module specified. If no objfile argument is
34 appended, a search will be made for an object file
35 consisting of the module name with a .o or .ko suffix,
36 starting at the /lib/modules/<release> directory on
37 the host system, or if not found there, starting at the
38 directory containing the kernel namelist file. If an
39 objfile argument is appended, then that file will be
40 used.
41 -d module Deletes the symbolic and debugging data of the module
42 specified.
43 -S [directory] Load symbolic and debugging data from the object file
44 for all loaded modules. For each module, a search
45 will be made for an object file consisting of the
46 module name with a .o or .ko suffix, starting at the
47 /lib/modules/<release> directory of the host system,
48 or if not found there, starting at the directory
49 containing the kernel namelist file. If a directory
50 argument is appended, then the search will be restricted
51 to that directory.
52 -D Deletes the symbolic and debugging data of all modules.
53 -t Display the contents of the module's "taints" bitmask
54 if it is non-zero. When possible, the "taints" bits
55 are translated to symbolic letters of the taint type;
56 otherwise the hexadecimal value is shown. In older
57 kernels, the contents of the "license_gplok" field is
58 displayed in hexadecimal; the field may be either a
59 bitmask or a boolean, depending upon the kernel version.
60 The relevant kernel sources should be consulted for the
61 meaning of the letter(s) or hexadecimal bit value(s).
62 For modules that have a "gpgsig_ok" field that is zero
63 (unsigned), the notation "(U)" is shown.
64 -r Passes the -readnow flag to the embedded gdb module,
65 which will override the two-stage strategy that it uses
66 for reading symbol tables from module object files.
67 -R Reinitialize module data. All currently-loaded symbolic
68 and debugging data will be deleted, and the installed
69 module list will be updated (live system only).
70 -g When used with -s or -S, add a module object's section
71 start and end addresses to its symbol list.
72 -o Load module symbols with old mechanism.
73
74 If the crash session was invoked with the "--mod <directory>" option, or
75 a CRASH_MODULE_PATH environment variable exists, then /lib/modules/<release>
76 will be overridden as the default directory tree that is searched for module
77 object files.
78
79 After symbolic and debugging data have been loaded, backtraces and text
80 disassembly will be displayed appropriately. Depending upon the processor
81 architecture, data may also printed symbolically with the "p" command;
82 at a minimum, the "rd" command may be used with module data symbols.
83
84 If crash can recognize that the set of modules has changed while running a
85 session on a live kernel, the module data will be reinitialized the next
86 time this command is run; the -r option forces the reinitialization.
23.3. EXAMPLES
1 Display the currently-installed modules:
2
3 crash> mod
4 MODULE NAME SIZE OBJECT FILE
5 c8019000 soundcore 2788 (not loaded)
6 c801b000 soundlow 336 (not loaded)
7 c801d000 sound 59864 (not loaded)
8 c802d000 ad1848 15728 (not loaded)
9 c8032000 uart401 6000 (not loaded)
10 c8035000 cs4232 2472 (not loaded)
11 c8043000 opl3 11048 (not loaded)
12 c8047000 3c59x 18152 (not loaded)
13 c804d000 sunrpc 53796 (not loaded)
14 c805c000 lockd 31528 (not loaded)
15 c8065000 nfsd 151896 (not loaded)
16 c8092000 nfs 29752 (not loaded)
17
18 Display the currently-installed modules on a system where all modules were
19 compiled with CONFIG_KALLSYMS:
20
21 crash> mod
22 MODULE NAME SIZE OBJECT FILE
23 e080d000 jbd 57016 (not loaded) [CONFIG_KALLSYMS]
24 e081e000 ext3 92360 (not loaded) [CONFIG_KALLSYMS]
25 e0838000 usbcore 83168 (not loaded) [CONFIG_KALLSYMS]
26 e0850000 usb-uhci 27532 (not loaded) [CONFIG_KALLSYMS]
27 e085a000 ehci-hcd 20904 (not loaded) [CONFIG_KALLSYMS]
28 e0865000 input 6208 (not loaded) [CONFIG_KALLSYMS]
29 e086a000 hid 22404 (not loaded) [CONFIG_KALLSYMS]
30 e0873000 mousedev 5688 (not loaded) [CONFIG_KALLSYMS]
31 e0878000 keybdev 2976 (not loaded) [CONFIG_KALLSYMS]
32 e08fd000 cdrom 34144 (not loaded) [CONFIG_KALLSYMS]
33 e0909000 ide-cd 35776 (not loaded) [CONFIG_KALLSYMS]
34 e0915000 scsi_mod 117928 (not loaded) [CONFIG_KALLSYMS]
35 e0935000 ide-scsi 12752 (not loaded) [CONFIG_KALLSYMS]
36 e093c000 microcode 5248 (not loaded) [CONFIG_KALLSYMS]
37 e0943000 sr_mod 18136 (not loaded) [CONFIG_KALLSYMS]
38 e0956000 floppy 59056 (not loaded) [CONFIG_KALLSYMS]
39 e0966000 sg 38060 (not loaded) [CONFIG_KALLSYMS]
40 e0971000 ip_tables 16544 (not loaded) [CONFIG_KALLSYMS]
41 e097d000 iptable_filter 2412 (not loaded) [CONFIG_KALLSYMS]
42 e097f000 e1000 76096 (not loaded) [CONFIG_KALLSYMS]
43 e09ba000 autofs 13780 (not loaded) [CONFIG_KALLSYMS]
44 e09c1000 parport 39072 (not loaded) [CONFIG_KALLSYMS]
45 e09ce000 lp 9220 (not loaded) [CONFIG_KALLSYMS]
46 e09d4000 parport_pc 19204 (not loaded) [CONFIG_KALLSYMS]
47 e09e2000 agpgart 59128 (not loaded) [CONFIG_KALLSYMS]
48 e0a1a000 radeon 117156 (not loaded) [CONFIG_KALLSYMS]
49 e2dc7000 sunrpc 91996 (not loaded) [CONFIG_KALLSYMS]
50 e2de1000 lockd 60624 (not loaded) [CONFIG_KALLSYMS]
51 e2df3000 nfs 96880 (not loaded) [CONFIG_KALLSYMS]
52
53 Load the symbolic and debugging data of all modules:
54
55 crash> mod -S
56 MODULE NAME SIZE OBJECT FILE
57 c8019000 soundcore 2788 /lib/modules/2.2.5-15/misc/soundcore.o
58 c801b000 soundlow 336 /lib/modules/2.2.5-15/misc/soundlow.o
59 c801d000 sound 59864 /lib/modules/2.2.5-15/misc/sound.o
60 c802d000 ad1848 15728 /lib/modules/2.2.5-15/misc/ad1848.o
61 c8032000 uart401 6000 /lib/modules/2.2.5-15/misc/uart401.o
62 c8035000 cs4232 2472 /lib/modules/2.2.5-15/misc/cs4232.o
63 c8043000 opl3 11048 /lib/modules/2.2.5-15/misc/opl3.o
64 c8047000 3c59x 18152 /lib/modules/2.2.5-15/net/3c59x.o
65 c804d000 sunrpc 53796 /lib/modules/2.2.5-15/misc/sunrpc.o
66 c805c000 lockd 31528 /lib/modules/2.2.5-15/fs/lockd.o
67 c8065000 nfsd 151896 /lib/modules/2.2.5-15/fs/nfsd.o
68 c8092000 nfs 29752 /lib/modules/2.2.5-15/fs/nfs.o
69
70 Load the symbolic and debugging data of the soundcore module from its
71 known location:
72
73 crash> mod -s soundcore
74 MODULE NAME SIZE OBJECT FILE
75 c8019000 soundcore 2788 /lib/modules/2.2.5-15/misc/soundcore.o
76
77 Delete the current symbolic and debugging data of the soundcore module,
78 and then re-load it from a specified object file:
79
80 crash> mod -d soundcore
81 crash> mod -s soundcore /tmp/soundcore.o
82 MODULE NAME SIZE OBJECT FILE
83 c8019000 soundcore 2788 /tmp/soundcore.o
84
85 After installing a new kernel module on a live system, reinitialize the
86 installed module list:
87
88 crash> !insmod mdacon
89 crash> mod
90 mod: NOTE: modules have changed on this system -- reinitializing
91 MODULE NAME SIZE OBJECT FILE
92 c8019000 soundcore 2788 (not loaded)
93 c801b000 soundlow 336 (not loaded)
94 c801d000 sound 59864 (not loaded)
95 c802d000 ad1848 15728 (not loaded)
96 c8032000 uart401 6000 (not loaded)
97 c8035000 cs4232 2472 (not loaded)
98 c8043000 opl3 11048 (not loaded)
99 c8047000 3c59x 18152 (not loaded)
100 c804d000 sunrpc 53796 (not loaded)
101 c805c000 lockd 31528 (not loaded)
102 c8065000 nfs 29752 (not loaded)
103 c806e000 autofs 9316 (not loaded)
104 c8072000 nfsd 151896 (not loaded)
105 c80a1000 mdacon 3556 (not loaded)
106
107 Display modules that are "tainted", where in this case
108 where they are proprietary and unsigned:
109
110 crash> mod -t
111 NAME TAINT
112 vxspec P(U)
113 vxportal P(U)
114 fdd P(U)
115 vxfs P(U)
116 vxdmp P(U)
117 vxio P(U)
118 vxglm P(U)
119 vxgms P(U)
120 vxodm P(U)
24. mount - mounted filesystem data
24.1. SYNOPSIS
1mount [-f][-i] [-n pid|task] [mount|vfsmount|superblock|dev|dir|dentry|inode]
24.2. DESCRIPTION
1 This command displays basic information about the currently-mounted
2 filesystems. The per-filesystem dirty inode list or list of open
3 files for the filesystem may also be displayed.
4
5 -f dump dentries and inodes for open files in each filesystem; only
6 supported on kernels prior to Linux 3.13.
7 -i dump all dirty inodes associated with each filesystem; only
8 supported on kernels prior to Linux 2.6.32.
9
10 For kernels supporting namespaces, the -n option may be used to
11 display the mounted filesystems with respect to the namespace of a
12 specified task:
13
14 -n pid a process PID.
15 -n task a hexadecimal task_struct pointer.
16
17 Specific filesystems may be selected using the following forms:
18
19 vfsmount hexadecimal address of a filesystem vfsmount structure.
20 mount hexadecimal address of a filesystem mount structure (Linux 3.3
21 and later).
22 superblock hexadecimal address of a filesystem super_block structure.
23 dev device name of a filesystem.
24 dir directory where a filesystem is mounted.
25 dentry hexadecimal address of an open dentry of a filesystem.
26 inode hexadecimal address of an open inode of a filesystem.
27
28 The first column of the command output displays the filesystem's vfsmount
29 structure address for kernels prior to Linux 3.3. For Linux 3.3 and later
30 kernels, the first column displays the filesystem's mount structure address,
31 which contains an embedded vfsmount structure.
24.3. EXAMPLES
1 Display mounted filesystem data:
2
3 crash> mount
4 VFSMOUNT SUPERBLK TYPE DEVNAME DIRNAME
5 c0089ea0 c0088a00 ext2 /dev/root /
6 c0089cf0 c0088c00 proc /proc /proc
7 c0089e10 c0088800 ext2 /dev/sda5 /boot
8 c0089d80 c0088600 ext2 /dev/sda6 /usr
9 c0089f30 c0088400 devpts none /dev/pts
10 c3f4b010 c0088200 ext2 /dev/sda1 /home
11
12 On Linux 3.3 and later kernels, the filesystem's mount structure address
13 is shown:
14
15 crash> mount
16 MOUNT SUPERBLK TYPE DEVNAME DIRNAME
17 ffff880212fb8200 ffff880212fc0800 rootfs rootfs /
18 ffff88020ffbea00 ffff880212fc2000 proc proc proc
19 ffff880211db7f00 ffff88020e01a800 sysfs sysfs /sys
20 ffff88020ffe1300 ffff880212a40000 devtmpfs devtmpfs /dev
21 ffff88020ff15000 ffff880212bbc800 devpts devpts /dev/pts
22 ffff88020e542800 ffff88020e62b800 tmpfs tmpfs /dev/shm
23 ...
24
25 Display the open files associated with each mounted filesystem:
26
27 crash> mount -f
28 VFSMOUNT SUPERBLK TYPE DEVNAME DIRNAME
29 c7fb2b80 c7fb3200 ext2 /dev/root /
30 OPEN FILES:
31 DENTRY INODE TYPE PATH
32 c6d02200 c6d0f7a0 REG usr/X11R6/lib/libX11.so.6.1
33 c6d02100 c6d0f9e0 REG usr/X11R6/lib/libXext.so.6.3
34 c6d02000 c6d0fc20 REG usr/X11R6/lib/libICE.so.6.3
35 c6d02680 c6d0f320 REG usr/X11R6/bin/xfs
36 c7106580 c70c5440 CHR dev/psaux
37 ...
38
39 Display the dirty inodes associated with each mounted filesystem:
40
41 crash> mount -i
42 VFSMOUNT SUPERBLK TYPE DEVNAME DIRNAME
43 c0089ea0 c0088a00 ext2 /dev/root /
44 DIRTY INODES
45 c7ad4008
46 c2233438
47 c72c4008
48 c7d6b548
49 c3af1a98
50 c7d6b768
51 c3c4e228
52 ...
53
54 Display the mounted filesystem containing inode c5000aa8:
55
56 crash> mount c5000aa8
57 VFSMOUNT SUPERBLK TYPE DEVNAME DIRNAME
58 c0089f30 c0088600 ext2 /dev/sda6 /usr
59
60 Display the mounted filesystem containing inode ffff8801f4245e40:
61
62 crash> mount ffff8801f4245e40
63 MOUNT SUPERBLK TYPE DEVNAME DIRNAME
64 ffff88020ffbea00 ffff880212fc2000 proc proc /proc
25. net - network command
25.1. SYNOPSIS
1net [[-s | -S] [-xd] [-R ref] [pid | task]] [-a] [ -n [pid | task]] [-N addr]
25.2. DESCRIPTION
1 Displays various network related data.
2
3 If no arguments are entered, the list of network devices, names and IP
4 addresses are displayed. For kernels supporting namespaces, the -n option
5 may be used to display the list of network devices with respect to the
6 network namespace of a current context or a task specified by pid or task:
7
8 -n the namespace of the current context.
9 -n pid a process PID.
10 -n task a hexadecimal task_struct pointer.
11
12 The -s and -S options display data with respect to the current context, but
13 may be appended with an argument to show the socket data with respect
14 to a specified task:
15
16 -s display open network socket/sock addresses, their family and type,
17 and for INET and INET6 families, their source and destination
18 addresses and ports.
19 -s pid same as above, for task with process PID pid.
20 -s task same as above, for task with hexadecimal task_struct pointer task.
21
22 -S displays open network socket/sock addresses followed by a dump
23 of both data structures.
24 -S pid same as above, with respect to process PID.
25 -S task same as above, with respect to hexadecimal task_struct pointer.
26
27 The -R option, typically invoked from "foreach net", and in conjunction
28 with the -s or -S options, searches for references to a socket address,
29 sock address, or a file descriptor; if found, only the referenced fd, socket
30 or sock data will be displayed:
31
32 -R ref socket or sock address, or file descriptor.
33
34 Other options:
35
36 -a display the ARP cache.
37 -N addr translates an IPv4 address expressed as a decimal or hexadecimal
38 value into a standard numbers-and-dots notation.
39 -x override default output format with hexadecimal format.
40 -d override default output format with decimal format.
25.3. EXAMPLES
1 Display the system's network device list:
2
3 crash> net
4 NET_DEVICE NAME IP ADDRESS(ES)
5 ffff8803741c0000 lo 127.0.0.1
6 fff88037059c0000 eth0 10.226.229.141
7 ffff8803705c0000 eth1 10.226.228.250
8 ffff880374ad6000 usb0 169.254.95.120
9
10 Display the network device list with respect to the network namespace
11 of PID 2618:
12
13 crash> net -n 2618
14 NET_DEVICE NAME IP ADDRESS(ES)
15 ffff880456ee7020 lo 127.0.0.1
16 ffff8804516a1020 eth0 10.1.9.223
17
18 Dump the ARP cache:
19
20 crash> net -a
21 NEIGHBOUR IP ADDRESS HW TYPE HW ADDRESS DEVICE STATE
22 f38d1b00 10.16.64.14 ETHER 00:16:3e:4b:a5:4a eth1 STALE
23 f38d1080 0.0.0.0 UNKNOWN 00 00 00 00 00 00 lo NOARP
24 f38d1bc0 10.16.71.254 ETHER 00:00:0c:07:ac:00 eth1 REACHABLE
25 f38d1200 10.16.64.21 ETHER 00:16:3e:51:d8:09 eth1 REACHABLE
26
27 Display the sockets for PID 2517, using both -s and -S output formats:
28
29 crash> net -s 2517
30 PID: 2517 TASK: c1598000 CPU: 1 COMMAND: "rlogin"
31 FD SOCKET SOCK FAMILY:TYPE SOURCE-PORT DESTINATION-PORT
32 3 c57375dc c1ff1850 INET:STREAM 10.1.8.20-1023 10.1.16.62-513
33
34 crash> net -S 2517
35 PID: 2517 TASK: c1598000 CPU: 1 COMMAND: "rlogin"
36 FD SOCKET SOCK
37 3 c57375dc c1ff1850
38
39 struct socket {
40 state = SS_CONNECTED,
41 flags = 131072,
42 ops = 0xc023f820,
43 inode = 0xc5737540,
44 fasync_list = 0x0,
45 file = 0xc58892b0,
46 sk = 0xc1ff1850,
47 wait = 0xc14d9ed4,
48 type = 1,
49 passcred = 0 '\000',
50 tli = 0 '\000'
51 }
52 struct sock {
53 sklist_next = 0xc1ff12f0,
54 sklist_prev = 0xc216bc00,
55 bind_next = 0x0,
56 bind_pprev = 0xc0918448,
57 daddr = 1041236234,
58 rcv_saddr = 336068874,
59 dport = 258,
60 num = 1023,
61 bound_dev_if = 0,
62 next = 0x0,
63 pprev = 0xc0286dd4,
64 state = 1 '\001',
65 zapped = 0 '\000',
66 sport = 65283,
67 family = 2,
68 reuse = 0 '\000',
69 ...
70 Translate the rcv_saddr from above into dotted-decimal notation:
71
72 crash> net -N 1041236234
73 10.1.16.62
74
75 From "foreach", find all tasks with references to socket c08ea3cc:
76
77 crash> foreach net -s -R c08ea3cc
78 PID: 2184 TASK: c7026000 CPU: 1 COMMAND: "klines.kss"
79 FD SOCKET SOCK FAMILY:TYPE SOURCE-PORT DESTINATION-PORT
80 5 c08ea3cc c50d3c80 INET:STREAM 0.0.0.0-1026 0.0.0.0-0
81
82 PID: 2200 TASK: c670a000 CPU: 1 COMMAND: "kpanel"
83 FD SOCKET SOCK FAMILY:TYPE SOURCE-PORT DESTINATION-PORT
84 5 c08ea3cc c50d3c80 INET:STREAM 0.0.0.0-1026 0.0.0.0-0
85
86 PID: 2201 TASK: c648a000 CPU: 1 COMMAND: "kbgndwm"
87 FD SOCKET SOCK FAMILY:TYPE SOURCE-PORT DESTINATION-PORT
88 5 c08ea3cc c50d3c80 INET:STREAM 0.0.0.0-1026 0.0.0.0-0
89
90 PID: 19294 TASK: c250a000 CPU: 0 COMMAND: "prefdm"
91 FD SOCKET SOCK FAMILY:TYPE SOURCE-PORT DESTINATION-PORT
92 5 c08ea3cc c50d3c80 INET:STREAM 0.0.0.0-1026 0.0.0.0-0
93
94 PID: 2194 TASK: c62dc000 CPU: 1 COMMAND: "kaudioserver"
95 FD SOCKET SOCK FAMILY:TYPE SOURCE-PORT DESTINATION-PORT
96 5 c08ea3cc c50d3c80 INET:STREAM 0.0.0.0-1026 0.0.0.0-0
97
98 PID: 2195 TASK: c6684000 CPU: 1 COMMAND: "maudio"
99 FD SOCKET SOCK FAMILY:TYPE SOURCE-PORT DESTINATION-PORT
100 5 c08ea3cc c50d3c80 INET:STREAM 0.0.0.0-1026 0.0.0.0-0
101
102 PID: 2196 TASK: c6b58000 CPU: 1 COMMAND: "kwmsound"
103 FD SOCKET SOCK FAMILY:TYPE SOURCE-PORT DESTINATION-PORT
104 5 c08ea3cc c50d3c80 INET:STREAM 0.0.0.0-1026 0.0.0.0-0
105
106 PID: 2197 TASK: c6696000 CPU: 0 COMMAND: "kfm"
107 FD SOCKET SOCK FAMILY:TYPE SOURCE-PORT DESTINATION-PORT
108 5 c08ea3cc c50d3c80 INET:STREAM 0.0.0.0-1026 0.0.0.0-0
109
110 PID: 2199 TASK: c65ec000 CPU: 0 COMMAND: "krootwm"
111 FD SOCKET SOCK FAMILY:TYPE SOURCE-PORT DESTINATION-PORT
112 5 c08ea3cc c50d3c80 INET:STREAM 0.0.0.0-1026 0.0.0.0-0
113
114 PID: 694 TASK: c1942000 CPU: 0 COMMAND: "prefdm"
115 FD SOCKET SOCK FAMILY:TYPE SOURCE-PORT DESTINATION-PORT
116 5 c08ea3cc c50d3c80 INET:STREAM 0.0.0.0-1026 0.0.0.0-0
117
118 PID: 698 TASK: c6a2c000 CPU: 1 COMMAND: "X"
119 FD SOCKET SOCK FAMILY:TYPE SOURCE-PORT DESTINATION-PORT
120 5 c08ea3cc c50d3c80 INET:STREAM 0.0.0.0-1026 0.0.0.0-0
121
122 PID: 2159 TASK: c4a5a000 CPU: 1 COMMAND: "kwm"
123 FD SOCKET SOCK FAMILY:TYPE SOURCE-PORT DESTINATION-PORT
124 5 c08ea3cc c50d3c80 INET:STREAM 0.0.0.0-1026 0.0.0.0-0
26. p - print the value of an expression
26.1. SYNOPSIS
1p [-x|-d][-u] [expression | symbol[:cpuspec]]
26.2. DESCRIPTION
1 This command passes its arguments on to gdb "print" command for evaluation.
2
3 expression an expression to be evaluated.
4 symbol a kernel symbol.
5 :cpuspec CPU specification for a per-cpu symbol:
6 : CPU of the currently selected task.
7 :a[ll] all CPUs.
8 :#[-#][,...] CPU list(s), e.g. "1,3,5", "1-3",
9 or "1,3,5-7,10".
10 -x override default output format with hexadecimal format.
11 -d override default output format with decimal format.
12 -u the expression evaluates to a user address reference.
13
14 The default output format is decimal, but that can be changed at any time
15 with the two built-in aliases "hex" and "dec". Alternatively, there
16 are two other built-in aliases, "px" and "pd", which force the command
17 output to be displayed in hexadecimal or decimal, without changing the
18 default mode.
26.3. EXAMPLES
1 Print the contents of jiffies:
2
3 crash> p jiffies
4 jiffies = $6 = 166532620
5 crash> px jiffies
6 jiffies = $7 = 0x9ed174b
7 crash> pd jiffies
8 jiffies = $8 = 166533160
9
10 Print the contents of the vm_area_struct "init_mm":
11
12 crash> p init_mm
13 init_mm = $5 = {
14 mmap = 0xc022d540,
15 mmap_avl = 0x0,
16 mmap_cache = 0x0,
17 pgd = 0xc0101000,
18 count = {
19 counter = 0x6
20 },
21 map_count = 0x1,
22 mmap_sem = {
23 count = {
24 counter = 0x1
25 },
26 waking = 0x0,
27 wait = 0x0
28 },
29 context = 0x0,
30 start_code = 0xc0000000,
31 end_code = 0xc022b4c8,
32 start_data = 0x0,
33 end_data = 0xc0250388,
34 start_brk = 0x0,
35 brk = 0xc02928d8,
36 start_stack = 0x0,
37 arg_start = 0x0,
38 arg_end = 0x0,
39 env_start = 0x0,
40 env_end = 0x0,
41 rss = 0x0,
42 total_vm = 0x0,
43 locked_vm = 0x0,
44 def_flags = 0x0,
45 cpu_vm_mask = 0x0,
46 swap_cnt = 0x0,
47 swap_address = 0x0,
48 segments = 0x0
49 }
50
51 If a per-cpu symbol is entered as a argument, its data type
52 and all of its per-cpu addresses are displayed:
53
54 crash> p irq_stat
55 PER-CPU DATA TYPE:
56 irq_cpustat_t irq_stat;
57 PER-CPU ADDRESSES:
58 [0]: ffff88021e211540
59 [1]: ffff88021e251540
60 [2]: ffff88021e291540
61 [3]: ffff88021e2d1540
62
63 To display the contents a per-cpu symbol for CPU 1, append
64 a cpu-specifier:
65
66 crash> p irq_stat:1
67 per_cpu(irq_stat, 1) = $29 = {
68 __softirq_pending = 0,
69 __nmi_count = 209034,
70 apic_timer_irqs = 597509876,
71 irq_spurious_count = 0,
72 icr_read_retry_count = 2,
73 x86_platform_ipis = 0,
74 apic_perf_irqs = 209034,
75 apic_irq_work_irqs = 0,
76 irq_resched_count = 264922233,
77 irq_call_count = 7036692,
78 irq_tlb_count = 4750442,
79 irq_thermal_count = 0,
80 irq_threshold_count = 0
81 }
27. ps - display process status information
27.1. SYNOPSIS
1ps [-k|-u|-G|-y policy] [-s] [-p|-c|-t|-[l|m][-C cpu]|-a|-g|-r|-S|-A]
2 [pid | task | command] ...
27.2. DESCRIPTION
1 This command displays process status for selected, or all, processes
2 in the system. If no arguments are entered, the process data is
3 is displayed for all processes. Specific processes may be selected
4 by using the following identifier formats:
5
6 pid a process PID.
7 task a hexadecimal task_struct pointer.
8 command a command name. If a command name is made up of letters that
9 are all numerical values, precede the name string with a "\".
10 If the command string is enclosed within "'" characters, then
11 the encompassed string must be a POSIX extended regular expression
12 that will be used to match task names.
13
14 The process list may be further restricted by the following options:
15
16 -k restrict the output to only kernel threads.
17 -u restrict the output to only user tasks.
18 -G display only the thread group leader in a thread group.
19 -y policy restrict the output to tasks having a specified scheduling policy
20 expressed by its integer value or by its (case-insensitive) name;
21 multiple policies may be entered in a comma-separated list:
22 0 or NORMAL
23 1 or FIFO
24 2 or RR
25 3 or BATCH
26 4 or ISO
27 5 or IDLE
28 6 or DEADLINE
29
30 The process identifier types may be mixed. For each task, the following
31 items are displayed:
32
33 1. the process PID.
34 2. the parent process PID.
35 3. the CPU number that the task ran on last.
36 4. the task_struct address or the kernel stack pointer of the process.
37 (see -s option below)
38 5. the task state (RU, IN, UN, ZO, ST, TR, DE, SW, WA, PA, ID, NE).
39 6. the percentage of physical memory being used by this task.
40 7. the virtual address size of this task in kilobytes.
41 8. the resident set size of this task in kilobytes.
42 9. the command name.
43
44 The default output shows the task_struct address of each process under a
45 column titled "TASK". This can be changed to show the kernel stack
46 pointer under a column titled "KSTACKP".
47
48 -s replace the TASK column with the KSTACKP column.
49
50 On SMP machines, the active task on each CPU will be highlighted by an
51 angle bracket (">") preceding its information. If the crash variable
52 "offline" is set to "hide", the active task on an offline CPU will
53 be highlighted by a "-" preceding its information.
54
55 Alternatively, information regarding parent-child relationships,
56 per-task time usage data, argument/environment data, thread groups,
57 or resource limits may be displayed:
58
59 -p display the parental hierarchy of selected, or all, tasks.
60 -c display the children of selected, or all, tasks.
61 -t display the task run time, start time, and cumulative user
62 and system times.
63 -l display the task's last-run timestamp value, using either the
64 task_struct's last_run value, the task_struct's timestamp value
65 or the task_struct's sched_entity last_arrival value, whichever
66 applies, of selected, or all, tasks; the list is sorted with the
67 most recently-run task (with the largest timestamp) shown first,
68 followed by the task's current state.
69 -m similar to -l, but the timestamp value is translated into days,
70 hours, minutes, seconds, and milliseconds since the task was
71 last run on a cpu.
72 -C cpus only usable with the -l or -m options, dump the timestamp data
73 in per-cpu blocks, where the cpu[s] can be specified as "1,3,5",
74 "1-3", "1,3,5-7,10", "all", or "a" (shortcut for "all").
75 -a display the command line arguments and environment strings of
76 selected, or all, user-mode tasks.
77 -g display tasks by thread group, of selected, or all, tasks.
78 -r display resource limits (rlimits) of selected, or all, tasks.
79 -S display a summary consisting of the number of tasks in a task state.
80 -A display only the active task on each cpu.
27.3. EXAMPLES
1 Show the process status of all current tasks:
2
3 crash> ps
4 PID PPID CPU TASK ST %MEM VSZ RSS COMM
5 > 0 0 3 c024c000 RU 0.0 0 0 [swapper]
6 > 0 0 0 c0dce000 RU 0.0 0 0 [swapper]
7 0 0 1 c0fa8000 RU 0.0 0 0 [swapper]
8 > 0 0 2 c009a000 RU 0.0 0 0 [swapper]
9 1 0 1 c0098000 IN 0.0 1096 476 init
10 2 1 1 c0090000 IN 0.0 0 0 [kflushd]
11 3 1 1 c000e000 IN 0.0 0 0 [kpiod]
12 4 1 3 c000c000 IN 0.0 0 0 [kswapd]
13 5 1 1 c0008000 IN 0.0 0 0 [mdrecoveryd]
14 253 1 2 fbc4c000 IN 0.0 1088 376 portmap
15 268 1 2 fbc82000 IN 0.1 1232 504 ypbind
16 274 268 2 fa984000 IN 0.1 1260 556 ypbind
17 321 1 1 fabf6000 IN 0.1 1264 608 syslogd
18 332 1 1 fa9be000 RU 0.1 1364 736 klogd
19 346 1 2 fae88000 IN 0.0 1112 472 atd
20 360 1 2 faeb2000 IN 0.1 1284 592 crond
21 378 1 2 fafd6000 IN 0.1 1236 560 inetd
22 392 1 0 fb710000 IN 0.1 2264 1468 named
23 406 1 3 fb768000 IN 0.1 1284 560 lpd
24 423 1 1 fb8ac000 IN 0.1 1128 528 rpc.statd
25 434 1 2 fb75a000 IN 0.0 1072 376 rpc.rquotad
26 445 1 2 fb4a4000 IN 0.0 1132 456 rpc.mountd
27 460 1 1 fa938000 IN 0.0 0 0 [nfsd]
28 461 1 1 faa86000 IN 0.0 0 0 [nfsd]
29 462 1 0 fac48000 IN 0.0 0 0 [nfsd]
30 463 1 0 fb4ca000 IN 0.0 0 0 [nfsd]
31 464 1 0 fb4c8000 IN 0.0 0 0 [nfsd]
32 465 1 2 fba6e000 IN 0.0 0 0 [nfsd]
33 466 1 1 fba6c000 IN 0.0 0 0 [nfsd]
34 467 1 2 fac04000 IN 0.0 0 0 [nfsd]
35 468 461 2 fa93a000 IN 0.0 0 0 [lockd]
36 469 468 2 fa93e000 IN 0.0 0 0 [rpciod]
37 486 1 0 fab54000 IN 0.1 1596 880 amd
38 523 1 2 fa84e000 IN 0.1 1884 1128 sendmail
39 538 1 0 fa82c000 IN 0.0 1112 416 gpm
40 552 1 3 fa70a000 IN 0.1 2384 1220 httpd
41 556 552 3 fa776000 IN 0.1 2572 1352 httpd
42 557 552 2 faba4000 IN 0.1 2572 1352 httpd
43 558 552 1 fa802000 IN 0.1 2572 1352 httpd
44 559 552 3 fa6ee000 IN 0.1 2572 1352 httpd
45 560 552 3 fa700000 IN 0.1 2572 1352 httpd
46 561 552 0 fa6f0000 IN 0.1 2572 1352 httpd
47 562 552 3 fa6ea000 IN 0.1 2572 1352 httpd
48 563 552 0 fa67c000 IN 0.1 2572 1352 httpd
49 564 552 3 fa674000 IN 0.1 2572 1352 httpd
50 565 552 3 fa66a000 IN 0.1 2572 1352 httpd
51 582 1 2 fa402000 IN 0.2 2968 1916 xfs
52 633 1 2 fa1ec000 IN 0.2 5512 2248 innd
53 636 1 3 fa088000 IN 0.1 2536 804 actived
54 676 1 0 fa840000 IN 0.0 1060 384 mingetty
55 677 1 1 fa590000 IN 0.0 1060 384 mingetty
56 678 1 2 fa3b8000 IN 0.0 1060 384 mingetty
57 679 1 0 fa5b8000 IN 0.0 1060 384 mingetty
58 680 1 1 fa3a4000 IN 0.0 1060 384 mingetty
59 681 1 2 fa30a000 IN 0.0 1060 384 mingetty
60 683 1 3 fa5d8000 IN 0.0 1052 280 update
61 686 378 1 fa3aa000 IN 0.1 2320 1136 in.rlogind
62 687 686 2 f9e52000 IN 0.1 2136 1000 login
63 688 687 0 f9dec000 IN 0.1 1732 976 bash
64 > 700 688 1 f9d62000 RU 0.0 1048 256 gen12
65
66 Display the parental hierarchy of the "crash" process on a live system:
67
68 crash> ps -p 4249
69 PID: 0 TASK: c0252000 CPU: 0 COMMAND: "swapper"
70 PID: 1 TASK: c009a000 CPU: 1 COMMAND: "init"
71 PID: 632 TASK: c73b6000 CPU: 1 COMMAND: "prefdm"
72 PID: 637 TASK: c5a4a000 CPU: 1 COMMAND: "prefdm"
73 PID: 649 TASK: c179a000 CPU: 0 COMMAND: "kwm"
74 PID: 683 TASK: c1164000 CPU: 0 COMMAND: "kfm"
75 PID: 1186 TASK: c165a000 CPU: 0 COMMAND: "xterm"
76 PID: 1188 TASK: c705e000 CPU: 1 COMMAND: "bash"
77 PID: 4249 TASK: c6b9a000 CPU: 0 COMMAND: "crash"
78
79 Display all children of the "kwm" window manager:
80
81 crash> ps -c kwm
82 PID: 649 TASK: c179a000 CPU: 0 COMMAND: "kwm"
83 PID: 682 TASK: c2d58000 CPU: 1 COMMAND: "kwmsound"
84 PID: 683 TASK: c1164000 CPU: 1 COMMAND: "kfm"
85 PID: 685 TASK: c053c000 CPU: 0 COMMAND: "krootwm"
86 PID: 686 TASK: c13fa000 CPU: 0 COMMAND: "kpanel"
87 PID: 687 TASK: c13f0000 CPU: 1 COMMAND: "kbgndwm"
88
89 Display all threads in a firefox session:
90
91 crash> ps firefox
92 PID PPID CPU TASK ST %MEM VSZ RSS COMM
93 21273 21256 6 ffff81003ec15080 IN 46.3 1138276 484364 firefox
94 21276 21256 6 ffff81003f49e7e0 IN 46.3 1138276 484364 firefox
95 21280 21256 0 ffff81003ec1d7e0 IN 46.3 1138276 484364 firefox
96 21286 21256 6 ffff81000b0d1820 IN 46.3 1138276 484364 firefox
97 21287 21256 2 ffff81000b0d10c0 IN 46.3 1138276 484364 firefox
98 26975 21256 5 ffff81003b5c1820 IN 46.3 1138276 484364 firefox
99 26976 21256 5 ffff810023232820 IN 46.3 1138276 484364 firefox
100 26977 21256 4 ffff810021a11820 IN 46.3 1138276 484364 firefox
101 26978 21256 5 ffff810003159040 IN 46.3 1138276 484364 firefox
102 26979 21256 5 ffff81003a058820 IN 46.3 1138276 484364 firefox
103
104 Display only the thread group leader in the firefox session:
105
106 crash> ps -G firefox
107 PID PPID CPU TASK ST %MEM VSZ RSS COMM
108 21273 21256 0 ffff81003ec15080 IN 46.3 1138276 484364 firefox
109
110 Show the time usage data for pid 10318:
111
112 crash> ps -t 10318
113 PID: 10318 TASK: f7b85550 CPU: 5 COMMAND: "bash"
114 RUN TIME: 1 days, 01:35:32
115 START TIME: 5209
116 UTIME: 95
117 STIME: 57
118
119 Show the process status of PID 1, task f9dec000, and all nfsd tasks:
120
121 crash> ps 1 f9dec000 nfsd
122 PID PPID CPU TASK ST %MEM VSZ RSS COMM
123 1 0 1 c0098000 IN 0.0 1096 476 init
124 688 687 0 f9dec000 IN 0.1 1732 976 bash
125 460 1 1 fa938000 IN 0.0 0 0 [nfsd]
126 461 1 1 faa86000 IN 0.0 0 0 [nfsd]
127 462 1 0 fac48000 IN 0.0 0 0 [nfsd]
128 463 1 0 fb4ca000 IN 0.0 0 0 [nfsd]
129 464 1 0 fb4c8000 IN 0.0 0 0 [nfsd]
130 465 1 2 fba6e000 IN 0.0 0 0 [nfsd]
131 466 1 1 fba6c000 IN 0.0 0 0 [nfsd]
132 467 1 2 fac04000 IN 0.0 0 0 [nfsd]
133
134 Show all kernel threads:
135
136 crash> ps -k
137 PID PPID CPU TASK ST %MEM VSZ RSS COMM
138 0 0 1 c0fac000 RU 0.0 0 0 [swapper]
139 0 0 0 c0252000 RU 0.0 0 0 [swapper]
140 2 1 1 c0fa0000 IN 0.0 0 0 [kflushd]
141 3 1 1 c03de000 IN 0.0 0 0 [kpiod]
142 4 1 1 c03dc000 IN 0.0 0 0 [kswapd]
143 5 1 0 c0092000 IN 0.0 0 0 [mdrecoveryd]
144 336 1 0 c4a9a000 IN 0.0 0 0 [rpciod]
145 337 1 0 c4830000 IN 0.0 0 0 [lockd]
146 487 1 1 c4ba6000 IN 0.0 0 0 [nfsd]
147 488 1 0 c18c6000 IN 0.0 0 0 [nfsd]
148 489 1 0 c0cac000 IN 0.0 0 0 [nfsd]
149 490 1 0 c056a000 IN 0.0 0 0 [nfsd]
150 491 1 0 c0860000 IN 0.0 0 0 [nfsd]
151 492 1 1 c0254000 IN 0.0 0 0 [nfsd]
152 493 1 0 c0a86000 IN 0.0 0 0 [nfsd]
153 494 1 0 c0968000 IN 0.0 0 0 [nfsd]
154
155 Display a summary consisting of the number of tasks in a task state:
156
157 crash> ps -S
158 RU: 5
159 IN: 259
160 UN: 31
161 ZO: 1
162
163 Display only the active task, on each cpu:
164
165 crash> ps -A
166 PID PPID CPU TASK ST %MEM VSZ RSS COMM
167 > 10 2 1 ffff880212969710 IN 0.0 0 0 [migration/1]
168 > 0 0 3 ffff884026d43520 RU 0.0 0 0 [swapper]
169 > 6582 1 2 ffff880f49c52040 RU 0.0 42202472 33368 oracle
170 > 9497 1 0 ffff880549ec2ab0 RU 0.0 42314692 138664 oracle
171
172 Show all tasks sorted by their task_struct's last_run, timestamp, or
173 sched_entity last_arrival timestamp value, whichever applies:
174
175 crash> ps -l
176 [20811245123] [IN] PID: 37 TASK: f7153030 CPU: 2 COMMAND: "events/2"
177 [20811229959] [IN] PID: 1756 TASK: f2a5a570 CPU: 2 COMMAND: "ntpd"
178 [20800696644] [IN] PID: 1456 TASK: f2b1f030 CPU: 4 COMMAND: "irqbalance"
179 [20617047229] [IN] PID: 2324 TASK: f57f9570 CPU: 5 COMMAND: "flush-253:0"
180 [20617029209] [IN] PID: 49 TASK: f7167030 CPU: 4 COMMAND: "bdi-default"
181 [20438025365] [IN] PID: 345 TASK: f55c7ab0 CPU: 3 COMMAND: "mpt_poll_0"
182 [20103026046] [IN] PID: 728 TASK: f72ba570 CPU: 3 COMMAND: "edac-poller"
183 [20000189409] [IN] PID: 35 TASK: f7153ab0 CPU: 0 COMMAND: "events/0"
184 [20000179905] [IN] PID: 48 TASK: f7167570 CPU: 0 COMMAND: "sync_supers"
185 [19997120354] [IN] PID: 36 TASK: f7153570 CPU: 1 COMMAND: "events/1"
186 [19991059209] [IN] PID: 38 TASK: f715fab0 CPU: 3 COMMAND: "events/3"
187 [19988091608] [IN] PID: 39 TASK: f715f570 CPU: 4 COMMAND: "events/4"
188 [19985076530] [IN] PID: 40 TASK: f715f030 CPU: 5 COMMAND: "events/5"
189 [19982019106] [IN] PID: 41 TASK: f7161ab0 CPU: 6 COMMAND: "events/6"
190 [19982016294] [IN] PID: 29 TASK: f7109ab0 CPU: 6 COMMAND: "ksoftirqd/6"
191 [19838402345] [RU] PID: 2331 TASK: f297f570 CPU: 7 COMMAND: "bash"
192 [19837129436] [IN] PID: 2326 TASK: f2ad5030 CPU: 6 COMMAND: "sshd"
193 [19289476417] [IN] PID: 1772 TASK: f5665570 CPU: 5 COMMAND: "sendmail"
194 ...
195
196 Show the most-recently run tasks on cpu 0 using both the -l and the -m
197 options:
198
199 crash> ps -m -C0
200 CPU: 0
201 [ 0 00:00:00.003] [RU] PID: 1205 TASK: dee03f20 CPU: 0 COMMAND: "insmod"
202 [ 0 00:00:00.006] [RU] PID: 770 TASK: df9e9940 CPU: 0 COMMAND: "rsyslogd"
203 [ 0 00:00:00.009] [IN] PID: 603 TASK: df9bcbc0 CPU: 0 COMMAND: "udevd"
204 [ 0 00:00:00.010] [IN] PID: 348 TASK: df9ecbc0 CPU: 0 COMMAND: "udevd"
205 [ 0 00:00:00.013] [IN] PID: 934 TASK: df9171a0 CPU: 0 COMMAND: "hald"
206 [ 0 00:00:00.023] [IN] PID: 6 TASK: df443f20 CPU: 0 COMMAND: "events/0"
207 [ 0 00:00:00.029] [IN] PID: 15 TASK: df46b280 CPU: 0 COMMAND: "kblockd/0"
208 [ 0 00:00:00.101] [IN] PID: 1168 TASK: dee01940 CPU: 0 COMMAND: "bash"
209 [ 0 00:00:01.404] [IN] PID: 272 TASK: dfa48ca0 CPU: 0 COMMAND: "flush-8:0"
210 ...
211
212 crash> ps -l -C0
213 CPU: 0
214 [137146164748] [RU] PID: 1205 TASK: dee03f20 CPU: 0 COMMAND: "insmod"
215 [137142534372] [RU] PID: 770 TASK: df9e9940 CPU: 0 COMMAND: "rsyslogd"
216 [137140168469] [IN] PID: 603 TASK: df9bcbc0 CPU: 0 COMMAND: "udevd"
217 [137138826427] [IN] PID: 348 TASK: df9ecbc0 CPU: 0 COMMAND: "udevd"
218 [137135214599] [IN] PID: 934 TASK: df9171a0 CPU: 0 COMMAND: "hald"
219 [137125651275] [IN] PID: 6 TASK: df443f20 CPU: 0 COMMAND: "events/0"
220 [137119564815] [IN] PID: 15 TASK: df46b280 CPU: 0 COMMAND: "kblockd/0"
221 [137047715027] [IN] PID: 1168 TASK: dee01940 CPU: 0 COMMAND: "bash"
222 [135744209052] [IN] PID: 272 TASK: dfa48ca0 CPU: 0 COMMAND: "flush-8:0"
223 ...
224
225 Show the kernel stack pointer of each user task:
226
227 crash> ps -us
228 PID PPID CPU KSTACKP ST %MEM VSZ RSS COMM
229 1 0 0 c009bedc IN 0.0 1096 52 init
230 239 1 0 c15e7ed8 IN 0.2 1332 224 pump
231 280 1 1 c7cbdedc IN 0.2 1092 208 portmap
232 295 1 0 c7481edc IN 0.0 1232 0 ypbind
233 301 295 0 c7c7bf28 IN 0.1 1260 124 ypbind
234 376 1 1 c5053f28 IN 0.0 1316 40 automount
235 381 1 0 c34ddf28 IN 0.2 1316 224 automount
236 391 1 1 c2777f28 IN 0.2 1316 224 automount
237 ...
238
239 Display the argument and environment data for the automount task:
240
241 crash> ps -a automount
242 PID: 3948 TASK: f722ee30 CPU: 0 COMMAND: "automount"
243 ARG: /usr/sbin/automount --timeout=60 /net program /etc/auto.net
244 ENV: SELINUX_INIT=YES
245 CONSOLE=/dev/console
246 TERM=linux
247 INIT_VERSION=sysvinit-2.85
248 PATH=/sbin:/usr/sbin:/bin:/usr/bin
249 LC_MESSAGES=en_US
250 RUNLEVEL=3
251 runlevel=3
252 PWD=/
253 LANG=ja_JP.UTF-8
254 PREVLEVEL=N
255 previous=N
256 HOME=/
257 SHLVL=2
258 _=/usr/sbin/automount
259
260 Display the tasks in the thread group containing task c20ab0b0:
261
262 crash> ps -g c20ab0b0
263 PID: 6425 TASK: f72f50b0 CPU: 0 COMMAND: "firefox-bin"
264 PID: 6516 TASK: f71bf1b0 CPU: 0 COMMAND: "firefox-bin"
265 PID: 6518 TASK: d394b930 CPU: 0 COMMAND: "firefox-bin"
266 PID: 6520 TASK: c20aa030 CPU: 0 COMMAND: "firefox-bin"
267 PID: 6523 TASK: c20ab0b0 CPU: 0 COMMAND: "firefox-bin"
268 PID: 6614 TASK: f1f181b0 CPU: 0 COMMAND: "firefox-bin"
269
270 Display the tasks in the thread group for each instance of the
271 program named "multi-thread":
272
273 crash> ps -g multi-thread
274 PID: 2522 TASK: 1003f0dc7f0 CPU: 1 COMMAND: "multi-thread"
275 PID: 2523 TASK: 10037b13030 CPU: 1 COMMAND: "multi-thread"
276 PID: 2524 TASK: 1003e064030 CPU: 1 COMMAND: "multi-thread"
277 PID: 2525 TASK: 1003e13a7f0 CPU: 1 COMMAND: "multi-thread"
278
279 PID: 2526 TASK: 1002f82b7f0 CPU: 1 COMMAND: "multi-thread"
280 PID: 2527 TASK: 1003e1737f0 CPU: 1 COMMAND: "multi-thread"
281 PID: 2528 TASK: 10035b4b7f0 CPU: 1 COMMAND: "multi-thread"
282 PID: 2529 TASK: 1003f0c37f0 CPU: 1 COMMAND: "multi-thread"
283 PID: 2530 TASK: 10035597030 CPU: 1 COMMAND: "multi-thread"
284 PID: 2531 TASK: 100184be7f0 CPU: 1 COMMAND: "multi-thread"
285
286 Display the resource limits of "bash" task 13896:
287
288 crash> ps -r 13896
289 PID: 13896 TASK: cf402000 CPU: 0 COMMAND: "bash"
290 RLIMIT CURRENT MAXIMUM
291 CPU (unlimited) (unlimited)
292 FSIZE (unlimited) (unlimited)
293 DATA (unlimited) (unlimited)
294 STACK 10485760 (unlimited)
295 CORE (unlimited) (unlimited)
296 RSS (unlimited) (unlimited)
297 NPROC 4091 4091
298 NOFILE 1024 1024
299 MEMLOCK 4096 4096
300 AS (unlimited) (unlimited)
301 LOCKS (unlimited) (unlimited)
302
303 Search for task names matching a POSIX regular expression:
304
305 crash> ps 'migration*'
306 PID PPID CPU TASK ST %MEM VSZ RSS COMM
307 8 2 0 ffff8802128a2e20 IN 0.0 0 0 [migration/0]
308 10 2 1 ffff880212969710 IN 0.0 0 0 [migration/1]
309 15 2 2 ffff880212989710 IN 0.0 0 0 [migration/2]
310 20 2 3 ffff8802129a9710 IN 0.0 0 0 [migration/3]
28. pte - translate a page table entry
28.1. SYNOPSIS
1pte contents ...
28.2. DESCRIPTION
1 This command translates the hexadecimal contents of a PTE into its physical
2 page address and page bit settings. If the PTE references a swap location,
3 the swap device and offset are displayed.
28.3. EXAMPLES
1
2 crash> pte d8e067
3 PTE PHYSICAL FLAGS
4 d8e067 d8e000 (PRESENT|RW|USER|ACCESSED|DIRTY)
5
6 crash> pte 13f600
7 PTE SWAP OFFSET
8 13f600 /dev/hda2 5104
29. ptob - page to bytes
29.1. SYNOPSIS
1ptob page_number ...
29.2. DESCRIPTION
1 This command translates a page frame number to its byte value.
29.3. EXAMPLES
1 crash> ptob 512a
2 512a: 512a000
30. ptov - physical to virtual
per-cpu to virtual
30.1. SYNOPSIS
1ptov [address | offset:cpuspec]
30.2. DESCRIPTION
1 This command translates a hexadecimal physical address into a kernel
2 virtual address. Alternatively, a hexadecimal per-cpu offset and
3 cpu specifier will be translated into kernel virtual addresses for
4 each cpu specified.
5
6 address a physical address
7 offset:cpuspec a per-cpu offset with a CPU specifier:
8 : CPU of the currently selected task.
9 :a[ll] all CPUs.
10 :#[-#][,...] CPU list(s), e.g. "1,3,5", "1-3",
11 or "1,3,5-7,10".
30.3. EXAMPLES
1 Translate physical address 56e000 into a kernel virtual address:
2
3 crash> ptov 56e000
4 VIRTUAL PHYSICAL
5 ffff88000056e000 56e000
6
7 Translate per-cpu offset b0c0 into a kernel virtual address for
8 all cpus:
9
10 crash> ptov b0c0:a
11 PER-CPU OFFSET: b0c0
12 CPU VIRTUAL
13 [0] ffff88021e20b0c0
14 [1] ffff88021e24b0c0
15 [2] ffff88021e28b0c0
16 [3] ffff88021e2cb0c0
31. q - exit this session
31.1. SYNOPSIS
1q
31.2. DESCRIPTION
1 Bail out of the current crash session.
31.3. NOTE
1 This command is equivalent to the "exit" command.
32. rd - read memory
32.1. SYNOPSIS
1rd [-adDsSupxmfNR][-8|-16|-32|-64][-o offs][-e addr][-r file][address|symbol]
2 [count]
32.2. DESCRIPTION
1 This command displays the contents of memory, with the output formatted
2 in several different manners. The starting address may be entered either
3 symbolically or by address. The default output size is the size of a long
4 data type, and the default output format is hexadecimal. When hexadecimal
5 output is used, the output will be accompanied by an ASCII translation.
6
7 -p address argument is a physical address.
8 -u address argument is a user virtual address; only required on
9 processors with common user and kernel virtual address spaces.
10 -m address argument is a xen host machine address.
11 -f address argument is a dumpfile offset.
12 -d display output in signed decimal format (default is hexadecimal).
13 -D display output in unsigned decimal format (default is hexadecimal).
14 -s displays output symbolically when appropriate.
15 -S[S] displays output symbolically when appropriate; if the memory
16 contents reference a slab cache object, the name of the slab cache
17 will be displayed in brackets. If -S is entered twice, and the
18 memory contents reference a slab cache object, both the memory
19 contents and the name of the slab cache will be displayed in
20 brackets.
21 -x do not display ASCII translation at end of each line.
22 -8 display output in 8-bit values.
23 -16 display output in 16-bit values.
24 -32 display output in 32-bit values (default on 32-bit machines).
25 -64 display output in 64-bit values (default on 64-bit machines).
26 -a display output in ASCII characters if the memory contains printable
27 ASCII characters; if no count argument is entered, stop at the first
28 non-printable character.
29 -N display output in network byte order (only valid for 16- and 32-bit
30 values)
31 -R display memory in reverse order; memory will be displayed up to and
32 including the address argument, requiring the count argument to be
33 greater than 1 in order to display memory before the specified
34 address.
35 -o offs offset the starting address by offs.
36 -e addr display memory until reaching specified ending hexadecimal address.
37 -r file dumps raw data to the specified output file; the number of bytes that
38 are copied to the file must be specified either by a count argument
39 or by the -e option.
40 address starting hexadecimal address:
41 1 the default presumes a kernel virtual address.
42 2. -p specifies a physical address.
43 3. -u specifies a user virtual address, but is only necessary on
44 processors with common user and kernel virtual address spaces.
45 symbol symbol of starting address to read.
46 count number of memory locations to display; if entered, it must be the
47 last argument on the command line; if not entered, the count defaults
48 to 1, or unlimited for -a; when used with the -r option, it is the
49 number of bytes to be written to the file.
32.3. EXAMPLES
1 Display the kernel's version string:
2
3 crash> rd -a linux_banner
4 c082a020: Linux version 2.6.32-119.el6.i686 (mockbuild@hs20-bc2-4.buil
5 c082a05c: d.redhat.com) (gcc version 4.4.4 20100726 (Red Hat 4.4.4-13)
6 c082a098: (GCC) ) #1 SMP Tue Mar 1 18:16:57 EST 2011
7
8 Display the same block of memory, first without symbols, again
9 with symbols, and then with symbols and slab cache references:
10
11 crash> rd f6e31f70 28
12 f6e31f70: f6e31f6c f779c180 c04a4032 00a9dd40 l.....y.2@J.@...
13 f6e31f80: 00000fff c0472da0 f6e31fa4 f779c180 .....-G.......y.
14 f6e31f90: fffffff7 00a9b70f f6e31000 c04731ee .............1G.
15 f6e31fa0: f6e31fa4 00000000 00000000 00000000 ................
16 f6e31fb0: 00000000 00a9dd40 c0404f17 00000000 ....@....O@.....
17 f6e31fc0: 00a9dd40 00000fff 00a9dd40 00a9b70f @.......@.......
18 f6e31fd0: bf9e2718 ffffffda c040007b 0000007b .'......{.@.{...
19 crash> rd -s f6e31f70 28
20 f6e31f70: f6e31f6c f779c180 kmsg_read 00a9dd40
21 f6e31f80: 00000fff vfs_read+159 f6e31fa4 f779c180
22 f6e31f90: fffffff7 00a9b70f f6e31000 sys_read+60
23 f6e31fa0: f6e31fa4 00000000 00000000 00000000
24 f6e31fb0: 00000000 00a9dd40 syscall_call+7 00000000
25 f6e31fc0: 00a9dd40 00000fff 00a9dd40 00a9b70f
26 f6e31fd0: bf9e2718 ffffffda startup_32+123 0000007b
27 crash> rd -S f6e31f70 28
28 f6e31f70: [size-4096] [filp] kmsg_read 00a9dd40
29 f6e31f80: 00000fff vfs_read+159 [size-4096] [filp]
30 f6e31f90: fffffff7 00a9b70f [size-4096] sys_read+60
31 f6e31fa0: [size-4096] 00000000 00000000 00000000
32 f6e31fb0: 00000000 00a9dd40 syscall_call+7 00000000
33 f6e31fc0: 00a9dd40 00000fff 00a9dd40 00a9b70f
34 f6e31fd0: bf9e2718 ffffffda startup_32+123 0000007b
35 crash> rd -SS f6e31f70 28
36 f6e31f70: [f6e31f6c:size-4096] [f779c180:filp] kmsg_read 00a9dd40
37 f6e31f80: 00000fff vfs_read+159 [f6e31fa4:size-4096] [f779c180:filp]
38 f6e31f90: fffffff7 00a9b70f [f6e31000:size-4096] sys_read+60
39 f6e31fa0: [f6e31fa4:size-4096] 00000000 00000000 00000000
40 f6e31fb0: 00000000 00a9dd40 syscall_call+7 00000000
41 f6e31fc0: 00a9dd40 00000fff 00a9dd40 00a9b70f
42 f6e31fd0: bf9e2718 ffffffda startup_32+123 0000007b
43
44 Read jiffies in hexadecimal and decimal format:
45
46 crash> rd jiffies
47 c0213ae0: 0008cc3a :...
48
49 crash> rd -d jiffies
50 c0213ae0: 577376
51
52 Access the same memory in different sizes:
53
54 crash> rd -64 kernel_version
55 c0226a6c: 35312d352e322e32 2.2.5-15
56
57 crash> rd -32 kernel_version 2
58 c0226a6c: 2e322e32 35312d35 2.2.5-15
59
60 crash> rd -16 kernel_version 4
61 c0226a6c: 2e32 2e32 2d35 3531 2.2.5-15
62
63 crash> rd -8 kernel_version 8
64 c0226a6c: 32 2e 32 2e 35 2d 31 35 2.2.5-15
65
66 Read the range of memory from c009bf2c to c009bf60:
67
68 crash> rd c009bf2c -e c009bf60
69 c009bf2c: c009bf64 c01328c3 c009bf64 c0132838 d....(..d...8(..
70 c009bf3c: 0000002a 00000004 c57d77e8 00000104 *........w}.....
71 c009bf4c: 0000000b c009a000 7fffffff 00000000 ................
72 c009bf5c: 00000000 ....
33. repeat - repeat a command
33.1. SYNOPSIS
1repeat [-seconds] command
33.2. DESCRIPTION
1 This command repeats a command indefinitely, optionally delaying a given
2 number of seconds between each command execution.
3
4 -seconds The number of seconds to delay between command executions.
5 This option must precede the command name to be executed.
6
7 Command execution may be stopped with CTRL-C, or if scrolling is in effect,
8 by entering "q". This command is meant for use on a live system; it is
9 hard to conceive of a reason to use it when debugging a crash dump.
33.3. EXAMPLES
1 Display the value of jiffies once per second:
2
3 crash> repeat -1 p jiffies
4 jiffies = $1 = 155551079
5 jiffies = $2 = 155551180
6 jiffies = $3 = 155551281
7 jiffies = $4 = 155551382
8 jiffies = $5 = 155551483
9 jiffies = $6 = 155551584
10 jiffies = $7 = 155551685
11 jiffies = $8 = 155551786
12 jiffies = $9 = 155551887
13 jiffies = $10 = 155551988
14 jiffies = $11 = 155552089
15 jiffies = $12 = 155552190
16 jiffies = $13 = 155552291
17 jiffies = $14 = 155552392
18 jiffies = $15 = 155552493
19 jiffies = $16 = 155552594
20 jiffies = $17 = 155552695
21 jiffies = $18 = 155552796
22 ...
34. runq - run queue
34.1. SYNOPSIS
1runq [-t] [-T] [-m] [-g] [-c cpu(s)]
34.2. DESCRIPTION
1 With no argument, this command displays the tasks on the run queues
2 of each cpu.
3
4 -t Display the timestamp information of each cpu's runqueue, which is the
5 rq.clock, rq.most_recent_timestamp or rq.timestamp_last_tick value,
6 whichever applies; following each cpu timestamp is the last_run or
7 timestamp value of the active task on that cpu, whichever applies,
8 along with the task identification.
9 -T Display the time lag of each CPU relative to the most recent runqueue
10 timestamp.
11 -m Display the amount of time that the active task on each cpu has been
12 running, expressed in a format consisting of days, hours, minutes,
13 seconds and milliseconds.
14 -g Display tasks hierarchically by task_group. The task_group line shows
15 the task_group address, the cfs_rq or rt_rq address, the task_group
16 name (if any), and whether the task_group is throttled.
17 -c cpu restrict the output to the run queue data of one or more CPUs,
18 which can be specified using the format "3", "1,8,9", "1-23",
19 or "1,8,9-14".
34.3. EXAMPLES
1 Display the tasks on an O(1) scheduler run queue:
2
3 crash> runq
4 CPU 0 RUNQUEUE: ffff880001cdb460
5 CURRENT: PID: 2739 TASK: ffff8800320fa7e0 COMMAND: "bash"
6 ACTIVE PRIO_ARRAY: ffff880001cdb4d8
7 [115] PID: 2739 TASK: ffff8800320fa7e0 COMMAND: "bash"
8 PID: 1776 TASK: ffff88003217d820 COMMAND: "syslogd"
9 EXPIRED PRIO_ARRAY: ffff880001cdbdb8
10 [no tasks queued]
11
12 CPU 1 RUNQUEUE: ffff880001ce3460
13 CURRENT: PID: 1779 TASK: ffff88003207a860 COMMAND: "klogd"
14 ACTIVE PRIO_ARRAY: ffff880001ce34d8
15 [115] PID: 1779 TASK: ffff88003207a860 COMMAND: "klogd"
16 EXPIRED PRIO_ARRAY: ffff880001ce3db8
17 [no tasks queued]
18
19 Display the tasks on a CFS run queue:
20
21 crash> runq
22 CPU 0 RUNQUEUE: ffff8800090436c0
23 CURRENT: PID: 588 TASK: ffff88007e4877a0 COMMAND: "udevd"
24 RT PRIO_ARRAY: ffff8800090437c8
25 [no tasks queued]
26 CFS RB_ROOT: ffff880009043740
27 [118] PID: 2110 TASK: ffff88007d470860 COMMAND: "check-cdrom.sh"
28 [118] PID: 2109 TASK: ffff88007f1247a0 COMMAND: "check-cdrom.sh"
29 [118] PID: 2114 TASK: ffff88007f20e080 COMMAND: "udevd"
30
31 CPU 1 RUNQUEUE: ffff88000905b6c0
32 CURRENT: PID: 2113 TASK: ffff88007e8ac140 COMMAND: "udevd"
33 RT PRIO_ARRAY: ffff88000905b7c8
34 [no tasks queued]
35 CFS RB_ROOT: ffff88000905b740
36 [118] PID: 2092 TASK: ffff88007d7a4760 COMMAND: "MAKEDEV"
37 [118] PID: 1983 TASK: ffff88007e59f140 COMMAND: "udevd"
38 [118] PID: 2064 TASK: ffff88007e40f7a0 COMMAND: "udevd"
39 [115] PID: 2111 TASK: ffff88007e4278a0 COMMAND: "kthreadd"
40
41 Display run queue timestamp data:
42
43 crash> runq -t
44 CPU 0: 2680990637359
45 2680986653330 PID: 28228 TASK: ffff880037ca2ac0 COMMAND: "loop"
46 CPU 1: 2680940618478
47 2680940618478 PID: 28167 TASK: ffff880078130040 COMMAND: "bash"
48 CPU 2: 2680990763425
49 2680986785772 PID: 28227 TASK: ffff8800787780c0 COMMAND: "loop"
50 CPU 3: 2680990954469
51 2680986059540 PID: 28226 TASK: ffff880078778b00 COMMAND: "loop"
52
53 Display the amount of time the active task on each cpu has been running:
54
55 crash> runq -m
56 CPU 0: [0 00:00:00.014] PID: 5275 TASK: f5dbcaa0 COMMAND: "sh"
57 CPU 1: [0 00:00:00.002] PID: 5203 TASK: f5c7baa0 COMMAND: "cat"
58 CPU 2: [0 00:00:00.014] PID: 7971 TASK: f5c6c550 COMMAND: "khelper"
59 CPU 3: [0 00:00:00.002] PID: 0 TASK: f4ccd000 COMMAND: "swapper"
60
61 Display tasks hierarchically by task_group:
62
63 crash> runq -g
64 CPU 0
65 CURRENT: PID: 14734 TASK: ffff88010626f500 COMMAND: "sh"
66 ROOT_TASK_GROUP: ffffffff81ed93e0 RT_RQ: ffff880028216808
67 [ 0] TASK_GROUP: ffff88022c6bbc00 RT_RQ: ffff880139fc9800 (THROTTLED)
68 [ 0] PID: 14750 TASK: ffff88013a4dd540 COMMAND: "rtloop99"
69 [ 1] PID: 14748 TASK: ffff88013bbca040 COMMAND: "rtloop98"
70 [ 1] TASK_GROUP: ffff88012b0fb400 RT_RQ: ffff880089029000
71 [ 1] PID: 14752 TASK: ffff880088abf500 COMMAND: "rtloop98"
72 [ 54] PID: 14749 TASK: ffff880037a4e080 COMMAND: "rtloop45"
73 [ 98] PID: 14746 TASK: ffff88012678c080 COMMAND: "rtloop1"
74 ROOT_TASK_GROUP: ffffffff81ed93e0 CFS_RQ: ffff8800282166e8
75 [120] PID: 14740 TASK: ffff88013b1e6080 COMMAND: "sh"
76 [120] PID: 14738 TASK: ffff88012678d540 COMMAND: "sh"
77 [120] PID: 14734 TASK: ffff88010626f500 COMMAND: "sh" [CURRENT]
78 TASK_GROUP: ffff884052bc9800 CFS_RQ: ffff8831e4a1b000 (THROTTLED)
79 [120] PID: 14732 TASK: ffff88013bbcb500 COMMAND: "sh"
80 [120] PID: 14728 TASK: ffff8800b3496080 COMMAND: "sh"
81 [120] PID: 14730 TASK: ffff880037833540 COMMAND: "sh"
82 TASK_GROUP: ffff884058f1d000 CFS_RQ: ffff88120a101600 (THROTTLED)
83 [120] PID: 14726 TASK: ffff880138d42aa0 COMMAND: "sh"
84 ...
85
86 Display tasks hierarchically by task_group for cpu 3 only:
87
88 crash> runq -g -c3
89 CPU 3
90 CURRENT: PID: 2948 TASK: ffff88022af2a100 COMMAND: "bash"
91 INIT_TASK_GROUP: ffffffff81e1a780 RT_RQ: ffff880028216148
92 [no tasks queued]
93 INIT_TASK_GROUP: ffffffff81e1a780 CFS_RQ: ffff880028216028
94 [120] PID: 2948 TASK: ffff88022af2a100 COMMAND: "bash" [CURRENT]
95 TASK_GROUP: ffff88012b880800 CFS_RQ: ffff88012c5d1000 <libvirt>
96 TASK_GROUP: ffff88012c078000 CFS_RQ: ffff88012c663e00 <qemu>
97 TASK_GROUP: ffff88022c7f4c00 CFS_RQ: ffff88012bb56000 <guest2>
98 TASK_GROUP: ffff88022b621400 CFS_RQ: ffff88012b012000 <vcpu0>
99 [120] PID: 3248 TASK: ffff88012a9d4100 COMMAND: "qemu-kvm"
36. search - search memory
36.1. SYNOPSIS
1search [-s start] [ -[kKV] | -u | -p | -t | -T ] [-e end | -l length] [-m mask]
2 [-x count] -[cwh] [value | (expression) | symbol | string] ...
36.2. DESCRIPTION
1 This command searches for a given value within a range of user virtual, kernel
2 virtual, or physical memory space. If no end nor length value is entered,
3 then the search stops at the end of user virtual, kernel virtual, or physical
4 address space, whichever is appropriate.
5
6 An optional mask value may be entered to mask off "don't care" bits.
7
8 -s start Start the search at this hexadecimal user or kernel virtual
9 address, physical address, or kernel symbol. The start address
10 must be appropriate for the memory type specified; if no memory
11 type is specified, the default is kernel virtual address space.
12 -k If no start address is specified, start the search at the base
13 of kernel virtual address space. This option is the default.
14 -K Same as -k, except that mapped kernel virtual memory that was
15 allocated by vmalloc(), module memory, or virtual mem_map regions
16 will not be searched.
17 -V Same as -k, except that unity-mapped kernel virtual memory and
18 mapped kernel-text/static-data (x86_64 and ia64) will not be
19 searched.
20 -u If no start address is specified, start the search at the base
21 of the current context's user virtual address space. If a start
22 address is specified, then this option specifies that the start
23 address is a user virtual address.
24 -p If no start address is specified, start the search at the base
25 of physical address space. If a start address is specified,
26 then this option specifies that the start address is a physical
27 address.
28 -t Search only the kernel stack pages of every task. If one or more
29 matches are found in a task's kernel stack, precede the output
30 with a task-identifying header.
31 -T Same as -t, except only the active task(s) are considered.
32 -e end Stop the search at this hexadecimal user or kernel virtual
33 address, kernel symbol, or physical address. The end address
34 must be appropriate for the memory type specified.
35 -l length Length in bytes of address range to search.
36 -m mask Ignore the bits that are set in the hexadecimal mask value.
37 -c Search for character string values instead of unsigned longs. If
38 the string contains any space(s), it must be encompassed by double
39 quotes.
40 -w Search for unsigned hexadecimal ints instead of unsigned longs.
41 This is only meaningful on 64-bit systems in order to search both
42 the upper and lower 32-bits of each 64-bit long for the value.
43 -h Search for unsigned hexadecimal shorts instead of unsigned longs.
44 -x count Display the memory contents before and after any found value. The
45 before and after memory context will consist of "count" memory
46 items of the same size as the "value" argument. This option is
47 not applicable with the -c option.
48 value Search for this hexadecimal long, unless modified by the -c, -w,
49 or -h options.
50(expression) Search for the value of this expression; the expression value must
51 not overflow the designated size when -h or -w are used; not
52 applicable when used with the -c option.
53 symbol Search for this symbol value; the symbol value must not overflow
54 the designated size when -h or -w are used; not applicable when
55 used with the -c option.
56 string Search for character string values; if the string contains any
57 space(s), it must be encompassed by double quotes; only applicable
58 with the -c option.
59
60 If -k, -K, -V, -u, -p or -t are not used, then the search defaults to kernel
61 virtual address space. The starting address must be long-word aligned.
62 Address ranges that start in user space and end in kernel space are not
63 accepted.
36.3. EXAMPLES
1 Search the current context's address space for all instances of 0xdeadbeef:
2
3 crash> search -u deadbeef
4 81aba5c: deadbeef
5 81abaa8: deadbeef
6 bfffc698: deadbeef
7 bffff390: deadbeef
8
9 Search all kernel memory above the kernel text space for all instances
10 of 0xabcd occurring in the lower 16-bits of each 32-bit word:
11
12 crash> search -s _etext -m ffff0000 abcd
13 c071481c: abcd
14 c0c2b0fc: 804abcd
15 c0cf5e74: 7489abcd
16 c17c0b44: c012abcd
17 c1dac730: 3dbeabcd
18 c226d0e8: ffffabcd
19 c23ed5dc: abcd
20 c3022544: 3dbeabcd
21 c3069b58: 3dbeabcd
22 c3e86e84: aabcd
23 c3e88ed0: aabcd
24 c3e8ee5c: aabcd
25 c3e9df50: aabcd
26 c3e9e930: aabcd
27 c440a778: 804abcd
28 c486eb44: 3dbeabcd
29 c578f0fc: 804abcd
30 c6394f90: 8ababcd
31 c65219f0: 3abcd
32 c661399c: abcd
33 c68514ac: 8abcd
34 c7e036bc: 3dbeabcd
35 c7e12568: 5abcd
36 c7e1256c: 5abcd
37
38 Search the 4K page at c532c000 for all instances of 0xffffffff:
39
40 crash> search -s c532c000 -l 4096 ffffffff
41 c532c33c: ffffffff
42 c532c3fc: ffffffff
43
44 Search the static kernel data area for all instances of c2d400eb:
45
46 crash> search -s _etext -e _edata c2d400eb
47 c022b550: c2d400eb
48 c022b590: c2d400eb
49 c022b670: c2d400eb
50 c022b6e0: c2d400eb
51 c022b7b0: c2d400eb
52 c022b7e0: c2d400eb
53 c022b8b0: c2d400eb
54
55 Search physical memory for all instances of 0xbabe occurring in the
56 upper 16 bits of each 32-bit word:
57
58 crash> search -p babe0000 -m ffff
59 2a1dc4: babe671e
60 2b6928: babe3de1
61 2f99ac: babe0d54
62 31843c: babe70b9
63 3ba920: babeb5d7
64 413ce4: babe7540
65 482747c: babe2600
66 48579a4: babe2600
67 4864a68: babe2600
68 ...
69
70 Search physical memory for all instances of 0xbabe occurring in the
71 upper 16 bits of each 32-bit word on a 64-bit system:
72
73 crash> search -p babe0000 -m ffff -w
74 102e248: babe1174
75 11d2f90: babe813d
76 122d3ad70: babe6b27
77 124d8cd30: babe3dc8
78 124d8eefc: babef981
79 124d8f060: babe3dc8
80 124d8f17c: babefc81
81 ...
82
83 Search kernel memory for all instances of 32-bit value 0xbabe1174
84 on a 64-bit system:
85
86 crash> search -k -w babe1174
87 ffff88000102e248: babe1174
88 ffffffff8102e248: babe1174
89
90 Search kernel memory for two strings:
91
92 crash> search -k -c "can't allocate memory" "Failure to"
93 ffff8800013ddec1: can't allocate memory for key lists..<3>%s %s: error con
94 ffff8801258be748: Failure to install fence: %d..<3>[drm:%s] *ERROR* Failed
95 ffff880125f07ec9: can't allocate memory..<3>ACPI: Invalid data..Too many d
96 ffffffff813ddec1: can't allocate memory for key lists..<3>%s %s: error con
97
98 Search the kernel stacks of all tasks for those that contain the inode
99 address ffff81002c0a3050:
100
101 crash> search -t ffff81002c0a3050
102 PID: 4876 TASK: ffff81003e9f5860 CPU: 7 COMMAND: "automount"
103 ffff8100288fbe98: ffff81002c0a3050
104
105 PID: 4880 TASK: ffff81003ce967a0 CPU: 0 COMMAND: "automount"
106 ffff81002c0fbdd8: ffff81002c0a3050
107 ffff81002c0fbe78: ffff81002c0a3050
108
109 When a kernel symbol or an (expression) is used an argument, both the
110 resultant value and the input string are displayed:
111
112 crash> search anon_inode_inode (__down_interruptible+191)
113 ffff81000222a728: ffffffff80493d60 (anon_inode_inode)
114 ffff810005a1e918: ffffffff800649d6 (__down_interruptible+191)
115 ffff810005a1e9d0: ffffffff800649d6 (__down_interruptible+191)
116 ffff810005a1eb48: ffffffff800649d6 (__down_interruptible+191)
117 ffff81000b409c60: ffffffff80493d60 (anon_inode_inode)
118 ffff81000c155b98: ffffffff80493d60 (anon_inode_inode)
119 ffff8100194fac70: ffffffff80493d60 (anon_inode_inode)
120 ffff81001daa1008: ffffffff80493d60 (anon_inode_inode)
121 ffff810028b95830: ffffffff800649d6 (__down_interruptible+191)
122 ffff81002cea0c70: ffffffff80493d60 (anon_inode_inode)
123 ffff810031327268: ffffffff80493d60 (anon_inode_inode)
124 ffff810031327270: ffffffff800649d6 (__down_interruptible+191)
125 ffff810034b1ccd0: ffffffff800649d6 (__down_interruptible+191)
126 ffff8100399565a8: ffffffff80493d60 (anon_inode_inode)
127 ffff81003a278cd0: ffffffff800649d6 (__down_interruptible+191)
128 ffff81003cc23e08: ffffffff800649d6 (__down_interruptible+191)
37. set - set a process context or internal crash variable
37.1. SYNOPSIS
1set [[-a] [pid | taskp] | [-c cpu] | -p] | [crash_variable [setting]] | -v
37.2. DESCRIPTION
1 This command either sets a new context, or gets the current context for
2 display. The context can be set by the use of:
3
4 pid a process PID.
5 taskp a hexadecimal task_struct pointer.
6 -a sets the pid or task as the active task on its cpu (dumpfiles only).
7 -c cpu sets the context to the active task on a cpu (dumpfiles only).
8 -p sets the context to the panic task, or back to the crash task on
9 a live system.
10 -v display the current state of internal crash variables.
11
12 If no argument is entered, the current context is displayed. The context
13 consists of the PID, the task pointer, the CPU, and task state. The task
14 state shows the bits found in both the task_struct state and exit_state
15 fields.
16
17 This command may also be used to set internal crash variables. If no value
18 argument is entered, the current value of the crash variable is shown. These
19 are the crash variables, acceptable arguments, and purpose:
20
21 scroll on | off controls output scrolling.
22 scroll less /usr/bin/less as the output scrolling program.
23 scroll more /bin/more as the output scrolling program.
24 scroll CRASHPAGER use CRASHPAGER environment variable as the
25 output scrolling program.
26 radix 10 | 16 sets output radix to 10 or 16.
27 refresh on | off controls internal task list refresh.
28 print_max number set maximum number of array elements to print.
29 print_array on | off if on, set gdb's printing of arrays to "pretty"
30 format, with one line per element.
31 console device-name sets debug console device.
32 debug number sets crash debug level.
33 core on | off if on, drops core when the next error message
34 is displayed.
35 hash on | off controls internal list verification.
36 silent on | off turns off initialization messages; turns off
37 crash prompt during input file execution.
38 (scrolling is turned off if silent is on)
39 edit vi | emacs set line editing mode (from .crashrc file only).
40 namelist filename name of kernel (from .crashrc file only).
41 zero_excluded on | off controls whether excluded pages, or pages that
42 are missing from an incomplete dumpfile, should
43 return zero-filled memory when read.
44 null-stop on | off if on, gdb's printing of character arrays will
45 stop at the first NULL encountered.
46 gdb on | off if on, the crash session will be run in a mode
47 where all commands will be passed directly to
48 gdb, and the command prompt will change to
49 "gdb>"; when running in this mode, native crash
50 commands may be executed by preceding them with
51 the "crash" directive.
52 scope text-addr sets the text scope for viewing the definition
53 of data structures; the "text-addr" argument
54 must be a kernel or module text address, which
55 may be expressed symbolically or as a hexadecimal
56 value.
57 offline show | hide show or hide command output that is associated
58 with offline cpus.
59 redzone on | off if on, CONFIG_SLUB object addresses displayed by
60 the kmem command will point to the SLAB_RED_ZONE
61 padding inserted at the beginning of the object.
62 error default | redirect | filename set the destination of error messages.
63 "default": error messages are always displayed
64 on the console; if the output of a command is
65 piped to an external command or redirected
66 to a file, the error messages are also sent
67 to the pipe or file.
68 "redirect": if the output of a command is piped
69 to an external command or redirected to a file,
70 error messages are only sent to the pipe or
71 file; otherwise they are displayed on the
72 console.
73 "filename": error messages are only sent to the
74 specified filename; they are not displayed on
75 the console and are not sent to a pipe or file.
76
77 Internal variables may be set in four manners:
78
79 1. entering the set command in $HOME/.crashrc.
80 2. entering the set command in .crashrc in the current directory.
81 3. executing an input file containing the set command.
82 4. during runtime with this command.
83
84 During initialization, $HOME/.crashrc is read first, followed by the
85 .crashrc file in the current directory. Set commands in the .crashrc file
86 in the current directory override those in $HOME/.crashrc. Set commands
87 entered with this command or by runtime input file override those
88 defined in either .crashrc file. Multiple set command arguments or argument
89 pairs may be entered in one command line.
37.3. EXAMPLES
1 Set the current context to task c2fe8000:
2
3 crash> set c2fe8000
4 PID: 15917
5 COMMAND: "bash"
6 TASK: c2fe8000
7 CPU: 0
8 STATE: TASK_INTERRUPTIBLE
9
10 Set the context back to the panicking task:
11
12 crash> set -p
13 PID: 698
14 COMMAND: "gen12"
15 TASK: f9d78000
16 CPU: 2
17 STATE: TASK_RUNNING (PANIC)
18
19 Turn off output scrolling:
20
21 crash> set scroll off
22 scroll: off (/usr/bin/less)
23
24 Show the current state of crash internal variables:
25
26 crash> set -v
27 scroll: on (/usr/bin/less)
28 radix: 10 (decimal)
29 refresh: on
30 print_max: 256
31 print_array: off
32 console: /dev/pts/2
33 debug: 0
34 core: off
35 hash: on
36 silent: off
37 edit: vi
38 namelist: vmlinux
39 zero_excluded: off
40 null-stop: on
41 gdb: off
42 scope: (not set)
43 offline: show
44 redzone: on
45 error: default
46
47 Show the current context:
48
49 crash> set
50 PID: 1525
51 COMMAND: "bash"
52 TASK: c1ede000
53 CPU: 0
54 STATE: TASK_INTERRUPTIBLE
38. sig - task signal handling
38.1. SYNOPSIS
1sig [[-l] | [-s sigset]] | [-g] [pid | taskp] ...
38.2. DESCRIPTION
1 This command displays signal-handling data of one or more tasks. Multiple
2 task or PID numbers may be entered; if no arguments are entered, the signal
3 handling data of the current context will be displayed. The default display
4 shows:
5
6 1. A formatted dump of the "sig" signal_struct structure referenced by
7 the task_struct. For each defined signal, it shows the sigaction
8 structure address, the signal handler, the signal sigset_t mask
9 (also expressed as a 64-bit hexadecimal value), and the flags.
10 2. Whether the task has an unblocked signal pending.
11 3. The contents of the "blocked" and "signal" sigset_t structures
12 from the task_struct/signal_struct, both of which are represented
13 as a 64-bit hexadecimal value.
14 4. For each queued signal, private and/or shared, if any, its signal
15 number and associated siginfo structure address.
16
17 The -l option lists the signal numbers and their name(s). The -s option
18 translates a 64-bit hexadecimal value representing the contents of a
19 sigset_t structure into the signal names whose bits are set.
20
21 pid a process PID.
22 taskp a hexadecimal task_struct pointer.
23 -g displays signal information for all threads in a task's
24 thread group.
25 -l displays the defined signal numbers and names.
26 -s sigset translates a 64-bit hexadecimal value representing a sigset_t
27 into a list of signal names associated with the bits set.
38.3. EXAMPLES
1 Dump the signal-handling data of PID 8970:
2
3 crash> sig 8970
4 PID: 8970 TASK: f67d8560 CPU: 1 COMMAND: "procsig"
5 SIGNAL_STRUCT: f6018680 COUNT: 1
6 SIG SIGACTION HANDLER MASK FLAGS
7 [1] f7877684 SIG_DFL 0000000000000000 0
8 [2] f7877698 SIG_DFL 0000000000000000 0
9 ...
10 [8] f7877710 SIG_DFL 0000000000000000 0
11 [9] f7877724 SIG_DFL 0000000000000000 0
12 [10] f7877738 804867a 0000000000000000 80000000 (SA_RESETHAND)
13 [11] f787774c SIG_DFL 0000000000000000 0
14 [12] f7877760 804867f 0000000000000000 10000004 (SA_SIGINFO|SA_RESTART)
15 [13] f7877774 SIG_DFL 0000000000000000 0
16 ...
17 [31] f78778dc SIG_DFL 0000000000000000 0
18 [32] f78778f0 SIG_DFL 0000000000000000 0
19 [33] f7877904 SIG_DFL 0000000000000000 0
20 [34] f7877918 804867f 0000000000000000 10000004 (SA_SIGINFO|SA_RESTART)
21 [35] f787792c SIG_DFL 0000000000000000 0
22 [36] f7877940 SIG_DFL 0000000000000000 0
23 ...
24 [58] f7877af8 SIG_DFL 0000000000000000 0
25 [59] f7877b0c SIG_DFL 0000000000000000 0
26 [60] f7877b20 SIG_DFL 0000000000000000 0
27 [61] f7877b34 SIG_DFL 0000000000000000 0
28 [62] f7877b48 SIG_DFL 0000000000000000 0
29 [63] f7877b5c SIG_DFL 0000000000000000 0
30 [64] f7877b70 804867f 0000000000000000 10000004 (SA_SIGINFO|SA_RESTART)
31 SIGPENDING: no
32 BLOCKED: 8000000200000800
33 PRIVATE_PENDING
34 SIGNAL: 0000000200000800
35 SIGQUEUE: SIG SIGINFO
36 12 f51b9c84
37 34 f51b9594
38 SHARED_PENDING
39 SIGNAL: 8000000000000800
40 SIGQUEUE: SIG SIGINFO
41 12 f51b9188
42 64 f51b9d18
43 64 f51b9500
44
45 Dump the signal-handling data for all tasks in the thread group containing
46 PID 2578:
47
48 crash> sig -g 2578
49 PID: 2387 TASK: f617d020 CPU: 0 COMMAND: "slapd"
50 SIGNAL_STRUCT: f7dede00 COUNT: 6
51 SIG SIGACTION HANDLER MASK FLAGS
52 [1] c1f60c04 a258a7 0000000000000000 10000000 (SA_RESTART)
53 [2] c1f60c18 a258a7 0000000000000000 10000000 (SA_RESTART)
54 [3] c1f60c2c SIG_DFL 0000000000000000 0
55 [4] c1f60c40 SIG_DFL 0000000000000000 0
56 [5] c1f60c54 a258a7 0000000000000000 10000000 (SA_RESTART)
57 [6] c1f60c68 SIG_DFL 0000000000000000 0
58 [7] c1f60c7c SIG_DFL 0000000000000000 0
59 [8] c1f60c90 SIG_DFL 0000000000000000 0
60 [9] c1f60ca4 SIG_DFL 0000000000000000 0
61 [10] c1f60cb8 a25911 0000000000000000 10000000 (SA_RESTART)
62 ...
63 [64] c1f610f0 SIG_DFL 0000000000000000 0
64 SHARED_PENDING
65 SIGNAL: 0000000000000000
66 SIGQUEUE: (empty)
67
68 PID: 2387 TASK: f617d020 CPU: 0 COMMAND: "slapd"
69 SIGPENDING: no
70 BLOCKED: 0000000000000000
71 PRIVATE_PENDING
72 SIGNAL: 0000000000000000
73 SIGQUEUE: (empty)
74
75 PID: 2392 TASK: f6175aa0 CPU: 0 COMMAND: "slapd"
76 SIGPENDING: no
77 BLOCKED: 0000000000000000
78 PRIVATE_PENDING
79 SIGNAL: 0000000000000000
80 SIGQUEUE: (empty)
81
82 PID: 2523 TASK: f7cd4aa0 CPU: 1 COMMAND: "slapd"
83 SIGPENDING: no
84 BLOCKED: 0000000000000000
85 PRIVATE_PENDING
86 SIGNAL: 0000000000000000
87 SIGQUEUE: (empty)
88
89 ...
90
91 Translate the sigset_t mask value, cut-and-pasted from the signal handling
92 data from signals 1 and 10 above:
93
94 crash> sig -s 800A000000000201
95 SIGHUP SIGUSR1 SIGRTMAX-14 SIGRTMAX-12 SIGRTMAX
96
97 List the signal numbers and their names:
98
99 crash> sig -l
100 [1] SIGHUP
101 [2] SIGINT
102 [3] SIGQUIT
103 [4] SIGILL
104 [5] SIGTRAP
105 [6] SIGABRT/SIGIOT
106 [7] SIGBUS
107 [8] SIGFPE
108 [9] SIGKILL
109 [10] SIGUSR1
110 [11] SIGSEGV
111 [12] SIGUSR2
112 [13] SIGPIPE
113 [14] SIGALRM
114 [15] SIGTERM
115 [16] SIGSTKFLT
116 [17] SIGCHLD/SIGCLD
117 [18] SIGCONT
118 [19] SIGSTOP
119 [20] SIGTSTP
120 [21] SIGTTIN
121 [22] SIGTTOU
122 [23] SIGURG
123 [24] SIGXCPU
124 [25] SIGXFSZ
125 [26] SIGVTALRM
126 [27] SIGPROF
127 [28] SIGWINCH
128 [29] SIGIO/SIGPOLL
129 [30] SIGPWR
130 [31] SIGSYS
131 [32] SIGRTMIN
132 [33] SIGRTMIN+1
133 [34] SIGRTMIN+2
134 [35] SIGRTMIN+3
135 [36] SIGRTMIN+4
136 [37] SIGRTMIN+5
137 [38] SIGRTMIN+6
138 [39] SIGRTMIN+7
139 [40] SIGRTMIN+8
140 [41] SIGRTMIN+9
141 [42] SIGRTMIN+10
142 [43] SIGRTMIN+11
143 [44] SIGRTMIN+12
144 [45] SIGRTMIN+13
145 [46] SIGRTMIN+14
146 [47] SIGRTMIN+15
147 [48] SIGRTMIN+16
148 [49] SIGRTMAX-15
149 [50] SIGRTMAX-14
150 [51] SIGRTMAX-13
151 [52] SIGRTMAX-12
152 [53] SIGRTMAX-11
153 [54] SIGRTMAX-10
154 [55] SIGRTMAX-9
155 [56] SIGRTMAX-8
156 [57] SIGRTMAX-7
157 [58] SIGRTMAX-6
158 [59] SIGRTMAX-5
159 [60] SIGRTMAX-4
160 [61] SIGRTMAX-3
161 [62] SIGRTMAX-2
162 [63] SIGRTMAX-1
163 [64] SIGRTMAX
39. struct - structure contents
39.1. SYNOPSIS
1struct struct_name[.member[,member]][-o][-l offset][-rfuxdp]
2 [address | symbol][:cpuspec] [count | -c count]
39.2. DESCRIPTION
1 This command displays either a structure definition, or a formatted display
2 of the contents of a structure at a specified address. When no address is
3 specified, the structure definition is shown along with the structure size.
4 A structure member may be appended to the structure name in order to limit
5 the scope of the data displayed to that particular member; when no address
6 is specified, the member's offset and definition are shown.
7
8 struct_name name of a C-code structure used by the kernel.
9 .member name of a structure member; to display multiple members of a
10 structure, use a comma-separated list of members. If any
11 member contains an embedded structure, or the member is an
12 array, the output may be restricted to just the embedded
13 structure or an array element by expressing the member argument
14 as "member.member" or "member[index]"; embedded member
15 specifications may extend beyond one level deep, by expressing
16 the member argument as "member.member.member...".
17 -o show member offsets when displaying structure definitions;
18 if used with an address or symbol argument, each member will
19 be preceded by its virtual address.
20 -l offset if the address argument is a pointer to a structure member that
21 is contained by the target data structure, typically a pointer
22 to an embedded list_head, the offset to the embedded member may
23 be entered in either of the following manners:
24 1. in "structure.member" format.
25 2. a number of bytes.
26 -r raw dump of structure data.
27 -f address argument is a dumpfile offset.
28 -u address argument is a user virtual address in the current
29 context.
30 -x override default output format with hexadecimal format.
31 -d override default output format with decimal format.
32 -p if a structure member is a pointer value, show the member's
33 data type on the output line; and on the subsequent line(s),
34 dereference the pointer, display the pointer target's symbol
35 value in brackets if appropriate, and if possible, display the
36 target data; requires an address argument.
37 address hexadecimal address of a structure; if the address points
38 to an embedded list_head structure contained within the
39 target data structure, then the "-l" option must be used.
40 symbol symbolic reference to the address of a structure.
41 :cpuspec CPU specification for a per-cpu address or symbol:
42 : CPU of the currently selected task.
43 :a[ll] all CPUs.
44 :#[-#][,...] CPU list(s), e.g. "1,3,5", "1-3",
45 or "1,3,5-7,10".
46 count count of structures to dump from an array of structures;
47 if used, this must be the last argument entered.
48 -c count "-c" is only required if "count" is not the last argument
49 entered or if a negative number is entered; if a negative
50 value is entered, the (positive) "count" structures that
51 lead up to and include the target structure will be displayed.
52
53 Structure data, sizes, and member offsets are shown in the current output
54 radix unless the -x or -d option is specified.
55
56 Please note that in the vast majority of cases, the "struct" command
57 name may be dropped; if the structure name does not conflict with any crash
58 or gdb command name, then the "struct_name[.member]" argument will be
59 recognized as a structure name, and this command automatically executed.
60 See the NOTE below.
39.3. EXAMPLES
1 Display the vm_area_struct at address c1e44f10:
2
3 crash> struct vm_area_struct c1e44f10
4 struct vm_area_struct {
5 vm_mm = 0xc2857750,
6 vm_start = 0x8048000,
7 vm_end = 0x80a5000,
8 vm_next = 0xc1e44a10,
9 vm_page_prot = {
10 pgprot = 0x25
11 },
12 vm_flags = 0x1875,
13 vm_avl_height = 0x2,
14 vm_avl_left = 0xc30fe200,
15 vm_avl_right = 0xc30fed00,
16 vm_next_share = 0x0,
17 vm_pprev_share = 0xc1e44a30,
18 vm_ops = 0xc0215ca0,
19 vm_offset = 0x0,
20 vm_file = 0xc0bfdc70,
21 vm_pte = 0
22 }
23
24 Display the definition and size of a vm_area_struct structure. This first
25 example below displays just the structure and size. The second example
26 uses the -o option to also display member offsets. Both examples were
27 run with the output radix set to 10 (decimal):
28
29 crash> struct vm_area_struct
30 struct vm_area_struct {
31 struct mm_struct *vm_mm;
32 long unsigned int vm_start;
33 long unsigned int vm_end;
34 struct vm_area_struct *vm_next;
35 pgprot_t vm_page_prot;
36 short unsigned int vm_flags;
37 short int vm_avl_height;
38 struct vm_area_struct *vm_avl_left;
39 struct vm_area_struct *vm_avl_right;
40 struct vm_area_struct *vm_next_share;
41 struct vm_area_struct **vm_pprev_share;
42 struct vm_operations_struct *vm_ops;
43 long unsigned int vm_offset;
44 struct file *vm_file;
45 long unsigned int vm_pte;
46 }
47 SIZE: 56
48
49 crash> struct vm_area_struct -o
50 struct vm_area_struct {
51 [0] struct mm_struct *vm_mm;
52 [4] long unsigned int vm_start;
53 [8] long unsigned int vm_end;
54 [12] struct vm_area_struct *vm_next;
55 [16] pgprot_t vm_page_prot;
56 [20] short unsigned int vm_flags;
57 [22] short int vm_avl_height;
58 [24] struct vm_area_struct *vm_avl_left;
59 [28] struct vm_area_struct *vm_avl_right;
60 [32] struct vm_area_struct *vm_next_share;
61 [36] struct vm_area_struct **vm_pprev_share;
62 [40] struct vm_operations_struct *vm_ops;
63 [44] long unsigned int vm_offset;
64 [48] struct file *vm_file;
65 [52] long unsigned int vm_pte;
66 }
67 SIZE: 56
68
69 Display the definition and offset of the pgd member of an mm_struct:
70
71 crash> struct mm_struct.pgd
72 struct mm_struct {
73 [80] pgd_t *pgd;
74 }
75
76 Display the pgd member of the mm_struct at address ffff810022e7d080:
77
78 crash> struct mm_struct.pgd ffff810022e7d080
79 pgd = 0xffff81000e3ac000
80
81 Display the pgd_t pointed to by the mm_struct.pgd pointer above, forcing
82 the output to be expressed in hexadecimal:
83
84 crash> mm_struct.pgd ffff810022e7d080 -px
85 pgd_t *pgd = 0xffff81000e3ac000
86 -> {
87 pgd = 0x2c0a6067
88 }
89
90 Display the thread_info structure pointed to by the thread_info
91 member of the task_struct at ffff8100181190c0:
92
93 crash> task_struct.thread_info ffff8100181190c0 -p
94 struct thread_info *thread_info = 0xffff810023c06000
95 -> {
96 task = 0xffff8100181190c0,
97 exec_domain = 0xffffffff802f78e0,
98 flags = 128,
99 status = 1,
100 cpu = 3,
101 preempt_count = 0,
102 addr_limit = {
103 seg = 18446604435732824064
104 },
105 restart_block = {
106 fn = 0xffffffff80095a52 <do_no_restart_syscall>,
107 arg0 = 0,
108 arg1 = 0,
109 arg2 = 0,
110 arg3 = 0
111 }
112 }
113
114 Display the flags and virtual members of 4 contigous page structures
115 in the mem_map page structure array:
116
117 crash> page.flags,virtual c101196c 4
118 flags = 0x8000,
119 virtual = 0xc04b0000
120
121 flags = 0x8000,
122 virtual = 0xc04b1000
123
124 flags = 0x8000,
125 virtual = 0xc04b2000
126
127 flags = 0x8000,
128 virtual = 0xc04b3000
129
130 Display the array of tcp_sl_timer structures declared by tcp_slt_array[]:
131
132 crash> struct tcp_sl_timer tcp_slt_array 4
133 struct tcp_sl_timer {
134 count = {
135 counter = 0x0
136 },
137 period = 0x32,
138 last = 0x1419e4,
139 handler = 0xc0164854 <tcp_syn_recv_timer>
140 }
141 struct tcp_sl_timer {
142 count = {
143 counter = 0x2
144 },
145 period = 0x753,
146 last = 0x14a6df,
147 handler = 0xc01645b0 <tcp_keepalive>
148 }
149 struct tcp_sl_timer {
150 count = {
151 counter = 0x0
152 },
153 period = 0x2ee,
154 last = 0x143134,
155 handler = 0xc016447c <tcp_twkill>
156 }
157 struct tcp_sl_timer {
158 count = {
159 counter = 0x0
160 },
161 period = 0x64,
162 last = 0x143198,
163 handler = 0xc0164404 <tcp_bucketgc>
164 }
165
166 Without using the "struct" command name, display the the "d_child"
167 list_head member from a dentry structure:
168
169 crash> dentry.d_child 0xe813cb4
170 d_child = {
171 next = 0x3661344,
172 prev = 0xdea4bc4
173 },
174
175 Display the child dentry structure referenced by the "next" pointer above.
176 Since the "next" address of 0x3661344 above is a pointer to an embedded
177 list_head structure within the child dentry structure, the -l option
178 is required:
179
180 crash> dentry -l dentry.d_child 0x3661344
181 struct dentry {
182 d_count = {
183 counter = 1
184 },
185 d_flags = 0,
186 d_inode = 0xf9aa604,
187 d_parent = 0x11152b1c,
188 d_hash = {
189 next = 0x11fb3fc0,
190 prev = 0x11fb3fc0
191 },
192 d_lru = {
193 next = 0x366133c,
194 prev = 0x366133c
195 },
196 d_child = {
197 next = 0x36613cc,
198 prev = 0xe813cd4
199 },
200 d_subdirs = {
201 next = 0x366134c,
202 prev = 0x366134c
203 },
204 d_alias = {
205 next = 0xf9aa614,
206 prev = 0xf9aa614
207 },
208 d_mounted = 0,
209 d_name = {
210 name = 0x3661384 "boot.log",
211 len = 8,
212 hash = 1935169207
213 },
214 d_time = 1515870810,
215 d_op = 0x0,
216 d_sb = 0x11fc9c00,
217 d_vfs_flags = 0,
218 d_fsdata = 0x0,
219 d_extra_attributes = 0x0,
220 d_iname = "boot.log\000"
221 }
222
223 Display the virtual address of each member of the task_struct at
224 ffff8100145d2080:
225
226 crash> task_struct -o ffff8100145d2080
227 struct task_struct {
228 [ffff8100145d2080] volatile long int state;
229 [ffff8100145d2088] struct thread_info *thread_info;
230 [ffff8100145d2090] atomic_t usage;
231 [ffff8100145d2098] long unsigned int flags;
232 [ffff8100145d20a0] int lock_depth;
233 [ffff8100145d20a4] int load_weight;
234 [ffff8100145d20a8] int prio;
235 [ffff8100145d20ac] int static_prio;
236 [ffff8100145d20b0] int normal_prio;
237 [ffff8100145d20b8] struct list_head run_list;
238 [ffff8100145d20c8] struct prio_array *array;
239 ...
240
241 Display the embedded sched_entity structure's on_rq member and
242 the third pid_link structure in the embedded pids[] array of the
243 task_struct at ffff88011653e250:
244
245 crash> task_struct.se.on_rq,pids[2] ffff88011653e250
246 se.on_rq = 1,
247 pids[2] = {
248 node = {
249 next = 0xffff88011653aff0,
250 pprev = 0xffff88011653a860
251 },
252 pid = 0xffff88010d07ed00
253 }
254
255 For an example of displaying per-cpu variables, consider the
256 struct hd_struct.dkstats member, which is a percpu pointer to
257 a disk_stats structure:
258
259 crash> struct hd_struct.dkstats
260 struct hd_struct {
261 [1232] struct disk_stats *dkstats;
262 }
263
264 Taking an hd_struct at address ffff8802450e2848, display all
265 of the per-cpu disk_stats structures that it references:
266
267 crash> struct hd_struct.dkstats ffff8802450e2848
268 dkstats = 0x60fdb48026c8
269 crash> struct disk_stats 0x60fdb48026c8:a
270 [0]: ffffe8fefe6026c8
271 struct disk_stats {
272 sectors = {451376, 80468},
273 ios = {6041, 971},
274 merges = {386, 390},
275 ticks = {194877, 56131},
276 io_ticks = 12371,
277 time_in_queue = 309163
278 }
279 [1]: ffffe8fefe8026c8
280 struct disk_stats {
281 sectors = {0, 0},
282 ios = {0, 0},
283 merges = {7, 242},
284 ticks = {0, 0},
285 io_ticks = 23,
286 time_in_queue = 581
287 }
288 [2]: ffffe8fefea026c8
289 struct disk_stats {
290 sectors = {0, 0},
291 ios = {0, 0},
292 merges = {4, 112},
293 ticks = {0, 0},
294 io_ticks = 11,
295 time_in_queue = 305
296 }
297 [3]: ffffe8fefec026c8
298 struct disk_stats {
299 sectors = {0, 0},
300 ios = {0, 0},
301 merges = {5, 54},
302 ticks = {0, 0},
303 io_ticks = 17,
304 time_in_queue = 41
305 }
39.4. NOTE
1 If the structure name does not conflict with any crash command name, the
2 "struct" command may be dropped. Accordingly, the examples above could
3 also have been accomplished like so:
4
5 crash> vm_area_struct c1e44f10
6 crash> vm_area_struct
7 crash> vm_area_struct -o
8 crash> mm_struct.pgd ffff810022e7d080
9 crash> mm_struct.pgd
10 crash> tcp_sl_timer tcp_slt_array 4
11
12 Lastly, the short-cut "*" pointer-to command may also be used to negate
13 the need to enter the "struct" command name (enter "help *" for details).
40. swap - swap device information
40.1. SYNOPSIS
1swap
40.2. DESCRIPTION
1 This command displays information for each configured swap device.
2
3EXAMPLE
4 crash> swap
5 SWAP_INFO_STRUCT TYPE SIZE USED PCT PRI FILENAME
6 ffff880153d45f40 PARTITION 7192568k 1200580k 16% -1 /dev/dm-1
41. sym - translate a symbol to its virtual address, or vice-versa
41.1. SYNOPSIS
1sym [-l] | [-M] | [-m module] | [-p|-n] | [-q string] | [symbol | vaddr]
41.2. DESCRIPTION
1 This command translates a symbol to its virtual address, or a static
2 kernel virtual address to its symbol -- or to a symbol-plus-offset value,
3 if appropriate. Additionally, the symbol type is shown in parentheses,
4 and if the symbol is a known text value, the file and line number are shown.
5
6 -l dumps all symbols and their values.
7 -M dumps the current set of module symbols.
8 -m module dumps the current set of symbols for a specified module.
9 -p display the target symbol and the previous symbol.
10 -n display the target symbol and the next symbol.
11 -q string searches for all symbols containing "string".
12 symbol a kernel text or data symbol.
13 vaddr a kernel virtual address.
14
15 If the "symbol", "vaddr" or "string" argument resolves to a module
16 symbol, then the module name will be displayed in brackets following the
17 symbol value.
41.3. EXAMPLES
1 Translate data symbol jiffies to its value, and vice-versa:
2
3 crash> sym jiffies
4 c0213ae0 (D) jiffies
5
6 crash> sym c0213ae0
7 c0213ae0 (D) jiffies
8
9 Translate a text address to its symbolic value and source file:
10
11 crash> sym c0109944
12 c0109944 (T) system_call+0x34 ../linux-2.2.5/arch/i386/kernel/signal.c: 723
13
14 Dump the whole symbol table:
15
16 crash> sym -l
17 c0100000 (T) _stext
18 c0100000 (A) _text
19 c0100000 (t) startup_32
20 c0100000 (T) stext
21 c01000a4 (t) checkCPUtype
22 c0100139 (t) is486
23 c0100148 (t) is386
24 c01001b1 (t) L6
25 c01001b3 (t) ready
26 c01001b4 (t) check_x87
27 c01001da (t) setup_idt
28 c01001f7 (t) rp_sidt
29 c0100204 (T) stack_start
30 c010020c (t) int_msg
31 c0100220 (t) ignore_int
32 c0100242 (t) idt_descr
33 c0100244 (T) idt
34 c010024a (t) gdt_descr
35 c010024c (T) gdt
36 c0101000 (T) swapper_pg_dir
37 c0102000 (T) pg0
38 c0103000 (T) empty_bad_page
39 c0104000 (T) empty_bad_page_table
40 c0105000 (T) empty_zero_page
41 ...
42
43 Find all symbols containing the string "pipe":
44
45 crash> sym -q pipe
46 c010ec60 (T) sys_pipe
47 c012f660 (t) pipe_read
48 c012f7b8 (t) pipe_write
49 c012f9c0 (t) pipe_lseek
50 c012f9d0 (t) bad_pipe_r
51 c012f9dc (t) bad_pipe_w
52 c012f9e8 (t) pipe_ioctl
53 c012fa18 (t) pipe_poll
54 c012fb00 (t) pipe_release
55 c012fb48 (t) pipe_read_release
56 c012fb5c (t) pipe_write_release
57 c012fb70 (t) pipe_rdwr_release
58 c012fba0 (t) pipe_read_open
59 c012fbb0 (t) pipe_write_open
60 c012fbc0 (t) pipe_rdwr_open
61 c012fbec (t) get_pipe_inode
62 c012fcc4 (T) do_pipe
63 c023a920 (D) read_pipe_fops
64 c023a960 (D) write_pipe_fops
65 c023a9a0 (D) rdwr_pipe_fops
66 c023a9e0 (D) pipe_inode_operations
67
68 Dump the symbols of the uart401 module, both before, and then after,
69 the complete set of symbols are loaded with the "mod -s" command:
70
71 crash> sym -m uart401
72 c8032000 MODULE START: uart401
73 c8032138 (?) uart401intr
74 c803235c (?) attach_uart401
75 c8032638 (?) probe_uart401
76 c80326d4 (?) unload_uart401
77 c8033770 MODULE END: uart401
78 crash> mod -s uart401
79 MODULE NAME SIZE OBJECT FILE
80 c8032000 uart401 6000 /lib/modules/2.2.14/misc/uart401.o
81 crash> sym -m uart401
82 c8032000 MODULE START: uart401
83 c8032050 (t) my_notifier_call
84 c8032084 (t) uart401_status
85 c8032098 (t) uart401_cmd
86 c80320a8 (t) uart401_read
87 c80320bc (t) uart401_write
88 c80320cc (t) uart401_input_loop
89 c8032138 (T) uart401intr
90 c8032168 (t) uart401_open
91 c80321c8 (t) uart401_close
92 c80321f4 (t) uart401_out
93 c80322ac (t) uart401_start_read
94 c80322b4 (t) uart401_end_read
95 c80322bc (t) uart401_kick
96 c80322c4 (t) uart401_buffer_status
97 c80322cc (t) enter_uart_mode
98 c803235c (T) attach_uart401
99 c803259c (t) reset_uart401
100 c8032638 (T) probe_uart401
101 c80326d4 (T) unload_uart401
102 c8032760 (T) init_module
103 c80327cc (T) cleanup_module
104 c8032b00 (d) sound_notifier
105 c8032b0c (d) detected_devc
106 c8032b20 (d) std_synth_info
107 c8032bc0 (d) std_midi_synth
108 c8033600 (d) uart401_operations
109 c80336c4 (D) io
110 c80336c8 (D) irq
111 c80336e0 (b) hw_info.508
112 c8033770 MODULE END: uart401
113
114 Display the value of jiffies, along with the next and previous symbols:
115
116 crash> sym -np jiffies
117 c023027c (D) prof_shift
118 c0230280 (D) jiffies
119 c02302a0 (D) task
120
121 Translate a symbol value to its name and module:
122
123 crash> sym f88878d1
124 f88878d1 (t) ext3_readdir [ext3]
125 crash>
42. sys - system data
42.1. SYNOPSIS
1sys [-c [name|number]] [-t] [-i] config
42.2. DESCRIPTION
1 This command displays system-specific data. If no arguments are entered,
2 the same system data shown during crash invocation is shown.
3
4 -c [name|number] If no name or number argument is entered, dump all
5 sys_call_table entries. If a name string is entered,
6 search the table for all entries containing the string.
7 If a number is entered, the table entry associated with
8 that number is displayed. If the current output radix
9 has been set to 16, the system call numbers will be
10 displayed in hexadecimal.
11 config If the kernel was configured with CONFIG_IKCONFIG, then
12 dump the in-kernel configuration data.
13 -t Display kernel taint information. If the "tainted_mask"
14 symbol exists, show its hexadecimal value and translate
15 each bit set to the symbolic letter of the taint type.
16 On older kernels with the "tainted" symbol, only its
17 hexadecimal value is shown. The relevant kernel sources
18 should be consulted for the meaning of the letter(s) or
19 hexadecimal bit value(s).
20 -panic Panic a live system. Requires write permission to
21 /dev/mem. Results in the crash context causing an
22 "Attempted to kill the idle task!" panic. (The dump
23 will indicate that the crash context has a PID of 0).
24 -i Dump the DMI string data if available in the kernel.
42.3. EXAMPLES
1 Display essential system information:
2
3 crash> sys
4 KERNEL: vmlinux.4
5 DUMPFILE: lcore.cr.4
6 CPUS: 4
7 DATE: Mon Oct 11 18:48:55 1999
8 UPTIME: 10 days, 14:14:39
9 LOAD AVERAGE: 0.74, 0.23, 0.08
10 TASKS: 77
11 NODENAME: test.mclinux.com
12 RELEASE: 2.2.5-15smp
13 VERSION: #24 SMP Mon Oct 11 17:41:40 CDT 1999
14 MACHINE: i686 (500 MHz)
15 MEMORY: 1 GB
16
17 Dump the system configuration data (if CONFIG_IKCONFIG):
18
19 crash> sys config
20 #
21 # Automatically generated make config: don't edit
22 # Linux kernel version: 2.6.16
23 # Mon Apr 10 07:58:06 2006
24 #
25 CONFIG_X86_64=y
26 CONFIG_64BIT=y
27 CONFIG_X86=y
28 CONFIG_SEMAPHORE_SLEEPERS=y
29 CONFIG_MMU=y
30 CONFIG_RWSEM_GENERIC_SPINLOCK=y
31 CONFIG_GENERIC_CALIBRATE_DELAY=y
32 CONFIG_X86_CMPXCHG=y
33 CONFIG_EARLY_PRINTK=y
34 CONFIG_GENERIC_ISA_DMA=y
35 CONFIG_GENERIC_IOMAP=y
36 CONFIG_ARCH_MAY_HAVE_PC_FDC=y
37 CONFIG_DMI=y
38 ...
39
40 Display the kernel taint information, in this case where both the
41 TAINT_WARN and TAINT_PROPRIETARY_MODULE bits have been set:
42
43 crash> sys -t
44 TAINTED_MASK: 201 PW
45
46 Dump the system call table:
47
48 crash> sys -c
49 NUM SYSTEM CALL FILE AND LINE NUMBER
50 0 sys_ni_syscall ../kernel/sys.c: 48
51 1 sys_exit ../kernel/exit.c: 404
52 2 sys_fork ../arch/i386/kernel/process.c: 771
53 3 sys_read ../fs/read_write.c: 117
54 4 sys_write ../fs/read_write.c: 146
55 5 sys_open ../fs/open.c: 754
56 6 sys_close ../fs/open.c: 839
57 7 sys_waitpid ../kernel/exit.c: 503
58 8 sys_creat ../fs/open.c: 789
59 9 sys_link ../fs/namei.c: 1213
60 10 sys_unlink ../fs/namei.c: 1074
61 11 sys_execve ../arch/i386/kernel/process.c: 806
62 ...
63
64 Find the system call number of the select system call:
65
66 crash> sys -c select
67 NUM SYSTEM CALL FILE AND LINE NUMBER
68 65 sys_select ../fs/select.c: 259
69
70 If the current output radix has been set to 16, the system call numbers
71 will be displayed in hexadecimal.
72
73 Dump the DMI string data:
74
75 crash> sys -i
76 DMI_BIOS_VENDOR: LENOVO
77 DMI_BIOS_VERSION: G4ET37WW (1.12 )
78 DMI_BIOS_DATE: 05/29/2012
79 DMI_SYS_VENDOR: LENOVO
80 DMI_PRODUCT_NAME: 2429BQ1
81 DMI_PRODUCT_VERSION: ThinkPad T530
82 DMI_PRODUCT_SERIAL: R9R91HZ
83 DMI_PRODUCT_UUID: 568DFA01-5180-11CB-B851-BD06085ADDB0
84 DMI_BOARD_VENDOR: LENOVO
85 DMI_BOARD_NAME: 2429BQ1
86 DMI_BOARD_VERSION: Not Available
87 DMI_BOARD_SERIAL: 1ZLV127F17M
88 DMI_BOARD_ASSET_TAG: Not Available
89 DMI_CHASSIS_VENDOR: LENOVO
90 DMI_CHASSIS_TYPE: 10
91 DMI_CHASSIS_VERSION: Not Available
92 DMI_CHASSIS_SERIAL: R9R91HZ
93 DMI_CHASSIS_ASSET_TAG: RH0004111
43. task - task_struct and thread_info contents
43.1. SYNOPSIS
1task [-R member[,member]] [-dx] [pid | taskp] ...
43.2. DESCRIPTION
1 This command dumps a formatted display of the contents of a task's
2 task_struct and thread_info structures. Multiple task or PID numbers
3 may be entered; if no arguments are entered, the task_struct and
4 thread_info structures of the current context are displayed. The -R option,
5 which may also be invoked indirectly via "foreach task", pares the output
6 down to one or more structure members.
7
8 pid a process PID.
9 taskp a hexadecimal task_struct pointer.
10 -R member a comma-separated list of one or more task_struct and/or
11 thread_info structure members. If any member contains an embedded
12 structure, or is an array, the output may be restricted to the
13 embedded structure or an array element by expressing the member
14 argument as "member.member" or "member[index]"; embedded member
15 specifications may extend beyond one level deep, by expressing the
16 member argument as "member.member.member...".
17 -x override default output format with hexadecimal format.
18 -d override default output format with decimal format.
43.3. EXAMPLES
1 Dump the task_struct and thread_info structures of the current context
2 in hexadecimal format:
3
4 crash> task -x
5 PID: 3176 TASK: f2451550 CPU: 1 COMMAND: "memtest"
6 struct task_struct {
7 state = 0x0,
8 stack = 0xf05b6000,
9 usage = {
10 counter = 0x2
11 },
12 flags = 0x402040,
13 ptrace = 0x0,
14 lock_depth = 0xffffffff,
15 prio = 0x78,
16 static_prio = 0x78,
17 normal_prio = 0x78,
18 rt_priority = 0x0,
19 ...
20 perf_event_ctxp = {0x0, 0x0},
21 memcg_batch = {
22 do_batch = 0x0,
23 memcg = 0x0,
24 bytes = 0x0,
25 memsw_bytes = 0x0
26 }
27 }
28
29 struct thread_info {
30 task = 0xf2451550,
31 exec_domain = 0xc0a60860,
32 flags = 0x88,
33 status = 0x0,
34 cpu = 0x1,
35 preempt_count = 0x4010000,
36 addr_limit = {
37 seg = 0xc0000000
38 },
39 restart_block = {
40 ...
41
42 Display the ngroups and groups task_struct members for PID 2958:
43
44 crash> task -R ngroups,groups 2958
45 PID: 2958 TASK: c6718000 CPU: 0 COMMAND: "bash"
46 ngroups = 6,
47 groups = {504, 8, 9, 1000, 1007, 1006, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
48 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0},
49
50 Display the embedded sched_entity structure's on_rq member:
51
52 crash> task -R se.on_rq
53 PID: 6529 TASK: ffff880116538790 CPU: 1 COMMAND: "bash"
54 se.on_rq = 1,
55
56 Display the 3rd pid_link structure in the embedded pids[] array:
57
58 crash> task -R pids[2]
59 PID: 6529 TASK: ffff880116538790 CPU: 0 COMMAND: "bash"
60 pids[2] = {
61 node = {
62 next = 0xffff8801165391b0,
63 pprev = 0xffff880209d011b0
64 },
65 pid = 0xffff8801f0876e00
66 }
67
68 NOTE: When this command is invoked directly (i.e., not from "foreach"), it
69 is not necessary to include the "-R" before the task_struct/thread_info
70 member name(s).
44. tree - display radix tree, XArray or red-black tree
44.1. SYNOPSIS
1tree [-t [radix|xarray|rbtree]] [-r offset] [-[s|S] struct[.member[,member]]]
2 -[x|d] [-o offset] [-l] [-p] [-N] start
44.2. DESCRIPTION
1 This command dumps the contents of a radix tree, an XAarray, or a red-black
2 tree. The arguments are as follows:
3
4 -t type The type of tree to dump; the type string can be one of
5 "radix", "rbtree", or "xarray", or alternatively, "ra",
6 "rb" or "x" are acceptable. If not specified, rbtree is the
7 default type.
8 -r offset If the "start" argument is the address of a data structure that
9 contains an radix_tree_root, xarray or rb_root structure, then this
10 is the offset to that structure member. If the offset is non-zero,
11 then this option is required. The offset may be entered in either
12 of two manners:
13 1. In "structure.member" format.
14 2. A number of bytes.
15 -o offset For red-black trees only, the offset of the rb_node within its
16 containing data structure; if the offset is non-zero, then this
17 option is required. The offset may be entered in either of two
18 manners:
19 1. In "structure.member" format.
20 2. A number of bytes.
21 This option is not applicable to radix trees.
22 -s struct For each entry in a tree, format and print it as this type of data
23 structure; use the "struct.member" format in order to display a
24 particular member of the structure. To display multiple members
25 of a structure, use a comma-separated list of members. If any
26 structure member contains an embedded structure or is an array, the
27 the output may be restricted to the embedded structure or an array
28 element by expressing the member argument as "struct.member.member"
29 or "struct.member[index]"; embedded member specifications may
30 extend beyond one level deep by expressing the struct argument as
31 "struct.member.member.member...".
32 -S struct Similar to -s, but instead of parsing gdb output, member values
33 are read directly from memory, so the command works much faster
34 for 1-, 2-, 4-, and 8-byte members.
35 -l For red-black trees, dump the tree sorted in linear order starting
36 with the leftmost node and progressing to the right. This option
37 does not apply to radix trees.
38 -p Display the node's position information, showing the relationship
39 between it and the root. For red-black trees, a position that
40 indicates "root/l/r" means that the node is the right child
41 of the left child of the root node. For radix trees and xarrays,
42 the index, the height, and the slot index values are shown with
43 respect to the root.
44 -x Override default output format with hexadecimal format.
45 -d Override default output format with decimal format.
46
47 The meaning of the "start" argument, which can be expressed either in
48 hexadecimal format or symbolically, depends upon whether the -N option
49 is prepended:
50
51 start The address of a radix_tree_root, xarray or rb_root structure, or
52 the address of a structure containing the radix_tree_root, xarray
53 or rb_root structure; if the latter, then the "-r offset" option
54 must be used if the member offset of the root structure is
55 non-zero.
56
57 -N start The address of a radix_tree_node, xa_node or rb_node structure,
58 bypassing the radix_tree_root, xarray, or rb_root that points
59 to it.
44.3. EXAMPLES
1 The vmap_area_root is a standalone rb_root structure. Display the
2 virtual addresses of each vmap_area in its red-black tree:
3
4 crash> whatis vmap_area_root
5 struct rb_root vmap_area_root;
6 crash> tree -t rbtree -o vmap_area.rb_node vmap_area_root
7 ffff880128c508c0
8 ffff88012cb68140
9 ffff88012c9afec0
10 ffff88012d65c440
11 ...
12
13 Display the vmap_area's va_start and va_end members of each of
14 the entries above expressing the vmap_area.rb_node offset as a
15 number of bytes:
16
17 crash> tree -t rbtree -o 24 vmap_area_root -s vmap_area.va_start,va_end
18 ffff880128c508c0
19 va_start = 0xffffc90014900000
20 va_end = 0xffffc90014921000
21 ffff88012cb68140
22 va_start = 0xffffc900110c0000
23 va_end = 0xffffc900110d1000
24 ffff88012c9afec0
25 va_start = 0xffffc90000640000
26 va_end = 0xffffc90000642000
27 ffff88012d65c440
28 va_start = 0xffffc90000620000
29 va_end = 0xffffc90000622000
30 ...
31
32 Alternatively, use the -N option with the rb_node address contained
33 in the vmap_area_root structure:
34
35 crash> p vmap_area_root
36 vmap_area_root = $8 = {
37 rb_node = 0xffff880128c508d8
38 }
39 crash> tree -t rbtree -o vmap_area.rb_node -N 0xffff880128c508d8
40 ffff880128c508c0
41 ffff88012cb68140
42 ffff88012c9afec0
43 ffff88012d65c440
44
45 Display the virtual address of each vm_area_struct in the red-black
46 tree that has its root inside an mm_struct located at ffff880128b5a300.
47 The vm_area_struct.vm_rb rb_node member has an offset of 0x38 bytes:
48
49 crash> tree -t rbtree -r mm_struct.mm_rb ffff880128b5a300 -o 0x38
50 ffff88012a0de080
51 ffff880123e3ac78
52 ffff880123e3a700
53 ffff88012b2837c8
54 ...
55 ffff880128c02ed0
56 ffff8801292e7958
57 ffff880123e3a318
58 ffff880123e3ad40
59
60 Add the -p option to the command above to show position information:
61
62 crash> tree -t rbtree -r mm_struct.mm_rb ffff880128b5a300 -o 0x38 -p
63 ffff88012a0de080
64 position: root
65 ffff880123e3ac78
66 position: root/l
67 ffff880123e3a700
68 position: root/l/l
69 ffff88012b2837c8
70 position: root/l/l/l
71 ...
72 ffff880128c02ed0
73 position: root/r/r/l/r
74 ffff8801292e7958
75 position: root/r/r/l/r/r
76 ffff880123e3a318
77 position: root/r/r/r
78 ffff880123e3ad40
79 position: root/r/r/r/r
80
81 Given an mm_struct address of 0xffff880074b5be80, list the VMA tree in linear
82 order from the leftmost node progressing to the right using the -l option:
83
84 crash> tree -ls vm_area_struct.vm_start -o vm_area_struct.vm_rb \
85 -r mm_struct.mm_rb 0xffff880074b5be80 | paste - -
86 ffff88001f2c50e0 vm_start = 0x400000
87 ffff88001f2c5290 vm_start = 0xceb000
88 ffff880074bfc6c0 vm_start = 0xcec000
89 ffff88001f2c4bd0 vm_start = 0xd10000
90 ffff880074bfc948 vm_start = 0x1fe9000
91 ffff880036e54510 vm_start = 0x7ff6aa296000
92 ffff88001f2c5bd8 vm_start = 0x7ff6aa298000
93 ffff880036e54af8 vm_start = 0x7ff6aa497000
94 ffff880036e54f30 vm_start = 0x7ff6aa498000
95 ffff88000e06aa20 vm_start = 0x7ff6aa499000
96 ffff88000e06b368 vm_start = 0x7ff6ab95f000
97 ...
98 ffff88001f2c5e60 vm_start = 0x7ff6bc1af000
99 ffff88001f2c4ca8 vm_start = 0x7ff6bc1b6000
100 ffff88001f2c5008 vm_start = 0x7ff6bc200000
101 ffff88001f2c5d88 vm_start = 0x7ff6bc205000
102 ffff880074bfd6c8 vm_start = 0x7ff6bc206000
103 ffff88001f2c4288 vm_start = 0x7ff6bc207000
104 ffff88001f2c4510 vm_start = 0x7ffc7a5fc000
105 ffff88001f2c5b00 vm_start = 0x7ffc7a6d1000
106
107 Compared to the top/down root/leaves order:
108
109 crash> tree -s vm_area_struct.vm_start -o vm_area_struct.vm_rb \
110 -r mm_struct.mm_rb 0xffff880074b5be80 | paste - -
111 ffff88001f2c5a28 vm_start = 0x7ff6bbbb9000
112 ffff88001f2c55f0 vm_start = 0x7ff6bb252000
113 ffff88000e06a360 vm_start = 0x7ff6ac6c3000
114 ffff88001f2c4bd0 vm_start = 0xd10000
115 ffff88001f2c5290 vm_start = 0xceb000
116 ffff88001f2c50e0 vm_start = 0x400000
117 ffff880074bfc6c0 vm_start = 0xcec000
118 ffff88000e06b368 vm_start = 0x7ff6ab95f000
119 ffff88001f2c5bd8 vm_start = 0x7ff6aa298000
120 ffff880074bfc948 vm_start = 0x1fe9000
121 ffff880036e54510 vm_start = 0x7ff6aa296000
122 ffff880036e54f30 vm_start = 0x7ff6aa498000
123 ffff880036e54af8 vm_start = 0x7ff6aa497000
124 ffff88000e06aa20 vm_start = 0x7ff6aa499000
125 ffff88000e06ae58 vm_start = 0x7ff6ac1df000
126 ffff88000e06ba28 vm_start = 0x7ff6abefc000
127 ffff88000e06a6c0 vm_start = 0x7ff6ac41b000
128 ffff88001f2c4000 vm_start = 0x7ff6bac75000
129 ffff88000e06bd88 vm_start = 0x7ff6b2d00000
130 ffff88000e06b440 vm_start = 0x7ff6b28de000
131 ...
132 ffff880074bfd6c8 vm_start = 0x7ff6bc206000
133 ffff88001f2c4510 vm_start = 0x7ffc7a5fc000
134 ffff88001f2c5b00 vm_start = 0x7ffc7a6d1000
135
136 Display a list of the page structs in the radix tree of an address_space
137 structure located at ffff88012d364de0:
138
139 crash> tree -t radix -r address_space.page_tree ffff88012d364de0
140 ffffea00040d12c0
141 ffffea00040d9a60
142 ffffea00040d9b08
143 ffffea000407eda8
144 ffffea0004084288
145 ...
146 ffffea000407bc70
147 ffffea00040baf48
148 ffffea0004043f48
149 ffffea000407de58
150
151 Add the -p option to the command above to show position information:
152
153 crash> tree -t radix -r address_space.page_tree ffff88012d364de0 -p
154 ffffea00040d12c0
155 index: 0 position: root/0/0
156 ffffea00040d9a60
157 index: 1 position: root/0/1
158 ffffea00040d9b08
159 index: 2 position: root/0/2
160 ffffea000407eda8
161 index: 3 position: root/0/3
162 ffffea0004084288
163 index: 4 position: root/0/4
164 ...
165 ffffea000407bc70
166 index: 217 position: root/3/25
167 ffffea00040baf48
168 index: 218 position: root/3/26
169 ffffea0004043f48
170 index: 219 position: root/3/27
171 ffffea000407de58
172 index: 220 position: root/3/28
173
174 Alternatively, take the address of the radix_tree_node from the
175 radix_tree_root structure in the address_space structure above,
176 and display the tree with the -N option:
177
178 crash> struct address_space.page_tree ffff88012d364de0
179 page_tree = {
180 height = 0x2,
181 gfp_mask = 0x20,
182 rnode = 0xffff8801238add71
183 }
184 crash> tree -t radix -N 0xffff8801238add71
185 ffffea00040d12c0
186 ffffea00040d9a60
187 ffffea00040d9b08
188 ffffea000407eda8
189 ffffea0004084288
190 ffffea00040843a0
191 ...
192
193 Using the same radix tree as above, display the flags and _count
194 members of each page struct in the list, and force the output format
195 to be hexadecimal:
196
197 crash> tree -t radix -N 0xffff8801238add71 -s page.flags,_count -x
198 ffffea00040d12c0
199 flags = 0x4000000002006c
200 _count = {
201 counter = 0x7
202 }
203 ffffea00040d9a60
204 flags = 0x4000000002006c
205 _count = {
206 counter = 0x7
207 }
208 ffffea00040d9b08
209 flags = 0x4000000002006c
210 _count = {
211 counter = 0x7
212 }
213 ffffea000407eda8
214 flags = 0x4000000002006c
215 _count = {
216 counter = 0x7
217 }
218 ...
219
220 In more recent kernels, the XArray facility has replaced radix trees.
221 Display a list of the page structs in the XArray of an address_space
222 structure located at 0xffff94c235e76828, where the i_pages field is
223 an embedded xarray structure:
224
225 crash> tree -t xarray -r address_space.i_pages 0xffff94c235e76828
226 fffffcc005aa8380
227 fffffcc005cafa80
228 fffffcc005a79c80
229 fffffcc005ccad80
230 fffffcc005a72ec0
231 fffffcc005e27c00
232 fffffcc005ce3100
233 fffffcc005ff8dc0
234 fffffcc005c9a100
235 fffffcc005a49e40
236 fffffcc005c95a80
237
238 Add the -p option to the command above to show position information:
239
240 crash> tree -t xarray -r address_space.i_pages 0xffff94c235e76828 -p
241 fffffcc005aa8380
242 index: 90 position: root/1/26
243 fffffcc005cafa80
244 index: 91 position: root/1/27
245 fffffcc005a79c80
246 index: 92 position: root/1/28
247 fffffcc005ccad80
248 index: 93 position: root/1/29
249 fffffcc005a72ec0
250 index: 94 position: root/1/30
251 fffffcc005e27c00
252 index: 95 position: root/1/31
253 fffffcc005ce3100
254 index: 96 position: root/1/32
255 fffffcc005ff8dc0
256 index: 97 position: root/1/33
257 fffffcc005c9a100
258 index: 98 position: root/1/34
259 fffffcc005a49e40
260 index: 99 position: root/1/35
261 fffffcc005c95a80
262 index: 100 position: root/1/36
263
264 Alternatively, take the value found in the xa_head field from
265 the xarray structure, and display the tree with the -N option:
266
267 crash> address_space.i_pages 0xffff94c235e76828
268 i_pages = {
269 ... [ xa_lock field not shown ] ...
270 xa_flags = 1,
271 xa_head = 0xffff94c23c1566ca
272 }
273 crash> tree -t x -N 0xffff94c23c1566ca
274 fffffcc005aa8380
275 fffffcc005cafa80
276 fffffcc005a79c80
277 fffffcc005ccad80
278 fffffcc005a72ec0
279 fffffcc005e27c00
280 fffffcc005ce3100
281 fffffcc005ff8dc0
282 fffffcc005c9a100
283 fffffcc005a49e40
284 fffffcc005c95a80
285
286 Using the same xarray command as above, display the flags and _refcount
287 members of each page struct in the list, and force the output format
288 to be hexadecimal:
289
290 crash> tree -t x -N 0xffff94c23c1566ca -s page.flags,_refcount -x
291 fffffcc005aa8380
292 flags = 0x57ffffc0000014
293 _refcount = {
294 counter = 0x1
295 }
296 fffffcc005cafa80
297 flags = 0x57ffffc0000014
298 _refcount = {
299 counter = 0x1
300 }
301 fffffcc005a79c80
302 flags = 0x57ffffc0000014
303 _refcount = {
304 counter = 0x1
305 }
306 fffffcc005ccad80
307 flags = 0x57ffffc0000014
308 _refcount = {
309 counter = 0x1
310 }
311 fffffcc005a72ec0
312 flags = 0x57ffffc0000014
313 _refcount = {
314 counter = 0x1
315 }
316 fffffcc005e27c00
317 flags = 0x57ffffc0000014
318 _refcount = {
319 counter = 0x1
320 }
321 fffffcc005ce3100
322 flags = 0x57ffffc0000014
323 _refcount = {
324 counter = 0x1
325 }
326 fffffcc005ff8dc0
327 flags = 0x57ffffc0000014
328 _refcount = {
329 counter = 0x1
330 }
331 fffffcc005c9a100
332 flags = 0x57ffffc0000014
333 _refcount = {
334 counter = 0x1
335 }
336 fffffcc005a49e40
337 flags = 0x57ffffc0000014
338 _refcount = {
339 counter = 0x1
340 }
341 fffffcc005c95a80
342 flags = 0x57ffffc0000014
343 _refcount = {
344 counter = 0x1
345 }
45. timer - timer queue data
45.1. SYNOPSIS
1timer [-r][-C cpu]
45.2. DESCRIPTION
1 This command displays the timer queue entries, both old- and new-style,
2 in chronological order. In the case of the old-style timers, the
3 timer_table array index is shown; in the case of the new-style timers,
4 the timer_list address is shown. On later kernels, the timer data is
5 per-cpu.
6
7 -r Display hrtimer timer queue entries, both old- and new-style, in
8 chronological order. In the case of the old-style hrtimers, the
9 expiration time is a single value; in the new-style hrtimers, the
10 expiration time is a range.
11 -C cpu Restrict the output to one or more CPUs, where multiple cpu[s] can
12 be specified, for example, as "1,3,5", "1-3", or "1,3,5-7,10".
45.3. EXAMPLES
1 Display the timer queue on an SMP system:
2
3 crash> timer
4 JIFFIES
5 4296291038
6 ...
7 TIMER_BASES[1][BASE_STD]: ffff9801aba5aa00
8 EXPIRES TTE TIMER_LIST FUNCTION
9 4296282997 -8041 ffff9801aba55ce0 ffffffff83a3bda0 <mce_timer_fn>
10 4296283104 -7934 ffff97fd84bd35e0 ffffffff83ac6b70 <delayed_work_timer_fn>
11 4296291061 23 ffffa6b283967de0 ffffffff83b29880 <process_timeout>
12 4296291112 74 ffff9800c9b62ad8 ffffffff83e6b550 <cursor_timer_handler>
13 4296291345 307 ffff980186d5ef88 ffffffff84146b80 <tcp_keepalive_timer>
14 4296291484 446 ffff9801a7c54740 ffffffff84147f50 <tcp_write_timer>
15 4296291997 959 ffffffffc073f880 ffffffff83ac6b70 <delayed_work_timer_fn>
16 4296296213 5175 ffffa6b28339be18 ffffffff83b29880 <process_timeout>
17 4296304383 13345 ffff980194ca72a8 ffffffff8412e4e0 <tw_timer_handler>
18 4296305724 14686 ffff980194ca6918 ffffffff8412e4e0 <tw_timer_handler>
19 4296306036 14998 ffff980194ca6d58 ffffffff8412e4e0 <tw_timer_handler>
20 4296306883 15845 ffff980194ca7e58 ffffffff8412e4e0 <tw_timer_handler>
21 4296307588 16550 ffff9801aaa27e58 ffffffff8412e4e0 <tw_timer_handler>
22 4296307625 16587 ffff980194ca6a28 ffffffff8412e4e0 <tw_timer_handler>
23 4296313542 22504 ffff980194ca7c38 ffffffff8412e4e0 <tw_timer_handler>
24 4296317680 26642 ffff9800c9149c58 ffffffff840da870 <neigh_timer_handler>
25 4296317744 26706 ffff9801a5354468 ffffffff83ac6b70 <delayed_work_timer_fn>
26 4296343322 52284 ffff980194ca63c8 ffffffff8412e4e0 <tw_timer_handler>
27 4296343581 52543 ffff980194ca7088 ffffffff8412e4e0 <tw_timer_handler>
28 4296343597 52559 ffff9801aaa274c8 ffffffff8412e4e0 <tw_timer_handler>
29 4296714205 423167 ffffffff84caf3c0 ffffffff83ac6b70 <delayed_work_timer_fn>
30 TIMER_BASES[1][BASE_DEF]: ffff9801aba5bc80
31 EXPIRES TTE TIMER_LIST FUNCTION
32 4296291264 226 ffffffff855eb238 ffffffff83c08fb0 <writeout_period>
33 4296319997 28959 ffffffffc06ede40 ffffffff83ac6b70 <delayed_work_timer_fn>
34 4296506084 215046 ffff9801aba629c8 ffffffff83ac5ea0 <idle_worker_timeout>
35 ...
36
37 Display a new-style hrtimer queue:
38
39 crash> timer -r
40 ...
41 CPU: 2 HRTIMER_CPU_BASE: ffff9801aba9cf00
42 CLOCK: 0 HRTIMER_CLOCK_BASE: ffff9801aba9cf40 [ktime_get]
43 CURRENT
44 1623742000000
45 SOFTEXPIRES EXPIRES TTE HRTIMER FUNCTION
46 1623741000000 1623741000000 -1000000 ffff9801aba9d540 ffffffff83b3c8e0 <tick_sched_timer>
47 1624024000000 1624024000000 282000000 ffff9801aba9d720 ffffffff83b7e7a0 <watchdog_timer_fn>
48 1626000939806 1626010929804 2268929804 ffffa6b28399fa40 ffffffff83b2c1e0 <hrtimer_wakeup>
49 1627576915615 1627576915615 3834915615 ffff9801a5727978 ffffffff83b365c0 <posix_timer_fn>
50 1627637194488 1627647194487 3905194487 ffffa6b283977db0 ffffffff83b2c1e0 <hrtimer_wakeup>
51 1629937423000 1629937423000 6195423000 ffff9801a9af2900 ffffffff83cf3d30 <timerfd_tmrproc>
52
53 CLOCK: 1 HRTIMER_CLOCK_BASE: ffff9801aba9cf80 [ktime_get_real]
54 CURRENT
55 1558362388334558243
56 SOFTEXPIRES EXPIRES TTE HRTIMER FUNCTION
57 1558362389331238000 1558362389331288000 996729757 ffffa6b28574bcf0 ffffffff83b2c1e0 <hrtimer_wakeup>
58 1558364372000000000 1558364372000000000 1983665441757 ffff9801a3513278 ffffffff83b365c0 <posix_timer_fn>
59
60 CLOCK: 2 HRTIMER_CLOCK_BASE: ffff9801aba9cfc0 [ktime_get_boottime]
61 (empty)
62 ...
46. union - union contents
46.1. SYNOPSIS
1union union_name[.member[,member]] [-o][-l offset][-rfuxdp]
2 [address | symbol][:cpuspec] [count | -c count]
46.2. DESCRIPTION
1 This command displays either a union definition, or a formatted display
2 of the contents of a union at a specified address. When no address is
3 specified, the union definition is shown along with the union size.
4 A union member may be appended to the structure name in order to limit
5 the scope of the data displayed to that particular member; when no address
6 is specified, the member's offset (always 0) and definition are shown.
7
8 union_name name of a C-code union used by the kernel.
9 .member name of a union member; to display multiple members of a
10 union, use a comma-separated list of members. If any member
11 contains an embedded structure, or the member is an array, the
12 output may be restricted to just the embedded structure or an
13 array element by expressing the argument as "member.member"
14 or "member[index]"; embedded member specifications may extend
15 beyond one level deep, by expressing the member argument as
16 "member.member.member...".
17 -o show member offsets when displaying union definitions; the
18 offset is always 0 unless used with an address or symbol
19 argument, in which case each member will be preceded by its
20 virtual address.
21 -l offset if the address argument is a pointer to a list_head structure
22 that is embedded in the target union structure, the offset
23 to the list_head member may be entered in either of the
24 following manners:
25 1. in "structure.member" format.
26 2. a number of bytes.
27 -r raw dump of union data.
28 -f address argument is a dumpfile offset.
29 -x override default output format with hexadecimal format.
30 -d override default output format with decimal format.
31 -p if a union member is a pointer value, show the member's
32 data type on the output line; and on the subsequent line(s),
33 dereference the pointer, display the pointer target's symbol
34 value in brackets if appropriate, and if possible, display the
35 target data; requires an address argument.
36 -u address argument is a user virtual address in the current
37 context.
38 address hexadecimal address of a union; if the address points
39 to an embedded list_head structure contained within the
40 target union structure, then the "-l" option must be used.
41 symbol symbolic reference to the address of a union.
42 :cpuspec CPU specification for a per-cpu address or symbol:
43 : CPU of the currently selected task.
44 :a[ll] all CPUs.
45 :#[-#][,...] CPU list(s), e.g. "1,3,5", "1-3",
46 or "1,3,5-7,10".
47 count count of unions to dump from an array of unions; if used,
48 this must be the last argument entered.
49 -c count "-c" is only required if "count" is not the last argument
50 entered or if a negative number is entered; if a negative
51 value is entered, the (positive) "count" structures that
52 lead up to and include the target structure will be displayed.
53
54 Union data, sizes, and member offsets are shown in the current output radix
55 unless the -x or -d option is specified.
56
57 Please note that in the vast majority of cases, the "union" command
58 name may be dropped; if the union name does not conflict with any crash
59 or gdb command name, then the "union_name[.member]" argument will be
60 recognized as a union name, and this command automatically executed.
61 See the NOTE below.
46.3. EXAMPLES
1
2 Display the bdflush_param union definition, and then an instance of it:
3
4 crash> union bdflush_param
5 union bdflush_param {
6 struct {
7 int nfract;
8 int ndirty;
9 int nrefill;
10 int nref_dirt;
11 int dummy1;
12 int age_buffer;
13 int age_super;
14 int dummy2;
15 int dummy3;
16 } b_un;
17 unsigned int data[9];
18 }
19
20 SIZE: 36 (0x24)
21
22 crash> union bdflush_param bdf_prm
23 union bdflush_param {
24 b_un = {
25 nfract = 40,
26 ndirty = 500,
27 nrefill = 64,
28 nref_dirt = 256,
29 dummy1 = 15,
30 age_buffer = 3000,
31 age_super = 500,
32 dummy2 = 1884,
33 dummy3 = 2
34 },
35 data = {40, 500, 64, 256, 15, 3000, 500, 1884, 2}
36 }
46.4. NOTE
1 If the union name does not conflict with any crash command name, the
2 "union" command may be dropped. Accordingly, the examples above could
3 also have been accomplished like so:
4
5 crash> bdflush_param
6 crash> bdflush_param bdf_prm
7
8 Lastly, the short-cut "*" (pointer-to) command may also be used to negate
9 the need to enter the "union" command name (enter "help *" for details).
47. vm - virtual memory
47.1. SYNOPSIS
1vm [-p | -P vma | -M mm | -v | -m | -x | -d | [-R reference] [pid | task]]
2 [-f vm_flags]
47.2. DESCRIPTION
1 This command displays basic virtual memory information of a context,
2 consisting of a pointer to its mm_struct and page dirctory, its RSS and
3 total virtual memory size; and a list of pointers to each vm_area_struct,
4 its starting and ending address, vm_flags value, and file pathname. If no
5 arguments are entered, the current context is used. Additionally, the -p
6 option translates each virtual page of each VM area to its physical address.
7 The -R option, typically invoked from "foreach vm", searches for references
8 to a supplied number, address, or filename argument, and prints only the
9 essential information leading up to and including the reference.
10 Alternatively, the -m or -v options may be used to dump the task's mm_struct
11 or all of its vm_area_structs respectively. The -p, -v, -m, -R and -f
12 options are all mutually exclusive.
13
14 -p translate each virtual page to its physical address, or if
15 the page is not mapped, its swap device and offset, or
16 filename and offset.
17 -P vma similar to -p, but only translate the pages belonging to the
18 specified VM area of a context.
19 -M mm if the mm_struct address has been removed from the task_struct
20 of an exiting task, the virtual memory data cannot be displayed.
21 However, if the address can be determined from the kernel stack,
22 it can be entered manually in order to try to resurrect the
23 virtual memory data of the task.
24 -R reference search for references to this number or filename.
25 -m dump the mm_struct assocated with the task.
26 -v dump all of the vm_area_structs associated with the task.
27 -x override the default output format for the -m or -v options
28 with hexadecimal format.
29 -d override the default output format for the -m or -v options
30 with decimal format.
31 -f vm_flags translate the bits of a FLAGS (vm_flags) value.
32 pid a process PID.
33 task a hexadecimal task_struct pointer.
47.3. EXAMPLES
1 Display the virtual memory data of the current context:
2
3 crash> vm
4 PID: 30986 TASK: c0440000 CPU: 0 COMMAND: "bash"
5 MM PGD RSS TOTAL_VM
6 c303fe20 c4789000 88k 1728k
7 VMA START END FLAGS FILE
8 c0d1f540 8048000 80ad000 1875 /bin/bash
9 c0d1f400 80ad000 80b3000 1873 /bin/bash
10 c0d1f880 80b3000 80ec000 77
11 c0d1f0c0 40000000 40012000 875 /lib/ld-2.1.1.so
12 c0d1f700 40012000 40013000 873 /lib/ld-2.1.1.so
13 c0d1fe00 40013000 40014000 77
14 c0d1f580 40014000 40016000 73
15 c0d1f280 4001a000 4004b000 75 /usr/lib/libncurses.so.4.2
16 c0d1f100 4004b000 40054000 73 /usr/lib/libncurses.so.4.2
17 c0d1f600 40054000 40057000 73
18 c0d1f9c0 40057000 40059000 75 /lib/libdl-2.1.1.so
19 c0d1f800 40059000 4005a000 73 /lib/libdl-2.1.1.so
20 c0d1fd00 4005a000 40140000 75 /lib/libc-2.1.1.so
21 c0d1fe40 40140000 40145000 73 /lib/libc-2.1.1.so
22 c0d1f780 40145000 40148000 73
23 c0d1f140 40148000 40150000 75 /lib/libnss_files-2.1.1.so
24 c0d1fa80 40150000 40151000 73 /lib/libnss_files-2.1.1.so
25 c0d1fb00 40151000 4015a000 75 /lib/libnss_nisplus-2.1.1.so
26 c5f754e0 4015a000 4015b000 73 /lib/libnss_nisplus-2.1.1.so
27 c0d1fec0 4015b000 4016d000 75 /lib/libnsl-2.1.1.so
28 c5f75460 4016d000 4016e000 73 /lib/libnsl-2.1.1.so
29 c5f75420 4016e000 40170000 73
30 c5f753e0 40170000 40178000 75 /lib/libnss_nis-2.1.1.so
31 c5f753a0 40178000 40179000 73 /lib/libnss_nis-2.1.1.so
32 c0d1f240 bfffc000 c0000000 177
33
34 Display the virtual memory data along with page translations for PID 386:
35
36 crash> vm -p 386
37 PID: 386 TASK: c11cc000 CPU: 0 COMMAND: "atd"
38 MM PGD RSS TOTAL_VM
39 c7e30560 c10e5000 104k 1112k
40 VMA START END FLAGS FILE
41 c0fbe6a0 8048000 804b000 1875 /usr/sbin/atd
42 VIRTUAL PHYSICAL
43 8048000 20e1000
44 8049000 17c6000
45 804a000 1f6f000
46 VMA START END FLAGS FILE
47 c61e0ba0 804b000 804d000 1873 /usr/sbin/atd
48 VIRTUAL PHYSICAL
49 804b000 254d000
50 804c000 6a9c000
51 VMA START END FLAGS FILE
52 c61e04e0 804d000 8050000 77
53 VIRTUAL PHYSICAL
54 804d000 219d000
55 804e000 2617000
56 804f000 SWAP: /dev/sda8 OFFSET: 24225
57 VMA START END FLAGS FILE
58 c61e0720 40000000 40012000 875 /lib/ld-2.1.1.so
59 VIRTUAL PHYSICAL
60 40000000 FILE: /lib/ld-2.1.1.so OFFSET: 0
61 40001000 FILE: /lib/ld-2.1.1.so OFFSET: 1000
62 40002000 FILE: /lib/ld-2.1.1.so OFFSET: 2000
63 40003000 FILE: /lib/ld-2.1.1.so OFFSET: 3000
64 40004000 FILE: /lib/ld-2.1.1.so OFFSET: 4000
65 40005000 FILE: /lib/ld-2.1.1.so OFFSET: 5000
66 ...
67
68 Although the -R option is typically invoked from "foreach vm", it can be
69 executed directly. This example displays all VM areas with vm_flags of 75:
70
71 crash> vm -R 75
72 PID: 694 TASK: c0c76000 CPU: 1 COMMAND: "crash"
73 MM PGD RSS TOTAL_VM
74 c6c43110 c0fe9000 8932k 10720k
75 VMA START END FLAGS FILE
76 c322c0d0 40019000 4004a000 75 /usr/lib/libncurses.so.4.2
77 c67537c0 40056000 40071000 75 /lib/libm-2.1.1.so
78 c6753d00 40072000 40074000 75 /lib/libdl-2.1.1.so
79 c6753540 40075000 40081000 75 /usr/lib/libz.so.1.1.3
80 c6753740 40085000 4016b000 75 /lib/libc-2.1.1.so
81
82 One reason to use -R directly is to pare down the output associated with
83 the -p option on a task with a huge address space. This example displays
84 the page data associated with virtual address 40121000:
85
86 crash> vm -R 40121000
87 PID: 694 TASK: c0c76000 CPU: 0 COMMAND: "crash"
88 MM PGD RSS TOTAL_VM
89 c6c43110 c0fe9000 8928k 10720k
90 VMA START END FLAGS FILE
91 c6753740 40085000 4016b000 75 /lib/libc-2.1.1.so
92 VIRTUAL PHYSICAL
93 40121000 FILE: /lib/libc-2.1.1.so OFFSET: 9c000
94
95 Display the mm_struct for PID 4777:
96
97 crash> vm -m 4777
98 PID: 4777 TASK: c0896000 CPU: 0 COMMAND: "bash"
99 struct mm_struct {
100 mmap = 0xc6caa1c0,
101 mmap_avl = 0x0,
102 mmap_cache = 0xc6caabc0,
103 pgd = 0xc100a000,
104 count = {
105 counter = 0x1
106 },
107 map_count = 0x14,
108 mmap_sem = {
109 count = {
110 counter = 0x1
111 },
112 waking = 0x0,
113 wait = 0x0
114 },
115 context = 0x0,
116 start_code = 0x8048000,
117 end_code = 0x809c6f7,
118 start_data = 0x0,
119 end_data = 0x80a2090,
120 start_brk = 0x80a5420,
121 brk = 0x80b9000,
122 start_stack = 0xbffff9d0,
123 arg_start = 0xbffffad1,
124 arg_end = 0xbffffad7,
125 env_start = 0xbffffad7,
126 env_end = 0xbffffff2,
127 rss = 0xf6,
128 total_vm = 0x1a3,
129 locked_vm = 0x0,
130 def_flags = 0x0,
131 cpu_vm_mask = 0x0,
132 swap_cnt = 0x23d,
133 swap_address = 0x0,
134 segments = 0x0
135 }
136
137 Display all of the vm_area_structs for task c47d4000:
138
139 crash> vm -v c47d4000
140 PID: 4971 TASK: c47d4000 CPU: 1 COMMAND: "login"
141 struct vm_area_struct {
142 vm_mm = 0xc4b0d200,
143 vm_start = 0x8048000,
144 vm_end = 0x804d000,
145 vm_next = 0xc3e3abd0,
146 vm_page_prot = {
147 pgprot = 0x25
148 },
149 vm_flags = 0x1875,
150 vm_avl_height = 0x1,
151 vm_avl_left = 0x0,
152 vm_avl_right = 0x0,
153 vm_next_share = 0x0,
154 vm_pprev_share = 0xc3e3abf0,
155 vm_ops = 0xc02392a0,
156 vm_offset = 0x0,
157 vm_file = 0xc1e23660,
158 vm_pte = 0x0
159 }
160 struct vm_area_struct {
161 vm_mm = 0xc4b0d200,
162 vm_start = 0x804d000,
163 vm_end = 0x804e000,
164 vm_next = 0xc3e3a010,
165 vm_page_prot = {
166 pgprot = 0x25
167 },
168 vm_flags = 0x1873,
169 vm_avl_height = 0x2,
170 vm_avl_left = 0xc3e3a810,
171 vm_avl_right = 0xc3e3a010,
172 vm_next_share = 0xc3e3a810,
173 vm_pprev_share = 0xc3699c14
174 ...
175
176 Translate a FLAGS value:
177
178 crash> vm -f 3875
179 3875: (READ|EXEC|MAYREAD|MAYWRITE|MAYEXEC|DENYWRITE|EXECUTABLE|LOCKED)
180
181 Display the page translations of the VM area at address f5604f2c:
182
183 crash> vm -P f5604f2c
184 PID: 5508 TASK: f56a9570 CPU: 0 COMMAND: "crond"
185 VMA START END FLAGS FILE
186 f5604f2c f5b000 f67000 8000075 /lib/libnss_files-2.12.so
187 VIRTUAL PHYSICAL
188 f5b000 3fec1000
189 f5c000 3d3a4000
190 f5d000 FILE: /lib/libnss_files-2.12.so OFFSET: 2000
191 f5e000 FILE: /lib/libnss_files-2.12.so OFFSET: 3000
192 f5f000 FILE: /lib/libnss_files-2.12.so OFFSET: 4000
193 f60000 3fd31000
194 f61000 3fd32000
195 f62000 FILE: /lib/libnss_files-2.12.so OFFSET: 7000
196 f63000 FILE: /lib/libnss_files-2.12.so OFFSET: 8000
197 f64000 3ff35000
198 f65000 FILE: /lib/libnss_files-2.12.so OFFSET: a000
199 f66000 FILE: /lib/libnss_files-2.12.so OFFSET: b000
48. vtop - virtual to physical
48.1. SYNOPSIS
1vtop [-c [pid | taskp]] [-u|-k] address ...
48.2. DESCRIPTION
1 This command translates a user or kernel virtual address to its physical
2 address. Also displayed is the PTE translation, the vm_area_struct data
3 for user virtual addresses, the mem_map page data associated with the
4 physical page, and the swap location or file location if the page is
5 not mapped. The -u and -k options specify that the address is a user
6 or kernel virtual address; -u and -k are not necessary on processors whose
7 virtual addresses self-define themselves as user or kernel. User addresses
8 are translated with respect to the current context unless the -c option
9 is used. Kernel virtual addresses are translated using the swapper_pg_dir
10 as the base page directory unless the -c option is used.
11
12 -u The address is a user virtual address; only required
13 on processors with overlapping user and kernel virtual
14 address spaces.
15 -k The address is a kernel virtual address; only required
16 on processors with overlapping user and kernel virtual
17 address spaces.
18 -c [pid | taskp] Translate the virtual address from the page directory
19 of the specified PID or hexadecimal task_struct pointer.
20 However, if this command is invoked from "foreach vtop",
21 the pid or taskp argument should NOT be entered; the
22 address will be translated using the page directory of
23 each task specified by "foreach".
24 address A hexadecimal user or kernel virtual address.
48.3. EXAMPLES
1 Translate user virtual address 80b4000:
2
3 crash> vtop 80b4000
4 VIRTUAL PHYSICAL
5 80b4000 660f000
6
7 PAGE DIRECTORY: c37f0000
8 PGD: c37f0080 => e0d067
9 PMD: c37f0080 => e0d067
10 PTE: c0e0d2d0 => 660f067
11 PAGE: 660f000
12
13 PTE PHYSICAL FLAGS
14 660f067 660f000 (PRESENT|RW|USER|ACCESSED|DIRTY)
15
16 VMA START END FLAGS FILE
17 c773daa0 80b4000 810c000 77
18
19 PAGE PHYSICAL INODE OFFSET CNT FLAGS
20 c0393258 660f000 0 17000 1 uptodate
21
22 Translate kernel virtual address c806e000, first using swapper_pg_dir
23 as the page directory base, and secondly, using the page table base
24 of PID 1359:
25
26 crash> vtop c806e000
27 VIRTUAL PHYSICAL
28 c806e000 2216000
29
30 PAGE DIRECTORY: c0101000
31 PGD: c0101c80 => 94063
32 PMD: c0101c80 => 94063
33 PTE: c00941b8 => 2216063
34 PAGE: 2216000
35
36 PTE PHYSICAL FLAGS
37 2216063 2216000 (PRESENT|RW|ACCESSED|DIRTY)
38
39 PAGE PHYSICAL INODE OFFSET CNT FLAGS
40 c02e9370 2216000 0 0 1
41
42 crash> vtop -c 1359 c806e000
43 VIRTUAL PHYSICAL
44 c806e000 2216000
45
46 PAGE DIRECTORY: c5caf000
47 PGD: c5cafc80 => 94063
48 PMD: c5cafc80 => 94063
49 PTE: c00941b8 => 2216063
50 PAGE: 2216000
51
52 PTE PHYSICAL FLAGS
53 2216063 2216000 (PRESENT|RW|ACCESSED|DIRTY)
54
55 PAGE PHYSICAL INODE OFFSET CNT FLAGS
56 c02e9370 2216000 0 0 1
57
58 Determine swap location of user virtual address 40104000:
59
60 crash> vtop 40104000
61 VIRTUAL PHYSICAL
62 40104000 (not mapped)
63
64 PAGE DIRECTORY: c40d8000
65 PGD: c40d8400 => 6bbe067
66 PMD: c40d8400 => 6bbe067
67 PTE: c6bbe410 => 58bc00
68
69 PTE SWAP OFFSET
70 58bc00 /dev/sda8 22716
71
72 VMA START END FLAGS FILE
73 c7200ae0 40104000 40b08000 73
74
75 SWAP: /dev/sda8 OFFSET: 22716
49. waitq - list tasks queued on a wait queue
49.1. SYNOPSIS
1waitq [ symbol ] | [ struct.member struct_addr ] | [ address ]
49.2. DESCRIPTION
1 This command walks the wait queue list displaying the tasks which
2 are blocked on the specified wait queue. The command differentiates
3 between the old- and new-style wait queue structures used by the kernel.
4 It can be invoked with the following argument types:
5
6 symbol a global symbol of a wait queue.
7 struct.member struct_addr a structure name and wait queue member combination
8 followed by the structure's hexadecimal address.
9 address a hexadecimal wait queue pointer.
49.3. EXAMPLES
1
2 Find out if any tasks are blocked on the "buffer_wait" wait queue:
3
4 crash> waitq buffer_wait
5 wait queue "buffer_wait" (c02927f0) is empty
6
7 See who is blocked on the "wait_chldexit" queue of task c5496000:
8
9 crash> waitq task_struct.wait_chldexit c5496000
10 PID: 30879 TASK: c5496000 CPU: 0 COMMAND: "bash"
11
12 Display the task list waiting on a known task queue:
13
14 crash> waitq c3534098
15 PID: 13691 TASK: c3534000 CPU: 1 COMMAND: "bash"
50. whatis - search symbol table for data or type information
50.1. SYNOPSIS
1whatis [[-o] [struct | union | typedef | symbol]] |
2 [[-r [size|range]] [-m member]]
50.2. DESCRIPTION
1 This command displays the definition of structures, unions, typedefs or
2 text/data symbols:
3
4 struct a structure name. The output is the same as if the "struct"
5 command was used.
6 union a union name. The output is the same as if the "union" command
7 was used.
8 -o display the offsets of structure/union members.
9 typedef a typedef name. If the typedef translates to a structure or union
10 the output is the same as if the "struct" or "union" command
11 was used. If the typedef is a primitive datatype, the one-line
12 declaration is displayed.
13 symbol a kernel symbol.
14
15 Alternatively, a search can be made for data structures of a given size or
16 size range, that contain a member of a given type, or contain a pointer to
17 given type. The -r and -m options may be used alone or in conjunction with
18 one another:
19
20 -r size search for structures of this exact size.
21 -r range search for structures of a range of sizes, expressed as "low-high".
22 -m member search for structures that contain a member of this data type, or
23 that contain a pointer to this data type; if a structure contains
24 another structure, the members of the embedded structure will also
25 be subject to the search. The member argument may also be expressed
26 as a substring of a member's data type.
50.3. EXAMPLES
1 Display the definition of a linux_binfmt structure:
2
3 crash> whatis linux_binfmt
4 struct linux_binfmt {
5 struct list_head lh;
6 struct module *module;
7 int (*load_binary)(struct linux_binprm *);
8 int (*load_shlib)(struct file *);
9 int (*core_dump)(struct coredump_params *);
10 unsigned long min_coredump;
11 }
12 SIZE: 56
13
14 Display the same structure with member offsets:
15
16 crash> whatis -o linux_binfmt
17 struct linux_binfmt {
18 [0] struct list_head lh;
19 [16] struct module *module;
20 [24] int (*load_binary)(struct linux_binprm *);
21 [32] int (*load_shlib)(struct file *);
22 [40] int (*core_dump)(struct coredump_params *);
23 [48] unsigned long min_coredump;
24 }
25 SIZE: 56
26
27 Since a kmem_bufctl_t is typedef'd to be a kmem_bufctl_s structure, the
28 output of the following two commands is identical:
29
30 crash> whatis kmem_bufctl_s
31 struct kmem_bufctl_s {
32 union {
33 struct kmem_bufctl_s *buf_nextp;
34 kmem_slab_t *buf_slabp;
35 void *buf_objp;
36 } u;
37 };
38
39 crash> whatis kmem_bufctl_t
40 struct kmem_bufctl_s {
41 union {
42 struct kmem_bufctl_s *buf_nextp;
43 kmem_slab_t *buf_slabp;
44 void *buf_objp;
45 } u;
46 };
47 SIZE: 4 (0x4)
48
49 Display the type data of sys_read() and jiffies text and data symbols:
50
51 crash> whatis sys_read
52 ssize_t sys_read(unsigned int, char *, size_t);
53
54 crash> whatis jiffies
55 long unsigned int jiffies;
56
57 Display definition of a kdev_t typedef:
58
59 crash> whatis kdev_t
60 typedef short unsigned int kdev_t;
61 SIZE: 2 (0x2)
62
63 Display all structures which have a size of 192 bytes:
64
65 crash> whatis -r 192
66 SIZE TYPE
67 192 _intel_private
68 192 blkcg_gq
69 192 clock_event_device
70 192 cper_sec_proc_generic
71 192 dentry
72 192 dst_ops
73 192 ehci_itd
74 192 ethtool_rxnfc
75 192 fb_ops
76 192 file_lock
77 192 inode_operations
78 192 input_device_id
79 192 ip_vs_stats
80 192 numa_group
81 192 parallel_data
82 192 pcie_port_service_driver
83 192 pebs_record_hsw
84 192 pnp_driver
85 192 regmap_config
86 192 sched_entity
87 192 tcp_timewait_sock
88 192 timerfd_ctx
89 192 tpm_vendor_specific
90 192 urb
91
92 Display all structures that contain members that point to
93 an mm_struct:
94
95 crash> whatis -m mm_struct
96 SIZE TYPE
97 16 tlb_state
98 24 flush_tlb_info
99 24 ftrace_raw_xen_mmu_pgd
100 24 futex_key
101 24 map_info
102 32 ftrace_raw_xen_mmu_alloc_ptpage
103 32 ftrace_raw_xen_mmu_pte_clear
104 40 ftrace_raw_xen_mmu_flush_tlb_others
105 40 ftrace_raw_xen_mmu_ptep_modify_prot
106 40 ftrace_raw_xen_mmu_set_pte_at
107 40 mm_slot
108 64 mm_walk
109 64 rmap_item
110 104 userfaultfd_ctx
111 128 mmu_gather
112 216 vm_area_struct
113 256 linux_binprm
114 2616 rq
115 2936 task_struct
116
117 Display all structures sized from 256 to 512 bytes that
118 contain members that point to a task_struct:
119
120 crash> whatis -r 256-512 -m task_struct
121 SIZE TYPE
122 256 file
123 256 od_cpu_dbs_info_s
124 264 srcu_notifier_head
125 272 protection_domain
126 288 clk_notifier
127 288 fsnotify_group
128 296 quota_info
129 312 tty_port
130 320 workqueue_struct
131 344 trace_array
132 344 uart_state
133 352 cpufreq_policy
134 352 elf_thread_core_info
135 376 perf_event_context
136 384 rcu_data
137 400 cgroup
138 408 subsys_private
139 424 hvc_struct
140 496 psmouse
51. wr - write memory
51.1. SYNOPSIS
1wr [-u|-k|-p] [-8|-16|-32|-64] [address|symbol] value
51.2. DESCRIPTION
1 This command modifies the contents of memory. The starting address may be
2 entered either symbolically or by address. The default modification size
3 is the size of a long data type. Write permission must exist on the
4 /dev/mem. When writing to memory on a live system, this command should
5 obviously be used with great care.
6
7 -u address argument is a user virtual address.
8 -k address argument is a kernel virtual address.
9 -p address argument is a physical address.
10 -8 write data in an 8-bit value.
11 -16 write data in a 16-bit value.
12 -32 write data in a 32-bit values (default on 32-bit machines).
13 -64 write data in a 64-bit values (default on 64-bit machines).
14 address address to write. The address is considered virtual unless the
15 -p option is used. If a virtual address is specified, the
16 -u or -k options are necessary only if the address space cannot
17 be determined from the address value itself. If a user virtual
18 address is specified, the address space of the current context
19 implied. The address must be expressed in hexadecimal format.
20 symbol symbol of starting address to write.
21 value the value of the data to write.
51.3. EXAMPLES
1 Turn on a debug flag:
2
3 crash> wr my_debug_flag 1