mdb tab completion

Last October, the first illumos hack-a-thon took place. Out of that a lot of interesting things were done and have since been integrated into illumos. Two of the more interesting gems were Adam Leventhal and Matt Ahrens adding dtrace -x temporal and Eric Schrock adding the DTrace print() action. Already print() is in the ranks of things where once you have it you really miss it when you don’t. During the hack-a-thon I had the chance to work Matt Amdur. Together we worked on another one of those nice to haves that has finally landed in illumos: tab completion for mdb.

md-what?

For those who have never used it, mdb is the Modular Debugger that comes as a part of illumos and was originally developed for Solaris 8. mdb is primarily used for post-mortem of user and kernel applications and kernel debugger. mdb isn’t a source level debugger, but it works quite well on core dumps from userland, inspects and modifies live kernel state without stopping the system, and provides facilities for traditional debugging where a program is stopped, stepped over, and inspected. mdb replaced adb which came out of AT&T. While mdb isn’t 100% compatible with adb, it does remind you that there’s ‘No algol 68 here’. For the full history, take a look at Mike Shapiro’s talk that he gave at the Brown CS 37th IPP Symposium.

One of the more useful pieces of mdb is its module API which allows other modules to deliver specifically tailored commands and walkers. This is used for everything from the v8 Javascript Engine to understanding cyclics. Between that, pipelines, and other niceties, there isn’t too much else you could want from your debugger.

What’s involved

The work that we’ve done falls into three parts:

• A tab completion engine.
• Changes to the module API and several new functions to allow dcmds
  to implement their own tab completion.
• Tab completion support for several dcmds

Thanks to CTF data in the kernel, we can tab complete everything from walker names, to types and their members. We went and added tab completion to the following dcmds:

• ::walk
• ::sizeof
• ::list
• ::offsetof
• ::print
• The dcmds themselves

Seeing is believing: Tab completion in action

Completing dcmds

> ::pr[tab]
print
printf
project
prov_tab
prtconf
prtusb

Completing walkers

> ::walk ar[tab]
arc_buf_hdr_t
arc_buf_t
> ::walk arc_buf_

Completing types

> ::sizeof struct dt[tab]
struct dtrace_actdesc
struct dtrace_action
struct dtrace_aggbuffer
struct dtrace_aggdesc
struct dtrace_aggkey
struct dtrace_aggregation
struct dtrace_anon
struct dtrace_argdesc
struct dtrace_attribute
struct dtrace_bufdesc
struct dtrace_buffer
struct dtrace_conf
struct dtrace_cred
struct dtrace_difo
struct dtrace_diftype
struct dtrace_difv
struct dtrace_dstate
struct dtrace_dstate_percpu
struct dtrace_dynhash
struct dtrace_dynvar
struct dtrace_ecb
struct dtrace_ecbdesc
struct dtrace_enabling
struct dtrace_eprobedesc
struct dtrace_fmtdesc
struct dtrace_hash
...

Completing members

> ::offsetof zio_t io_v[tab]
io_vd
io_vdev_tree
io_vsd
io_vsd_ops

Walking across types with ::print

> p0::print proc_t p_zone->zone_n[tab]
zone_name
zone_ncpus
zone_ncpus_online
zone_netstack
zone_nlwps
zone_nlwps_ctl
zone_nlwps_lock
zone_nodename
zone_nprocs
zone_nprocs_ctl
zone_nprocs_kstat
zone_ntasks

In addition, just as you can walk across structure (.) and array ([]) dereferences in ::print invocations, you can also do the same with tab completion.

What’s next?

Now that mdb tab completion change is in illumos there’s already some work to add backends to new dcmds including:

• ::printf
• ::help
• ::bp

What else would you like to see? Let us know in a comment or better yet, go ahead and implement it yourself!