Tales from a Core File

Close this search box.

mdb tab completion

May 15, 2012

Last October, the first illumos hack-a-thon took place. Out of that a lot of interesting things were done and have since been integrated into illumos. Two of the more interesting gems were Adam Leventhal and Matt Ahrens adding dtrace -x temporal and Eric Schrock adding the DTrace print() action. Already print() is in the ranks of things where once you have it you really miss it when you don’t. During the hack-a-thon I had the chance to work Matt Amdur. Together we worked on another one of those nice to haves that has finally landed in illumos: tab completion for mdb.


For those who have never used it, mdb is the Modular Debugger that comes as a part of illumos and was originally developed for Solaris 8. mdb is primarily used for post-mortem of user and kernel applications and kernel debugger. mdb isn’t a source level debugger, but it works quite well on core dumps from userland, inspects and modifies live kernel state without stopping the system, and provides facilities for traditional debugging where a program is stopped, stepped over, and inspected. mdb replaced adb which came out of AT&T. While mdb isn’t 100% compatible with adb, it does remind you that there’s ‘No algol 68 here’. For the full history, take a look at Mike Shapiro’s talk that he gave at the Brown CS 37th IPP Symposium.

One of the more useful pieces of mdb is its module API which allows other modules to deliver specifically tailored commands and walkers. This is used for everything from the v8 Javascript Engine to understanding cyclics. Between that, pipelines, and other niceties, there isn’t too much else you could want from your debugger.

What’s involved

The work that we’ve done falls into three parts:

  • A tab completion engine.
  • Changes to the module API and several new functions to allow dcmds
    to implement their own tab completion.
  • Tab completion support for several dcmds

Thanks to CTF data in the kernel, we can tab complete everything from walker names, to types and their members. We went and added tab completion to the following dcmds:

  • ::walk
  • ::sizeof
  • ::list
  • ::offsetof
  • ::print
  • The dcmds themselves

Seeing is believing: Tab completion in action

Completing dcmds

> ::pr[tab]

Completing walkers

> ::walk ar[tab]
> ::walk arc_buf_

Completing types

> ::sizeof struct dt[tab]
struct dtrace_actdesc
struct dtrace_action
struct dtrace_aggbuffer
struct dtrace_aggdesc
struct dtrace_aggkey
struct dtrace_aggregation
struct dtrace_anon
struct dtrace_argdesc
struct dtrace_attribute
struct dtrace_bufdesc
struct dtrace_buffer
struct dtrace_conf
struct dtrace_cred
struct dtrace_difo
struct dtrace_diftype
struct dtrace_difv
struct dtrace_dstate
struct dtrace_dstate_percpu
struct dtrace_dynhash
struct dtrace_dynvar
struct dtrace_ecb
struct dtrace_ecbdesc
struct dtrace_enabling
struct dtrace_eprobedesc
struct dtrace_fmtdesc
struct dtrace_hash

Completing members

> ::offsetof zio_t io_v[tab]

Walking across types with ::print

> p0::print proc_t p_zone->zone_n[tab]

In addition, just as you can walk across structure (.) and array ([]) dereferences in ::print invocations, you can also do the same with tab completion.

What’s next?

Now that mdb tab completion change is in illumos there’s already some work to add backends to new dcmds including:

  • ::printf
  • ::help
  • ::bp

What else would you like to see? Let us know in a comment or better yet, go ahead and implement it yourself!

2 Responses

  1. This looks very, very useful. Thanks! I’m looking forward to using it, once I build the latest illumos sources.

    Are there any plans for vim key-bindings? Or customizable key-bindings for mdb?

    1. While I’ve longed to do the equivalent of set -o vi in mdb, I don’t have any plans to do that. Maybe someone else out there does.

Recent Posts

September 27, 2019
September 6, 2019
October 1, 2014