Faster ipdb Setup

I did some work to reduce the friction involved with setting up the python debugger (ipdb). I ended up getting it down from 10-30 seconds to about 1.5 seconds, which brings ipdb from rarely worth using (relative to print debugging) to almost always worth using.

Before working on this, my workflow was to add the lines import ipdb and ipdb.set_trace() where I wanted a breakpoint, then run the code and use the debugger, repeat with more breakpoints as needed, and then find all the spots with breakpoints and delete them. Adding a breakpoint like this takes me 5-6 seconds (kind of a clunky string of text to type), and finding and deleting the spots with breakpoints takes anywhere from 3 to 20 seconds (because I may have hopped around a bunch of files during the debugging process). Goal is to shrink both of these as much as possible.

To improve creating a breakpoint, I realized I could hack my way to not having to do an import by adding something to builtins. I defined a setup function like

import builtins
def setup():
    def DEBUG(cond=1):
        import ipdb
        ipdb.set_trace(cond=cond)
    builtins.DEBUG = DEBUG

and I updated my entrypoint file (similar to django’s manage.py) to call this function. This means that if I run a command or test or server or anything, it’ll add DEBUG to builtins. So then in the place where I want a breakpoint, I just add one DEBUG() line (or DEBUG(x==0) if I want the breakpoint conditional on x=0). This brings me down from 5ish seconds to about 1-1.5 seconds, which is a big win. I can probably use an even shorter name to get down to half a second if I start finding the 1.5 seconds annoying.

You have to do one more small step to get this to work in ipython, because even if you start it with the same entrypoint, ipython gets its own python environment. You just need to add c.InteractiveShellApp.exec_lines = ["from settings import setup", "setup()"], where c is your python config. (If you don’t have any ipython config yet it’s just an ipython_config.py file with c = get_config().)

The improvement to cleanup was an even bigger win. I just added this to my vimrc

nnoremap <C-b> :let current_buf = bufnr('%') \| execute 'silent bufdo g/^\s*DEBUG(/d \| update' \| execute 'buffer' current_buf<CR>

This maps ctrl-b to a command that goes through each open buffer, searches for lines that start with whitespace followed by DEBUG(, deletes them, and saves the file (that’s what update is for, I’d never used that before). silent suppresses all the output so that doing this is less disruptive. We also record the current buffer number and reset the current window to that buffer, because otherwise we would leave it at whatever buffer it got to last.

This improvement brings cleanup from 3-20s down to pretty much instant.

Quick larger point, these improvements made me start choosing ipdb over print debugging around 80% of the time, compared to ~5% before. This is why the mindset in this xkcd comic is bad. It assumes how often you use the thing is independent of how long it takes to use, but really when you eliminate friction you increase usage. So following the table in that comic will make you severely under invest in automation.

Update 8/27: Change command from

nnoremap <C-b> :let current_bn = bufnr('%') \| silent bufdo g/^\s*DEBUG(/d \| update \| execute 'buffer' current_bn<CR>  

to

 nnoremap <C-b> :let current_buf = bufnr('%') \| execute 'silent bufdo g/^\s*DEBUG(/d \| update' \| execute 'buffer' current_buf<CR>  

to stop it from waiting until the last buffer before saving.