Short version

If your remote-pdb session fails to allow interactive history, try using socat to connect to it. An example command is:

socat -d -d readline,history=$HOME/.pdb_history tcp4:127.0.0.1:4444

Details

Sometimes, you want an interactive python debugger, but pdb doesn’t quite fit the bill.

This comes up if you work with Docker containers. For me, recently, it’s because we run an application locally using the Serverless framework, which can simulate an AWS-lambda-like environment locally. In both cases, the relevant problem is that the process running the python application doesn’t have a standard input pipe attached, so there’s no straightforward way to drop into an interactive debugging session.

The remote-pdb package is a nice solution to this problem: when you configure it properly, and it hits a breakpoint, it waits for a connection on a network socket, and you can interact with pdb through that network connection.

For a long time, I used telnet for this purpose. I’d set up remote_pdb to listen on 127.0.0.1:4444 and then when it awaited a connection, I’d run telnet 127.0.0.1 4444. remote_pdb would accept the connection, things would basically work. But, annoyingly, pdb’s interactive history wouldn’t work. I’d hit the “up” key on the keyboard to amend a command I’d just run, and the terminal would instead print ^[[A. I worked around this by not using the history, and suffered the indignity of dull tools.

But no more! Researching the issue led me to socat, which is a very powerful and generic piece of software that handles this correctly. Instead of telnet 127.0.0.1 4444 I run socat -d -d READLINE,history=$HOME/.pdb_history TCP4:127.0.0.1:4444, and everything works like a dream.

But why?

readline is the central concept here. I’d never really thought about how a shell knew to turn the “up arrow key” input into “scroll up to the last-entered command”, but the GNU Readline library is the answer. It’s a piece of software written in C, maintained since 1989, and linked into bash and cpython, among other things.

The GNU Readline library provides a set of functions for use by applications that allow users to edit command lines as they are typed in. Both Emacs and vi editing modes are available. The Readline library includes additional functions to maintain a list of previously-entered command lines, to recall and perhaps reedit those lines, and perform csh-like history expansion on previous commands.

Anyway, telnet is even older, and doesn’t support readline, though it can be retrofitted to do so with rlwrap, a clever piece of software written by Hans Lub. The README for rlwrap says the following:

rlwrap is a ‘readline wrapper’, a small utility that uses the GNU Readline library to allow the editing of keyboard input for any command.

I couldn’t find anything like it when I needed it, so I wrote this one back in 1999. By now, there are (and, in hindsight, even then there were) a number of good readline wrappers around, like rlfe, distributed as part of the GNU readline library, and the amazing socat.

“Amazing” sounded pretty good to me, so I installed socat and read through the man page. Like a good man page should, it includes some examples:

socat - TCP4:www.domain.org:80

    transfers data between STDIO (-) and a
    TCP4 connection to port 80 of host
    www.domain.org. This example results
    in an interactive connection similar
    to telnet or netcat. The stdin terminal
    parameters are not changed, so you
    may close the relay with ^D or abort
    it with ^C.

socat -d -d READLINE,history=$HOME/.http_history \
    TCP4:www.domain.org:www,crnl

    this is similar to the previous
    example, but you can edit the current
    line in a bash like manner (READLINE)
    and use the history file .http_history;
    socat prints messages about progress
    (-d -d). The  port is specified
    by service name (www), and correct
    network line termination characters
    (crnl) instead of NL are used.

Adapting that is straightforward, leading to my chosen solution of socat -d -d readline,history=$HOME/.pdb_history tcp4:127.0.0.1:4444.

Links