Embedded debugging with LLDB? Sure!

LLDB?

You haven’t heard about LLDB? In a nutshell it’s new debugger, part of the ubiqitious LLVM project, aiming to replace the venerable GDB which is used everywhere. It has tons of interesting features, some of which also found their way into GDB by now, and is the standard debugger on macOS, though many people will not even know due to the very nice integration into XCode. It is very actively supported and developped, especially by Apple and Google. However what seems to be the largest benefit is at the same time the biggest downside: Anything not of interest to Apple (macOS and iOS) or Google (Android and Linux in general) won’t receive not anywhere near as much attention as those prime areas. Especially well-established GDB specifics which are not needed for Android or iOS development are really lacking, so support for remote embedded targets is not anywhere near as good as in GDB.

So why still trying to use LLDB you ask? There’re a whole number of reasons why:

  1. If you’ve ever worked on a Mac recently you will have noticed that there’s a quite strict software signing requirement, which is especially true for software with extended system abilities like debuggers.
  2. LLDB comes out of the box with a Mac that has XCode installed, sounds like a small thing but it means it’s readily available on any Mac used for development – no questions asked. And thanks to a no-nonsense policy it always comes with support for all architectures and OSes it can support out-of-the-box; say goodby to those ten-and-something foo-bar-baz-gdb binaries coming with some toolchain in varying version and compile-time options.
  3. It has a really good CLI GUI built-in. No really, it’s night and day compared to GDBs tui mode.
  4. It offers outstanding inside into data structures in memory for pretty much any language you want to see supported.
  5. The descendance of LLVM allows you to really modify program flow on the fly using dynamically compiled code.
  6. While still supporting most of the commonly known GDB cryptic mnemonics it offers a much clearer (and easily extensible) command structure with full help built-in.
  7. It is completely scriptable in Python. Find youself typing the same stuff over and over again? Hack it in Python code and load it as a new command. Want to find a detail burried deep in technical details? Use Python to access all data structures and easily drill down to the required information using the very powerful Python interpreter and standard library.

Sounds great. What’s the problem then?

The problem is that GDB has been around for a whole lot longer than lldb so people figured out ways to make GDB do what they needed to get done. One of the “solutions” to a specific problem, namely how to remotely talk to a piece of hardware under test (called the target) from your debugging console (called the host), is the Remote Serial Protocol. In a nutshell this protocol is what connects the programming adapter/debugging interface/probe which speaks some native protocol of the target (which could be JTAG or debugWIRE or SWD or …) to your host running the debugging software. This protocol is the defacto standard in the OpenSource world so anyone attempts to speak it with largely variable amounts of success and close to none interoperability. This is not a problem if the two components are from the same project (like gdb and gdbserver or lldb and lldb-server) but it starts to be a problem when you start mixing and matching components and the server is not GDB which in reality is the only software people test against…

(cue in the LLDB march…)

Now, LLDB is not GDB, so this is really painful if you do want to use the former instead of the latter. To add insult to the injury, people tend to point out this very fact to you and suggest to go bugger off and/or fix LLDB so it behaves exactly like GDB because – you know – it’s the “standard”.

So is there a way then?

Yes, actually there is… after lots of attempts with different hardware and software, plenty of fruitless discussions and many wasted hours, I actually found something that actually does with LLDB.

To be perfectly clear: This approach might not be for everyone and might not be applicable outside of STM32 hardware at all, but it least it shows that not everything is lost and interoperability is possible without changes to LLDB (it would certainly take a while until changs in LLDB would actually show up in the field, e.g. new versions of XCode).

And the way is ST-LINK.

ST-LINK is the name of a programming/debugging adapter offered by STMicroelectronics to program STM8 and STM32 (i.e. Cortex-M) MCUs. Those programming adapter are offered in standalone editions by ST but are also available integrated on ST evaluation boards (on Nucleo as snap-off PCB variant, on the Discovery they’re on the main PCB). It is also possible to get cheap knock-offs of an standalone version in those special Chinese shops…

I’ve not the slightest idea whether they’ll work with other vendor chips as well but STM32 is very widely available in huge varieties and somewhat cheap so at least there’s a start.

Since ST-LINK only speaks a proprietary protocol, mostly ST tools are available to use ST-LINKs which is not that useful for our purpose. However a clever person under the pseudonym texane figured out how the protocol works and made a toolset available on GitHub. Meanwhile the code has spread to other projects like OpenOCD, too, but our focus really is just on stlink for now.

The stlink toolkit contains 3 (actually more) really interesting utilities, the first of while is st-info. st-info allows you to query whether an ST-LINK is connected and a target is connected and detected by it, like:

# st-info --probe
Found 1 stlink programmers
# st-info --flash
0x20000

The next one is st-flash which allows you to flash softeware into your MCU, something that can not be done with LLDB directly at the moment:

# st-flash --format ihex write out
st-flash 1.3.1
2017-07-01T23:12:20 INFO src/common.c: Loading device parameters....
2017-07-01T23:12:20 INFO src/common.c: Device connected is: F1 Medium-density device, id 0x20036410
2017-07-01T23:12:20 INFO src/common.c: SRAM size: 0x5000 bytes (20 KiB), Flash: 0x20000 bytes (128 KiB) in pages of 1024 bytes
2017-07-01T23:12:20 INFO src/common.c: Attempting to write 20052 (0x4e54) bytes to stm32 address: 134217728 (0x8000000)
Flash page at addr: 0x08004c00 erased
2017-07-01T23:12:20 INFO src/common.c: Finished erasing 20 pages of 1024 (0x400) bytes
2017-07-01T23:12:20 INFO src/common.c: Starting Flash write for VL/F0/F3 core id
2017-07-01T23:12:20 INFO src/flash_loader.c: Successfully loaded flash loader in sram
 19/19 pages written
2017-07-01T23:12:21 INFO src/common.c: Starting verification of write complete
2017-07-01T23:12:21 INFO src/common.c: Flash written and verified! jolly good!

And last but not least there’s st-util which provides the bridge to LLDB we’ve been waiting for so long. First you need to start st-util with some options which will open a TCP port which we’ll point LLDB towards (the same also works for GDB but is not the topic here):

# st-util -m
st-util 1.3.1
2017-07-02T01:06:31 INFO src/common.c: Loading device parameters....
2017-07-02T01:06:31 INFO src/common.c: Device connected is: F1 Medium-density device, id 0x20036410
2017-07-02T01:06:31 INFO src/common.c: SRAM size: 0x5000 bytes (20 KiB), Flash: 0x20000 bytes (128 KiB) in pages of 1024 bytes
2017-07-02T01:06:31 INFO src/gdbserver/gdb-server.c: Chip ID is 00000410, Core ID is  1ba01477.
2017-07-02T01:06:31 INFO src/gdbserver/gdb-server.c: Listening at *:4242...

This will launch the st-util process as a blocking process connecting to the ST-LINK while at the same time opening a TCP networking port for the debugging console to connect to. After the process is started we can easily connect via LLDB to it:

# lldb target/thumbv7m-none-eabi/debug/examples/blinky
(lldb) target create "target/thumbv7m-none-eabi/debug/examples/blinky"
Current executable set to 'target/thumbv7m-none-eabi/debug/examples/blinky' (arm).
(lldb) gdb-remote 4242
Process 1 stopped
* thread #1: tid = 0x0001, 0xe0001058, stop reason = signal SIGTRAP
    frame #0: 0xe0001058
->  0xe0001058: stmdals r6, {r0, r1, r2, r12, pc}
    0xe000105c: stmdals r8, {r3, r12, pc}
    0xe0001060: .long  0xf04f9907                ; unknown opcode
    0xe0001064: addmi  r0, r8, #256

and we’re in. In case you’re wondering about the name of the binary: yes I’m actually debugging a Rust programm running on a MCU on a Nucluo-Board connected via the on-board ST-LINK adapter in the native lldb coming with my Mac running El Capitan.

And that’s really it for this post. I could rant about which other interfaces do not work at this time but maybe I manage to get them fixed instead which would be a lot more helpful than a rant… ;)