Yeah I tried that multiple times, it wasn’t a port problem, since VisualGDB could happily flash the firmware over to the device, so communication was happening. But the gdb connection never worked no matter what I tried.
In the end I bought an Olimex ARM-USB-TINY-H jtag debugger and I now have that up and running instead.