Hi,
This is likely caused by a particular combination of settings, and the challenge would be narrowing down the exact settings.
You can start by logging the J-Link GDB server command line and the GDB commands via VisualGDB (View->Other Windows->VisualGDB Diagnostics Console for command lines and VisualGDB Project Properties -> Advanced GDB Settings to enable GDB logging). This should help you reproduce the problem with just command-line tools from Segger, so you can share the repro steps to them and see if they have any suggestions.