Thanks for clarifying this. Is it a remotely built Advanced CMake project? If this is the case, VisualGDB tries connecting to the remote machine in order to query the code structure from CMake (this ensures that the project structure shown in Solution Explorer exactly matches what will be actually built).
If you have previously opened the same project on the same machine, you can cancel the connection so that VisualGDB will display the cached project structure, however the project editing functionality that depends on getting updated information from CMake (e.g. adding new files to targets) won’t work.
If you don’t want to have continuous access to the Linux VM, please consider using a Windows cross-toolchain instead. In this case CMake will run on the Windows machine, so no remote connection will be required until you start debugging.