Based on what we could tell, the Azure RTOS integration has been moved out of Beta status, but is still somewhat experimental, only supporting a few select devices, and is not a part of the mainstream SDKs, requiring a separate downloadable package.
Until it is included in the regular SDKs, we would advise using the STM32CubeMX workflow together with the downloadable package you linked. As the project generation will be handled by the STM32CubeMX tool, there is no need for any special support for it on the VisualGDB side.