What is this? Why does it always take so long? (20+ seconds)

Sysprogs forums Forums VisualGDB What is this? Why does it always take so long? (20+ seconds)

Viewing 13 posts - 1 through 13 (of 13 total)
  • Author
    Posts
  • #12566
    Ophidian14
    Participant

    #12567
    support
    Keymaster

    Hi,

    This could be caused by some toolchains-specific or connection-specific issues. We can help you troubleshoot this if you could let us know the email associated with your license so that we could verify your support status.

    #12572
    Ophidian14
    Participant

    I will send my email to support directly to verify my support status.  Note, when this display is on the screen, gdb on the backend is at 100% CPU and its memory usage is climbing rapidly.

    #12576
    support
    Keymaster

    Hi,

    Thanks for verifying your support status.

    It looks like the gdb executable is either hanging while VisualGDB is trying to execute initial commands, or the communication between VisualGDB and gdb is broken.

    Please try enabling the gdb logging via VisualGDB Project Properties -> Advanced GDB Settings, then reproduce the problem, cancel the session and view the gdb log.

    Do you see just one ‘-data-evaluate-expression’ command or multiple commands sent rapidly? If it is just one command, please try running gdb manually using the command line shown in the log and re-running the same commands. Does this trigger the gdb hang?

    #12589
    Ophidian14
    Participant

    Ran the logger and observed this:

    [   33344 ms] -data-evaluate-expression "sizeof(void *)"
    [   59611 ms] ^done,value="8"
    [   59753 ms] -data-evaluate-expression "sizeof(int)"
    [   59764 ms] ^done,value="4"
    [   59764 ms] -data-evaluate-expression "sizeof(short)"
    [   59772 ms] ^done,value="2"
    [   59772 ms] -data-evaluate-expression "sizeof(long)"
    [   59780 ms] ^done,value="8"
    [   59782 ms] -data-evaluate-expression "sizeof(long long)"
    [   59790 ms] ^done,value="8"
    [   59790 ms] -data-evaluate-expression "sizeof(char)"
    [   59794 ms] ^done,value="1"
    [   59794 ms] -data-evaluate-expression "sizeof(wchar_t)"
    [   59801 ms] ^done,value="4"
    [   59801 ms] -data-evaluate-expression "sizeof(float)"
    [   59808 ms] ^done,value="4"
    [   59808 ms] -data-evaluate-expression "sizeof(double)"
    [   59813 ms] ^done,value="8"
    [   59813 ms] -data-evaluate-expression "sizeof(long double)"
    [   59819 ms] ^done,value="16"
    [   61535 ms] -exec-continue

    Ran the command by hand with gdb –interpreter mi.  Saw the same thing, 20+ second delay but subsequent commands returned immediately.

    In terms of what was going on during the delay.  In “top” I could see gdb going from virt 549MB / res 190MB, to, 2583MB virt / 2256MB res.  And during this time, strace -f on the gdb process just showed a long stream of brk() system calls (consistent with the memory growth, obviously) interspersed with a few calls to mmap().

    So I guess the question is, what would cause

    -data-evaluate-expression "sizeof(void *)"

    to balloon the size of the GDB process?  What is it trying to do, and what about my debug target is causing it to be so expensive?  How would I figure that out, would I need to debug gdb?

    Also is there a way to cache the results of these -data-evaluate-expression calls so that I can avoid this problem altogether?

    #12590
    Ophidian14
    Participant

    Debugged gdb and broke in a few times while it was in this loop.  Here are a few select stack frames.

     

    (gdb) bt
    #0  0x00007ffff5c71370 in free () from /lib64/libc.so.6
    #1  0x00000000006ae06c in cp_canonicalize_string ()
    #2  0x0000000000628bbf in dwarf2_canonicalize_name.isra.84.part.85 ()
    #3  0x000000000063412c in dwarf2_compute_name ()
    #4  0x0000000000634614 in dwarf2_physname ()
    #5  0x00000000006347b7 in new_symbol_full ()
    #6  0x0000000000636a73 in process_die ()
    #7  0x0000000000639080 in process_structure_scope ()
    #8  0x000000000063720f in process_die ()
    #9  0x000000000063659d in process_die ()
    #10 0x000000000063659d in process_die ()
    #11 0x000000000063659d in process_die ()
    #12 0x0000000000636f51 in process_die ()
    #13 0x000000000063b668 in dw2_do_instantiate_symtab ()
    #14 0x000000000063bb8c in dw2_instantiate_symtab ()
    #15 0x000000000063c2f9 in dw2_lookup_symbol ()
    #16 0x00000000005a99ed in lookup_symbol_aux_quick ()
    #17 0x00000000005a9bb0 in lookup_symbol_global_iterator_cb ()
    #18 0x0000000000611a31 in default_iterate_over_objfiles_in_search_order ()
    #19 0x00000000005a9605 in lookup_symbol_global ()
    #20 0x00000000006af956 in lookup_symbol_file ()
    #21 0x00000000006afb30 in cp_lookup_symbol_in_namespace ()
    #22 0x00000000006afc26 in lookup_namespace_scope ()
    #23 0x00000000006b0112 in cp_lookup_symbol_nonlocal ()
    #24 0x00000000005a9d4f in lookup_symbol_in_language ()
    #25 0x00000000005f30f8 in lookup_typename ()
    #26 0x00000000005f3282 in lookup_signed_typename ()
    #27 0x0000000000547a60 in c_parse_internal ()
    #28 0x0000000000549e86 in c_parse ()
    #29 0x00000000006081ba in parse_exp_in_context.constprop.2 ()
    #30 0x00000000006083e2 in parse_exp_1 ()
    #31 0x0000000000608449 in parse_expression ()
    #32 0x000000000051a51c in mi_cmd_data_evaluate_expression ()
    #33 0x000000000051b979 in mi_execute_command ()
    #34 0x0000000000516f0d in mi_execute_command_input_handler ()
    #35 0x00000000005e2bb4 in process_event ()
    #36 0x00000000005e2f47 in gdb_do_one_event ()
    #37 0x00000000005e3177 in start_event_loop ()
    #38 0x00000000005dbfd3 in captured_command_loop ()
    #39 0x00000000005da7ba in catch_errors ()
    #40 0x00000000005dcc86 in captured_main ()
    #41 0x00000000005da7ba in catch_errors ()
    #42 0x00000000005dd8c4 in gdb_main ()
    #43 0x000000000045771e in main ()
    (gdb) cont
    Continuing.
    ^C
    Program received signal SIGINT, Interrupt.
    0x00000000005a72ca in eq_demangled_name_entry ()
    (gdb) bt
    #0  0x00000000005a72ca in eq_demangled_name_entry ()
    #1  0x00000000007a5710 in htab_find_slot_with_hash ()
    #2  0x00000000005a8457 in symbol_set_names ()
    #3  0x00000000006347e9 in new_symbol_full ()
    #4  0x00000000006390b8 in process_structure_scope ()
    #5  0x000000000063720f in process_die ()
    #6  0x000000000063659d in process_die ()
    #7  0x0000000000636f51 in process_die ()
    #8  0x000000000063b668 in dw2_do_instantiate_symtab ()
    #9  0x000000000063bb8c in dw2_instantiate_symtab ()
    #10 0x000000000063c2f9 in dw2_lookup_symbol ()
    #11 0x00000000005a99ed in lookup_symbol_aux_quick ()
    #12 0x00000000005a9bb0 in lookup_symbol_global_iterator_cb ()
    #13 0x0000000000611a31 in default_iterate_over_objfiles_in_search_order ()
    #14 0x00000000005a9605 in lookup_symbol_global ()
    #15 0x00000000006af956 in lookup_symbol_file ()
    #16 0x00000000006afb30 in cp_lookup_symbol_in_namespace ()
    #17 0x00000000006afc26 in lookup_namespace_scope ()
    #18 0x00000000006b0112 in cp_lookup_symbol_nonlocal ()
    #19 0x00000000005a9d4f in lookup_symbol_in_language ()
    #20 0x00000000005f30f8 in lookup_typename ()
    #21 0x00000000005f3282 in lookup_signed_typename ()
    #22 0x0000000000547a60 in c_parse_internal ()
    #23 0x0000000000549e86 in c_parse ()
    #24 0x00000000006081ba in parse_exp_in_context.constprop.2 ()
    #25 0x00000000006083e2 in parse_exp_1 ()
    #26 0x0000000000608449 in parse_expression ()
    #27 0x000000000051a51c in mi_cmd_data_evaluate_expression ()
    #28 0x000000000051b979 in mi_execute_command ()
    #29 0x0000000000516f0d in mi_execute_command_input_handler ()
    #30 0x00000000005e2bb4 in process_event ()
    #31 0x00000000005e2f47 in gdb_do_one_event ()
    #32 0x00000000005e3177 in start_event_loop ()
    #33 0x00000000005dbfd3 in captured_command_loop ()
    #34 0x00000000005da7ba in catch_errors ()
    #35 0x00000000005dcc86 in captured_main ()
    #36 0x00000000005da7ba in catch_errors ()
    #37 0x00000000005dd8c4 in gdb_main ()
    #38 0x000000000045771e in main ()
    (gdb)
    #12591
    Ophidian14
    Participant

    One other point of information.  Running gdb 7.12 from devtoolset-6 doesn’t seem to incur this delay.  That is, it can evaluate sizeof(void *) immediately.  I will try integrating this into my config.

    #12593
    Ophidian14
    Participant

    Just one followup, I eventually experienced the “evaluate sizeof” hang later on in my debugging session on one of my own types, so I don’t think gdb 7.12 is the solution necessarily.

    #12597
    support
    Keymaster

    Hi,

    This looks like a buggy gdb build or some incompatibility between gdb and the debug information format. Were you able to debug your programs on that machine successfully before? If yes, we would advise trying to revert to the gdb version that worked.

    Another option would be to check if the gdb hang is related to the debugged binary (e.g. does it hang with no debugged binary or with a simple “hello, world” program?). If yes, changing the debug information format (e.g. -gdwarf-2) might solve this. If the program consists of many large shared libraries, stripping the ones you don’t want to debug (using the ‘strip’ command) will reduce the amount of symbols loaded in gdb and could also improve performance.

    Yet another option would be to try running gdb on the Windows side instead (note to other readers: this requires the Custom edition). With VisualGDB 5.3 it’s as easy as selecting “allow changing build/debug command host” on the first page of VisualGDB Project Properties, adding an action to download the built binary on Windows and then changing “gdb executable” on the Debug Settings page to your Windows gdb.exe (it needs to match the Linux target type). VisualGDB will then figure out how to use gdbserver and update the startup commands automatically.

    #12600
    Ophidian14
    Participant

    I tried -gdwarf-2 and shrinking the binary as much as I could, didn’t make a difference.

    One thing I feel I have should have mentioned, but didn’t:  all this code was compiled with Clang i.e. LLVM.  When I compile with gcc, I haven’t been able to reproduce the problem.  The problem is, my larger code base does not compile with gcc due to errors, so I can’t really switch.

    It is probably just some incompatibility/bug between Clang and gdb.

    #12614
    support
    Keymaster

    Hi,

    This could be the case. We would advise trying lldb-mi instead of gdb (LLDB is the debugger from the clang/llvm suite) that might work with clang-generated code better. VisualGDB supports it, although it is generally less feature-complete and less stable than gdb.

    #12631
    Ophidian14
    Participant

    I tried lldb, setting a breakpoint in main(), running my program and hitting the breakpoint took almost 1 minute.  So that’s not going to work.  Thanks though.

    #12632
    support
    Keymaster

    Hi,

    Thanks for confirming this. The only other advice would be to ask in the clang/llvm mailing lists. This might be a known issue and someone from the clang community might be able to suggest a better workaround.

Viewing 13 posts - 1 through 13 (of 13 total)
  • You must be logged in to reply to this topic.