Ticket #101 (closed defect)

Opened 8 years ago

Last modified 7 years ago

Deadlock in libssh2_session_free

Reported by: anonymous Owned by: bagder
Priority: normal Milestone:
Component: misc Version:
Keywords: Cc: bagder
Blocked By: Blocks:


I don't really know if the problem originates in libssh2 code, but it's the best guess I could think of.

I have a Perl program (using Net::SSH2) running as a CORBA deamon (using mico XS-bindings from http://sourceforge.net/projects/corba-mico/). The CORBA server creates SSH tunnels to remote hosts and communicates (non blocking mode) with other servers beyond the other end of the tunnel. SSH Tunnels are kept open as long as possible and are reinitialized if the connection is lost.
This works fine, usually. But every now and then the whole process blocks.

This happens on SuSE Linux Enterprise 10.2, 64Bit, gcc 4.1.2 20070115 (SUSE Linux) using Perl 5.10.0, Net::SSH2 0.19, and libssh2 1.1.

I've attached the stack trace of the (still running) process below.

Thanks in advance for your effort!

#0 0x00002ab6fe7d0918 in lll_mutex_lock_wait () from /lib64/libc.so.6
#1 0x00002ab6fe774bff in _L_lock_3988 () from /lib64/libc.so.6
#2 0x00002ab6fe771b71 in free () from /lib64/libc.so.6
#3 0x00002ab6fdcb9426 in _dl_map_object_deps () from /lib64/ld-linux-x86-64.so.2
#4 0x00002ab6fdcbe02d in dl_open_worker () from /lib64/ld-linux-x86-64.so.2
#5 0x00002ab6fdcba1f6 in _dl_catch_error () from /lib64/ld-linux-x86-64.so.2
#6 0x00002ab6fdcbdacb in _dl_open () from /lib64/ld-linux-x86-64.so.2
#7 0x00002ab6fe7f7600 in do_dlopen () from /lib64/libc.so.6
#8 0x00002ab6fdcba1f6 in _dl_catch_error () from /lib64/ld-linux-x86-64.so.2
#9 0x00002ab6fe7f76b5 in dlerror_run () from /lib64/libc.so.6
#10 0x00002ab6fe7f7787 in
libc_dlopen_mode () from /lib64/libc.so.6
#11 0x00002ab6fe7d5d7a in init () from /lib64/libc.so.6
#12 0x00002ab6fe5f711d in pthread_once () from /lib64/libpthread.so.0
#13 0x00002ab6fe7d5e05 in backtrace () from /lib64/libc.so.6
#14 0x00002ab6fe76b51f in libc_message () from /lib64/libc.so.6
#15 0x00002ab6fe7704fe in malloc_printerr () from /lib64/libc.so.6
#16 0x00002ab6fe771b7c in free () from /lib64/libc.so.6
#17 0x00002ab700a4a379 in libssh2_session_free (session=0x1522d90) at session.c:843
#18 0x00002ab7009315cc in XS_Net

from /digibib/tools/lib/perl5/site_perl/5.10.0/x86_64-linux-thread-multi-ld/auto/Net/SSH2/SSH2.so

#19 0x00002ab6fde6c2db in Perl_pp_entersub () from /digibib/ips/lib/libperl.so
#20 0x00002ab6fde65c57 in Perl_call_sv () from /digibib/ips/lib/libperl.so
#21 0x00002ab6fde7d5f0 in Perl_sv_clear () from /digibib/ips/lib/libperl.so
#22 0x00002ab6fde7ddb2 in Perl_sv_free2 () from /digibib/ips/lib/libperl.so
#23 0x00002ab6fde97fac in Perl_free_tmps () from /digibib/ips/lib/libperl.so
#24 0x00002ab6fde6d9e0 in Perl_pp_nextstate () from /digibib/ips/lib/libperl.so
#25 0x00002ab6fde6a8ee in Perl_runops_standard () from /digibib/ips/lib/libperl.so
#26 0x00002ab6fde65b65 in Perl_call_sv () from /digibib/ips/lib/libperl.so
#27 0x00002ab6ff422275 in pmico_call_method (name=0x14a54b8 "perform", return_items=1, exceptions=0x1274aa0)

at server.cc:288

#28 0x00002ab6ff42354b in PMicoServant::invoke (this=0x686a60, _req=0x14a7c60) at server.cc:834
#29 0x00002ab6ff8a4193 in MICOPOA::POA_impl::perform_invoke () from /digibib/tools/lib/libmico2.3.13.so
#30 0x00002ab6ff8a70a2 in MICOPOA::POA_impl::local_invoke () from /digibib/tools/lib/libmico2.3.13.so
#31 0x00002ab6ff8a773a in MICOPOA::POA_impl::invoke () from /digibib/tools/lib/libmico2.3.13.so
#32 0x00002ab6ff7fd8aa in CORBA::ORB::invoke_async () from /digibib/tools/lib/libmico2.3.13.so
#33 0x00002ab6ff81e5f0 in MICO::IIOPServer::exec_invoke_request () from /digibib/tools/lib/libmico2.3.13.so
#34 0x00002ab6ff8290f6 in MICO::IIOPServer::handle_invoke_request () from /digibib/tools/lib/libmico2.3.13.so
#35 0x00002ab6ff8297be in MICO::IIOPServer::handle_input () from /digibib/tools/lib/libmico2.3.13.so
#36 0x00002ab6ff829e98 in MICO::IIOPServer::input_callback () from /digibib/tools/lib/libmico2.3.13.so
#37 0x00002ab6ff82a2f7 in MICO::GIOPConn::do_read () from /digibib/tools/lib/libmico2.3.13.so
#38 0x00002ab6ff7e420b in MICO::SelectDispatcher::handle_fevents () from /digibib/tools/lib/libmico2.3.13.so
#39 0x00002ab6ff7e5823 in MICO::SelectDispatcher::run () from /digibib/tools/lib/libmico2.3.13.so
#40 0x00002ab6ff7fb9cf in CORBA::ORB::run () from /digibib/tools/lib/libmico2.3.13.so
#41 0x00002ab6ff4074c6 in XS_CORBAORB_run (my_perl=<value optimized out>, cv=<value optimized out>) at MICO.xs:406
#42 0x00002ab6fde6c2db in Perl_pp_entersub () from /digibib/ips/lib/libperl.so
#43 0x00002ab6fde6a8ee in Perl_runops_standard () from /digibib/ips/lib/libperl.so
#44 0x00002ab6fde6631c in perl_run () from /digibib/ips/lib/libperl.so
#45 0x0000000000400d2c in main ()

Change History

comment:1 Changed 8 years ago by bagder

I'm sorry, but exactly how can a single-threaded lib such as libssh2 dead-lock on this? I realize you have a problem but I don't' see libssh2 causes this.

comment:2 Changed 8 years ago by bagder

Thanks for your help on improving libssh2!

We need more details on this bug entry to be able to sort it out properly, and until more info is provided this entry will be set to 'pending' status and will get closed automatically at a later date unless feedback has been given.

comment:3 Changed 8 years ago by anonymous

I have yet to find more evidence for this, but the problem might be related to LIBSSH2_FREEing session->userauth_list_data when the libssh2 object gets destroyed.
After eliminating a call to "$ssh2->auth_list($sshUser)" (with $ssh2 being an object of type Net::SSH2) in my Perl code (right after object creation in preparation of establishing a new connection) the problem went away. Well, at least until now - it might take another day or two to be completely sure.
I still have no idea why there's a "pthread_once" call in that stack.


comment:4 Changed 7 years ago by sf-robot

This Tracker item was closed automatically by the system. It was
previously set to a Pending status, and the original submitter
did not respond within 14 days (the time period specified by
the administrator of this Tracker).

Note: See TracTickets for help on using tickets.