log4cxx crashes on application exit

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

log4cxx crashes on application exit

Bonneau, Guy

We have used log4cxx V0_10_0 since a few years with both Linux and Windows without any issues until now. We have a setup to compile and build the log4cxx shared library on both Linux and Window. We made a solution (sln) that can build with Microsoft VS 2008 up to VS 2013. Our log4cxx is build under gentoo for Linux. 


We recently started to have a crash with a testing application under Linux Gentoo when a testing application exit and the destruction of the static singletons of the log4cxx library proceed. However the same testing application doesn't crash under Windows with VS 2013. We would appreciate at any hints that could help us fix the crash issue under Linux. The stack of the crash is:

#0  0x00007f2cde67d805 in ?? () from /lib64/libc.so.6
#1  0x00007f2cde67e878 in ?? () from /lib64/libc.so.6
#2  0x00007f2ce7b829b3 in std::_Rb_tree<std::string, std::pair<std::string const, log4cxx::helpers::ObjectPtrT<log4cxx::Logger> >, std::_Select1st<std::pair<std::string const, log4cxx::helpers::ObjectPtrT<log4cxx::Logger> > >, std::less<std::string>, std::allocator<std::pair<std::string const, log4cxx::helpers::ObjectPtrT<log4cxx::Logger> > > >::_M_erase(std::_Rb_tree_node<std::pair<std::string const, log4cxx::helpers::ObjectPtrT<log4cxx::Logger> > >*) () from /usr/lib64/liblog4cxx.so.10
#3  0x00007f2ce7b80940 in log4cxx::Hierarchy::~Hierarchy() () from /usr/lib64/liblog4cxx.so.10
#4  0x00007f2ce7b80b99 in log4cxx::Hierarchy::~Hierarchy() () from /usr/lib64/liblog4cxx.so.10
#5  0x00007f2ce7b98c46 in log4cxx::spi::DefaultRepositorySelector::~DefaultRepositorySelector() () from /usr/lib64/liblog4cxx.so.10
#6  0x00007f2ce7b989c0 in log4cxx::helpers::ObjectPtrT<log4cxx::spi::RepositorySelector>::~ObjectPtrT() () from /usr/lib64/liblog4cxx.so.10
#7  0x00007f2cde63beef in __cxa_finalize () from /lib64/libc.so.6
#8  0x00007f2ce7b4ab33 in ?? () from /usr/lib64/liblog4cxx.so.10​


Thanks

Guy Bonneau



Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

RE: log4cxx crashes on application exit

Dave Birdsall

I am not a log4cxx expert, but this looks a bit like the old global-object-destructor problem.

 

After a C++ program calls exit(), the destructors for all global objects are called. The order is non-deterministic, and cannot be influenced by the programmer. The order is determined at link edit time. Up until now, you have been lucky and the destructors were called in a nice order. But now it appears they are not.

 

In your case, it looks like an ObjectPtrT object is being destroyed.

 

The best way to avoid these kinds of problems is to avoid using global C++ objects. If you simply must, one workaround is to use a global pointer to a C++ object, initialized to null, and wrap all accesses to that pointer with code that checks for null, creates the object if so, and stores the pointer. Then at exit time, you can write code to explicitly destroy the object (if you care), before calling exit(). This workaround is not ideal; it is not thread-safe, for example. But you can cause object instances to be created in your main before spinning up other threads.

 

Dave

 

From: Bonneau, Guy [mailto:[hidden email]]
Sent: Monday, May 22, 2017 11:31 AM
To: [hidden email]
Subject: log4cxx crashes on application exit

 

We have used log4cxx V0_10_0 since a few years with both Linux and Windows without any issues until now. We have a setup to compile and build the log4cxx shared library on both Linux and Window. We made a solution (sln) that can build with Microsoft VS 2008 up to VS 2013. Our log4cxx is build under gentoo for Linux. 

 

We recently started to have a crash with a testing application under Linux Gentoo when a testing application exit and the destruction of the static singletons of the log4cxx library proceed. However the same testing application doesn't crash under Windows with VS 2013. We would appreciate at any hints that could help us fix the crash issue under Linux. The stack of the crash is:

#0  0x00007f2cde67d805 in ?? () from /lib64/libc.so.6
#1  0x00007f2cde67e878 in ?? () from /lib64/libc.so.6
#2  0x00007f2ce7b829b3 in std::_Rb_tree<std::string, std::pair<std::string const, log4cxx::helpers::ObjectPtrT<log4cxx::Logger> >, std::_Select1st<std::pair<std::string const, log4cxx::helpers::ObjectPtrT<log4cxx::Logger> > >, std::less<std::string>, std::allocator<std::pair<std::string const, log4cxx::helpers::ObjectPtrT<log4cxx::Logger> > > >::_M_erase(std::_Rb_tree_node<std::pair<std::string const, log4cxx::helpers::ObjectPtrT<log4cxx::Logger> > >*) () from /usr/lib64/liblog4cxx.so.10
#3  0x00007f2ce7b80940 in log4cxx::Hierarchy::~Hierarchy() () from /usr/lib64/liblog4cxx.so.10
#4  0x00007f2ce7b80b99 in log4cxx::Hierarchy::~Hierarchy() () from /usr/lib64/liblog4cxx.so.10
#5  0x00007f2ce7b98c46 in log4cxx::spi::DefaultRepositorySelector::~DefaultRepositorySelector() () from /usr/lib64/liblog4cxx.so.10
#6  0x00007f2ce7b989c0 in log4cxx::helpers::ObjectPtrT<log4cxx::spi::RepositorySelector>::~ObjectPtrT() () from /usr/lib64/liblog4cxx.so.10
#7  0x00007f2cde63beef in __cxa_finalize () from /lib64/libc.so.6
#8  0x00007f2ce7b4ab33 in ?? () from /usr/lib64/liblog4cxx.so.10​

 

Thanks

Guy Bonneau

 

 

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

RE: log4cxx crashes on application exit

Dave Birdsall

Hi,

 

Looking at your stack more carefully, it looks like the ObjectPtrT global object is created by the log4cxx code itself. Which makes things harder. It looks like it is trying to destroy some underlying objects. The Hierarchy destructor seems to be making use of ObjectPtrT and some other things. Perhaps there is a way to clean out the Hierarchy object first so that there is nothing for it to destroy?

 

Dave

 

From: Dave Birdsall [mailto:[hidden email]]
Sent: Monday, May 22, 2017 11:43 AM
To: Log4CXX User <[hidden email]>
Subject: RE: log4cxx crashes on application exit

 

I am not a log4cxx expert, but this looks a bit like the old global-object-destructor problem.

 

After a C++ program calls exit(), the destructors for all global objects are called. The order is non-deterministic, and cannot be influenced by the programmer. The order is determined at link edit time. Up until now, you have been lucky and the destructors were called in a nice order. But now it appears they are not.

 

In your case, it looks like an ObjectPtrT object is being destroyed.

 

The best way to avoid these kinds of problems is to avoid using global C++ objects. If you simply must, one workaround is to use a global pointer to a C++ object, initialized to null, and wrap all accesses to that pointer with code that checks for null, creates the object if so, and stores the pointer. Then at exit time, you can write code to explicitly destroy the object (if you care), before calling exit(). This workaround is not ideal; it is not thread-safe, for example. But you can cause object instances to be created in your main before spinning up other threads.

 

Dave

 

From: Bonneau, Guy [[hidden email]]
Sent: Monday, May 22, 2017 11:31 AM
To: [hidden email]
Subject: log4cxx crashes on application exit

 

We have used log4cxx V0_10_0 since a few years with both Linux and Windows without any issues until now. We have a setup to compile and build the log4cxx shared library on both Linux and Window. We made a solution (sln) that can build with Microsoft VS 2008 up to VS 2013. Our log4cxx is build under gentoo for Linux. 

 

We recently started to have a crash with a testing application under Linux Gentoo when a testing application exit and the destruction of the static singletons of the log4cxx library proceed. However the same testing application doesn't crash under Windows with VS 2013. We would appreciate at any hints that could help us fix the crash issue under Linux. The stack of the crash is:

#0  0x00007f2cde67d805 in ?? () from /lib64/libc.so.6
#1  0x00007f2cde67e878 in ?? () from /lib64/libc.so.6
#2  0x00007f2ce7b829b3 in std::_Rb_tree<std::string, std::pair<std::string const, log4cxx::helpers::ObjectPtrT<log4cxx::Logger> >, std::_Select1st<std::pair<std::string const, log4cxx::helpers::ObjectPtrT<log4cxx::Logger> > >, std::less<std::string>, std::allocator<std::pair<std::string const, log4cxx::helpers::ObjectPtrT<log4cxx::Logger> > > >::_M_erase(std::_Rb_tree_node<std::pair<std::string const, log4cxx::helpers::ObjectPtrT<log4cxx::Logger> > >*) () from /usr/lib64/liblog4cxx.so.10
#3  0x00007f2ce7b80940 in log4cxx::Hierarchy::~Hierarchy() () from /usr/lib64/liblog4cxx.so.10
#4  0x00007f2ce7b80b99 in log4cxx::Hierarchy::~Hierarchy() () from /usr/lib64/liblog4cxx.so.10
#5  0x00007f2ce7b98c46 in log4cxx::spi::DefaultRepositorySelector::~DefaultRepositorySelector() () from /usr/lib64/liblog4cxx.so.10
#6  0x00007f2ce7b989c0 in log4cxx::helpers::ObjectPtrT<log4cxx::spi::RepositorySelector>::~ObjectPtrT() () from /usr/lib64/liblog4cxx.so.10
#7  0x00007f2cde63beef in __cxa_finalize () from /lib64/libc.so.6
#8  0x00007f2ce7b4ab33 in ?? () from /usr/lib64/liblog4cxx.so.10​

 

Thanks

Guy Bonneau

 

 

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: log4cxx crashes on application exit

Bonneau, Guy

​The issue happens indeed when the Hierarchy destructor attempts to destroy the loggers or ProvisionNodes in that code:

Hierarchy::~Hierarchy()
{
    delete loggers;
    delete provisionNodes;
}

Since both of these are containers of ObjectPtrT which are shared pointer implemented by the library I would expect to nicely exit. Thus I am a little bit baffled at this point.



From: Dave Birdsall <[hidden email]>
Sent: Monday, May 22, 2017 2:48 PM
To: Log4CXX User
Subject: RE: log4cxx crashes on application exit
 

Hi,

 

Looking at your stack more carefully, it looks like the ObjectPtrT global object is created by the log4cxx code itself. Which makes things harder. It looks like it is trying to destroy some underlying objects. The Hierarchy destructor seems to be making use of ObjectPtrT and some other things. Perhaps there is a way to clean out the Hierarchy object first so that there is nothing for it to destroy?

 

Dave

 

From: Dave Birdsall [mailto:[hidden email]]
Sent: Monday, May 22, 2017 11:43 AM
To: Log4CXX User <[hidden email]>
Subject: RE: log4cxx crashes on application exit

 

I am not a log4cxx expert, but this looks a bit like the old global-object-destructor problem.

 

After a C++ program calls exit(), the destructors for all global objects are called. The order is non-deterministic, and cannot be influenced by the programmer. The order is determined at link edit time. Up until now, you have been lucky and the destructors were called in a nice order. But now it appears they are not.

 

In your case, it looks like an ObjectPtrT object is being destroyed.

 

The best way to avoid these kinds of problems is to avoid using global C++ objects. If you simply must, one workaround is to use a global pointer to a C++ object, initialized to null, and wrap all accesses to that pointer with code that checks for null, creates the object if so, and stores the pointer. Then at exit time, you can write code to explicitly destroy the object (if you care), before calling exit(). This workaround is not ideal; it is not thread-safe, for example. But you can cause object instances to be created in your main before spinning up other threads.

 

Dave

 

From: Bonneau, Guy [[hidden email]]
Sent: Monday, May 22, 2017 11:31 AM
To: [hidden email]
Subject: log4cxx crashes on application exit

 

We have used log4cxx V0_10_0 since a few years with both Linux and Windows without any issues until now. We have a setup to compile and build the log4cxx shared library on both Linux and Window. We made a solution (sln) that can build with Microsoft VS 2008 up to VS 2013. Our log4cxx is build under gentoo for Linux. 

 

We recently started to have a crash with a testing application under Linux Gentoo when a testing application exit and the destruction of the static singletons of the log4cxx library proceed. However the same testing application doesn't crash under Windows with VS 2013. We would appreciate at any hints that could help us fix the crash issue under Linux. The stack of the crash is:

#0  0x00007f2cde67d805 in ?? () from /lib64/libc.so.6
#1  0x00007f2cde67e878 in ?? () from /lib64/libc.so.6
#2  0x00007f2ce7b829b3 in std::_Rb_tree<std::string, std::pair<std::string const, log4cxx::helpers::ObjectPtrT<log4cxx::Logger> >, std::_Select1st<std::pair<std::string const, log4cxx::helpers::ObjectPtrT<log4cxx::Logger> > >, std::less<std::string>, std::allocator<std::pair<std::string const, log4cxx::helpers::ObjectPtrT<log4cxx::Logger> > > >::_M_erase(std::_Rb_tree_node<std::pair<std::string const, log4cxx::helpers::ObjectPtrT<log4cxx::Logger> > >*) () from /usr/lib64/liblog4cxx.so.10
#3  0x00007f2ce7b80940 in log4cxx::Hierarchy::~Hierarchy() () from /usr/lib64/liblog4cxx.so.10
#4  0x00007f2ce7b80b99 in log4cxx::Hierarchy::~Hierarchy() () from /usr/lib64/liblog4cxx.so.10
#5  0x00007f2ce7b98c46 in log4cxx::spi::DefaultRepositorySelector::~DefaultRepositorySelector() () from /usr/lib64/liblog4cxx.so.10
#6  0x00007f2ce7b989c0 in log4cxx::helpers::ObjectPtrT<log4cxx::spi::RepositorySelector>::~ObjectPtrT() () from /usr/lib64/liblog4cxx.so.10
#7  0x00007f2cde63beef in __cxa_finalize () from /lib64/libc.so.6
#8  0x00007f2ce7b4ab33 in ?? () from /usr/lib64/liblog4cxx.so.10​

 

Thanks

Guy Bonneau

 

 

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: log4cxx crashes on application exit

Robert Middleton
Are these issues the same at all?  The stack trace doesn't look the
same, but they may point you in the right direction.

https://issues.apache.org/jira/browse/LOGCXX-216
https://issues.apache.org/jira/browse/LOGCXX-352

-Robert Middleton

On Mon, May 22, 2017 at 3:02 PM, Bonneau, Guy
<[hidden email]> wrote:

> The issue happens indeed when the Hierarchy destructor attempts to destroy
> the loggers or ProvisionNodes in that code:
>
> Hierarchy::~Hierarchy()
> {
>     delete loggers;
>     delete provisionNodes;
> }
>
> Since both of these are containers of ObjectPtrT which are shared pointer
> implemented by the library I would expect to nicely exit. Thus I am a little
> bit baffled at this point.
>
>
> ________________________________
> From: Dave Birdsall <[hidden email]>
> Sent: Monday, May 22, 2017 2:48 PM
> To: Log4CXX User
>
> Subject: RE: log4cxx crashes on application exit
>
>
> Hi,
>
>
>
> Looking at your stack more carefully, it looks like the ObjectPtrT global
> object is created by the log4cxx code itself. Which makes things harder. It
> looks like it is trying to destroy some underlying objects. The Hierarchy
> destructor seems to be making use of ObjectPtrT and some other things.
> Perhaps there is a way to clean out the Hierarchy object first so that there
> is nothing for it to destroy?
>
>
>
> Dave
>
>
>
> From: Dave Birdsall [mailto:[hidden email]]
> Sent: Monday, May 22, 2017 11:43 AM
> To: Log4CXX User <[hidden email]>
> Subject: RE: log4cxx crashes on application exit
>
>
>
> I am not a log4cxx expert, but this looks a bit like the old
> global-object-destructor problem.
>
>
>
> After a C++ program calls exit(), the destructors for all global objects are
> called. The order is non-deterministic, and cannot be influenced by the
> programmer. The order is determined at link edit time. Up until now, you
> have been lucky and the destructors were called in a nice order. But now it
> appears they are not.
>
>
>
> In your case, it looks like an ObjectPtrT object is being destroyed.
>
>
>
> The best way to avoid these kinds of problems is to avoid using global C++
> objects. If you simply must, one workaround is to use a global pointer to a
> C++ object, initialized to null, and wrap all accesses to that pointer with
> code that checks for null, creates the object if so, and stores the pointer.
> Then at exit time, you can write code to explicitly destroy the object (if
> you care), before calling exit(). This workaround is not ideal; it is not
> thread-safe, for example. But you can cause object instances to be created
> in your main before spinning up other threads.
>
>
>
> Dave
>
>
>
> From: Bonneau, Guy [mailto:[hidden email]]
> Sent: Monday, May 22, 2017 11:31 AM
> To: [hidden email]
> Subject: log4cxx crashes on application exit
>
>
>
> We have used log4cxx V0_10_0 since a few years with both Linux and Windows
> without any issues until now. We have a setup to compile and build the
> log4cxx shared library on both Linux and Window. We made a solution (sln)
> that can build with Microsoft VS 2008 up to VS 2013. Our log4cxx is build
> under gentoo for Linux.
>
>
>
> We recently started to have a crash with a testing application under Linux
> Gentoo when a testing application exit and the destruction of the static
> singletons of the log4cxx library proceed. However the same testing
> application doesn't crash under Windows with VS 2013. We would appreciate at
> any hints that could help us fix the crash issue under Linux. The stack of
> the crash is:
>
> #0  0x00007f2cde67d805 in ?? () from /lib64/libc.so.6
> #1  0x00007f2cde67e878 in ?? () from /lib64/libc.so.6
> #2  0x00007f2ce7b829b3 in std::_Rb_tree<std::string, std::pair<std::string
> const, log4cxx::helpers::ObjectPtrT<log4cxx::Logger> >,
> std::_Select1st<std::pair<std::string const,
> log4cxx::helpers::ObjectPtrT<log4cxx::Logger> > >, std::less<std::string>,
> std::allocator<std::pair<std::string const,
> log4cxx::helpers::ObjectPtrT<log4cxx::Logger> > >
>>::_M_erase(std::_Rb_tree_node<std::pair<std::string const,
> log4cxx::helpers::ObjectPtrT<log4cxx::Logger> > >*) () from
> /usr/lib64/liblog4cxx.so.10
> #3  0x00007f2ce7b80940 in log4cxx::Hierarchy::~Hierarchy() () from
> /usr/lib64/liblog4cxx.so.10
> #4  0x00007f2ce7b80b99 in log4cxx::Hierarchy::~Hierarchy() () from
> /usr/lib64/liblog4cxx.so.10
> #5  0x00007f2ce7b98c46 in
> log4cxx::spi::DefaultRepositorySelector::~DefaultRepositorySelector() ()
> from /usr/lib64/liblog4cxx.so.10
> #6  0x00007f2ce7b989c0 in
> log4cxx::helpers::ObjectPtrT<log4cxx::spi::RepositorySelector>::~ObjectPtrT()
> () from /usr/lib64/liblog4cxx.so.10
> #7  0x00007f2cde63beef in __cxa_finalize () from /lib64/libc.so.6
> #8  0x00007f2ce7b4ab33 in ?? () from /usr/lib64/liblog4cxx.so.10
>
>
>
> Thanks
>
> Guy Bonneau
>
>
>
>
Loading...