How to configure ObjectMapper of Log4j2 JSON Layout

classic Classic list List threaded Threaded
13 messages Options
Reply | Threaded
Open this post in threaded view
|

How to configure ObjectMapper of Log4j2 JSON Layout

Dominik Sandjaja
Hello,

we are using the JSON layout in our Spring Boot application to easily ingest the logs in Elasticsearch.

One message that we log is about failed conversions, when an incoming request parameter cannot be converted correctly. As we use Spring’s Converter functionality for this, these `IllegalArgumentException`s are wrapped in a `ConversionFailedException`. This exception has fields of type `TypeDescriptor` which hold information about the source and the target type of the conversion.

The problem is, that these `TypeDescriptor` objects cannot be serialized to JSON. The simple call

com.fasterxml.jackson.databind.ObjectMapper().writeValueAsString(org.springframework.core.convert.TypeDescriptor.valueOf(String.javaClass))

throws an exception like

ERROR StatusLogger com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Direct self-reference leading to cycle (through reference chain: org.springframework.core.ResolvableType["componentType"]->org.springframework.core.ResolvableType["componentType"])
com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Direct self-reference leading to cycle (through reference chain: org.springframework.core.ResolvableType["componentType"]->org.springframework.core.ResolvableType["componentType"])
    at com.fasterxml.jackson.databind.exc.InvalidDefinitionException.from(InvalidDefinitionException.java:77)
    at com.fasterxml.jackson.databind.SerializerProvider.reportBadDefinition(SerializerProvider.java:1191)
    at com.fasterxml.jackson.databind.ser.BeanPropertyWriter._handleSelfReference(BeanPropertyWriter.java:944)
    at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:721)
    at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719)
    at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155)

The error can be prevented if `Mixin`s are added to the `ObjectMapper` which ignore the corresponding class.

The `Log4jJsonObjectMapper` is initialized in the `JacksonFactory.JSON` class and I currently do not see any way to somehow configure this ObjectMapper with additional MixIns.

Hence, my question finally is:
How can I configure Log4J’s JSON mapping to not break on such non-serializable log content?

Thank you!
Dominik
Reply | Threaded
Open this post in threaded view
|

Re: How to configure ObjectMapper of Log4j2 JSON Layout

Volkan Yazıcı
Hello Dominik,

I don't know how you can achieve that using JsonLayout. That said, it
sounds like a really valid use case and I have already created a
log4j2-logstash-layout ticket for it
<https://github.com/vy/log4j2-logstash-layout/issues/35>. I plan to
implement it in a couple of weeks and cut a new release. I will post a
follow-up to this thread when that happens.

Cheers.

On Fri, Jul 12, 2019 at 2:38 PM Dominik Sandjaja <
[hidden email]> wrote:

> Hello,
>
> we are using the JSON layout in our Spring Boot application to easily
> ingest the logs in Elasticsearch.
>
> One message that we log is about failed conversions, when an incoming
> request parameter cannot be converted correctly. As we use Spring’s
> Converter functionality for this, these `IllegalArgumentException`s are
> wrapped in a `ConversionFailedException`. This exception has fields of type
> `TypeDescriptor` which hold information about the source and the target
> type of the conversion.
>
> The problem is, that these `TypeDescriptor` objects cannot be serialized
> to JSON. The simple call
>
>
> com.fasterxml.jackson.databind.ObjectMapper().writeValueAsString(org.springframework.core.convert.TypeDescriptor.valueOf(String.javaClass))
>
> throws an exception like
>
> ERROR StatusLogger
> com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Direct
> self-reference leading to cycle (through reference chain:
> org.springframework.core.ResolvableType["componentType"]->org.springframework.core.ResolvableType["componentType"])
> com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Direct
> self-reference leading to cycle (through reference chain:
> org.springframework.core.ResolvableType["componentType"]->org.springframework.core.ResolvableType["componentType"])
>     at
> com.fasterxml.jackson.databind.exc.InvalidDefinitionException.from(InvalidDefinitionException.java:77)
>     at
> com.fasterxml.jackson.databind.SerializerProvider.reportBadDefinition(SerializerProvider.java:1191)
>     at
> com.fasterxml.jackson.databind.ser.BeanPropertyWriter._handleSelfReference(BeanPropertyWriter.java:944)
>     at
> com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:721)
>     at
> com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719)
>     at
> com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155)
>
> The error can be prevented if `Mixin`s are added to the `ObjectMapper`
> which ignore the corresponding class.
>
> The `Log4jJsonObjectMapper` is initialized in the `JacksonFactory.JSON`
> class and I currently do not see any way to somehow configure this
> ObjectMapper with additional MixIns.
>
> Hence, my question finally is:
> How can I configure Log4J’s JSON mapping to not break on such
> non-serializable log content?
>
> Thank you!
> Dominik
>
Reply | Threaded
Open this post in threaded view
|

Re: How to configure ObjectMapper of Log4j2 JSON Layout

Gary Gregory-4
In reply to this post by Dominik Sandjaja
Hi Dominik

I see two ways of addressing this:
1) Add a setObjectMapper(ObjectMapper)
to org.apache.logging.log4j.core.layout.JsonLayout.Builder
  This would work well for programmatic configurations
and/or:
2) Add a setObjectMapperFactory(String)
to org.apache.logging.log4j.core.layout.JsonLayout.Builder
  This would add a class name for implementors of a new interface
ObjectMapperFactory.
  This would work nicely for configuration files.

To keep all users in play, 2) seems best.

Thoughts?

Gary


On Fri, Jul 12, 2019 at 8:38 AM Dominik Sandjaja <
[hidden email]> wrote:

> Hello,
>
> we are using the JSON layout in our Spring Boot application to easily
> ingest the logs in Elasticsearch.
>
> One message that we log is about failed conversions, when an incoming
> request parameter cannot be converted correctly. As we use Spring’s
> Converter functionality for this, these `IllegalArgumentException`s are
> wrapped in a `ConversionFailedException`. This exception has fields of type
> `TypeDescriptor` which hold information about the source and the target
> type of the conversion.
>
> The problem is, that these `TypeDescriptor` objects cannot be serialized
> to JSON. The simple call
>
>
> com.fasterxml.jackson.databind.ObjectMapper().writeValueAsString(org.springframework.core.convert.TypeDescriptor.valueOf(String.javaClass))
>
> throws an exception like
>
> ERROR StatusLogger
> com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Direct
> self-reference leading to cycle (through reference chain:
> org.springframework.core.ResolvableType["componentType"]->org.springframework.core.ResolvableType["componentType"])
> com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Direct
> self-reference leading to cycle (through reference chain:
> org.springframework.core.ResolvableType["componentType"]->org.springframework.core.ResolvableType["componentType"])
>     at
> com.fasterxml.jackson.databind.exc.InvalidDefinitionException.from(InvalidDefinitionException.java:77)
>     at
> com.fasterxml.jackson.databind.SerializerProvider.reportBadDefinition(SerializerProvider.java:1191)
>     at
> com.fasterxml.jackson.databind.ser.BeanPropertyWriter._handleSelfReference(BeanPropertyWriter.java:944)
>     at
> com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:721)
>     at
> com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719)
>     at
> com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155)
>
> The error can be prevented if `Mixin`s are added to the `ObjectMapper`
> which ignore the corresponding class.
>
> The `Log4jJsonObjectMapper` is initialized in the `JacksonFactory.JSON`
> class and I currently do not see any way to somehow configure this
> ObjectMapper with additional MixIns.
>
> Hence, my question finally is:
> How can I configure Log4J’s JSON mapping to not break on such
> non-serializable log content?
>
> Thank you!
> Dominik
>
Reply | Threaded
Open this post in threaded view
|

Re: How to configure ObjectMapper of Log4j2 JSON Layout

Matt Sicker
Plugin maybe? I think we could potentially extend the plugin system as a
general dependency and configuration injection system. We can make less
special case methods of pseudo components and options outside that.

On Fri, Jul 12, 2019 at 17:34, Gary Gregory <[hidden email]> wrote:

> Hi Dominik
>
> I see two ways of addressing this:
> 1) Add a setObjectMapper(ObjectMapper)
> to org.apache.logging.log4j.core.layout.JsonLayout.Builder
>   This would work well for programmatic configurations
> and/or:
> 2) Add a setObjectMapperFactory(String)
> to org.apache.logging.log4j.core.layout.JsonLayout.Builder
>   This would add a class name for implementors of a new interface
> ObjectMapperFactory.
>   This would work nicely for configuration files.
>
> To keep all users in play, 2) seems best.
>
> Thoughts?
>
> Gary
>
>
> On Fri, Jul 12, 2019 at 8:38 AM Dominik Sandjaja <
> [hidden email]> wrote:
>
> > Hello,
> >
> > we are using the JSON layout in our Spring Boot application to easily
> > ingest the logs in Elasticsearch.
> >
> > One message that we log is about failed conversions, when an incoming
> > request parameter cannot be converted correctly. As we use Spring’s
> > Converter functionality for this, these `IllegalArgumentException`s are
> > wrapped in a `ConversionFailedException`. This exception has fields of
> type
> > `TypeDescriptor` which hold information about the source and the target
> > type of the conversion.
> >
> > The problem is, that these `TypeDescriptor` objects cannot be serialized
> > to JSON. The simple call
> >
> >
> >
> com.fasterxml.jackson.databind.ObjectMapper().writeValueAsString(org.springframework.core.convert.TypeDescriptor.valueOf(String.javaClass))
> >
> > throws an exception like
> >
> > ERROR StatusLogger
> > com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Direct
> > self-reference leading to cycle (through reference chain:
> >
> org.springframework.core.ResolvableType["componentType"]->org.springframework.core.ResolvableType["componentType"])
> > com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Direct
> > self-reference leading to cycle (through reference chain:
> >
> org.springframework.core.ResolvableType["componentType"]->org.springframework.core.ResolvableType["componentType"])
> >     at
> >
> com.fasterxml.jackson.databind.exc.InvalidDefinitionException.from(InvalidDefinitionException.java:77)
> >     at
> >
> com.fasterxml.jackson.databind.SerializerProvider.reportBadDefinition(SerializerProvider.java:1191)
> >     at
> >
> com.fasterxml.jackson.databind.ser.BeanPropertyWriter._handleSelfReference(BeanPropertyWriter.java:944)
> >     at
> >
> com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:721)
> >     at
> >
> com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719)
> >     at
> >
> com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155)
> >
> > The error can be prevented if `Mixin`s are added to the `ObjectMapper`
> > which ignore the corresponding class.
> >
> > The `Log4jJsonObjectMapper` is initialized in the `JacksonFactory.JSON`
> > class and I currently do not see any way to somehow configure this
> > ObjectMapper with additional MixIns.
> >
> > Hence, my question finally is:
> > How can I configure Log4J’s JSON mapping to not break on such
> > non-serializable log content?
> >
> > Thank you!
> > Dominik
> >
>
--
Matt Sicker <[hidden email]>
Reply | Threaded
Open this post in threaded view
|

Re: How to configure ObjectMapper of Log4j2 JSON Layout

Matt Sicker
And by plugin, see for example the various BlockingQueueFactory plugins.

On Fri, Jul 12, 2019 at 18:09, Matt Sicker <[hidden email]> wrote:

> Plugin maybe? I think we could potentially extend the plugin system as a
> general dependency and configuration injection system. We can make less
> special case methods of pseudo components and options outside that.
>
> On Fri, Jul 12, 2019 at 17:34, Gary Gregory <[hidden email]>
> wrote:
>
>> Hi Dominik
>>
>> I see two ways of addressing this:
>> 1) Add a setObjectMapper(ObjectMapper)
>> to org.apache.logging.log4j.core.layout.JsonLayout.Builder
>>   This would work well for programmatic configurations
>> and/or:
>> 2) Add a setObjectMapperFactory(String)
>> to org.apache.logging.log4j.core.layout.JsonLayout.Builder
>>   This would add a class name for implementors of a new interface
>> ObjectMapperFactory.
>>   This would work nicely for configuration files.
>>
>> To keep all users in play, 2) seems best.
>>
>> Thoughts?
>>
>> Gary
>>
>>
>> On Fri, Jul 12, 2019 at 8:38 AM Dominik Sandjaja <
>> [hidden email]> wrote:
>>
>> > Hello,
>> >
>> > we are using the JSON layout in our Spring Boot application to easily
>> > ingest the logs in Elasticsearch.
>> >
>> > One message that we log is about failed conversions, when an incoming
>> > request parameter cannot be converted correctly. As we use Spring’s
>> > Converter functionality for this, these `IllegalArgumentException`s are
>> > wrapped in a `ConversionFailedException`. This exception has fields of
>> type
>> > `TypeDescriptor` which hold information about the source and the target
>> > type of the conversion.
>> >
>> > The problem is, that these `TypeDescriptor` objects cannot be serialized
>> > to JSON. The simple call
>> >
>> >
>> >
>> com.fasterxml.jackson.databind.ObjectMapper().writeValueAsString(org.springframework.core.convert.TypeDescriptor.valueOf(String.javaClass))
>> >
>> > throws an exception like
>> >
>> > ERROR StatusLogger
>> > com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Direct
>> > self-reference leading to cycle (through reference chain:
>> >
>> org.springframework.core.ResolvableType["componentType"]->org.springframework.core.ResolvableType["componentType"])
>> > com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Direct
>> > self-reference leading to cycle (through reference chain:
>> >
>> org.springframework.core.ResolvableType["componentType"]->org.springframework.core.ResolvableType["componentType"])
>> >     at
>> >
>> com.fasterxml.jackson.databind.exc.InvalidDefinitionException.from(InvalidDefinitionException.java:77)
>> >     at
>> >
>> com.fasterxml.jackson.databind.SerializerProvider.reportBadDefinition(SerializerProvider.java:1191)
>> >     at
>> >
>> com.fasterxml.jackson.databind.ser.BeanPropertyWriter._handleSelfReference(BeanPropertyWriter.java:944)
>> >     at
>> >
>> com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:721)
>> >     at
>> >
>> com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719)
>> >     at
>> >
>> com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155)
>> >
>> > The error can be prevented if `Mixin`s are added to the `ObjectMapper`
>> > which ignore the corresponding class.
>> >
>> > The `Log4jJsonObjectMapper` is initialized in the `JacksonFactory.JSON`
>> > class and I currently do not see any way to somehow configure this
>> > ObjectMapper with additional MixIns.
>> >
>> > Hence, my question finally is:
>> > How can I configure Log4J’s JSON mapping to not break on such
>> > non-serializable log content?
>> >
>> > Thank you!
>> > Dominik
>> >
>>
> --
> Matt Sicker <[hidden email]>
>
--
Matt Sicker <[hidden email]>
Reply | Threaded
Open this post in threaded view
|

Re: How to configure ObjectMapper of Log4j2 JSON Layout

Carter Kozak-2
I like the idea of an ObjectMapperFactory plugin point. That gives us a cleaner builder API on JsonLayout where we can potentially pass in a supplier for an ObjectMapper instance from programmatic configuration.

On Fri, Jul 12, 2019, at 19:10, Matt Sicker wrote:

> And by plugin, see for example the various BlockingQueueFactory plugins.
>
> On Fri, Jul 12, 2019 at 18:09, Matt Sicker <[hidden email]> wrote:
>
> > Plugin maybe? I think we could potentially extend the plugin system as a
> > general dependency and configuration injection system. We can make less
> > special case methods of pseudo components and options outside that.
> >
> > On Fri, Jul 12, 2019 at 17:34, Gary Gregory <[hidden email]>
> > wrote:
> >
> >> Hi Dominik
> >>
> >> I see two ways of addressing this:
> >> 1) Add a setObjectMapper(ObjectMapper)
> >> to org.apache.logging.log4j.core.layout.JsonLayout.Builder
> >> This would work well for programmatic configurations
> >> and/or:
> >> 2) Add a setObjectMapperFactory(String)
> >> to org.apache.logging.log4j.core.layout.JsonLayout.Builder
> >> This would add a class name for implementors of a new interface
> >> ObjectMapperFactory.
> >> This would work nicely for configuration files.
> >>
> >> To keep all users in play, 2) seems best.
> >>
> >> Thoughts?
> >>
> >> Gary
> >>
> >>
> >> On Fri, Jul 12, 2019 at 8:38 AM Dominik Sandjaja <
> >> [hidden email]> wrote:
> >>
> >> > Hello,
> >> >
> >> > we are using the JSON layout in our Spring Boot application to easily
> >> > ingest the logs in Elasticsearch.
> >> >
> >> > One message that we log is about failed conversions, when an incoming
> >> > request parameter cannot be converted correctly. As we use Spring’s
> >> > Converter functionality for this, these `IllegalArgumentException`s are
> >> > wrapped in a `ConversionFailedException`. This exception has fields of
> >> type
> >> > `TypeDescriptor` which hold information about the source and the target
> >> > type of the conversion.
> >> >
> >> > The problem is, that these `TypeDescriptor` objects cannot be serialized
> >> > to JSON. The simple call
> >> >
> >> >
> >> >
> >> com.fasterxml.jackson.databind.ObjectMapper().writeValueAsString(org.springframework.core.convert.TypeDescriptor.valueOf(String.javaClass))
> >> >
> >> > throws an exception like
> >> >
> >> > ERROR StatusLogger
> >> > com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Direct
> >> > self-reference leading to cycle (through reference chain:
> >> >
> >> org.springframework.core.ResolvableType["componentType"]->org.springframework.core.ResolvableType["componentType"])
> >> > com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Direct
> >> > self-reference leading to cycle (through reference chain:
> >> >
> >> org.springframework.core.ResolvableType["componentType"]->org.springframework.core.ResolvableType["componentType"])
> >> > at
> >> >
> >> com.fasterxml.jackson.databind.exc.InvalidDefinitionException.from(InvalidDefinitionException.java:77)
> >> > at
> >> >
> >> com.fasterxml.jackson.databind.SerializerProvider.reportBadDefinition(SerializerProvider.java:1191)
> >> > at
> >> >
> >> com.fasterxml.jackson.databind.ser.BeanPropertyWriter._handleSelfReference(BeanPropertyWriter.java:944)
> >> > at
> >> >
> >> com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:721)
> >> > at
> >> >
> >> com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719)
> >> > at
> >> >
> >> com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155)
> >> >
> >> > The error can be prevented if `Mixin`s are added to the `ObjectMapper`
> >> > which ignore the corresponding class.
> >> >
> >> > The `Log4jJsonObjectMapper` is initialized in the `JacksonFactory.JSON`
> >> > class and I currently do not see any way to somehow configure this
> >> > ObjectMapper with additional MixIns.
> >> >
> >> > Hence, my question finally is:
> >> > How can I configure Log4J’s JSON mapping to not break on such
> >> > non-serializable log content?
> >> >
> >> > Thank you!
> >> > Dominik
> >> >
> >>
> > --
> > Matt Sicker <[hidden email]>
> >
> --
> Matt Sicker <[hidden email]>
>
Reply | Threaded
Open this post in threaded view
|

Re: How to configure ObjectMapper of Log4j2 JSON Layout

Ralph Goers
Actually, I was thinking that we can keep the existing ObjectMapper but supply a Function that can “enhance” the ObjectMapper either at the end of the Log4jJsonObjectMapper constructor, at the end of JacksonFactory.newObjectMapper, or after the ObjectMapper is constructed in JacksonFactory.newWriter. Whether that is done via a Plugin or some other way I hadn’t looked at too much. But I suppose it could either be a Plugin that is a parameter to the Layout and then passed into the newWriter method or it could be looked up by the JSON method in JacksonFactory. Passing it on the Layout probably makes it more flexible.

Ralph

> On Jul 12, 2019, at 4:13 PM, Carter Kozak <[hidden email]> wrote:
>
> I like the idea of an ObjectMapperFactory plugin point. That gives us a cleaner builder API on JsonLayout where we can potentially pass in a supplier for an ObjectMapper instance from programmatic configuration.
>
> On Fri, Jul 12, 2019, at 19:10, Matt Sicker wrote:
>> And by plugin, see for example the various BlockingQueueFactory plugins.
>>
>> On Fri, Jul 12, 2019 at 18:09, Matt Sicker <[hidden email]> wrote:
>>
>>> Plugin maybe? I think we could potentially extend the plugin system as a
>>> general dependency and configuration injection system. We can make less
>>> special case methods of pseudo components and options outside that.
>>>
>>> On Fri, Jul 12, 2019 at 17:34, Gary Gregory <[hidden email]>
>>> wrote:
>>>
>>>> Hi Dominik
>>>>
>>>> I see two ways of addressing this:
>>>> 1) Add a setObjectMapper(ObjectMapper)
>>>> to org.apache.logging.log4j.core.layout.JsonLayout.Builder
>>>> This would work well for programmatic configurations
>>>> and/or:
>>>> 2) Add a setObjectMapperFactory(String)
>>>> to org.apache.logging.log4j.core.layout.JsonLayout.Builder
>>>> This would add a class name for implementors of a new interface
>>>> ObjectMapperFactory.
>>>> This would work nicely for configuration files.
>>>>
>>>> To keep all users in play, 2) seems best.
>>>>
>>>> Thoughts?
>>>>
>>>> Gary
>>>>
>>>>
>>>> On Fri, Jul 12, 2019 at 8:38 AM Dominik Sandjaja <
>>>> [hidden email]> wrote:
>>>>
>>>>> Hello,
>>>>>
>>>>> we are using the JSON layout in our Spring Boot application to easily
>>>>> ingest the logs in Elasticsearch.
>>>>>
>>>>> One message that we log is about failed conversions, when an incoming
>>>>> request parameter cannot be converted correctly. As we use Spring’s
>>>>> Converter functionality for this, these `IllegalArgumentException`s are
>>>>> wrapped in a `ConversionFailedException`. This exception has fields of
>>>> type
>>>>> `TypeDescriptor` which hold information about the source and the target
>>>>> type of the conversion.
>>>>>
>>>>> The problem is, that these `TypeDescriptor` objects cannot be serialized
>>>>> to JSON. The simple call
>>>>>
>>>>>
>>>>>
>>>> com.fasterxml.jackson.databind.ObjectMapper().writeValueAsString(org.springframework.core.convert.TypeDescriptor.valueOf(String.javaClass))
>>>>>
>>>>> throws an exception like
>>>>>
>>>>> ERROR StatusLogger
>>>>> com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Direct
>>>>> self-reference leading to cycle (through reference chain:
>>>>>
>>>> org.springframework.core.ResolvableType["componentType"]->org.springframework.core.ResolvableType["componentType"])
>>>>> com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Direct
>>>>> self-reference leading to cycle (through reference chain:
>>>>>
>>>> org.springframework.core.ResolvableType["componentType"]->org.springframework.core.ResolvableType["componentType"])
>>>>> at
>>>>>
>>>> com.fasterxml.jackson.databind.exc.InvalidDefinitionException.from(InvalidDefinitionException.java:77)
>>>>> at
>>>>>
>>>> com.fasterxml.jackson.databind.SerializerProvider.reportBadDefinition(SerializerProvider.java:1191)
>>>>> at
>>>>>
>>>> com.fasterxml.jackson.databind.ser.BeanPropertyWriter._handleSelfReference(BeanPropertyWriter.java:944)
>>>>> at
>>>>>
>>>> com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:721)
>>>>> at
>>>>>
>>>> com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719)
>>>>> at
>>>>>
>>>> com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155)
>>>>>
>>>>> The error can be prevented if `Mixin`s are added to the `ObjectMapper`
>>>>> which ignore the corresponding class.
>>>>>
>>>>> The `Log4jJsonObjectMapper` is initialized in the `JacksonFactory.JSON`
>>>>> class and I currently do not see any way to somehow configure this
>>>>> ObjectMapper with additional MixIns.
>>>>>
>>>>> Hence, my question finally is:
>>>>> How can I configure Log4J’s JSON mapping to not break on such
>>>>> non-serializable log content?
>>>>>
>>>>> Thank you!
>>>>> Dominik
>>>>>
>>>>
>>> --
>>> Matt Sicker <[hidden email]>
>>>
>> --
>> Matt Sicker <[hidden email]>
>>



---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: How to configure ObjectMapper of Log4j2 JSON Layout

Matt Sicker
Being a plugin means you can inject it as a @PluginElement wherever. Plus,
that makes it easier for users to write a custom plugin class to configure
any other options they want. Comparable to the DataSourceFactory or
whatever it’s called for JDBC drivers in that plugin.

Keeping it as an element should make it more flexible with future
programmatic configuration ideas. The more component based everything is,
the simpler it’ll be to make customizations without users digging into the
internals of LoggerConfig et al.

On Fri, Jul 12, 2019 at 18:36, Ralph Goers <[hidden email]>
wrote:

> Actually, I was thinking that we can keep the existing ObjectMapper but
> supply a Function that can “enhance” the ObjectMapper either at the end of
> the Log4jJsonObjectMapper constructor, at the end of
> JacksonFactory.newObjectMapper, or after the ObjectMapper is constructed in
> JacksonFactory.newWriter. Whether that is done via a Plugin or some other
> way I hadn’t looked at too much. But I suppose it could either be a Plugin
> that is a parameter to the Layout and then passed into the newWriter method
> or it could be looked up by the JSON method in JacksonFactory. Passing it
> on the Layout probably makes it more flexible.
>
> Ralph
>
> > On Jul 12, 2019, at 4:13 PM, Carter Kozak <[hidden email]> wrote:
> >
> > I like the idea of an ObjectMapperFactory plugin point. That gives us a
> cleaner builder API on JsonLayout where we can potentially pass in a
> supplier for an ObjectMapper instance from programmatic configuration.
> >
> > On Fri, Jul 12, 2019, at 19:10, Matt Sicker wrote:
> >> And by plugin, see for example the various BlockingQueueFactory plugins.
> >>
> >> On Fri, Jul 12, 2019 at 18:09, Matt Sicker <[hidden email]> wrote:
> >>
> >>> Plugin maybe? I think we could potentially extend the plugin system as
> a
> >>> general dependency and configuration injection system. We can make less
> >>> special case methods of pseudo components and options outside that.
> >>>
> >>> On Fri, Jul 12, 2019 at 17:34, Gary Gregory <[hidden email]>
> >>> wrote:
> >>>
> >>>> Hi Dominik
> >>>>
> >>>> I see two ways of addressing this:
> >>>> 1) Add a setObjectMapper(ObjectMapper)
> >>>> to org.apache.logging.log4j.core.layout.JsonLayout.Builder
> >>>> This would work well for programmatic configurations
> >>>> and/or:
> >>>> 2) Add a setObjectMapperFactory(String)
> >>>> to org.apache.logging.log4j.core.layout.JsonLayout.Builder
> >>>> This would add a class name for implementors of a new interface
> >>>> ObjectMapperFactory.
> >>>> This would work nicely for configuration files.
> >>>>
> >>>> To keep all users in play, 2) seems best.
> >>>>
> >>>> Thoughts?
> >>>>
> >>>> Gary
> >>>>
> >>>>
> >>>> On Fri, Jul 12, 2019 at 8:38 AM Dominik Sandjaja <
> >>>> [hidden email]> wrote:
> >>>>
> >>>>> Hello,
> >>>>>
> >>>>> we are using the JSON layout in our Spring Boot application to easily
> >>>>> ingest the logs in Elasticsearch.
> >>>>>
> >>>>> One message that we log is about failed conversions, when an incoming
> >>>>> request parameter cannot be converted correctly. As we use Spring’s
> >>>>> Converter functionality for this, these `IllegalArgumentException`s
> are
> >>>>> wrapped in a `ConversionFailedException`. This exception has fields
> of
> >>>> type
> >>>>> `TypeDescriptor` which hold information about the source and the
> target
> >>>>> type of the conversion.
> >>>>>
> >>>>> The problem is, that these `TypeDescriptor` objects cannot be
> serialized
> >>>>> to JSON. The simple call
> >>>>>
> >>>>>
> >>>>>
> >>>>
> com.fasterxml.jackson.databind.ObjectMapper().writeValueAsString(org.springframework.core.convert.TypeDescriptor.valueOf(String.javaClass))
> >>>>>
> >>>>> throws an exception like
> >>>>>
> >>>>> ERROR StatusLogger
> >>>>> com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Direct
> >>>>> self-reference leading to cycle (through reference chain:
> >>>>>
> >>>>
> org.springframework.core.ResolvableType["componentType"]->org.springframework.core.ResolvableType["componentType"])
> >>>>> com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Direct
> >>>>> self-reference leading to cycle (through reference chain:
> >>>>>
> >>>>
> org.springframework.core.ResolvableType["componentType"]->org.springframework.core.ResolvableType["componentType"])
> >>>>> at
> >>>>>
> >>>>
> com.fasterxml.jackson.databind.exc.InvalidDefinitionException.from(InvalidDefinitionException.java:77)
> >>>>> at
> >>>>>
> >>>>
> com.fasterxml.jackson.databind.SerializerProvider.reportBadDefinition(SerializerProvider.java:1191)
> >>>>> at
> >>>>>
> >>>>
> com.fasterxml.jackson.databind.ser.BeanPropertyWriter._handleSelfReference(BeanPropertyWriter.java:944)
> >>>>> at
> >>>>>
> >>>>
> com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:721)
> >>>>> at
> >>>>>
> >>>>
> com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719)
> >>>>> at
> >>>>>
> >>>>
> com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155)
> >>>>>
> >>>>> The error can be prevented if `Mixin`s are added to the
> `ObjectMapper`
> >>>>> which ignore the corresponding class.
> >>>>>
> >>>>> The `Log4jJsonObjectMapper` is initialized in the
> `JacksonFactory.JSON`
> >>>>> class and I currently do not see any way to somehow configure this
> >>>>> ObjectMapper with additional MixIns.
> >>>>>
> >>>>> Hence, my question finally is:
> >>>>> How can I configure Log4J’s JSON mapping to not break on such
> >>>>> non-serializable log content?
> >>>>>
> >>>>> Thank you!
> >>>>> Dominik
> >>>>>
> >>>>
> >>> --
> >>> Matt Sicker <[hidden email]>
> >>>
> >> --
> >> Matt Sicker <[hidden email]>
> >>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
> --
Matt Sicker <[hidden email]>
Reply | Threaded
Open this post in threaded view
|

Re: How to configure ObjectMapper of Log4j2 JSON Layout

Ralph Goers
Just because it is a plugin doesn’t mean it can be used anywhere. The JacksonFactory could collect plugins for its own category, which would mean they couldn’t be specified in the configuration. Only “core” plugins can directly be used as parameters in the configuration. But yes, once it is a core plugin it could be passed into the newWriter method as I describe.

Ralph

> On Jul 13, 2019, at 10:12 AM, Matt Sicker <[hidden email]> wrote:
>
> Being a plugin means you can inject it as a @PluginElement wherever. Plus,
> that makes it easier for users to write a custom plugin class to configure
> any other options they want. Comparable to the DataSourceFactory or
> whatever it’s called for JDBC drivers in that plugin.
>
> Keeping it as an element should make it more flexible with future
> programmatic configuration ideas. The more component based everything is,
> the simpler it’ll be to make customizations without users digging into the
> internals of LoggerConfig et al.
>
> On Fri, Jul 12, 2019 at 18:36, Ralph Goers <[hidden email]>
> wrote:
>
>> Actually, I was thinking that we can keep the existing ObjectMapper but
>> supply a Function that can “enhance” the ObjectMapper either at the end of
>> the Log4jJsonObjectMapper constructor, at the end of
>> JacksonFactory.newObjectMapper, or after the ObjectMapper is constructed in
>> JacksonFactory.newWriter. Whether that is done via a Plugin or some other
>> way I hadn’t looked at too much. But I suppose it could either be a Plugin
>> that is a parameter to the Layout and then passed into the newWriter method
>> or it could be looked up by the JSON method in JacksonFactory. Passing it
>> on the Layout probably makes it more flexible.
>>
>> Ralph
>>
>>> On Jul 12, 2019, at 4:13 PM, Carter Kozak <[hidden email]> wrote:
>>>
>>> I like the idea of an ObjectMapperFactory plugin point. That gives us a
>> cleaner builder API on JsonLayout where we can potentially pass in a
>> supplier for an ObjectMapper instance from programmatic configuration.
>>>
>>> On Fri, Jul 12, 2019, at 19:10, Matt Sicker wrote:
>>>> And by plugin, see for example the various BlockingQueueFactory plugins.
>>>>
>>>> On Fri, Jul 12, 2019 at 18:09, Matt Sicker <[hidden email]> wrote:
>>>>
>>>>> Plugin maybe? I think we could potentially extend the plugin system as
>> a
>>>>> general dependency and configuration injection system. We can make less
>>>>> special case methods of pseudo components and options outside that.
>>>>>
>>>>> On Fri, Jul 12, 2019 at 17:34, Gary Gregory <[hidden email]>
>>>>> wrote:
>>>>>
>>>>>> Hi Dominik
>>>>>>
>>>>>> I see two ways of addressing this:
>>>>>> 1) Add a setObjectMapper(ObjectMapper)
>>>>>> to org.apache.logging.log4j.core.layout.JsonLayout.Builder
>>>>>> This would work well for programmatic configurations
>>>>>> and/or:
>>>>>> 2) Add a setObjectMapperFactory(String)
>>>>>> to org.apache.logging.log4j.core.layout.JsonLayout.Builder
>>>>>> This would add a class name for implementors of a new interface
>>>>>> ObjectMapperFactory.
>>>>>> This would work nicely for configuration files.
>>>>>>
>>>>>> To keep all users in play, 2) seems best.
>>>>>>
>>>>>> Thoughts?
>>>>>>
>>>>>> Gary
>>>>>>
>>>>>>
>>>>>> On Fri, Jul 12, 2019 at 8:38 AM Dominik Sandjaja <
>>>>>> [hidden email]> wrote:
>>>>>>
>>>>>>> Hello,
>>>>>>>
>>>>>>> we are using the JSON layout in our Spring Boot application to easily
>>>>>>> ingest the logs in Elasticsearch.
>>>>>>>
>>>>>>> One message that we log is about failed conversions, when an incoming
>>>>>>> request parameter cannot be converted correctly. As we use Spring’s
>>>>>>> Converter functionality for this, these `IllegalArgumentException`s
>> are
>>>>>>> wrapped in a `ConversionFailedException`. This exception has fields
>> of
>>>>>> type
>>>>>>> `TypeDescriptor` which hold information about the source and the
>> target
>>>>>>> type of the conversion.
>>>>>>>
>>>>>>> The problem is, that these `TypeDescriptor` objects cannot be
>> serialized
>>>>>>> to JSON. The simple call
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>> com.fasterxml.jackson.databind.ObjectMapper().writeValueAsString(org.springframework.core.convert.TypeDescriptor.valueOf(String.javaClass))
>>>>>>>
>>>>>>> throws an exception like
>>>>>>>
>>>>>>> ERROR StatusLogger
>>>>>>> com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Direct
>>>>>>> self-reference leading to cycle (through reference chain:
>>>>>>>
>>>>>>
>> org.springframework.core.ResolvableType["componentType"]->org.springframework.core.ResolvableType["componentType"])
>>>>>>> com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Direct
>>>>>>> self-reference leading to cycle (through reference chain:
>>>>>>>
>>>>>>
>> org.springframework.core.ResolvableType["componentType"]->org.springframework.core.ResolvableType["componentType"])
>>>>>>> at
>>>>>>>
>>>>>>
>> com.fasterxml.jackson.databind.exc.InvalidDefinitionException.from(InvalidDefinitionException.java:77)
>>>>>>> at
>>>>>>>
>>>>>>
>> com.fasterxml.jackson.databind.SerializerProvider.reportBadDefinition(SerializerProvider.java:1191)
>>>>>>> at
>>>>>>>
>>>>>>
>> com.fasterxml.jackson.databind.ser.BeanPropertyWriter._handleSelfReference(BeanPropertyWriter.java:944)
>>>>>>> at
>>>>>>>
>>>>>>
>> com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:721)
>>>>>>> at
>>>>>>>
>>>>>>
>> com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719)
>>>>>>> at
>>>>>>>
>>>>>>
>> com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155)
>>>>>>>
>>>>>>> The error can be prevented if `Mixin`s are added to the
>> `ObjectMapper`
>>>>>>> which ignore the corresponding class.
>>>>>>>
>>>>>>> The `Log4jJsonObjectMapper` is initialized in the
>> `JacksonFactory.JSON`
>>>>>>> class and I currently do not see any way to somehow configure this
>>>>>>> ObjectMapper with additional MixIns.
>>>>>>>
>>>>>>> Hence, my question finally is:
>>>>>>> How can I configure Log4J’s JSON mapping to not break on such
>>>>>>> non-serializable log content?
>>>>>>>
>>>>>>> Thank you!
>>>>>>> Dominik
>>>>>>>
>>>>>>
>>>>> --
>>>>> Matt Sicker <[hidden email]>
>>>>>
>>>> --
>>>> Matt Sicker <[hidden email]>
>>>>
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>>
>> --
> Matt Sicker <[hidden email]>



---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: How to configure ObjectMapper of Log4j2 JSON Layout

Matt Sicker
Oh, that's a part of the plugin system I often forget about since most
plugins are core plugins. In any case, a plugin of some sort would be
the most flexible here I think.

On Sat, 13 Jul 2019 at 15:21, Ralph Goers <[hidden email]> wrote:

>
> Just because it is a plugin doesn’t mean it can be used anywhere. The JacksonFactory could collect plugins for its own category, which would mean they couldn’t be specified in the configuration. Only “core” plugins can directly be used as parameters in the configuration. But yes, once it is a core plugin it could be passed into the newWriter method as I describe.
>
> Ralph
>
> > On Jul 13, 2019, at 10:12 AM, Matt Sicker <[hidden email]> wrote:
> >
> > Being a plugin means you can inject it as a @PluginElement wherever. Plus,
> > that makes it easier for users to write a custom plugin class to configure
> > any other options they want. Comparable to the DataSourceFactory or
> > whatever it’s called for JDBC drivers in that plugin.
> >
> > Keeping it as an element should make it more flexible with future
> > programmatic configuration ideas. The more component based everything is,
> > the simpler it’ll be to make customizations without users digging into the
> > internals of LoggerConfig et al.
> >
> > On Fri, Jul 12, 2019 at 18:36, Ralph Goers <[hidden email]>
> > wrote:
> >
> >> Actually, I was thinking that we can keep the existing ObjectMapper but
> >> supply a Function that can “enhance” the ObjectMapper either at the end of
> >> the Log4jJsonObjectMapper constructor, at the end of
> >> JacksonFactory.newObjectMapper, or after the ObjectMapper is constructed in
> >> JacksonFactory.newWriter. Whether that is done via a Plugin or some other
> >> way I hadn’t looked at too much. But I suppose it could either be a Plugin
> >> that is a parameter to the Layout and then passed into the newWriter method
> >> or it could be looked up by the JSON method in JacksonFactory. Passing it
> >> on the Layout probably makes it more flexible.
> >>
> >> Ralph
> >>
> >>> On Jul 12, 2019, at 4:13 PM, Carter Kozak <[hidden email]> wrote:
> >>>
> >>> I like the idea of an ObjectMapperFactory plugin point. That gives us a
> >> cleaner builder API on JsonLayout where we can potentially pass in a
> >> supplier for an ObjectMapper instance from programmatic configuration.
> >>>
> >>> On Fri, Jul 12, 2019, at 19:10, Matt Sicker wrote:
> >>>> And by plugin, see for example the various BlockingQueueFactory plugins.
> >>>>
> >>>> On Fri, Jul 12, 2019 at 18:09, Matt Sicker <[hidden email]> wrote:
> >>>>
> >>>>> Plugin maybe? I think we could potentially extend the plugin system as
> >> a
> >>>>> general dependency and configuration injection system. We can make less
> >>>>> special case methods of pseudo components and options outside that.
> >>>>>
> >>>>> On Fri, Jul 12, 2019 at 17:34, Gary Gregory <[hidden email]>
> >>>>> wrote:
> >>>>>
> >>>>>> Hi Dominik
> >>>>>>
> >>>>>> I see two ways of addressing this:
> >>>>>> 1) Add a setObjectMapper(ObjectMapper)
> >>>>>> to org.apache.logging.log4j.core.layout.JsonLayout.Builder
> >>>>>> This would work well for programmatic configurations
> >>>>>> and/or:
> >>>>>> 2) Add a setObjectMapperFactory(String)
> >>>>>> to org.apache.logging.log4j.core.layout.JsonLayout.Builder
> >>>>>> This would add a class name for implementors of a new interface
> >>>>>> ObjectMapperFactory.
> >>>>>> This would work nicely for configuration files.
> >>>>>>
> >>>>>> To keep all users in play, 2) seems best.
> >>>>>>
> >>>>>> Thoughts?
> >>>>>>
> >>>>>> Gary
> >>>>>>
> >>>>>>
> >>>>>> On Fri, Jul 12, 2019 at 8:38 AM Dominik Sandjaja <
> >>>>>> [hidden email]> wrote:
> >>>>>>
> >>>>>>> Hello,
> >>>>>>>
> >>>>>>> we are using the JSON layout in our Spring Boot application to easily
> >>>>>>> ingest the logs in Elasticsearch.
> >>>>>>>
> >>>>>>> One message that we log is about failed conversions, when an incoming
> >>>>>>> request parameter cannot be converted correctly. As we use Spring’s
> >>>>>>> Converter functionality for this, these `IllegalArgumentException`s
> >> are
> >>>>>>> wrapped in a `ConversionFailedException`. This exception has fields
> >> of
> >>>>>> type
> >>>>>>> `TypeDescriptor` which hold information about the source and the
> >> target
> >>>>>>> type of the conversion.
> >>>>>>>
> >>>>>>> The problem is, that these `TypeDescriptor` objects cannot be
> >> serialized
> >>>>>>> to JSON. The simple call
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>
> >> com.fasterxml.jackson.databind.ObjectMapper().writeValueAsString(org.springframework.core.convert.TypeDescriptor.valueOf(String.javaClass))
> >>>>>>>
> >>>>>>> throws an exception like
> >>>>>>>
> >>>>>>> ERROR StatusLogger
> >>>>>>> com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Direct
> >>>>>>> self-reference leading to cycle (through reference chain:
> >>>>>>>
> >>>>>>
> >> org.springframework.core.ResolvableType["componentType"]->org.springframework.core.ResolvableType["componentType"])
> >>>>>>> com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Direct
> >>>>>>> self-reference leading to cycle (through reference chain:
> >>>>>>>
> >>>>>>
> >> org.springframework.core.ResolvableType["componentType"]->org.springframework.core.ResolvableType["componentType"])
> >>>>>>> at
> >>>>>>>
> >>>>>>
> >> com.fasterxml.jackson.databind.exc.InvalidDefinitionException.from(InvalidDefinitionException.java:77)
> >>>>>>> at
> >>>>>>>
> >>>>>>
> >> com.fasterxml.jackson.databind.SerializerProvider.reportBadDefinition(SerializerProvider.java:1191)
> >>>>>>> at
> >>>>>>>
> >>>>>>
> >> com.fasterxml.jackson.databind.ser.BeanPropertyWriter._handleSelfReference(BeanPropertyWriter.java:944)
> >>>>>>> at
> >>>>>>>
> >>>>>>
> >> com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:721)
> >>>>>>> at
> >>>>>>>
> >>>>>>
> >> com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719)
> >>>>>>> at
> >>>>>>>
> >>>>>>
> >> com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155)
> >>>>>>>
> >>>>>>> The error can be prevented if `Mixin`s are added to the
> >> `ObjectMapper`
> >>>>>>> which ignore the corresponding class.
> >>>>>>>
> >>>>>>> The `Log4jJsonObjectMapper` is initialized in the
> >> `JacksonFactory.JSON`
> >>>>>>> class and I currently do not see any way to somehow configure this
> >>>>>>> ObjectMapper with additional MixIns.
> >>>>>>>
> >>>>>>> Hence, my question finally is:
> >>>>>>> How can I configure Log4J’s JSON mapping to not break on such
> >>>>>>> non-serializable log content?
> >>>>>>>
> >>>>>>> Thank you!
> >>>>>>> Dominik
> >>>>>>>
> >>>>>>
> >>>>> --
> >>>>> Matt Sicker <[hidden email]>
> >>>>>
> >>>> --
> >>>> Matt Sicker <[hidden email]>
> >>>>
> >>
> >>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: [hidden email]
> >> For additional commands, e-mail: [hidden email]
> >>
> >> --
> > Matt Sicker <[hidden email]>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>


--
Matt Sicker <[hidden email]>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: How to configure ObjectMapper of Log4j2 JSON Layout

Dominik Sandjaja
Hello everybody,

thank you all for the discussion about this topic.

My initial thought was that I was maybe missing some configuration option for setting the classname of one of the initialization classes to override the object creation.
But after looking a bit into the Log4J Plugin system I agree, that this is the cleanest way forward from an overall architectural point of view.

I don't think that I will be able to provide a Pull Request within the next few weeks (hooray for holidays!), but if time permits, I could have a shot at it in September.

Until then, we will most likely remove this one exception from the Exception chain.

Thank you all!

On 13.07.19, 22:34, "Matt Sicker" <[hidden email]> wrote:

    Oh, that's a part of the plugin system I often forget about since most
    plugins are core plugins. In any case, a plugin of some sort would be
    the most flexible here I think.
   
    On Sat, 13 Jul 2019 at 15:21, Ralph Goers <[hidden email]> wrote:
    >
    > Just because it is a plugin doesn’t mean it can be used anywhere. The JacksonFactory could collect plugins for its own category, which would mean they couldn’t be specified in the configuration. Only “core” plugins can directly be used as parameters in the configuration. But yes, once it is a core plugin it could be passed into the newWriter method as I describe.
    >
    > Ralph
    >
    > > On Jul 13, 2019, at 10:12 AM, Matt Sicker <[hidden email]> wrote:
    > >
    > > Being a plugin means you can inject it as a @PluginElement wherever. Plus,
    > > that makes it easier for users to write a custom plugin class to configure
    > > any other options they want. Comparable to the DataSourceFactory or
    > > whatever it’s called for JDBC drivers in that plugin.
    > >
    > > Keeping it as an element should make it more flexible with future
    > > programmatic configuration ideas. The more component based everything is,
    > > the simpler it’ll be to make customizations without users digging into the
    > > internals of LoggerConfig et al.
    > >
    > > On Fri, Jul 12, 2019 at 18:36, Ralph Goers <[hidden email]>
    > > wrote:
    > >
    > >> Actually, I was thinking that we can keep the existing ObjectMapper but
    > >> supply a Function that can “enhance” the ObjectMapper either at the end of
    > >> the Log4jJsonObjectMapper constructor, at the end of
    > >> JacksonFactory.newObjectMapper, or after the ObjectMapper is constructed in
    > >> JacksonFactory.newWriter. Whether that is done via a Plugin or some other
    > >> way I hadn’t looked at too much. But I suppose it could either be a Plugin
    > >> that is a parameter to the Layout and then passed into the newWriter method
    > >> or it could be looked up by the JSON method in JacksonFactory. Passing it
    > >> on the Layout probably makes it more flexible.
    > >>
    > >> Ralph
    > >>
    > >>> On Jul 12, 2019, at 4:13 PM, Carter Kozak <[hidden email]> wrote:
    > >>>
    > >>> I like the idea of an ObjectMapperFactory plugin point. That gives us a
    > >> cleaner builder API on JsonLayout where we can potentially pass in a
    > >> supplier for an ObjectMapper instance from programmatic configuration.
    > >>>
    > >>> On Fri, Jul 12, 2019, at 19:10, Matt Sicker wrote:
    > >>>> And by plugin, see for example the various BlockingQueueFactory plugins.
    > >>>>
    > >>>> On Fri, Jul 12, 2019 at 18:09, Matt Sicker <[hidden email]> wrote:
    > >>>>
    > >>>>> Plugin maybe? I think we could potentially extend the plugin system as
    > >> a
    > >>>>> general dependency and configuration injection system. We can make less
    > >>>>> special case methods of pseudo components and options outside that.
    > >>>>>
    > >>>>> On Fri, Jul 12, 2019 at 17:34, Gary Gregory <[hidden email]>
    > >>>>> wrote:
    > >>>>>
    > >>>>>> Hi Dominik
    > >>>>>>
    > >>>>>> I see two ways of addressing this:
    > >>>>>> 1) Add a setObjectMapper(ObjectMapper)
    > >>>>>> to org.apache.logging.log4j.core.layout.JsonLayout.Builder
    > >>>>>> This would work well for programmatic configurations
    > >>>>>> and/or:
    > >>>>>> 2) Add a setObjectMapperFactory(String)
    > >>>>>> to org.apache.logging.log4j.core.layout.JsonLayout.Builder
    > >>>>>> This would add a class name for implementors of a new interface
    > >>>>>> ObjectMapperFactory.
    > >>>>>> This would work nicely for configuration files.
    > >>>>>>
    > >>>>>> To keep all users in play, 2) seems best.
    > >>>>>>
    > >>>>>> Thoughts?
    > >>>>>>
    > >>>>>> Gary
    > >>>>>>
    > >>>>>>
    > >>>>>> On Fri, Jul 12, 2019 at 8:38 AM Dominik Sandjaja <
    > >>>>>> [hidden email]> wrote:
    > >>>>>>
    > >>>>>>> Hello,
    > >>>>>>>
    > >>>>>>> we are using the JSON layout in our Spring Boot application to easily
    > >>>>>>> ingest the logs in Elasticsearch.
    > >>>>>>>
    > >>>>>>> One message that we log is about failed conversions, when an incoming
    > >>>>>>> request parameter cannot be converted correctly. As we use Spring’s
    > >>>>>>> Converter functionality for this, these `IllegalArgumentException`s
    > >> are
    > >>>>>>> wrapped in a `ConversionFailedException`. This exception has fields
    > >> of
    > >>>>>> type
    > >>>>>>> `TypeDescriptor` which hold information about the source and the
    > >> target
    > >>>>>>> type of the conversion.
    > >>>>>>>
    > >>>>>>> The problem is, that these `TypeDescriptor` objects cannot be
    > >> serialized
    > >>>>>>> to JSON. The simple call
    > >>>>>>>
    > >>>>>>>
    > >>>>>>>
    > >>>>>>
    > >> com.fasterxml.jackson.databind.ObjectMapper().writeValueAsString(org.springframework.core.convert.TypeDescriptor.valueOf(String.javaClass))
    > >>>>>>>
    > >>>>>>> throws an exception like
    > >>>>>>>
    > >>>>>>> ERROR StatusLogger
    > >>>>>>> com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Direct
    > >>>>>>> self-reference leading to cycle (through reference chain:
    > >>>>>>>
    > >>>>>>
    > >> org.springframework.core.ResolvableType["componentType"]->org.springframework.core.ResolvableType["componentType"])
    > >>>>>>> com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Direct
    > >>>>>>> self-reference leading to cycle (through reference chain:
    > >>>>>>>
    > >>>>>>
    > >> org.springframework.core.ResolvableType["componentType"]->org.springframework.core.ResolvableType["componentType"])
    > >>>>>>> at
    > >>>>>>>
    > >>>>>>
    > >> com.fasterxml.jackson.databind.exc.InvalidDefinitionException.from(InvalidDefinitionException.java:77)
    > >>>>>>> at
    > >>>>>>>
    > >>>>>>
    > >> com.fasterxml.jackson.databind.SerializerProvider.reportBadDefinition(SerializerProvider.java:1191)
    > >>>>>>> at
    > >>>>>>>
    > >>>>>>
    > >> com.fasterxml.jackson.databind.ser.BeanPropertyWriter._handleSelfReference(BeanPropertyWriter.java:944)
    > >>>>>>> at
    > >>>>>>>
    > >>>>>>
    > >> com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:721)
    > >>>>>>> at
    > >>>>>>>
    > >>>>>>
    > >> com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719)
    > >>>>>>> at
    > >>>>>>>
    > >>>>>>
    > >> com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155)
    > >>>>>>>
    > >>>>>>> The error can be prevented if `Mixin`s are added to the
    > >> `ObjectMapper`
    > >>>>>>> which ignore the corresponding class.
    > >>>>>>>
    > >>>>>>> The `Log4jJsonObjectMapper` is initialized in the
    > >> `JacksonFactory.JSON`
    > >>>>>>> class and I currently do not see any way to somehow configure this
    > >>>>>>> ObjectMapper with additional MixIns.
    > >>>>>>>
    > >>>>>>> Hence, my question finally is:
    > >>>>>>> How can I configure Log4J’s JSON mapping to not break on such
    > >>>>>>> non-serializable log content?
    > >>>>>>>
    > >>>>>>> Thank you!
    > >>>>>>> Dominik
    > >>>>>>>
    > >>>>>>
    > >>>>> --
    > >>>>> Matt Sicker <[hidden email]>
    > >>>>>
    > >>>> --
    > >>>> Matt Sicker <[hidden email]>
    > >>>>
    > >>
    > >>
    > >>
    > >> ---------------------------------------------------------------------
    > >> To unsubscribe, e-mail: [hidden email]
    > >> For additional commands, e-mail: [hidden email]
    > >>
    > >> --
    > > Matt Sicker <[hidden email]>
    >
    >
    >
    > ---------------------------------------------------------------------
    > To unsubscribe, e-mail: [hidden email]
    > For additional commands, e-mail: [hidden email]
    >
   
   
    --
    Matt Sicker <[hidden email]>
   
    ---------------------------------------------------------------------
    To unsubscribe, e-mail: [hidden email]
    For additional commands, e-mail: [hidden email]
   
   


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: How to configure ObjectMapper of Log4j2 JSON Layout

Ralph Goers
Did you create a Jira issue for this? One of us might get around to it but we never will without the Jira issue reminding us.

Ralph

> On Jul 16, 2019, at 8:12 AM, Dominik Sandjaja <[hidden email]> wrote:
>
> Hello everybody,
>
> thank you all for the discussion about this topic.
>
> My initial thought was that I was maybe missing some configuration option for setting the classname of one of the initialization classes to override the object creation.
> But after looking a bit into the Log4J Plugin system I agree, that this is the cleanest way forward from an overall architectural point of view.
>
> I don't think that I will be able to provide a Pull Request within the next few weeks (hooray for holidays!), but if time permits, I could have a shot at it in September.
>
> Until then, we will most likely remove this one exception from the Exception chain.
>
> Thank you all!
>
> On 13.07.19, 22:34, "Matt Sicker" <[hidden email]> wrote:
>
>    Oh, that's a part of the plugin system I often forget about since most
>    plugins are core plugins. In any case, a plugin of some sort would be
>    the most flexible here I think.
>
>    On Sat, 13 Jul 2019 at 15:21, Ralph Goers <[hidden email]> wrote:
>>
>> Just because it is a plugin doesn’t mean it can be used anywhere. The JacksonFactory could collect plugins for its own category, which would mean they couldn’t be specified in the configuration. Only “core” plugins can directly be used as parameters in the configuration. But yes, once it is a core plugin it could be passed into the newWriter method as I describe.
>>
>> Ralph
>>
>>> On Jul 13, 2019, at 10:12 AM, Matt Sicker <[hidden email]> wrote:
>>>
>>> Being a plugin means you can inject it as a @PluginElement wherever. Plus,
>>> that makes it easier for users to write a custom plugin class to configure
>>> any other options they want. Comparable to the DataSourceFactory or
>>> whatever it’s called for JDBC drivers in that plugin.
>>>
>>> Keeping it as an element should make it more flexible with future
>>> programmatic configuration ideas. The more component based everything is,
>>> the simpler it’ll be to make customizations without users digging into the
>>> internals of LoggerConfig et al.
>>>
>>> On Fri, Jul 12, 2019 at 18:36, Ralph Goers <[hidden email]>
>>> wrote:
>>>
>>>> Actually, I was thinking that we can keep the existing ObjectMapper but
>>>> supply a Function that can “enhance” the ObjectMapper either at the end of
>>>> the Log4jJsonObjectMapper constructor, at the end of
>>>> JacksonFactory.newObjectMapper, or after the ObjectMapper is constructed in
>>>> JacksonFactory.newWriter. Whether that is done via a Plugin or some other
>>>> way I hadn’t looked at too much. But I suppose it could either be a Plugin
>>>> that is a parameter to the Layout and then passed into the newWriter method
>>>> or it could be looked up by the JSON method in JacksonFactory. Passing it
>>>> on the Layout probably makes it more flexible.
>>>>
>>>> Ralph
>>>>
>>>>> On Jul 12, 2019, at 4:13 PM, Carter Kozak <[hidden email]> wrote:
>>>>>
>>>>> I like the idea of an ObjectMapperFactory plugin point. That gives us a
>>>> cleaner builder API on JsonLayout where we can potentially pass in a
>>>> supplier for an ObjectMapper instance from programmatic configuration.
>>>>>
>>>>> On Fri, Jul 12, 2019, at 19:10, Matt Sicker wrote:
>>>>>> And by plugin, see for example the various BlockingQueueFactory plugins.
>>>>>>
>>>>>> On Fri, Jul 12, 2019 at 18:09, Matt Sicker <[hidden email]> wrote:
>>>>>>
>>>>>>> Plugin maybe? I think we could potentially extend the plugin system as
>>>> a
>>>>>>> general dependency and configuration injection system. We can make less
>>>>>>> special case methods of pseudo components and options outside that.
>>>>>>>
>>>>>>> On Fri, Jul 12, 2019 at 17:34, Gary Gregory <[hidden email]>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi Dominik
>>>>>>>>
>>>>>>>> I see two ways of addressing this:
>>>>>>>> 1) Add a setObjectMapper(ObjectMapper)
>>>>>>>> to org.apache.logging.log4j.core.layout.JsonLayout.Builder
>>>>>>>> This would work well for programmatic configurations
>>>>>>>> and/or:
>>>>>>>> 2) Add a setObjectMapperFactory(String)
>>>>>>>> to org.apache.logging.log4j.core.layout.JsonLayout.Builder
>>>>>>>> This would add a class name for implementors of a new interface
>>>>>>>> ObjectMapperFactory.
>>>>>>>> This would work nicely for configuration files.
>>>>>>>>
>>>>>>>> To keep all users in play, 2) seems best.
>>>>>>>>
>>>>>>>> Thoughts?
>>>>>>>>
>>>>>>>> Gary
>>>>>>>>
>>>>>>>>
>>>>>>>> On Fri, Jul 12, 2019 at 8:38 AM Dominik Sandjaja <
>>>>>>>> [hidden email]> wrote:
>>>>>>>>
>>>>>>>>> Hello,
>>>>>>>>>
>>>>>>>>> we are using the JSON layout in our Spring Boot application to easily
>>>>>>>>> ingest the logs in Elasticsearch.
>>>>>>>>>
>>>>>>>>> One message that we log is about failed conversions, when an incoming
>>>>>>>>> request parameter cannot be converted correctly. As we use Spring’s
>>>>>>>>> Converter functionality for this, these `IllegalArgumentException`s
>>>> are
>>>>>>>>> wrapped in a `ConversionFailedException`. This exception has fields
>>>> of
>>>>>>>> type
>>>>>>>>> `TypeDescriptor` which hold information about the source and the
>>>> target
>>>>>>>>> type of the conversion.
>>>>>>>>>
>>>>>>>>> The problem is, that these `TypeDescriptor` objects cannot be
>>>> serialized
>>>>>>>>> to JSON. The simple call
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>> com.fasterxml.jackson.databind.ObjectMapper().writeValueAsString(org.springframework.core.convert.TypeDescriptor.valueOf(String.javaClass))
>>>>>>>>>
>>>>>>>>> throws an exception like
>>>>>>>>>
>>>>>>>>> ERROR StatusLogger
>>>>>>>>> com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Direct
>>>>>>>>> self-reference leading to cycle (through reference chain:
>>>>>>>>>
>>>>>>>>
>>>> org.springframework.core.ResolvableType["componentType"]->org.springframework.core.ResolvableType["componentType"])
>>>>>>>>> com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Direct
>>>>>>>>> self-reference leading to cycle (through reference chain:
>>>>>>>>>
>>>>>>>>
>>>> org.springframework.core.ResolvableType["componentType"]->org.springframework.core.ResolvableType["componentType"])
>>>>>>>>> at
>>>>>>>>>
>>>>>>>>
>>>> com.fasterxml.jackson.databind.exc.InvalidDefinitionException.from(InvalidDefinitionException.java:77)
>>>>>>>>> at
>>>>>>>>>
>>>>>>>>
>>>> com.fasterxml.jackson.databind.SerializerProvider.reportBadDefinition(SerializerProvider.java:1191)
>>>>>>>>> at
>>>>>>>>>
>>>>>>>>
>>>> com.fasterxml.jackson.databind.ser.BeanPropertyWriter._handleSelfReference(BeanPropertyWriter.java:944)
>>>>>>>>> at
>>>>>>>>>
>>>>>>>>
>>>> com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:721)
>>>>>>>>> at
>>>>>>>>>
>>>>>>>>
>>>> com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719)
>>>>>>>>> at
>>>>>>>>>
>>>>>>>>
>>>> com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155)
>>>>>>>>>
>>>>>>>>> The error can be prevented if `Mixin`s are added to the
>>>> `ObjectMapper`
>>>>>>>>> which ignore the corresponding class.
>>>>>>>>>
>>>>>>>>> The `Log4jJsonObjectMapper` is initialized in the
>>>> `JacksonFactory.JSON`
>>>>>>>>> class and I currently do not see any way to somehow configure this
>>>>>>>>> ObjectMapper with additional MixIns.
>>>>>>>>>
>>>>>>>>> Hence, my question finally is:
>>>>>>>>> How can I configure Log4J’s JSON mapping to not break on such
>>>>>>>>> non-serializable log content?
>>>>>>>>>
>>>>>>>>> Thank you!
>>>>>>>>> Dominik
>>>>>>>>>
>>>>>>>>
>>>>>>> --
>>>>>>> Matt Sicker <[hidden email]>
>>>>>>>
>>>>>> --
>>>>>> Matt Sicker <[hidden email]>
>>>>>>
>>>>
>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: [hidden email]
>>>> For additional commands, e-mail: [hidden email]
>>>>
>>>> --
>>> Matt Sicker <[hidden email]>
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>>
>
>
>    --
>    Matt Sicker <[hidden email]>
>
>    ---------------------------------------------------------------------
>    To unsubscribe, e-mail: [hidden email]
>    For additional commands, e-mail: [hidden email]
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>



---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: How to configure ObjectMapper of Log4j2 JSON Layout

Dominik Sandjaja
Argh, good point:
https://issues.apache.org/jira/browse/LOG4J2-2670

On 30.07.19, 20:30, "Ralph Goers" <[hidden email]> wrote:

    Did you create a Jira issue for this? One of us might get around to it but we never will without the Jira issue reminding us.



---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]