Max index limit in DefaultRolloverStrategy

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Max index limit in DefaultRolloverStrategy

Chandra
Hi guys,

Something I noticed in my production, the maximum limit of index (i.e., the max #files to be rolled before an overwrite occurs) is 7 by default.
This is not mentioned anywhere in any of the documentation. I was only able to confirm this by actually looking into the source code [1].  Unfortunately, by the time we noticed there was  huge data loss.

I wanted to confirm:
 * Why is the default max index set to 7? I was naive to think that the default max index would be set to a large value (may be no max index at all, perhaps)
 * Why wasn’t this mentioned anywhere in the documentation (if it is, can you point me to it. may be I wasn’t paying attention). If it happened to me, perhaps it might for others.



Best,
Chandra



[1] https://github.com/apache/logging-log4j2/blob/master/log4j-core/src/main/java/org/apache/logging/log4j/core/appender/rolling/DefaultRolloverStrategy.java#L126
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Max index limit in DefaultRolloverStrategy

Remko Popma-2
Sorry to hear that!

You are right that the docs currently don't mention the default value for
the max attribute.
We should fix the docs. Can I ask you to raise a Jira for this?

As for why not a large default value, there is a performance impact: all
files within the range need to be renamed (bump version by one), and the
current logic scans the directory to look for files that match the pattern.

You may be interested in the custom Delete action which is more efficient (
https://logging.apache.org/log4j/2.x/manual/appenders.html#CustomDeleteOnRollover
).


On Mon, Apr 10, 2017 at 2:50 PM, Chandra <
[hidden email]> wrote:

> Hi guys,
>
> Something I noticed in my production, the maximum limit of index (i.e.,
> the max #files to be rolled before an overwrite occurs) is 7 by default.
> This is not mentioned anywhere in any of the documentation. I was only
> able to confirm this by actually looking into the source code [1].
> Unfortunately, by the time we noticed there was  huge data loss.
>
> I wanted to confirm:
> * Why is the default max index set to 7? I was naive to think that the
> default max index would be set to a large value (may be no max index at
> all, perhaps)
> * Why wasn’t this mentioned anywhere in the documentation (if it is, can
> you point me to it. may be I wasn’t paying attention). If it happened to
> me, perhaps it might for others.
>
>
>
> Best,
> Chandra
>
>
>
> [1] https://github.com/apache/logging-log4j2/blob/master/
> log4j-core/src/main/java/org/apache/logging/log4j/core/appender/rolling/
> DefaultRolloverStrategy.java#L126
>
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Max index limit in DefaultRolloverStrategy

Ralph Goers
I believe starting with 2.8.1 we no longer scan the directory so it is much faster. We could probably raise the default limit.

Ralph

> On Apr 11, 2017, at 11:12 AM, Remko Popma <[hidden email]> wrote:
>
> Sorry to hear that!
>
> You are right that the docs currently don't mention the default value for
> the max attribute.
> We should fix the docs. Can I ask you to raise a Jira for this?
>
> As for why not a large default value, there is a performance impact: all
> files within the range need to be renamed (bump version by one), and the
> current logic scans the directory to look for files that match the pattern.
>
> You may be interested in the custom Delete action which is more efficient (
> https://logging.apache.org/log4j/2.x/manual/appenders.html#CustomDeleteOnRollover
> ).
>
>
> On Mon, Apr 10, 2017 at 2:50 PM, Chandra <
> [hidden email]> wrote:
>
>> Hi guys,
>>
>> Something I noticed in my production, the maximum limit of index (i.e.,
>> the max #files to be rolled before an overwrite occurs) is 7 by default.
>> This is not mentioned anywhere in any of the documentation. I was only
>> able to confirm this by actually looking into the source code [1].
>> Unfortunately, by the time we noticed there was  huge data loss.
>>
>> I wanted to confirm:
>> * Why is the default max index set to 7? I was naive to think that the
>> default max index would be set to a large value (may be no max index at
>> all, perhaps)
>> * Why wasn’t this mentioned anywhere in the documentation (if it is, can
>> you point me to it. may be I wasn’t paying attention). If it happened to
>> me, perhaps it might for others.
>>
>>
>>
>> Best,
>> Chandra
>>
>>
>>
>> [1] https://github.com/apache/logging-log4j2/blob/master/
>> log4j-core/src/main/java/org/apache/logging/log4j/core/appender/rolling/
>> DefaultRolloverStrategy.java#L126
>>



---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Max index limit in DefaultRolloverStrategy

Chandra
Yes, that’d be great.  7 seems to too low for frequently filled logs.
And also guys, please document it. I am raising a JIRA ticket in this regards ( thanks Remko for the reminder).
Also yes, I have set a default purge policy, this is 90 days in our use case.

Best,
Chandra

On 12 Apr 2017, 12:07 AM +0530, Ralph Goers <[hidden email]>, wrote:

> I believe starting with 2.8.1 we no longer scan the directory so it is much faster. We could probably raise the default limit.
>
> Ralph
>
> > On Apr 11, 2017, at 11:12 AM, Remko Popma <[hidden email]> wrote:
> >
> > Sorry to hear that!
> >
> > You are right that the docs currently don't mention the default value for
> > the max attribute.
> > We should fix the docs. Can I ask you to raise a Jira for this?
> >
> > As for why not a large default value, there is a performance impact: all
> > files within the range need to be renamed (bump version by one), and the
> > current logic scans the directory to look for files that match the pattern.
> >
> > You may be interested in the custom Delete action which is more efficient (
> > https://logging.apache.org/log4j/2.x/manual/appenders.html#CustomDeleteOnRollover
> > ).
> >
> >
> > On Mon, Apr 10, 2017 at 2:50 PM, Chandra <
> > [hidden email]> wrote:
> >
> > > Hi guys,
> > >
> > > Something I noticed in my production, the maximum limit of index (i.e.,
> > > the max #files to be rolled before an overwrite occurs) is 7 by default.
> > > This is not mentioned anywhere in any of the documentation. I was only
> > > able to confirm this by actually looking into the source code [1].
> > > Unfortunately, by the time we noticed there was huge data loss.
> > >
> > > I wanted to confirm:
> > > * Why is the default max index set to 7? I was naive to think that the
> > > default max index would be set to a large value (may be no max index at
> > > all, perhaps)
> > > * Why wasn’t this mentioned anywhere in the documentation (if it is, can
> > > you point me to it. may be I wasn’t paying attention). If it happened to
> > > me, perhaps it might for others.
> > >
> > >
> > >
> > > Best,
> > > Chandra
> > >
> > >
> > >
> > > [1] https://github.com/apache/logging-log4j2/blob/master/
> > > log4j-core/src/main/java/org/apache/logging/log4j/core/appender/rolling/
> > > DefaultRolloverStrategy.java#L126
> > >
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Max index limit in DefaultRolloverStrategy

Chandra
Hi guys, any update on this ticket?
https://issues.apache.org/jira/browse/LOG4J2-1877


thanks,
Chandra

On 12 Apr 2017, 12:14 AM +0530, Chandra <[hidden email]>, wrote:

> Yes, that’d be great.  7 seems to too low for frequently filled logs.
> And also guys, please document it. I am raising a JIRA ticket in this regards ( thanks Remko for the reminder).
> Also yes, I have set a default purge policy, this is 90 days in our use case.
>
> Best,
> Chandra
>
> On 12 Apr 2017, 12:07 AM +0530, Ralph Goers <[hidden email]>, wrote:
> > I believe starting with 2.8.1 we no longer scan the directory so it is much faster. We could probably raise the default limit.
> >
> > Ralph
> >
> > > On Apr 11, 2017, at 11:12 AM, Remko Popma <[hidden email]> wrote:
> > >
> > > Sorry to hear that!
> > >
> > > You are right that the docs currently don't mention the default value for
> > > the max attribute.
> > > We should fix the docs. Can I ask you to raise a Jira for this?
> > >
> > > As for why not a large default value, there is a performance impact: all
> > > files within the range need to be renamed (bump version by one), and the
> > > current logic scans the directory to look for files that match the pattern.
> > >
> > > You may be interested in the custom Delete action which is more efficient (
> > > https://logging.apache.org/log4j/2.x/manual/appenders.html#CustomDeleteOnRollover
> > > ).
> > >
> > >
> > > On Mon, Apr 10, 2017 at 2:50 PM, Chandra <
> > > [hidden email]> wrote:
> > >
> > > > Hi guys,
> > > >
> > > > Something I noticed in my production, the maximum limit of index (i.e.,
> > > > the max #files to be rolled before an overwrite occurs) is 7 by default.
> > > > This is not mentioned anywhere in any of the documentation. I was only
> > > > able to confirm this by actually looking into the source code [1].
> > > > Unfortunately, by the time we noticed there was huge data loss.
> > > >
> > > > I wanted to confirm:
> > > > * Why is the default max index set to 7? I was naive to think that the
> > > > default max index would be set to a large value (may be no max index at
> > > > all, perhaps)
> > > > * Why wasn’t this mentioned anywhere in the documentation (if it is, can
> > > > you point me to it. may be I wasn’t paying attention). If it happened to
> > > > me, perhaps it might for others.
> > > >
> > > >
> > > >
> > > > Best,
> > > > Chandra
> > > >
> > > >
> > > >
> > > > [1] https://github.com/apache/logging-log4j2/blob/master/
> > > > log4j-core/src/main/java/org/apache/logging/log4j/core/appender/rolling/
> > > > DefaultRolloverStrategy.java#L126
> > > >
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: [hidden email]
> > For additional commands, e-mail: [hidden email]
> >
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Max index limit in DefaultRolloverStrategy

Gary Gregory-4
Patches welcome ;-)

Gary


On Apr 18, 2017 1:18 AM, "Chandra" <[hidden email]>
wrote:

Hi guys, any update on this ticket?
https://issues.apache.org/jira/browse/LOG4J2-1877


thanks,
Chandra

On 12 Apr 2017, 12:14 AM +0530, Chandra <[hidden email]>,
wrote:
> Yes, that’d be great.  7 seems to too low for frequently filled logs.
> And also guys, please document it. I am raising a JIRA ticket in this
regards ( thanks Remko for the reminder).
> Also yes, I have set a default purge policy, this is 90 days in our use
case.
>
> Best,
> Chandra
>
> On 12 Apr 2017, 12:07 AM +0530, Ralph Goers <[hidden email]>,
wrote:
> > I believe starting with 2.8.1 we no longer scan the directory so it is
much faster. We could probably raise the default limit.
> >
> > Ralph
> >
> > > On Apr 11, 2017, at 11:12 AM, Remko Popma <[hidden email]>
wrote:
> > >
> > > Sorry to hear that!
> > >
> > > You are right that the docs currently don't mention the default value
for
> > > the max attribute.
> > > We should fix the docs. Can I ask you to raise a Jira for this?
> > >
> > > As for why not a large default value, there is a performance impact:
all
> > > files within the range need to be renamed (bump version by one), and
the
> > > current logic scans the directory to look for files that match the
pattern.
> > >
> > > You may be interested in the custom Delete action which is more
efficient (
> > > https://logging.apache.org/log4j/2.x/manual/appenders.
html#CustomDeleteOnRollover
> > > ).
> > >
> > >
> > > On Mon, Apr 10, 2017 at 2:50 PM, Chandra <
> > > [hidden email]> wrote:
> > >
> > > > Hi guys,
> > > >
> > > > Something I noticed in my production, the maximum limit of index
(i.e.,
> > > > the max #files to be rolled before an overwrite occurs) is 7 by
default.
> > > > This is not mentioned anywhere in any of the documentation. I was
only
> > > > able to confirm this by actually looking into the source code [1].
> > > > Unfortunately, by the time we noticed there was huge data loss.
> > > >
> > > > I wanted to confirm:
> > > > * Why is the default max index set to 7? I was naive to think that
the
> > > > default max index would be set to a large value (may be no max
index at
> > > > all, perhaps)
> > > > * Why wasn’t this mentioned anywhere in the documentation (if it
is, can
> > > > you point me to it. may be I wasn’t paying attention). If it
happened to

> > > > me, perhaps it might for others.
> > > >
> > > >
> > > >
> > > > Best,
> > > > Chandra
> > > >
> > > >
> > > >
> > > > [1] https://github.com/apache/logging-log4j2/blob/master/
> > > > log4j-core/src/main/java/org/apache/logging/log4j/core/
appender/rolling/
> > > > DefaultRolloverStrategy.java#L126
> > > >
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: [hidden email]
> > For additional commands, e-mail: [hidden email]
> >
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Max index limit in DefaultRolloverStrategy

Chandra
Hi Gary,

I’d love to, but this ticket is primarily to change the documentation and it’d have a minor amendments in the document on the website. I’m not exactly sure how to send a patch to this :-/

1)
https://logging.apache.org/log4j/2.x/manual/appenders.html#DefaultRolloverStrategy

Particularly, the section "DefaultRolloverStrategy Parameters” :

From:
max - Integer -  The maximum value of the counter. Once this values is reached older archives will be deleted on subsequent rollovers.

To :
max - Integer - The maximum value of the counter. Once this values is reached older archives will be deleted on subsequent rollovers (The default value is 7).

2) Also, I’d also _recommend_ to update the Javadoc (this I can send a patch :) ) but, I think it’s not required.

thanks,
Chandra

On 18 Apr 2017, 8:53 PM +0530, Gary Gregory <[hidden email]>, wrote:

> Patches welcome ;-)
>
> Gary
>
>
> On Apr 18, 2017 1:18 AM, "Chandra" <[hidden email]
> wrote:
>
> Hi guys, any update on this ticket?
> https://issues.apache.org/jira/browse/LOG4J2-1877
>
>
> thanks,
> Chandra
>
> On 12 Apr 2017, 12:14 AM +0530, Chandra <[hidden email]>,
> wrote:
> > Yes, that’d be great. 7 seems to too low for frequently filled logs.
> > And also guys, please document it. I am raising a JIRA ticket in this
> regards ( thanks Remko for the reminder).
> > Also yes, I have set a default purge policy, this is 90 days in our use
> case.
> >
> > Best,
> > Chandra
> >
> > On 12 Apr 2017, 12:07 AM +0530, Ralph Goers <[hidden email]>,
> wrote:
> > > I believe starting with 2.8.1 we no longer scan the directory so it is
> much faster. We could probably raise the default limit.
> > >
> > > Ralph
> > >
> > > > On Apr 11, 2017, at 11:12 AM, Remko Popma <[hidden email]
> wrote:
> > > >
> > > > Sorry to hear that!
> > > >
> > > > You are right that the docs currently don't mention the default value
> for
> > > > the max attribute.
> > > > We should fix the docs. Can I ask you to raise a Jira for this?
> > > >
> > > > As for why not a large default value, there is a performance impact:
> all
> > > > files within the range need to be renamed (bump version by one), and
> the
> > > > current logic scans the directory to look for files that match the
> pattern.
> > > >
> > > > You may be interested in the custom Delete action which is more
> efficient (
> > > > https://logging.apache.org/log4j/2.x/manual/appenders.
> html#CustomDeleteOnRollover
> > > > ).
> > > >
> > > >
> > > > On Mon, Apr 10, 2017 at 2:50 PM, Chandra <
> > > > [hidden email]> wrote:
> > > >
> > > > > Hi guys,
> > > > >
> > > > > Something I noticed in my production, the maximum limit of index
> (i.e.,
> > > > > the max #files to be rolled before an overwrite occurs) is 7 by
> default.
> > > > > This is not mentioned anywhere in any of the documentation. I was
> only
> > > > > able to confirm this by actually looking into the source code [1].
> > > > > Unfortunately, by the time we noticed there was huge data loss.
> > > > >
> > > > > I wanted to confirm:
> > > > > * Why is the default max index set to 7? I was naive to think that
> the
> > > > > default max index would be set to a large value (may be no max
> index at
> > > > > all, perhaps)
> > > > > * Why wasn’t this mentioned anywhere in the documentation (if it
> is, can
> > > > > you point me to it. may be I wasn’t paying attention). If it
> happened to
> > > > > me, perhaps it might for others.
> > > > >
> > > > >
> > > > >
> > > > > Best,
> > > > > Chandra
> > > > >
> > > > >
> > > > >
> > > > > [1] https://github.com/apache/logging-log4j2/blob/master/
> > > > > log4j-core/src/main/java/org/apache/logging/log4j/core/
> appender/rolling/
> > > > > DefaultRolloverStrategy.java#L126
> > > > >
> > >
> > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: [hidden email]
> > > For additional commands, e-mail: [hidden email]
> > >
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Max index limit in DefaultRolloverStrategy

Ralph Goers
FYI - the documentation for the web site is in the project at src/site/xdoc/manual.  You can create a patch for that the same way you would patch any of the java source.

Ralph

> On Apr 18, 2017, at 10:20 AM, Chandra <[hidden email]> wrote:
>
> Hi Gary,
>
> I’d love to, but this ticket is primarily to change the documentation and it’d have a minor amendments in the document on the website. I’m not exactly sure how to send a patch to this :-/
>
> 1)
> https://logging.apache.org/log4j/2.x/manual/appenders.html#DefaultRolloverStrategy
>
> Particularly, the section "DefaultRolloverStrategy Parameters” :
>
> From:
> max - Integer -  The maximum value of the counter. Once this values is reached older archives will be deleted on subsequent rollovers.
>
> To :
> max - Integer - The maximum value of the counter. Once this values is reached older archives will be deleted on subsequent rollovers (The default value is 7).
>
> 2) Also, I’d also _recommend_ to update the Javadoc (this I can send a patch :) ) but, I think it’s not required.
>
> thanks,
> Chandra
>
> On 18 Apr 2017, 8:53 PM +0530, Gary Gregory <[hidden email]>, wrote:
>> Patches welcome ;-)
>>
>> Gary
>>
>>
>> On Apr 18, 2017 1:18 AM, "Chandra" <[hidden email]
>> wrote:
>>
>> Hi guys, any update on this ticket?
>> https://issues.apache.org/jira/browse/LOG4J2-1877
>>
>>
>> thanks,
>> Chandra
>>
>> On 12 Apr 2017, 12:14 AM +0530, Chandra <[hidden email]>,
>> wrote:
>>> Yes, that’d be great. 7 seems to too low for frequently filled logs.
>>> And also guys, please document it. I am raising a JIRA ticket in this
>> regards ( thanks Remko for the reminder).
>>> Also yes, I have set a default purge policy, this is 90 days in our use
>> case.
>>>
>>> Best,
>>> Chandra
>>>
>>> On 12 Apr 2017, 12:07 AM +0530, Ralph Goers <[hidden email]>,
>> wrote:
>>>> I believe starting with 2.8.1 we no longer scan the directory so it is
>> much faster. We could probably raise the default limit.
>>>>
>>>> Ralph
>>>>
>>>>> On Apr 11, 2017, at 11:12 AM, Remko Popma <[hidden email]
>> wrote:
>>>>>
>>>>> Sorry to hear that!
>>>>>
>>>>> You are right that the docs currently don't mention the default value
>> for
>>>>> the max attribute.
>>>>> We should fix the docs. Can I ask you to raise a Jira for this?
>>>>>
>>>>> As for why not a large default value, there is a performance impact:
>> all
>>>>> files within the range need to be renamed (bump version by one), and
>> the
>>>>> current logic scans the directory to look for files that match the
>> pattern.
>>>>>
>>>>> You may be interested in the custom Delete action which is more
>> efficient (
>>>>> https://logging.apache.org/log4j/2.x/manual/appenders.
>> html#CustomDeleteOnRollover
>>>>> ).
>>>>>
>>>>>
>>>>> On Mon, Apr 10, 2017 at 2:50 PM, Chandra <
>>>>> [hidden email]> wrote:
>>>>>
>>>>>> Hi guys,
>>>>>>
>>>>>> Something I noticed in my production, the maximum limit of index
>> (i.e.,
>>>>>> the max #files to be rolled before an overwrite occurs) is 7 by
>> default.
>>>>>> This is not mentioned anywhere in any of the documentation. I was
>> only
>>>>>> able to confirm this by actually looking into the source code [1].
>>>>>> Unfortunately, by the time we noticed there was huge data loss.
>>>>>>
>>>>>> I wanted to confirm:
>>>>>> * Why is the default max index set to 7? I was naive to think that
>> the
>>>>>> default max index would be set to a large value (may be no max
>> index at
>>>>>> all, perhaps)
>>>>>> * Why wasn’t this mentioned anywhere in the documentation (if it
>> is, can
>>>>>> you point me to it. may be I wasn’t paying attention). If it
>> happened to
>>>>>> me, perhaps it might for others.
>>>>>>
>>>>>>
>>>>>>
>>>>>> Best,
>>>>>> Chandra
>>>>>>
>>>>>>
>>>>>>
>>>>>> [1] https://github.com/apache/logging-log4j2/blob/master/
>>>>>> log4j-core/src/main/java/org/apache/logging/log4j/core/
>> appender/rolling/
>>>>>> DefaultRolloverStrategy.java#L126
>>>>>>
>>>>
>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: [hidden email]
>>>> For additional commands, e-mail: [hidden email]
>>>>



---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Loading...