FAQ
Replace infoSteram by a logging framework (SLF4J)
-------------------------------------------------

Key: LUCENE-1482
URL: https://issues.apache.org/jira/browse/LUCENE-1482
Project: Lucene - Java
Issue Type: Improvement
Components: Index
Reporter: Shai Erera
Priority: Minor
Fix For: 2.4.1, 2.9


Lucene makes use of infoStream to output messages in its indexing code only. For debugging purposes, when the search application is run on the customer side, getting messages from other code flows, like search, query parsing, analysis etc can be extremely useful.
There are two main problems with infoStream today:
1. It is owned by IndexWriter, so if I want to add logging capabilities to other classes I need to either expose an API or propagate infoStream to all classes (see for example DocumentsWriter, which receives its infoStream instance from IndexWriter).
2. I can either turn debugging on or off, for the entire code.

Introducing a logging framework can allow each class to control its logging independently, and more importantly, allows the application to turn on logging for only specific areas in the code (i.e., org.apache.lucene.index.*).

I've investigated SLF4J (stands for Simple Logging Facade for Java) which is, as it names states, a facade over different logging frameworks. As such, you can include the slf4j.jar in your application, and it recognizes at deploy time what is the actual logging framework you'd like to use. SLF4J comes with several adapters for Java logging, Log4j and others. If you know your application uses Java logging, simply drop slf4j.jar and slf4j-jdk14.jar in your classpath, and your logging statements will use Java logging underneath the covers.

This makes the logging code very simple. For a class A the logger will be instantiated like this:
public class A {
private static final logger = LoggerFactory.getLogger(A.class);
}

And will later be used like this:
public class A {
private static final logger = LoggerFactory.getLogger(A.class);

public void foo() {
if (logger.isDebugEnabled()) {
logger.debug("message");
}
}
}

That's all !

Checking for isDebugEnabled is very quick, at least using the JDK14 adapter (but I assume it's fast also over other logging frameworks).

The important thing is, every class controls its own logger. Not all classes have to output logging messages, and we can improve Lucene's logging gradually, w/o changing the API, by adding more logging messages to interesting classes.

I will submit a patch shortly

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org

Search Discussions

  • Shai Erera (JIRA) at Dec 8, 2008 at 2:25 pm
    [ https://issues.apache.org/jira/browse/LUCENE-1482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Shai Erera updated LUCENE-1482:
    -------------------------------

    Attachment: LUCENE-1482.patch

    This patch covers:
    - Using SLF4J in all the classes that used infoStream
    - A test which uses the JDK14 adapter to make sure it works, as well as fixing few tests which relied on some messages
    - Deprecation of setInfoStream(), getInfoStream() etc. in several classes who exposed this API.

    Few notes:
    - As in many customer environments I know of the INFO level is usually turned on, and we were forbidden to output any message in the INFO level, unless it's really INFO, WARNING or SEVERE, I assumed Lucene logging messages should be in the DEBUG level (which is one less than TRACE).

    - I wasn't sure what to do with the set/get infoStream methods, so I just deprecated them and do nothing (i.e., setInfoStream does nothing and getInfoStream always returns a null).
    Not sure how's that align with Lucene's back-compat policy, but on the other hand I didn't think I should keep both infoStream and SLF4J logging in the code.

    - Should I attach the slf4j jars separately?

    - I didn't go as far as measuring performance because:
    1. The code uses isDebugEnabled everywhere, which is at least judging by the JDK14 adapter very fast (just checks a member on the actual logger instance) and is almost equivalent to infoStream != null check.
    2. It really depends on the adapter that's being used. I used JDK14, but perhaps some other adapter will perform worse on these calls, although I expect these calls to be executed quickly, if not even being inlined by the compiler.
    Replace infoSteram by a logging framework (SLF4J)
    -------------------------------------------------

    Key: LUCENE-1482
    URL: https://issues.apache.org/jira/browse/LUCENE-1482
    Project: Lucene - Java
    Issue Type: Improvement
    Components: Index
    Reporter: Shai Erera
    Priority: Minor
    Fix For: 2.4.1, 2.9

    Attachments: LUCENE-1482.patch


    Lucene makes use of infoStream to output messages in its indexing code only. For debugging purposes, when the search application is run on the customer side, getting messages from other code flows, like search, query parsing, analysis etc can be extremely useful.
    There are two main problems with infoStream today:
    1. It is owned by IndexWriter, so if I want to add logging capabilities to other classes I need to either expose an API or propagate infoStream to all classes (see for example DocumentsWriter, which receives its infoStream instance from IndexWriter).
    2. I can either turn debugging on or off, for the entire code.
    Introducing a logging framework can allow each class to control its logging independently, and more importantly, allows the application to turn on logging for only specific areas in the code (i.e., org.apache.lucene.index.*).
    I've investigated SLF4J (stands for Simple Logging Facade for Java) which is, as it names states, a facade over different logging frameworks. As such, you can include the slf4j.jar in your application, and it recognizes at deploy time what is the actual logging framework you'd like to use. SLF4J comes with several adapters for Java logging, Log4j and others. If you know your application uses Java logging, simply drop slf4j.jar and slf4j-jdk14.jar in your classpath, and your logging statements will use Java logging underneath the covers.
    This makes the logging code very simple. For a class A the logger will be instantiated like this:
    public class A {
    private static final logger = LoggerFactory.getLogger(A.class);
    }
    And will later be used like this:
    public class A {
    private static final logger = LoggerFactory.getLogger(A.class);
    public void foo() {
    if (logger.isDebugEnabled()) {
    logger.debug("message");
    }
    }
    }
    That's all !
    Checking for isDebugEnabled is very quick, at least using the JDK14 adapter (but I assume it's fast also over other logging frameworks).
    The important thing is, every class controls its own logger. Not all classes have to output logging messages, and we can improve Lucene's logging gradually, w/o changing the API, by adding more logging messages to interesting classes.
    I will submit a patch shortly
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-dev-help@lucene.apache.org
  • Doug Cutting (JIRA) at Dec 8, 2008 at 7:45 pm
    [ https://issues.apache.org/jira/browse/LUCENE-1482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12654545#action_12654545 ]

    Doug Cutting commented on LUCENE-1482:
    --------------------------------------
    Should I attach the slf4j jars separately?
    If we go with SLF4J, we'd want to include the -api jar in Lucene for sure, along with a single implementation. My vote would be for the -nop implementation. Then, folks who want logging can include the implementation they like.

    Replace infoSteram by a logging framework (SLF4J)
    -------------------------------------------------

    Key: LUCENE-1482
    URL: https://issues.apache.org/jira/browse/LUCENE-1482
    Project: Lucene - Java
    Issue Type: Improvement
    Components: Index
    Reporter: Shai Erera
    Priority: Minor
    Fix For: 2.4.1, 2.9

    Attachments: LUCENE-1482.patch


    Lucene makes use of infoStream to output messages in its indexing code only. For debugging purposes, when the search application is run on the customer side, getting messages from other code flows, like search, query parsing, analysis etc can be extremely useful.
    There are two main problems with infoStream today:
    1. It is owned by IndexWriter, so if I want to add logging capabilities to other classes I need to either expose an API or propagate infoStream to all classes (see for example DocumentsWriter, which receives its infoStream instance from IndexWriter).
    2. I can either turn debugging on or off, for the entire code.
    Introducing a logging framework can allow each class to control its logging independently, and more importantly, allows the application to turn on logging for only specific areas in the code (i.e., org.apache.lucene.index.*).
    I've investigated SLF4J (stands for Simple Logging Facade for Java) which is, as it names states, a facade over different logging frameworks. As such, you can include the slf4j.jar in your application, and it recognizes at deploy time what is the actual logging framework you'd like to use. SLF4J comes with several adapters for Java logging, Log4j and others. If you know your application uses Java logging, simply drop slf4j.jar and slf4j-jdk14.jar in your classpath, and your logging statements will use Java logging underneath the covers.
    This makes the logging code very simple. For a class A the logger will be instantiated like this:
    public class A {
    private static final logger = LoggerFactory.getLogger(A.class);
    }
    And will later be used like this:
    public class A {
    private static final logger = LoggerFactory.getLogger(A.class);
    public void foo() {
    if (logger.isDebugEnabled()) {
    logger.debug("message");
    }
    }
    }
    That's all !
    Checking for isDebugEnabled is very quick, at least using the JDK14 adapter (but I assume it's fast also over other logging frameworks).
    The important thing is, every class controls its own logger. Not all classes have to output logging messages, and we can improve Lucene's logging gradually, w/o changing the API, by adding more logging messages to interesting classes.
    I will submit a patch shortly
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-dev-help@lucene.apache.org
  • Shai Erera (JIRA) at Dec 9, 2008 at 11:51 am
    [ https://issues.apache.org/jira/browse/LUCENE-1482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Shai Erera updated LUCENE-1482:
    -------------------------------

    Attachment: (was: LUCENE-1482.patch)
    Replace infoSteram by a logging framework (SLF4J)
    -------------------------------------------------

    Key: LUCENE-1482
    URL: https://issues.apache.org/jira/browse/LUCENE-1482
    Project: Lucene - Java
    Issue Type: Improvement
    Components: Index
    Reporter: Shai Erera
    Priority: Minor
    Fix For: 2.4.1, 2.9

    Attachments: LUCENE-1482.patch, slf4j-api-1.5.6.jar, slf4j-nop-1.5.6.jar


    Lucene makes use of infoStream to output messages in its indexing code only. For debugging purposes, when the search application is run on the customer side, getting messages from other code flows, like search, query parsing, analysis etc can be extremely useful.
    There are two main problems with infoStream today:
    1. It is owned by IndexWriter, so if I want to add logging capabilities to other classes I need to either expose an API or propagate infoStream to all classes (see for example DocumentsWriter, which receives its infoStream instance from IndexWriter).
    2. I can either turn debugging on or off, for the entire code.
    Introducing a logging framework can allow each class to control its logging independently, and more importantly, allows the application to turn on logging for only specific areas in the code (i.e., org.apache.lucene.index.*).
    I've investigated SLF4J (stands for Simple Logging Facade for Java) which is, as it names states, a facade over different logging frameworks. As such, you can include the slf4j.jar in your application, and it recognizes at deploy time what is the actual logging framework you'd like to use. SLF4J comes with several adapters for Java logging, Log4j and others. If you know your application uses Java logging, simply drop slf4j.jar and slf4j-jdk14.jar in your classpath, and your logging statements will use Java logging underneath the covers.
    This makes the logging code very simple. For a class A the logger will be instantiated like this:
    public class A {
    private static final logger = LoggerFactory.getLogger(A.class);
    }
    And will later be used like this:
    public class A {
    private static final logger = LoggerFactory.getLogger(A.class);
    public void foo() {
    if (logger.isDebugEnabled()) {
    logger.debug("message");
    }
    }
    }
    That's all !
    Checking for isDebugEnabled is very quick, at least using the JDK14 adapter (but I assume it's fast also over other logging frameworks).
    The important thing is, every class controls its own logger. Not all classes have to output logging messages, and we can improve Lucene's logging gradually, w/o changing the API, by adding more logging messages to interesting classes.
    I will submit a patch shortly
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-dev-help@lucene.apache.org
  • Shai Erera (JIRA) at Dec 9, 2008 at 11:51 am
    [ https://issues.apache.org/jira/browse/LUCENE-1482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Shai Erera updated LUCENE-1482:
    -------------------------------

    Attachment: slf4j-nop-1.5.6.jar
    slf4j-api-1.5.6.jar
    LUCENE-1482.patch

    Thanks Doug,

    I've replaced the JDK14 jar with the NOP jar and deleted the logging test I added (since NOP does not log anything).
    Replace infoSteram by a logging framework (SLF4J)
    -------------------------------------------------

    Key: LUCENE-1482
    URL: https://issues.apache.org/jira/browse/LUCENE-1482
    Project: Lucene - Java
    Issue Type: Improvement
    Components: Index
    Reporter: Shai Erera
    Priority: Minor
    Fix For: 2.4.1, 2.9

    Attachments: LUCENE-1482.patch, slf4j-api-1.5.6.jar, slf4j-nop-1.5.6.jar


    Lucene makes use of infoStream to output messages in its indexing code only. For debugging purposes, when the search application is run on the customer side, getting messages from other code flows, like search, query parsing, analysis etc can be extremely useful.
    There are two main problems with infoStream today:
    1. It is owned by IndexWriter, so if I want to add logging capabilities to other classes I need to either expose an API or propagate infoStream to all classes (see for example DocumentsWriter, which receives its infoStream instance from IndexWriter).
    2. I can either turn debugging on or off, for the entire code.
    Introducing a logging framework can allow each class to control its logging independently, and more importantly, allows the application to turn on logging for only specific areas in the code (i.e., org.apache.lucene.index.*).
    I've investigated SLF4J (stands for Simple Logging Facade for Java) which is, as it names states, a facade over different logging frameworks. As such, you can include the slf4j.jar in your application, and it recognizes at deploy time what is the actual logging framework you'd like to use. SLF4J comes with several adapters for Java logging, Log4j and others. If you know your application uses Java logging, simply drop slf4j.jar and slf4j-jdk14.jar in your classpath, and your logging statements will use Java logging underneath the covers.
    This makes the logging code very simple. For a class A the logger will be instantiated like this:
    public class A {
    private static final logger = LoggerFactory.getLogger(A.class);
    }
    And will later be used like this:
    public class A {
    private static final logger = LoggerFactory.getLogger(A.class);
    public void foo() {
    if (logger.isDebugEnabled()) {
    logger.debug("message");
    }
    }
    }
    That's all !
    Checking for isDebugEnabled is very quick, at least using the JDK14 adapter (but I assume it's fast also over other logging frameworks).
    The important thing is, every class controls its own logger. Not all classes have to output logging messages, and we can improve Lucene's logging gradually, w/o changing the API, by adding more logging messages to interesting classes.
    I will submit a patch shortly
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-dev-help@lucene.apache.org
  • Shai Erera (JIRA) at Dec 9, 2008 at 1:11 pm
    [ https://issues.apache.org/jira/browse/LUCENE-1482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Shai Erera updated LUCENE-1482:
    -------------------------------

    Attachment: (was: LUCENE-1482.patch)
    Replace infoSteram by a logging framework (SLF4J)
    -------------------------------------------------

    Key: LUCENE-1482
    URL: https://issues.apache.org/jira/browse/LUCENE-1482
    Project: Lucene - Java
    Issue Type: Improvement
    Components: Index
    Reporter: Shai Erera
    Priority: Minor
    Fix For: 2.4.1, 2.9

    Attachments: slf4j-api-1.5.6.jar, slf4j-nop-1.5.6.jar


    Lucene makes use of infoStream to output messages in its indexing code only. For debugging purposes, when the search application is run on the customer side, getting messages from other code flows, like search, query parsing, analysis etc can be extremely useful.
    There are two main problems with infoStream today:
    1. It is owned by IndexWriter, so if I want to add logging capabilities to other classes I need to either expose an API or propagate infoStream to all classes (see for example DocumentsWriter, which receives its infoStream instance from IndexWriter).
    2. I can either turn debugging on or off, for the entire code.
    Introducing a logging framework can allow each class to control its logging independently, and more importantly, allows the application to turn on logging for only specific areas in the code (i.e., org.apache.lucene.index.*).
    I've investigated SLF4J (stands for Simple Logging Facade for Java) which is, as it names states, a facade over different logging frameworks. As such, you can include the slf4j.jar in your application, and it recognizes at deploy time what is the actual logging framework you'd like to use. SLF4J comes with several adapters for Java logging, Log4j and others. If you know your application uses Java logging, simply drop slf4j.jar and slf4j-jdk14.jar in your classpath, and your logging statements will use Java logging underneath the covers.
    This makes the logging code very simple. For a class A the logger will be instantiated like this:
    public class A {
    private static final logger = LoggerFactory.getLogger(A.class);
    }
    And will later be used like this:
    public class A {
    private static final logger = LoggerFactory.getLogger(A.class);
    public void foo() {
    if (logger.isDebugEnabled()) {
    logger.debug("message");
    }
    }
    }
    That's all !
    Checking for isDebugEnabled is very quick, at least using the JDK14 adapter (but I assume it's fast also over other logging frameworks).
    The important thing is, every class controls its own logger. Not all classes have to output logging messages, and we can improve Lucene's logging gradually, w/o changing the API, by adding more logging messages to interesting classes.
    I will submit a patch shortly
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-dev-help@lucene.apache.org
  • Shai Erera (JIRA) at Dec 9, 2008 at 1:13 pm
    [ https://issues.apache.org/jira/browse/LUCENE-1482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Shai Erera updated LUCENE-1482:
    -------------------------------

    Attachment: LUCENE-1482.patch

    Forgot to clean up some code in tests which made use of JDK logging.
    Replace infoSteram by a logging framework (SLF4J)
    -------------------------------------------------

    Key: LUCENE-1482
    URL: https://issues.apache.org/jira/browse/LUCENE-1482
    Project: Lucene - Java
    Issue Type: Improvement
    Components: Index
    Reporter: Shai Erera
    Priority: Minor
    Fix For: 2.4.1, 2.9

    Attachments: LUCENE-1482.patch, slf4j-api-1.5.6.jar, slf4j-nop-1.5.6.jar


    Lucene makes use of infoStream to output messages in its indexing code only. For debugging purposes, when the search application is run on the customer side, getting messages from other code flows, like search, query parsing, analysis etc can be extremely useful.
    There are two main problems with infoStream today:
    1. It is owned by IndexWriter, so if I want to add logging capabilities to other classes I need to either expose an API or propagate infoStream to all classes (see for example DocumentsWriter, which receives its infoStream instance from IndexWriter).
    2. I can either turn debugging on or off, for the entire code.
    Introducing a logging framework can allow each class to control its logging independently, and more importantly, allows the application to turn on logging for only specific areas in the code (i.e., org.apache.lucene.index.*).
    I've investigated SLF4J (stands for Simple Logging Facade for Java) which is, as it names states, a facade over different logging frameworks. As such, you can include the slf4j.jar in your application, and it recognizes at deploy time what is the actual logging framework you'd like to use. SLF4J comes with several adapters for Java logging, Log4j and others. If you know your application uses Java logging, simply drop slf4j.jar and slf4j-jdk14.jar in your classpath, and your logging statements will use Java logging underneath the covers.
    This makes the logging code very simple. For a class A the logger will be instantiated like this:
    public class A {
    private static final logger = LoggerFactory.getLogger(A.class);
    }
    And will later be used like this:
    public class A {
    private static final logger = LoggerFactory.getLogger(A.class);
    public void foo() {
    if (logger.isDebugEnabled()) {
    logger.debug("message");
    }
    }
    }
    That's all !
    Checking for isDebugEnabled is very quick, at least using the JDK14 adapter (but I assume it's fast also over other logging frameworks).
    The important thing is, every class controls its own logger. Not all classes have to output logging messages, and we can improve Lucene's logging gradually, w/o changing the API, by adding more logging messages to interesting classes.
    I will submit a patch shortly
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-dev-help@lucene.apache.org
  • Doug Cutting (JIRA) at Dec 9, 2008 at 6:16 pm
    [ https://issues.apache.org/jira/browse/LUCENE-1482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12654875#action_12654875 ]

    Doug Cutting commented on LUCENE-1482:
    --------------------------------------

    safeDebugMsg is protected in a public class, which means it will appear in the javadoc, which it should not. Also, logging the thread ID should be done by the logging system, not by Lucene. So that method should just be removed, no?

    Also, you've added braces to all of the log statements. This is in conformance with our style guidelines, but I prefer that logging add a minimum of vertical space, so that more real logic is visible at once. I suggest you not make this style change in this patch, but propose it separately, if at all.

    Replace infoSteram by a logging framework (SLF4J)
    -------------------------------------------------

    Key: LUCENE-1482
    URL: https://issues.apache.org/jira/browse/LUCENE-1482
    Project: Lucene - Java
    Issue Type: Improvement
    Components: Index
    Reporter: Shai Erera
    Priority: Minor
    Fix For: 2.4.1, 2.9

    Attachments: LUCENE-1482.patch, slf4j-api-1.5.6.jar, slf4j-nop-1.5.6.jar


    Lucene makes use of infoStream to output messages in its indexing code only. For debugging purposes, when the search application is run on the customer side, getting messages from other code flows, like search, query parsing, analysis etc can be extremely useful.
    There are two main problems with infoStream today:
    1. It is owned by IndexWriter, so if I want to add logging capabilities to other classes I need to either expose an API or propagate infoStream to all classes (see for example DocumentsWriter, which receives its infoStream instance from IndexWriter).
    2. I can either turn debugging on or off, for the entire code.
    Introducing a logging framework can allow each class to control its logging independently, and more importantly, allows the application to turn on logging for only specific areas in the code (i.e., org.apache.lucene.index.*).
    I've investigated SLF4J (stands for Simple Logging Facade for Java) which is, as it names states, a facade over different logging frameworks. As such, you can include the slf4j.jar in your application, and it recognizes at deploy time what is the actual logging framework you'd like to use. SLF4J comes with several adapters for Java logging, Log4j and others. If you know your application uses Java logging, simply drop slf4j.jar and slf4j-jdk14.jar in your classpath, and your logging statements will use Java logging underneath the covers.
    This makes the logging code very simple. For a class A the logger will be instantiated like this:
    public class A {
    private static final logger = LoggerFactory.getLogger(A.class);
    }
    And will later be used like this:
    public class A {
    private static final logger = LoggerFactory.getLogger(A.class);
    public void foo() {
    if (logger.isDebugEnabled()) {
    logger.debug("message");
    }
    }
    }
    That's all !
    Checking for isDebugEnabled is very quick, at least using the JDK14 adapter (but I assume it's fast also over other logging frameworks).
    The important thing is, every class controls its own logger. Not all classes have to output logging messages, and we can improve Lucene's logging gradually, w/o changing the API, by adding more logging messages to interesting classes.
    I will submit a patch shortly
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-dev-help@lucene.apache.org
  • Shai Erera (JIRA) at Dec 10, 2008 at 6:51 am
    [ https://issues.apache.org/jira/browse/LUCENE-1482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Shai Erera updated LUCENE-1482:
    -------------------------------

    Attachment: LUCENE-1482-2.patch

    I kept safeDebugMsg because it was used by a class which extended IndexWriter and relied on that method to be called. However, I fixed the class by overriding testPoint instead. So I can now remove safeDebugMsg.

    As for the output format, I agree that it should be handled by the logging system, but wanted to confirm that with other members before I change it. I'm glad that you agree to that too.

    Attached is a new patch which removes the method.
    Replace infoSteram by a logging framework (SLF4J)
    -------------------------------------------------

    Key: LUCENE-1482
    URL: https://issues.apache.org/jira/browse/LUCENE-1482
    Project: Lucene - Java
    Issue Type: Improvement
    Components: Index
    Reporter: Shai Erera
    Priority: Minor
    Fix For: 2.4.1, 2.9

    Attachments: LUCENE-1482-2.patch, LUCENE-1482.patch, slf4j-api-1.5.6.jar, slf4j-nop-1.5.6.jar


    Lucene makes use of infoStream to output messages in its indexing code only. For debugging purposes, when the search application is run on the customer side, getting messages from other code flows, like search, query parsing, analysis etc can be extremely useful.
    There are two main problems with infoStream today:
    1. It is owned by IndexWriter, so if I want to add logging capabilities to other classes I need to either expose an API or propagate infoStream to all classes (see for example DocumentsWriter, which receives its infoStream instance from IndexWriter).
    2. I can either turn debugging on or off, for the entire code.
    Introducing a logging framework can allow each class to control its logging independently, and more importantly, allows the application to turn on logging for only specific areas in the code (i.e., org.apache.lucene.index.*).
    I've investigated SLF4J (stands for Simple Logging Facade for Java) which is, as it names states, a facade over different logging frameworks. As such, you can include the slf4j.jar in your application, and it recognizes at deploy time what is the actual logging framework you'd like to use. SLF4J comes with several adapters for Java logging, Log4j and others. If you know your application uses Java logging, simply drop slf4j.jar and slf4j-jdk14.jar in your classpath, and your logging statements will use Java logging underneath the covers.
    This makes the logging code very simple. For a class A the logger will be instantiated like this:
    public class A {
    private static final logger = LoggerFactory.getLogger(A.class);
    }
    And will later be used like this:
    public class A {
    private static final logger = LoggerFactory.getLogger(A.class);
    public void foo() {
    if (logger.isDebugEnabled()) {
    logger.debug("message");
    }
    }
    }
    That's all !
    Checking for isDebugEnabled is very quick, at least using the JDK14 adapter (but I assume it's fast also over other logging frameworks).
    The important thing is, every class controls its own logger. Not all classes have to output logging messages, and we can improve Lucene's logging gradually, w/o changing the API, by adding more logging messages to interesting classes.
    I will submit a patch shortly
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-dev-help@lucene.apache.org
  • Shai Erera (JIRA) at Dec 14, 2008 at 12:17 pm
    [ https://issues.apache.org/jira/browse/LUCENE-1482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12656382#action_12656382 ]

    Shai Erera commented on LUCENE-1482:
    ------------------------------------

    Is there anything else I can do in order to help drive this issue forward?
    Replace infoSteram by a logging framework (SLF4J)
    -------------------------------------------------

    Key: LUCENE-1482
    URL: https://issues.apache.org/jira/browse/LUCENE-1482
    Project: Lucene - Java
    Issue Type: Improvement
    Components: Index
    Reporter: Shai Erera
    Priority: Minor
    Fix For: 2.4.1, 2.9

    Attachments: LUCENE-1482-2.patch, LUCENE-1482.patch, slf4j-api-1.5.6.jar, slf4j-nop-1.5.6.jar


    Lucene makes use of infoStream to output messages in its indexing code only. For debugging purposes, when the search application is run on the customer side, getting messages from other code flows, like search, query parsing, analysis etc can be extremely useful.
    There are two main problems with infoStream today:
    1. It is owned by IndexWriter, so if I want to add logging capabilities to other classes I need to either expose an API or propagate infoStream to all classes (see for example DocumentsWriter, which receives its infoStream instance from IndexWriter).
    2. I can either turn debugging on or off, for the entire code.
    Introducing a logging framework can allow each class to control its logging independently, and more importantly, allows the application to turn on logging for only specific areas in the code (i.e., org.apache.lucene.index.*).
    I've investigated SLF4J (stands for Simple Logging Facade for Java) which is, as it names states, a facade over different logging frameworks. As such, you can include the slf4j.jar in your application, and it recognizes at deploy time what is the actual logging framework you'd like to use. SLF4J comes with several adapters for Java logging, Log4j and others. If you know your application uses Java logging, simply drop slf4j.jar and slf4j-jdk14.jar in your classpath, and your logging statements will use Java logging underneath the covers.
    This makes the logging code very simple. For a class A the logger will be instantiated like this:
    public class A {
    private static final logger = LoggerFactory.getLogger(A.class);
    }
    And will later be used like this:
    public class A {
    private static final logger = LoggerFactory.getLogger(A.class);
    public void foo() {
    if (logger.isDebugEnabled()) {
    logger.debug("message");
    }
    }
    }
    That's all !
    Checking for isDebugEnabled is very quick, at least using the JDK14 adapter (but I assume it's fast also over other logging frameworks).
    The important thing is, every class controls its own logger. Not all classes have to output logging messages, and we can improve Lucene's logging gradually, w/o changing the API, by adding more logging messages to interesting classes.
    I will submit a patch shortly
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-dev-help@lucene.apache.org
  • Grant Ingersoll (JIRA) at Dec 15, 2008 at 8:30 pm
    [ https://issues.apache.org/jira/browse/LUCENE-1482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12656739#action_12656739 ]

    Grant Ingersoll commented on LUCENE-1482:
    -----------------------------------------

    In the mail thread, Yonik said (http://lucene.markmail.org/message/j3r53azx2tmljijg?q=Java+Logging+in+Lucene):
    {quote}
    I'm leery of going down this logging road because people may add
    logging statements in inappropriate places, believing that
    isLoggable() is about the same as infoStream != null

    They seem roughly equivalent because of the context in which they are
    tested: coarse grained logging where the surrounding operations
    eclipse the logging check.

    isLoggable() involves volatile reads, which prevent optimizations and
    instruction reordering across the read. On current x86 platforms, no
    memory barrier instructions are needed for a volatile read, but that's
    not true of other architectures.
    {quote}

    Thoughts on how to address this? Have you done any performance testing of this patch versus the current system, both w/ infoStream == null and infoStream != null.

    I also think it is important to address Yonik's point about "inappropriate" places. In other words, we need guidelines about where and when to using logging and committers need to be on the lookout for logging uses. I realize that is as much a community policing problem as a patch problem, but, we should address them before we adopt logging.
    Replace infoSteram by a logging framework (SLF4J)
    -------------------------------------------------

    Key: LUCENE-1482
    URL: https://issues.apache.org/jira/browse/LUCENE-1482
    Project: Lucene - Java
    Issue Type: Improvement
    Components: Index
    Reporter: Shai Erera
    Priority: Minor
    Fix For: 2.4.1, 2.9

    Attachments: LUCENE-1482-2.patch, LUCENE-1482.patch, slf4j-api-1.5.6.jar, slf4j-nop-1.5.6.jar


    Lucene makes use of infoStream to output messages in its indexing code only. For debugging purposes, when the search application is run on the customer side, getting messages from other code flows, like search, query parsing, analysis etc can be extremely useful.
    There are two main problems with infoStream today:
    1. It is owned by IndexWriter, so if I want to add logging capabilities to other classes I need to either expose an API or propagate infoStream to all classes (see for example DocumentsWriter, which receives its infoStream instance from IndexWriter).
    2. I can either turn debugging on or off, for the entire code.
    Introducing a logging framework can allow each class to control its logging independently, and more importantly, allows the application to turn on logging for only specific areas in the code (i.e., org.apache.lucene.index.*).
    I've investigated SLF4J (stands for Simple Logging Facade for Java) which is, as it names states, a facade over different logging frameworks. As such, you can include the slf4j.jar in your application, and it recognizes at deploy time what is the actual logging framework you'd like to use. SLF4J comes with several adapters for Java logging, Log4j and others. If you know your application uses Java logging, simply drop slf4j.jar and slf4j-jdk14.jar in your classpath, and your logging statements will use Java logging underneath the covers.
    This makes the logging code very simple. For a class A the logger will be instantiated like this:
    public class A {
    private static final logger = LoggerFactory.getLogger(A.class);
    }
    And will later be used like this:
    public class A {
    private static final logger = LoggerFactory.getLogger(A.class);
    public void foo() {
    if (logger.isDebugEnabled()) {
    logger.debug("message");
    }
    }
    }
    That's all !
    Checking for isDebugEnabled is very quick, at least using the JDK14 adapter (but I assume it's fast also over other logging frameworks).
    The important thing is, every class controls its own logger. Not all classes have to output logging messages, and we can improve Lucene's logging gradually, w/o changing the API, by adding more logging messages to interesting classes.
    I will submit a patch shortly
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-dev-help@lucene.apache.org
  • Grant Ingersoll (JIRA) at Dec 15, 2008 at 8:32 pm
    [ https://issues.apache.org/jira/browse/LUCENE-1482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

    Grant Ingersoll updated LUCENE-1482:
    ------------------------------------

    Priority: Major (was: Minor)
    Lucene Fields: [New, Patch Available] (was: [Patch Available, New])

    While the patch itself is likely minor, the implications of applying the patch are significant, IMO
    Replace infoSteram by a logging framework (SLF4J)
    -------------------------------------------------

    Key: LUCENE-1482
    URL: https://issues.apache.org/jira/browse/LUCENE-1482
    Project: Lucene - Java
    Issue Type: Improvement
    Components: Index
    Reporter: Shai Erera
    Fix For: 2.4.1, 2.9

    Attachments: LUCENE-1482-2.patch, LUCENE-1482.patch, slf4j-api-1.5.6.jar, slf4j-nop-1.5.6.jar


    Lucene makes use of infoStream to output messages in its indexing code only. For debugging purposes, when the search application is run on the customer side, getting messages from other code flows, like search, query parsing, analysis etc can be extremely useful.
    There are two main problems with infoStream today:
    1. It is owned by IndexWriter, so if I want to add logging capabilities to other classes I need to either expose an API or propagate infoStream to all classes (see for example DocumentsWriter, which receives its infoStream instance from IndexWriter).
    2. I can either turn debugging on or off, for the entire code.
    Introducing a logging framework can allow each class to control its logging independently, and more importantly, allows the application to turn on logging for only specific areas in the code (i.e., org.apache.lucene.index.*).
    I've investigated SLF4J (stands for Simple Logging Facade for Java) which is, as it names states, a facade over different logging frameworks. As such, you can include the slf4j.jar in your application, and it recognizes at deploy time what is the actual logging framework you'd like to use. SLF4J comes with several adapters for Java logging, Log4j and others. If you know your application uses Java logging, simply drop slf4j.jar and slf4j-jdk14.jar in your classpath, and your logging statements will use Java logging underneath the covers.
    This makes the logging code very simple. For a class A the logger will be instantiated like this:
    public class A {
    private static final logger = LoggerFactory.getLogger(A.class);
    }
    And will later be used like this:
    public class A {
    private static final logger = LoggerFactory.getLogger(A.class);
    public void foo() {
    if (logger.isDebugEnabled()) {
    logger.debug("message");
    }
    }
    }
    That's all !
    Checking for isDebugEnabled is very quick, at least using the JDK14 adapter (but I assume it's fast also over other logging frameworks).
    The important thing is, every class controls its own logger. Not all classes have to output logging messages, and we can improve Lucene's logging gradually, w/o changing the API, by adding more logging messages to interesting classes.
    I will submit a patch shortly
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-dev-help@lucene.apache.org
  • Shai Erera (JIRA) at Dec 15, 2008 at 8:38 pm
    [ https://issues.apache.org/jira/browse/LUCENE-1482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12656741#action_12656741 ]

    Shai Erera commented on LUCENE-1482:
    ------------------------------------

    I haven't done any performance test since after following Doug's proposal, I switched to using the NOP logger which all of its isLoggable()-like calls return false. That means that OOtB calling isDebugEnabled returns immediately w/ false, w/o doing any checks. This may even be inlined by the compiler.

    If someone decides to drop the slf4j-jdk14.jar (for example), then that means that someone already uses Java logging in other places in his/her application (not necessarily the search parts only) and therefore the overhead of checking whether logging is enabled is already been taken into consideration.
    However, I don't think it's the Lucene's community's responsibility to take care of these situations, because like I wrote before, someone might extend SLF4J and provide a very inefficient isDebugEnabled() implementation, and it wouldn't (and shouldn't) be the community's responsibility.

    What's important is that Lucene uses the NOP adapter, which really does nothing.

    If you insist, I can run some performance tests, but I really don't think it's necessary.
    Replace infoSteram by a logging framework (SLF4J)
    -------------------------------------------------

    Key: LUCENE-1482
    URL: https://issues.apache.org/jira/browse/LUCENE-1482
    Project: Lucene - Java
    Issue Type: Improvement
    Components: Index
    Reporter: Shai Erera
    Fix For: 2.4.1, 2.9

    Attachments: LUCENE-1482-2.patch, LUCENE-1482.patch, slf4j-api-1.5.6.jar, slf4j-nop-1.5.6.jar


    Lucene makes use of infoStream to output messages in its indexing code only. For debugging purposes, when the search application is run on the customer side, getting messages from other code flows, like search, query parsing, analysis etc can be extremely useful.
    There are two main problems with infoStream today:
    1. It is owned by IndexWriter, so if I want to add logging capabilities to other classes I need to either expose an API or propagate infoStream to all classes (see for example DocumentsWriter, which receives its infoStream instance from IndexWriter).
    2. I can either turn debugging on or off, for the entire code.
    Introducing a logging framework can allow each class to control its logging independently, and more importantly, allows the application to turn on logging for only specific areas in the code (i.e., org.apache.lucene.index.*).
    I've investigated SLF4J (stands for Simple Logging Facade for Java) which is, as it names states, a facade over different logging frameworks. As such, you can include the slf4j.jar in your application, and it recognizes at deploy time what is the actual logging framework you'd like to use. SLF4J comes with several adapters for Java logging, Log4j and others. If you know your application uses Java logging, simply drop slf4j.jar and slf4j-jdk14.jar in your classpath, and your logging statements will use Java logging underneath the covers.
    This makes the logging code very simple. For a class A the logger will be instantiated like this:
    public class A {
    private static final logger = LoggerFactory.getLogger(A.class);
    }
    And will later be used like this:
    public class A {
    private static final logger = LoggerFactory.getLogger(A.class);
    public void foo() {
    if (logger.isDebugEnabled()) {
    logger.debug("message");
    }
    }
    }
    That's all !
    Checking for isDebugEnabled is very quick, at least using the JDK14 adapter (but I assume it's fast also over other logging frameworks).
    The important thing is, every class controls its own logger. Not all classes have to output logging messages, and we can improve Lucene's logging gradually, w/o changing the API, by adding more logging messages to interesting classes.
    I will submit a patch shortly
    --
    This message is automatically generated by JIRA.
    -
    You can reply to this email to add a comment to the issue online.


    ---------------------------------------------------------------------
    To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
    For additional commands, e-mail: java-dev-help@lucene.apache.org

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupjava-dev @
categorieslucene
postedDec 8, '08 at 2:15p
activeDec 15, '08 at 8:38p
posts13
users1
websitelucene.apache.org

1 user in discussion

Shai Erera (JIRA): 13 posts

People

Translate

site design / logo © 2021 Grokbase