The log4mongo-java project includes several [Log4J|http://logging.apache.org/log4j/] appenders for storing log messages in a MongoDB collection.
* MongoDbAppender - stores the LogEvent in a BSONified form
* ExtendedMongoDbAppender - Extends MongoDbAppender by allowing you to add top level elements
* MongoDbPatternLayoutAppender - supports logging data in a custom format
[Downloads|http://log4mongo.org/display/PUB/Downloads]
h2. QuickStart
# Download and install [MongoDB|http://www.mongodb.org/downloads] from [http://www.mongodb.org/downloads]
# Start mongod running on localhost on the default port
# Download the [Mongo Java driver|http://github.com/mongodb/mongo-java-driver/downloads] jarfile
# Download [Log4J|http://logging.apache.org/log4j/1.2/download.html] jarfile
# Download [log4mongo-java|http://log4mongo.org/display/PUB/Downloads] jarfile
# Place these three jar files in your classpath
# Create a file called log4j.properties in your classpath
# Add the following lines to the log4j.properties file
## log4j.rootLogger=INFO, MongoDB
## log4j.appender.MongoDB=org.log4mongo.MongoDbAppender
## log4j.appender.MongoDB.databaseName=appname
## log4j.appender.MongoDB.collectionName=log
# Create a Java class like:
{newcode:java}
import org.apache.log4j.Logger;
public class LogTest {
public static void main(String[] args) {
Logger logger = Logger.getLogger(LogTest.class);
logger.error("Don't panic");
}
}
{newcode}
Compile and run the class. Then, start a mongo shell and confirm that an event was logged.
{code}
$ mongo
MongoDB shell version: 2.0.3
connecting to: testconnecting to: test
> use appname
switched to db appname
> db.log.findOne()
{{
"_id" : ObjectId("4c60e21be54744706960af09"),
"timestamp" : "Thu Mar 29 2012 22:22:35 GMT-0700 (PDT)",
"level" : "ERROR",
"thread" : "main",
"message" : "Don't panic",
"fileName" : "LogTest.java",
"method" : "main",
"lineNumber" : "8",
"class" : {
"fullyQualifiedClassName" : "LogTest",
"package" : [
""
],
"className" : "LogTest"
}
"loggerName" : {
"fullyQualifiedClassName" : "LogTest",
"package" : [
""
],
"className" : "LogTest"
}
}
{code}
h2. Appenders
An abstract BsonAppender class (introduced in 0.5) provides basic functionality for serializing a Log4J LogEvent to a BSON DBObject. The MongoDB specific appenders extend this class. This abstract class is useful if you want to store the same information outside of MongoDB.
h3. MongoDbAppender
MongoDbAppender stores the Log4J LogEvent object as a document in a MongoDB collection. The document is serialized in a format that is very similar to the structure of the LogEvent class. Where possible, the data is stored as appropriate BSON objects, rather than just strings.
A future enhancement will optionally store the Map in the MDC (Mapped Diagnostic Context) as part of the document.
h3. MongoDbPatternLayoutAppender
MongoDbPatternLayoutAppender stores a log message as a document in a MongoDB collection based on a user-specified conversion pattern using the standard Log4J pattern layout, parser and converter functionality. While requiring some additional configuration, this appender allows significant flexibility over the format of the stored document. The MongoDbPatternLayoutAppender also allows custom information to be stored with the document. The custom values are determined at the time an event is logged.
h2. Log4J Properties
The root directory of the project distribution contains a sample properties file named log4j.properties.sample. All of the properties used by log4mongo-java are documented in the sample file.
h2. Replica Sets
The log4mongo-java 0.6 release added full support for replica sets. While earlier releases would work with replica sets, you could specify only one host in the hostname property. If the appender happened to be initialized when that host was not available, the appender would not be able to connect to other members of the set.
With the 0.6 release, you should specify the names of all the hosts that can become master. The Mongo Java driver will automatically find the current master. For example:
{code}
log4j.appender.MongoDB.hostname=mongo1.example.com mongo2.example.com
{code}
If MongoDB is listening on the same port on each host, you can specify just that port. If it's the default port of 27017, you can omit the property.
{code}
log4j.appender.MongoDB.port=27000
{code}
If the same port is not used by each host, then you must specify the port for all hosts. For example, if the hosts mongo1 and mongo2 are listening on 27000 and 28000, respectively:
{code}
log4j.appender.MongoDB.port=27000 28000
{code}
When a master becomes available, the next attempt to log will fail and the insert will be lost. The Mongo driver will then check for the new master and the next insert should go to the new master. Log inserts between the time of the original master's failure and the election of the new master will be lost.
h2. MongoDB Collection vs. Capped Collection
Although events can be logged to any MongoDB [collection|http://www.mongodb.org/display/DOCS/Collections], it's common to use a [capped collection|http://www.mongodb.org/display/DOCS/Capped+Collections]. Capped collections provide very high performance and automatic removal of old information. One drawback, though, is that the collection is capped by size in bytes or in objects, rather than by a time period. A common desire is to keep logs for a certain period of time. Capped collections cannot be used for this purpose, though. If you are confident that logging data will rarely exceed a certain size over the desired period of time, though, capped collections can be a big win.
Another advantage of capped collections is that they allow you to control the amount of disk space used for logging. History is full of sad stories of apps grinding to a halt because the log files filled up a disk partition to which the app needed to write other data.
Assuming you are using a database named *appname* and a collection named *log*, you can use the following commands in the mongo shell to create a capped collection of size 100,000,000 bytes.
{code}
> use appname
> db.createCollection('log', {capped:true, size:100000000})
{code}
h2. Authentication
MongoDB authentication is optional. If the userName property is specified, the appender will attempt to authenticate against the specified MongoDB database using the values for the userName and password properties. (TODO: Add description of behavior and a test case for when userName is specified, but password isn't).
For example, if an appender is defined as:
{code}
log4j.appender.MongoDB=com.google.code.log4mongo.MongoDbAppender
{code}
and the host, database, user name and password are specified as:
{code}
log4j.appender.MongoDB.hostname=localhost
log4j.appender.MongoDB.databaseName=appname
log4j.appender.MongoDB.userName=open
log4j.appender.MongoDB.password=sesame
{code}
then when the appender is initialized, it will attempt to authenticate as user *open* with password *sesame* against the *appname* database in a MongoDB instance listening on the *default port* on *localhost*.
h2. Troubleshooting
h3. When logging an event, this exception is thrown - java.lang.NoSuchMethodError: com.mongodb.DBCollection.insert(Lcom/mongodb/DBObject\;)
If you use log4mongo-java 0.3.2 or earlier with MongoDB Java driver 2.1 or later, you will probably get this error.
A source compatible, but binary incompatible, change was made in the 2.1 driver to the DBCollection.insert() method used by log4mongo-java. That method now takes a varargs argument instead of a single DBCollection object. In client code, the Java compiler converts the varargs argument to an array whether you pass one DBCollection or more. However, log4mongo-java 0.3.2 was compiled against the 2.0 Java driver, so it passes a standalone DBCollection, instead of the array that is expected by the compiled 2.1 driver code.
The workaround is to either upgrade to log4mongo-java 0.4.0\+ or to compile your own log4mongo-java jarfile against MongoDB Java driver 2.1+.
h3. When logging an event, this exception is thrown - java.lang.NoSuchMethodError: java.util.Arrays.copyOf(\[Ljava/lang/Object;I)\[Ljava/lang/Object;
Versions of log4mongo-java up to and including 0.6.0 contain Java 1.6 specific code. You may see the above exception if you are using Java 1.5 or earlier. This has been fixed on GitHub and will be included in the next release of log4mongo-java, likely to be 0.6.1.
h3. NullPointerException when app exits and appender is closed when using MongoDB Java Driver 2.2
The MongoDB Java Driver 2.2 has a bug, [JAVA-180|http://jira.mongodb.org/browse/JAVA-180], that causes a NullPointerException if you call Mongo.close() and are not using MongoDB in a replica set. The bug is fixed in [Robert's fork of the 2.2 driver|http://github.com/RobertStewart/mongo-java-driver], as well as in the [trunk for the 2.3 release|http://github.com/mongodb/mongo-java-driver]. The bug was not present in the 2.1 driver.
h3. Can't Serialize Proxied Object
Libraries such as Hibernate may use libraries like CGLib for generating class proxies. In your code it will look like you are passing a regular object to one of the Logger methods for logging a message, but at runtime a proxy will be passed. The BSONEncoder can't serialize this proxy.
Here's an example stack trace:
{code}
java.lang.IllegalArgumentException: can't serialize class com.example.MyClass$$EnhancerByCGLIB$$8a4801c9
at org.bson.BSONEncoder._putObjectField(BSONEncoder.java:205)
at org.bson.BSONEncoder.putObject(BSONEncoder.java:121)
...
{code}
The simplest workaround is usually to call .toString() on the object before passing it to one of the Logger methods.
...