Hi,
After a small delay, version 0.8.6 is available in both google code page as well as in github.com. I don't like to publish latest code until I finish with all intended features, so, sorry for those who's been waiting for particular feature.
This release includes XML appender, with very flexible configuration. You can basically add whatever attribute/element you want in the output xml record for logs. You can configure XML appender using file-based configuration but unfortunately, it's not possible to send XMLSpecs (look at Appenders_API.txt for more info) in file-based configuration yet.
Another change in 0.8.6 is to have sync logging instead of async (use sync_notify instead of notify in gen_event). This actually should be faster and prevent crashing erlang in high loads.
There are also small changes, most notably is the introduction of error_logger handler. Check out documentation in this release for more information.
As usual, I'll be happy to hear your comments/suggestions. You know my email!
Best regards,
Ahmed
Saturday, May 2, 2009
Sunday, April 19, 2009
Delays in the next version...
Hi,
I've promised previously that I'll deliver the next version of log4erl about now but due to unforeseeable work, I won't be able to. My apologies if I kept you waiting. All new features and bug fixes are already done but xml_appender is still under development. However, don't be too depressed, it's not going to take a long time. I think it'll be ready by next week :).
Ahmed
I've promised previously that I'll deliver the next version of log4erl about now but due to unforeseeable work, I won't be able to. My apologies if I kept you waiting. All new features and bug fixes are already done but xml_appender is still under development. However, don't be too depressed, it's not going to take a long time. I think it'll be ready by next week :).
Ahmed
Wednesday, April 8, 2009
Log4erl issue with high load
Hi,
I've got a case in Log4erl google group mailing list regarding a "out of memory" crash in log4erl. This is an issue that I've been aware of for some time now. It's actually the downside of using concurrent language where you can spawn lots of processes. A similar case can happen with error_logger.
The problem is that many processes may be sending log requests to a logger that has to perform IO on it. Given that IO is slow, requests coming from many processes will accumulate in the process inbox and logger will be overwhelmed eventually causing "out of memory" crash.
In log4erl case, and presumably error_logger, log requests are asynchronous. Basically, log_manager module will use gen_enent:notify/2 to dispatch requests to the appropriate logger and this operation is non-blocking.
You can create multiple loggers, degrade log level to 'warn' or 'fatal' for example. But this will not really solve the problem. A solution is to change notify/2 to sync_notify/2, but this will slow logging down. Also, I think this is the easy but not the right way to proceed.
I think the best solution is to use a persistent db (queue) to offload requests from logger's inbox. Until now, I can only think of using something like memcacheq so that log_manager will save requests into a queue and loggers will fetch them from there. I'm still thinking about this but haven't made my mind yet. Implementation will not be easy as we have to take into account many cases (e.g. queue is empty). If you have any suggestions on how to best do this, please let me know.
Best regards,
Ahmed
Update 01-May-2009:
It turned out that this is mainly because in Eralng selective receive is very slow when there are so many messages in process messages. That is why synchronous logging has better performance. I'll change the default behavior in the next version of log4erl to sync instead of async event notification. This should solve the erlang crash reported in log4erl google group.
I've got a case in Log4erl google group mailing list regarding a "out of memory" crash in log4erl. This is an issue that I've been aware of for some time now. It's actually the downside of using concurrent language where you can spawn lots of processes. A similar case can happen with error_logger.
The problem is that many processes may be sending log requests to a logger that has to perform IO on it. Given that IO is slow, requests coming from many processes will accumulate in the process inbox and logger will be overwhelmed eventually causing "out of memory" crash.
In log4erl case, and presumably error_logger, log requests are asynchronous. Basically, log_manager module will use gen_enent:notify/2 to dispatch requests to the appropriate logger and this operation is non-blocking.
You can create multiple loggers, degrade log level to 'warn' or 'fatal' for example. But this will not really solve the problem. A solution is to change notify/2 to sync_notify/2, but this will slow logging down. Also, I think this is the easy but not the right way to proceed.
I think the best solution is to use a persistent db (queue) to offload requests from logger's inbox. Until now, I can only think of using something like memcacheq so that log_manager will save requests into a queue and loggers will fetch them from there. I'm still thinking about this but haven't made my mind yet. Implementation will not be easy as we have to take into account many cases (e.g. queue is empty). If you have any suggestions on how to best do this, please let me know.
Best regards,
Ahmed
Update 01-May-2009:
It turned out that this is mainly because in Eralng selective receive is very slow when there are so many messages in process messages. That is why synchronous logging has better performance. I'll change the default behavior in the next version of log4erl to sync instead of async event notification. This should solve the erlang crash reported in log4erl google group.
Friday, April 3, 2009
Log4erl group
Hi,
As a result of many questions/email regarding log4erl, I decided to start a new google group for log4erl project. Please subscribe to our mailing list on the URL below and participate for the continued support of log4erl.
http://groups.google.com/group/log4erl
Have a nice day.
Ahmed
As a result of many questions/email regarding log4erl, I decided to start a new google group for log4erl project. Please subscribe to our mailing list on the URL below and participate for the continued support of log4erl.
http://groups.google.com/group/log4erl
Have a nice day.
Ahmed
Tuesday, March 31, 2009
Feedback for log4erl
Hi,
I've just seen some comments to say about a couple of emails I received from log4erl users. The first was from Loren of StackHub requesting XML appneder and the other from Kuba submitting an update to have log4erl handle calls to error_logger and requesting a fix to file_appender.
The bug in file_appender is that it copies files during rotation instead of just renaming them. For large files, this could results in timeouts or even crashes the program. I don't know what was I thinking but it's need a fix.
XML appender should be available in the next release and I'll also include Kuba's code and a fix to file_appender along. Thanks to Loren and Kuba for the feedback. Keep your suggestions coming and good luck.
Ahmed
I've just seen some comments to say about a couple of emails I received from log4erl users. The first was from Loren of StackHub requesting XML appneder and the other from Kuba submitting an update to have log4erl handle calls to error_logger and requesting a fix to file_appender.
The bug in file_appender is that it copies files during rotation instead of just renaming them. For large files, this could results in timeouts or even crashes the program. I don't know what was I thinking but it's need a fix.
XML appender should be available in the next release and I'll also include Kuba's code and a fix to file_appender along. Thanks to Loren and Kuba for the feedback. Keep your suggestions coming and good luck.
Ahmed
Tuesday, March 24, 2009
Syslog and file-based configuration in version 0.8.5
Hi,
A new version of log4erl is updated in both google code and github.com repositories. The new features here are introduction of file-based configuration and syslog_appender. To get more details, please read updated documentations in files 'README.txt', 'API.txt', 'Appenders_API.txt', 'CHANGELOG.txt' and the new file 'CONFIGURATION.txt' for details of the file-based configuration. You can also find an example in 'priv/log4erl.conf'.
I've also made changes to the format with the addition of %I, %S, and %Z specifiers to output ISO-8601 standard output and timezone information. Also, %d and %j specifiers are changed to be ISO-compatible.
Enjoy the new release and keep the bug reports coming.
Regards,
Ahmed
A new version of log4erl is updated in both google code and github.com repositories. The new features here are introduction of file-based configuration and syslog_appender. To get more details, please read updated documentations in files 'README.txt', 'API.txt', 'Appenders_API.txt', 'CHANGELOG.txt' and the new file 'CONFIGURATION.txt' for details of the file-based configuration. You can also find an example in 'priv/log4erl.conf'.
I've also made changes to the format with the addition of %I, %S, and %Z specifiers to output ISO-8601 standard output and timezone information. Also, %d and %j specifiers are changed to be ISO-compatible.
Enjoy the new release and keep the bug reports coming.
Regards,
Ahmed
Sunday, March 1, 2009
Log4erl v. 0.8.4 is out with SMTP logs
Hi,
Today, version 0.8.4 has been released. The main addition here, as promised, is smtp_appender. As the name imply, this allows for sending log messages to emails.
For this, I've used smtp_client written by Michael Bradford (http://erlang.org/user.html#smtp_client-1.1) with some modifications that allows it to work with Erlang/OTP R12B. I don't think the additions will allow it to work with Erlang/OPT R11B or earlier. If so, you can download and use the original smtp_client from erlang.org instead of the included module in Log4erl distribution.
smtp_client doesn't support SSL authentication, so smtp_appender might not work with gmail or some other email servers that require SSL, but for everything else, this should be enough. Please send me your feedback on this.
Note however that you should use smtp_appender with fatal or warn messages and shouldn't be used with more verbose log levels as the current implementation connects every time to the email server when sending email. I think this should be clear but just wanted everyone to know.
Also, there are 2 other additions, the most important is the feedback when error. For example, if you tried to add an appender with name already registered, you should get an error. Please check README.txt and CHANGELOG.txt for more info.
Log4erl is evolving rapidly in recent days. Next I'm going to add syslog appender and, possibly, html appender. Also, I'll try to add the much needed configuration file for log4erl.
Anyways, we still need to do a lot of work before reaching the 1.0 milestone on July.
Good luck,
Ahmed
Today, version 0.8.4 has been released. The main addition here, as promised, is smtp_appender. As the name imply, this allows for sending log messages to emails.
For this, I've used smtp_client written by Michael Bradford (http://erlang.org/user.html#smtp_client-1.1) with some modifications that allows it to work with Erlang/OTP R12B. I don't think the additions will allow it to work with Erlang/OPT R11B or earlier. If so, you can download and use the original smtp_client from erlang.org instead of the included module in Log4erl distribution.
smtp_client doesn't support SSL authentication, so smtp_appender might not work with gmail or some other email servers that require SSL, but for everything else, this should be enough. Please send me your feedback on this.
Note however that you should use smtp_appender with fatal or warn messages and shouldn't be used with more verbose log levels as the current implementation connects every time to the email server when sending email. I think this should be clear but just wanted everyone to know.
Also, there are 2 other additions, the most important is the feedback when error. For example, if you tried to add an appender with name already registered, you should get an error. Please check README.txt and CHANGELOG.txt for more info.
Log4erl is evolving rapidly in recent days. Next I'm going to add syslog appender and, possibly, html appender. Also, I'll try to add the much needed configuration file for log4erl.
Anyways, we still need to do a lot of work before reaching the 1.0 milestone on July.
Good luck,
Ahmed
Subscribe to:
Posts (Atom)