tag:blogger.com,1999:blog-556744574829494472024-02-08T04:48:41.931-08:00Log4erl's BlogLog4erlhttp://www.blogger.com/profile/07615112361079664417noreply@blogger.comBlogger14125tag:blogger.com,1999:blog-55674457482949447.post-44255152445333733272009-05-02T14:00:00.000-07:002009-05-02T14:14:45.362-07:00Log4erl 0.8.6 is released!Hi,<br /><br />After a small delay, version 0.8.6 is available in both google code page as well as in github.com. I don't like to publish latest code until I finish with all intended features, so, sorry for those who's been waiting for particular feature.<br /><br />This release includes XML appender, with very flexible configuration. You can basically add whatever attribute/element you want in the output xml record for logs. You can configure XML appender using file-based configuration but unfortunately, it's not possible to send XMLSpecs (look at Appenders_API.txt for more info) in file-based configuration yet. <br /><br />Another change in 0.8.6 is to have sync logging instead of async (use sync_notify instead of notify in gen_event). This actually should be faster and prevent crashing erlang in high loads.<br /><br />There are also small changes, most notably is the introduction of error_logger handler. Check out documentation in this release for more information.<br /><br />As usual, I'll be happy to hear your comments/suggestions. You know my email! <br /><br />Best regards,<br /><br />AhmedLog4erlhttp://www.blogger.com/profile/07615112361079664417noreply@blogger.com7tag:blogger.com,1999:blog-55674457482949447.post-23644317210967640712009-04-19T17:11:00.000-07:002009-04-19T17:17:01.271-07:00Delays in the next version...Hi,<br /><br />I've promised previously that I'll deliver the next version of log4erl about now but due to unforeseeable work, I won't be able to. My apologies if I kept you waiting. All new features and bug fixes are already done but xml_appender is still under development. However, don't be too depressed, it's not going to take a long time. I think it'll be ready by next week :).<br /><br />AhmedLog4erlhttp://www.blogger.com/profile/07615112361079664417noreply@blogger.com0tag:blogger.com,1999:blog-55674457482949447.post-1886445984435904032009-04-08T00:02:00.000-07:002009-05-01T03:54:41.297-07:00Log4erl issue with high loadHi,<br /><br />I've got a <a href="http://groups.google.com/group/log4erl/browse_thread/thread/a7104e77820065d7#">case</a> in Log4erl google group mailing list regarding a "out of memory" crash in log4erl. This is an issue that I've been aware of for some time now. It's actually the downside of using concurrent language where you can spawn lots of processes. A similar <a href="http://blogtrader.net/dcaoyuan/entry/a_case_study_of_scalable">case</a> can happen with error_logger.<br /><br />The problem is that many processes may be sending log requests to a logger that has to perform IO on it. Given that IO is slow, requests coming from many processes will accumulate in the process inbox and logger will be overwhelmed eventually causing "out of memory" crash.<br /><br />In log4erl case, and presumably error_logger, log requests are asynchronous. Basically, log_manager module will use gen_enent:notify/2 to dispatch requests to the appropriate logger and this operation is non-blocking. <br /><br />You can create multiple loggers, degrade log level to 'warn' or 'fatal' for example. But this will not really solve the problem. A solution is to change notify/2 to sync_notify/2, but this will slow logging down. Also, I think this is the easy but not the right way to proceed.<br /><br />I think the best solution is to use a persistent db (queue) to offload requests from logger's inbox. Until now, I can only think of using something like <a href="http://memcachedb.org/memcacheq/">memcacheq</a> so that log_manager will save requests into a queue and loggers will fetch them from there. I'm still thinking about this but haven't made my mind yet. Implementation will not be easy as we have to take into account many cases (e.g. queue is empty). If you have any suggestions on how to best do this, please let me know.<br /><br />Best regards,<br /><br />Ahmed<br /><br /><span style="font-style:italic;">Update 01-May-2009:</span><br />It turned out that this is mainly <a href="http://www.nabble.com/Low-disk-logging-performance-in-SMP-td23217639.html">because</a> in Eralng selective receive is very slow when there are so many messages in process messages. That is why synchronous logging has better performance. I'll change the default behavior in the next version of log4erl to sync instead of async event notification. This should solve the erlang crash reported in log4erl google group.Log4erlhttp://www.blogger.com/profile/07615112361079664417noreply@blogger.com0tag:blogger.com,1999:blog-55674457482949447.post-62976442561389537862009-04-03T13:54:00.000-07:002009-04-03T13:57:34.085-07:00Log4erl groupHi,<br /><br />As a result of many questions/email regarding log4erl, I decided to start a new google group for log4erl project. Please subscribe to our mailing list on the URL below and participate for the continued support of log4erl. <br /><br /><a href="http://groups.google.com/group/log4erl">http://groups.google.com/group/log4erl</a><br /><br />Have a nice day.<br /><br />AhmedLog4erlhttp://www.blogger.com/profile/07615112361079664417noreply@blogger.com0tag:blogger.com,1999:blog-55674457482949447.post-39864620761966083292009-03-31T11:34:00.000-07:002009-03-31T11:51:58.681-07:00Feedback for log4erlHi,<br /><br />I've just seen some comments to say about a couple of emails I received from log4erl users. The first was from Loren of <a href="http://www.stackhub.com/">StackHub</a> requesting XML appneder and the other from Kuba submitting an update to have log4erl handle calls to error_logger and requesting a fix to file_appender. <br /><br />The bug in file_appender is that it copies files during rotation instead of just renaming them. For large files, this could results in timeouts or even crashes the program. I don't know what was I thinking but it's need a fix.<br /><br />XML appender should be available in the next release and I'll also include Kuba's code and a fix to file_appender along. Thanks to Loren and Kuba for the feedback. Keep your suggestions coming and good luck.<br /><br />AhmedLog4erlhttp://www.blogger.com/profile/07615112361079664417noreply@blogger.com0tag:blogger.com,1999:blog-55674457482949447.post-68003190081903248202009-03-24T07:08:00.000-07:002009-03-24T07:17:04.234-07:00Syslog and file-based configuration in version 0.8.5Hi,<br /><br />A new version of log4erl is updated in both google code and github.com repositories. The new features here are introduction of file-based configuration and syslog_appender. To get more details, please read updated documentations in files 'README.txt', 'API.txt', 'Appenders_API.txt', 'CHANGELOG.txt' and the new file 'CONFIGURATION.txt' for details of the file-based configuration. You can also find an example in 'priv/log4erl.conf'.<br /><br />I've also made changes to the format with the addition of %I, %S, and %Z specifiers to output ISO-8601 standard output and timezone information. Also, %d and %j specifiers are changed to be ISO-compatible.<br /><br />Enjoy the new release and keep the bug reports coming. <br /><br />Regards,<br /><br />AhmedLog4erlhttp://www.blogger.com/profile/07615112361079664417noreply@blogger.com0tag:blogger.com,1999:blog-55674457482949447.post-48144868993087412062009-03-01T00:57:00.000-08:002009-03-01T01:33:16.413-08:00Log4erl v. 0.8.4 is out with SMTP logsHi,<br /><br />Today, version 0.8.4 has been released. The main addition here, as promised, is smtp_appender. As the name imply, this allows for sending log messages to emails. <br /><br />For this, I've used smtp_client written by Michael Bradford (http://erlang.org/user.html#smtp_client-1.1) with some modifications that allows it to work with Erlang/OTP R12B. I don't think the additions will allow it to work with Erlang/OPT R11B or earlier. If so, you can download and use the original smtp_client from erlang.org instead of the included module in Log4erl distribution.<br /><br />smtp_client doesn't support SSL authentication, so smtp_appender might not work with gmail or some other email servers that require SSL, but for everything else, this should be enough. Please send me your feedback on this. <br /><br />Note however that you should use smtp_appender with fatal or warn messages and shouldn't be used with more verbose log levels as the current implementation connects every time to the email server when sending email. I think this should be clear but just wanted everyone to know.<br /><br />Also, there are 2 other additions, the most important is the feedback when error. For example, if you tried to add an appender with name already registered, you should get an error. Please check README.txt and CHANGELOG.txt for more info.<br /><br />Log4erl is evolving rapidly in recent days. Next I'm going to add syslog appender and, possibly, html appender. Also, I'll try to add the much needed configuration file for log4erl. <br /><br />Anyways, we still need to do a lot of work before reaching the 1.0 milestone on July. <br /><br />Good luck,<br /><br />AhmedLog4erlhttp://www.blogger.com/profile/07615112361079664417noreply@blogger.com0tag:blogger.com,1999:blog-55674457482949447.post-13807611345880094902009-02-28T03:01:00.000-08:002009-02-28T03:04:58.751-08:00Log4erl in github.comHi,<br /><br />I have added log4erl to github.com repository. I've planned to adding this for sometime becuase it seems many people prefer git to svn and wanted to offer both choices. I will not stop updating google code's svn but I'll maintain both github.com and google code's. You can find the github page below<br /><br />http://github.com/ahmednawras/log4erl<br /><br />Have fun.<br /><br />AhmedLog4erlhttp://www.blogger.com/profile/07615112361079664417noreply@blogger.com0tag:blogger.com,1999:blog-55674457482949447.post-41294091679813654102009-02-10T03:24:00.000-08:002009-02-10T03:28:56.918-08:00Fixed a small bug in log4erl-0.8.3Hi,<br /><br />I've just fixed a bug in the latest log4erl version which caused a crash and multiple restarts by the supervisor for file_appenders when rotated. I don't know how I missed this one but it's fixed now. If you are currently using version 0.8.3, you must have noticed this by now. You can download the latest one with the fix in google code's. The file is named log4erl-0.8.3_2.tar.gz. Thanks to Mark for bringing this up to my attention.<br /><br />Best regards,<br /><br />AhmedLog4erlhttp://www.blogger.com/profile/07615112361079664417noreply@blogger.com0tag:blogger.com,1999:blog-55674457482949447.post-74978952701712886552009-02-07T05:29:00.000-08:002009-02-07T05:41:02.591-08:00Small step... log4erl 0.8.3Hi,<br /><br />A new version of log4erl (0.8.3) is out there. This release includes small additions and changes. This release has been staying in my disk for sometime now waiting for docs to be included in the release tar. I've just updated repository and added the tarball to the download section. An update to the manual is due soon.<br /><br />The new additions include a console appender, few additions to the log_formatter and small fixes. Refer to README.txt file in the tarball for updated API document. The new additions to log_formatter includes 3 new specifiers (%i, %b, %B) which outputs milliseconds, short month's name (e.g. Feb), month's name (e.g. February).<br /><br />Next release will include SMTP logger and perhaps a simple configuration file. <br /><br />As usual, please send your comments and suggestions to my email (can be found in README.txt file). Next is to update the manual. Until then, enjoy.<br /><br />AhmedLog4erlhttp://www.blogger.com/profile/07615112361079664417noreply@blogger.com0tag:blogger.com,1999:blog-55674457482949447.post-22230670185059891392008-12-04T15:03:00.000-08:002008-12-04T15:13:42.689-08:00Log4erl 0.8 manual is outHi,<br /><br />I've just added a new manual for log4erl 0.8. A functions I didn't talk about previously is log4erl:get_appenders/0,1, which basically returns what appenders are currently added to a logger.<br /><br />Following up to my previous post, I forgot to mention one missing feature for log4erl 1.0, which is file configuration. Currently, you can only interact with log4erl using specified API (check this in the manual). However, I'd like to have the ability to configure log4erl using a file (xml?, erlang term?, properties files?) so that users can just say log4erl:file("log.conf"). The format of the configuration file is not determined yet but it's not difficult to imagine having more than 1. <br /><br />Time is scarce and I'm busy but I'll try to do all of this by July '09 as expected. This is all for now. Have a good day!Log4erlhttp://www.blogger.com/profile/07615112361079664417noreply@blogger.com0tag:blogger.com,1999:blog-55674457482949447.post-73361907953236576012008-12-03T13:50:00.001-08:002008-12-04T14:12:57.629-08:00Log4erl 0.8.2 is outHi All,<br /><br />It's been a long time and as usual been busy with other projects. However, finally another release is out. I've just released version 0.8.2 of log4erl. It's a small step but, hopefully, in the right direction. This code has been in my computer for some time now.<br /><br />The most important addition here is log4erl:change_format/2,3. This is similar to PatternLayout in Log4j. A list of specifiers can be found in README.txt file. Using log4erl:change_format/2,3, you'll be able to change the layout of the outputed message. Also, there's log4erl:change_log_level/1,2 plus some bug fixes.<br /><br />I think this is a good time to talk about where this project is going and where I do intend to take it. I've got some questions regarding performance, email logging, Syslog...etc. All of these features are within the scope of log4erl. However, it's too early to say when they will be added. The main concern for me now is to freeze the API and make log_manager stable. Once this is done, adding new formatters will be simpler. Performance metrics and optimization should be done around the time log_manager is stable.<br /><br />Having said that, I think console_formatter is necessary and I'd like to add SMTP formatter soon. By log4erl 1.0, I intend to have at least 3 formatters: file_formatter, console_formatter, smtp_formatter. Also, a performance benchmark should be published by that time. The expected release of version 1.0 is around July 2009 but please don't bet on it. There are a lot of variables here and I don't want to give promises. This is just an educated guess.<br /><br />I had some questions about distributed logging. Such a feature is not intended to be part of the short-term plan for log4erl. I started log4erl to be used in my own projects, which are mainly run in single-server machines and decided to share it with the rest of the community. I believe distributed logging needs a more careful design and testing methods, although I believe implementing distributed logging is simpler in erlang than other languages. For now, the only distributed mechanism log4erl can do is through rpc:call/4.<br /><br />Anyways, back to work. The next step is the long-awaited manual for log4erl 0.8. Until then, have a good time.Log4erlhttp://www.blogger.com/profile/07615112361079664417noreply@blogger.com0tag:blogger.com,1999:blog-55674457482949447.post-74566305765470453172008-08-03T03:03:00.000-07:002008-08-03T03:21:26.656-07:00API changes in 0.8I haven't got a lot of time to explain the API changes in 0.8. I'll get around to this task when possible. For now, I'll quickly introduce some changes.<br /><br />In 0.7, Log4erl didn't have the concept of appender. For version 0.7, logger and appender where the same things. So, when you add a new logger with specific properties, you basically create a logger, add an appender with the properties. That's why version 0.7 had <span style="font-style:italic;">log4erl:add_file_logger/1,2</span> functions.<br /><br />The biggest change in 0.8, is the separation of both. So now, you can create a logger and add as many appenders to it. Currently, there are only file_appender and dummy_appender. Dummy_appender is basically dumps all messages sent to it. It can be useful during development. An example for using the new API, the following code adds 2 appenders to the default logger, one is a file_appender and the other is dummy_appender.<br /><br />> log4erl:add_appender({file_appeneder, appender1}, {"../logs", "logger1", {size, 100000}, 4, "elog", warn}).<br />> log4erl:add_appender({dummy_appender, dummy}, "").<br /><br />As you can see above, developer is required to supply a tuple of name of the module and a name for the appender as the first agrument to <span style="font-style:italic;">add_appender/2</span>. The 2nd item in the tuple is an identifier used by Log4erl. You can get all appenders for a logger using <span style="font-style:italic;">log4erl:get_appenders/0,1</span><br /><br />Users can also use <span style="font-style:italic;">add_file_appender/2,3</span> and <span style="font-style:italic;">add_dummy_appender/2,3</span> as convenient functios for add_appender.<br /><br />Another useful function in the API is <span style="font-style:italic;">change_log_level/1,2</span>, by which users can change log level without having to restart Log4erl.<br /><br />I hope this is useful for all of Log4erl 0.8 users. As I said, I'll get around to modifiying the manual in google code to reflect these changes. Until then, have a nice day :)Log4erlhttp://www.blogger.com/profile/07615112361079664417noreply@blogger.com0tag:blogger.com,1999:blog-55674457482949447.post-32045780437694995992008-07-28T14:24:00.001-07:002008-07-28T14:28:49.157-07:00Hi,<br /><br />This is the first blog for log4erl. Log4erl is a project aimed to implement a generic logger for Erlang. Erlang currently has its own logger and it has its merits. However, coming from Java land, I prefer something similar to Log4J and Log4* family of loggers. That's why I decided to start log4erl. I've just uploaded version 0.8.1 to google code's page of the project ("http://code.google.com/p/log4erl"). Please use it and let me know your feedback.<br /><br />Obviously, the project has just started and there's a lot to do. Any contributions, comments are welcome. Cheers!Log4erlhttp://www.blogger.com/profile/07615112361079664417noreply@blogger.com0