Testing Sanity

The core task of software development is sanity testing.

  • warning: Parameter 2 to gmap_gmap() expected to be a reference, value given in /home2/thepalls/public_html/cgpsoftware/includes/module.inc on line 483.
  • strict warning: Non-static method view::load() should not be called statically in /home2/thepalls/public_html/cgpsoftware/sites/all/modules/views/views.module on line 879.
Posted by cgp

There's a lot of ways this is referred to in the java world but they all essentially mean the above. The method described at dzone is good, but it really isn't runtime unless you provide a method for loading a class (which was probably beyond scope)

After really reading through what I could find on the web though, the consensus seems to be that reloading code is quirky and unpredictable. I don't like the idea of needing a custom loader either, it's all a bit overbearing when you compare it to something like scripting, so I'm using the scripting built into Java 6.

References

Posted by cgp

Often, there are times on a server you may want to install multiple applications, but you may want to update one of the applications without bringing down the others, so you need to run multiple instances. Not surprisingly, you can do this without having multiple instances of the binaries lying around. This can make it potentially "easier" (to update your tomcat, assuming there aren't any hangups with regards to dependencies/oddities on a version of tomcat.

Setting up multiple tomcats on a single server is pretty straightforward:

First, the base installation of tomcat should only need the following:

+tomcat (home)
|__bin
|__lib - contains server libraries, can be used for shared libraries

Everything else is a customizable piece of tomcat:

+tomcat (base_dir)
|___work - the deployed version of your application
|___temp - for whatever tomcat needs to do on the fly
|__ webapp - your web applications
|___log - the logs for your server
|___conf - configs for the server.

* - not all tomcats seem to have this folder

If you've created the above folder structure somewhere outside of the original tomcat installation, you are pretty much ready to go. Just set a CATALINA_BASE environment variable and you're ready to roll.

Catalina.out troubles

Catalina.out is the catch-all files for console output. (All those who have used System.out.println are familiar with it)

One thing I find a bit annoying about the catalina startup script is that it does not provide a way to change the default name of this file. There's a dozen environment variables, but none to change the console output filename. You can do this yourself by simply searching for catalina.out in your catalina.sh script and replcing it with an environment variable choice.

The other gotcha about this file is that there are no restrictions on it's output length, and it can eat your disk space up complete if you're not careful. A couple of ways to deal with it:

  • Setup a cron job to roll it over, compress it, or to delete it daily/weekly
  • Symbolic link to /dev/null :) -- naturally, assumes you have no interest in the console output

Customizing logging

Previously, I had talked some about customizing logging. Tomcat doesn't do Log4j, but uses a logging system called JULI. It's configuration files are a little bit different, but you should be able to pare it down without trouble.

Removing Unnecessary Parts

Under the work and webapps directory there's a series of applications that come with tomcat: (examples, manager, host-manager, docs) You can easily get rid of examples and docs, but manager and host-manager are up to you. They can be used by things like Maven to help deploy applications, so you may want to consider keeping them around.

References

  • http://lifeofhumanvirus.blogspot.com/2007/03/manually-deploying-new-tomcat-web.html
  • http://tomcat.apache.org/tomcat-5.5-doc/logging.html
  • http://brondsema.net/blog/index.php/2006/05/25/tomcat_juli_logging_properties
Posted by cgp

A lot of shops use the apache webserver in front of their tomcat servers to serve up their webservers. They use them essentially as proxies so that when the container server is taken down, the system can display a maintenance message or redirect to a web server which is up.

But, if you have something else doing this job, and have chosen to run just tomcat, how can you recreate those really nice apache logs to run analysis software like awstats against?

Well, actually, it's incredibly simple, and a quick look at the default configuration provides pretty much everything a server needs:

Just remember that the logger has attributes/options not shown on the valve documentation (but referred to), to control naming of the file logs.

Gotchas

You can't specify an absolute path in the directory attribute, so to get to the root or another path, you're going to have to go: "../../../../" until you get there.

References

  • Valve Reference Documentation
  • File Logging (controls naming of access logs)


  • Log4J

    18 Aug 2008
    Posted by cgp

    Everyone seems to use log4j. The configuration always seemed a bit cryptic to me.

    For instance, when I had the following entry:


    log4j.rootLogger=debug, stdout, daily

    Nothing logged to the debug "logger"! It had to come after stdout in the list as such:


    log4j.rootLogger=warn, stdout, daily, debug

    The reason for this is simple! The first parameter in this list specifies the logging threshhold for each of the appenders attached to the rootLogger. If you specify a valid value ("debug", "info", "warn", "error", "fatal"), it uses that as a threshhold. If you do not specify a valid threshhold value, it assumes a default (I'm not sure what the default is)

    So, specifying the names stdout/daily/debug for log4j.rootLogger creates appenders by that name under the rootLogger. Options are set for each appender as such:


    log4j.appender.stdout=....
    log4j.appender.daily=....
    log4j.appender.debug=....
    ...

    The root of those entry declares the class to be used (each has a different purpose):

    log4j.appender.stdout=org.apache.log4j.ConsoleAppender
    log4j.appender.daily=org.apache.log4j.DailyRollingFileAppender
    log4j.appender.debug=org.apache.log4j.RollingFileAppender

    And then, specify settings of each of those is just a matter of knowing the options available to the class:

    log4j.appender.debug.layout=org.apache.log4j.PatternLayout
    log4j.appender.debug.layout.ConversionPattern=%d{h:mm:ssa} %5p (%F:%L) - %m%n
    log4j.appender.debug.threshold=debug
    log4j.appender.debug.MaxFileSize=1024KB
    log4j.appender.debug.MaxBackupIndex=1
    log4j.appender.debug.File=/apps/decommerce_be/logs/decommerce_be_debug.log

    Pretty nifty. You can use XML to setup these configs too, to make it clear which appender owns each setting.

    One other thing, in addition to being able to specify a logger under log4j.rootLogger, you can specify a logger like so:


    log4j.logger.EJB.Logger=info,EJB

    This will set the default threshhold and also specify the name of the appender (the logger).

    The advantage of this being, that you can configure it to take only a particular subset of log messages (such as for EJB)

    Configuration remains the same.

    log4j.appender.EJB=org.apache.log4j.DailyRollingFileAppender
    log4j.appender.EJB.File=${LOCLOG}/logs.txt

    Additivity sets whether logs which went to that particular log will appear in the root log:


    log4j.additivity.EJB.Logger=false

    So, the only thing that confuses me here is, being that this is not set against any particular class, doesn't this mean ALL logs will not be sent to the root logger?

    This seems like it would work, and make more sense:
    You can also set a logger for a particular class, simply by specifying the class after logger:

    log4j.logger.com.cgpsoftware=debug,cgpLogs
    log4j.additivity.com.cgpsoftware=false
    log4j.appender.cgpLogs=org.apache.log4j.DailyRollingFileAppenderExt

    Now logs from com.cgpsoftware should appear in the appender log from this entry, but not in the root log (according to what I understand.

    References

    Posted by cgp

    I'm not certain why, but I was getting: could not reassociate uninitialized transient collection for the longest of times when trying to merge my hibernate object into the database. The odd thing is, it would work in one set of unit tests but not another.

    The thing that made it work?

    Setting my Fetch types from lazy to eager. Of course, this makes my queries run nice and slow as it is fetching all of the subrecords.

    I found this discussion to be interesting, but not necessarily helpful.

    Posted by cgp

    So how do you extract data from semantically weak sources?

    Did the presentation as a pirate. Interesting.

    1. Invent things like tag libs? I guess so, they are useful...
    2. You rate the data.
    3. User reviews?

    Semantics is wrapping meaning around data... Such as wrapping that a red light means food. Wrapping

    Learning is a change in behavior based on previous experience.

    Machine learning, algorithms that change based on input.

    Punctilio, a classic case of machine learning.

    tags alone=weak

    tags+people=weak (because they do the AI) :)

    Van Tapes Calculator

    08 Feb 2008
    Posted by cgp
    This is a response to the oft quoted remark:


    Basically, this tool calculates exactly what that bandwidth actually is. It does take into account the carrying capacity of the vehicle. For example, most vans won't be able to carry more than 2500lbs of hard drives, which works out to a surprisingly low number of hard drives.

    I probably could add in something to calculate the latency. See here for some comparison of bandwidth speeds.

    For some reason this is broken in Chrome... (turns out that I had variables/prototypes named media which is some sort of reserved word, changing the names cleared it all up)

    Media (Tape) Specs
    Storage per Tape (in gigs):
    Weight per Tape (lbs):
    Cost per Tape:
    Transfer Rate (mb/s):
    Tape Dimensions (inches) : x x
    Transportation (Van) Specs
    Van Max Weight Load (lbs) :
    Speed :
    Van Dimensions (inches) : x x
    # of Tapes:
    Drive Time (hours):
     
    Posted by cgp

    My goal can be to understand everything on these pages... Perhaps I should already :)