Not an actual Spring-related subject

We need help about a subject that came up in our project in a very annoying way.
I would like to say that the following has little to do with Spring properly, in the sense that it is a design/architecture decision rather than a technological problem, but I think that the Spring community is as good a place as every other to ask such questions in order to get feedback from experienced, talented and on-the-same-page developers. So, without further due, I’ll get to the point.

We are developing a large system; I mean a nation-wide large system, which of course manages a lot of business entities. There is a certain subset of said entities that must be audited every time one is added, deleted or updated. The problem here is the special nature of these audits. See, we are not looking at a text-only solution in the form of “User X did operation Y to entity Z” but we must actually store the entity being updated or saved in order to fulfill a detailed view requirement in one of our use cases that asks for the full history of changes of an user-selected attribute of one of the entities audited. For instance, if one such an entity were a Person class, an user could ask for all the states the name or hairColor attributes has had since the Person object with id 456 was created. This issue grows even bigger when we take into account that this is probably true ALSO for lists or sets of attributes.
We are using Spring 2.0.6 (our project framework ensemble was defined from above, using a template from a previous project, so we don’t have the choice to advocate for Spring 2.5 right now) Hibernate 3.x and JSF. We also use Acegi 1.0.x for security (although we heavily modified it for special requirements of security we have and also to support AJAX) and other less relevant frameworks. Our DBMS is Oracle 10g.
Initially we thought in serializing all the traffic between the application and the database, and coded a solution accordingly. But although we haven’t tested the code in real situations, it’s obvious this is a huge pull down on performance, not to mention database disk space. We are using a Hibernate Interceptor injected into the Transaction Manager to hold the problematic code. This way we have a chance to get in the way of every transaction that is made to the database via Hibernate, leaving out native SQL queries, HQL and stored procedures (the latest we handle using Spring support, it was by far easier to comprehend and to use than Hibernate’s).
So current solution is as follows:
Every time an Auditable entity is saved, we serialized it and stored it in this format in a BLOB field.
Every time an Auditable entity is updated, we loop through all of its attributes seeing which one of them changed (the Hibernate interceptor provides two arrays for this, one with current values and one with old values) and if it has, then if it’s a plain attribute or a wrapper, we store it as a ModificationAttribute object, containing the name of the property and its value; if it’s an Object, we serialize it into another implementation of ModificationAttribute, and if it’s a Collection of any kind, we serialize the object(s) that caused the Collection to change into ModificationAttribute(s). The the UpdateAudit and its accompanying list of Modification attributes is save to the database at the commit phase in the same interceptor. Problems with these approach are obvious, we are imposing a huge hit in performance in the application as well as in the storage capacity of the database. We have thought a lot about this, but the requirements are adamant and we are always forced to go through all the attributes in the entities. What I need is some insight of experience about something similar to these, or at the very least another way to store an object through several updates in a way that allows it to be rebuilt as it was at any given point of time.

The complete solution is larger than this, but this is the problematic part.
I’m sorry I’m posting this in the Spring forum which is supposed to be devoted to technical discussion of Spring features, but I thought this was as good a place to ask for help as any other. And after all, I’m also a Spring user, albeit a very lame one. I’m not posting code or XML due to confidentiality constraints, but I’ll be glad to address any concern you may have in the clearest way possible. I also apologize for my poor English.
Thanks in advance

This sounds like a job for AOP. A way to get started would be to examine all the places you would like your auditting to take place and express those as AspectJ pointcuts. From there you can start outlining the behavior of your auditting code at each pointcut. It might be different for different point cuts. This would be a type of specification of the cross cutting concern you have.

Once you have this information (or preferably subset to help you start with experiements early) you can start designing abstract and concrete aspects which implement your ideas.

At this point you have to decide if you want to use Spring AOP or AspectJ directly. AspectJ tends to perform better and can work on code that isn't managed by Spring. In such a large system I would be surprised if everything is (especially things like entities).

In addition to reading up on AspectJ pointcut language, the Spring reference manual on AOP and other general AOP topics, check out the following links. They don't match exactly what you are looking for, but I believe they will be helpful in setting the right frame of mind and give you some ideas:
Using AOP in the Enterprise
Capturing failures and system state (part I)

We are already using Spring AOP in other layers of the enterprise for other kinds of audittings, namely method calls, web services calls, login and logout audittings and several others. I restricted the post to only the Entities audits because these are the ones being problematic.

I do found interesting the assertion you make abour AspectJ being able to work even in code not directly managed by Spring. I'm afraid I haven't study AspectJ at all due to time constraints in our preparation for this project. Could you elaborate on how AspectJ accomplishes the weaving of aspects outside the Spring Context? In the mean time, I'll take a look at some AspectJ bibliography.

It uses some sort of Java Agent to load AspectJ aspects when classes are loaded. You can even use built in aspects to autowire your Entities using the @Configured annotation.

You should look at the Spring reference manual for more detail.

You will be able to mix Spring AOP with AspectJ AOP in the same application. You will just apply the AspectJ aspects to particular pointcuts that aren't Spring managed beans.

Spring 2.5 has even better for support for load time weaving, and depending on your application, can do away with having to modify your JVM parameters to accomplish load time weaving, using a runtime class instead.

So, if I understand correctly, I could weave aspects in the "setX" methods of my entities and then when we are about to persist said entity I would extract the list of modifications from the entity (this is state managed by the aspect) and then persist this list usign the Hibernate Interceptor?

This will effectively save me the trouble of going through all of the attributes of every updated entity, at the cost of making my entities fatter (each entity must now hold the list of modifications it had had in its current lifecycle)... I'm not sure if this is correct, I mean from a good design practices point of view, or even a practical one. It is a new idea though... Please tell me if what I'm saying is feasible....

Yes, that does sound feasible. You might even consider using an AOP concept of a mixin, where functionality can be added to your entities without changing your actual entity class. So when Hibernate persists the message, the modifications could go with it.

Saving you the trouble of going through all of the attributes is exactly the kind of problems AOP is designed to solve. This type of functionality (often called concerns) cuts across multiple methods or even classes, hence the AOP term "cross cutting concerns".

This idea was one of my firsts in our early brainstorm sessions on the subject, but we rejected as it would require (at least with the knowledge we have) to prototype our business entities inside the Spring Context and retrieve them every time (an awful thought) because Spring AOP as of Spring 2.0 requires the object being woven to be managed by a Context.
But looking at the problem as weaving the aspect in bytecode via AspectJ changes a lot of things....

I'm not sure if AspectJ has something akin to the Introduction Advice Spring uses for mixins... I guess it has, since the very reference recommends it as a more complete pointcut language. I guess that all that is left is to start training myself in AspectJ and fast... we are already falling behind schedule here...

Your help have been priceless... If I manage to get this right, I'll let you know. On to work then...