Architecture advice involving multiple projects/spring context files?

[Fatefree]

I am developing an application that in essence is built out of many smaller but isolated applications, with one web project tying them all together and I am debating about the architecture of my eclipse projects and context files.

Currently I have (and have traditionally used) an ApplicationFramework and ApplicationWeb project. Service/Dao/Domain classes reside in the application framework along with the service/dao context files so they can be unit tested and isolated from the web.

In the web project web.xml, I refer to the context files from the framework project and also any context files the web might be using.

This has worked well in the past, but I am noticing that since I will have multiple smaller projects, each with their own services and domain objects, it may make sense to divide them into their own projects as follows:

ApplicationFramework
/ \
SomeAppFramework AnotherAppFramework

With ApplicationWeb referring to all of them. This would of course mean each subproject has its own context files, which may bring a new set of issues, such as do I then need multiple cache managers to configure the application caches separately and would that be inefficient (or more efficient?), how would the ehcache MBean handle multiple managers etc.

The alternative is to have the code reside in different packages within the main application framework, although I have a feeling that will grow very fast. Some of these smaller applications will not have much code, some may have a lot more, although I expect them all to have persistent objects of some kind.

From an architecture point of view I believe there is value in isolating the subprojects, and typing this out I realize my only big hang up is the multiple cachemanagers, so I will look into that to start. But I was wondering if anyone had any high level insight they could share.

[Fatefree]

Another problem I just realized with multiple context files is the session factory mappings. I don't know how it would be possible to share one session factory without some sort of coupling between the individual dao objects that needed to be mapped.

It seems like I almost need a centralized place for the context files, and I fiddled with the idea of an ApplicationContext project which basically holds all the context files, and I suppose every project would refer to it when unit testing, and the web when running the application. I don't know though as I've never heard anything about this and it seems hacky by my initial impression.

Another sort of hack would be to extend the local session factory bean to maybe search for a directory of mappings thats specified in the xml, and also any sub packages to pull in all of the mappings of the individual applications..

[constv]

1) Keep your "business" components outside the webapp module; I understand you are doing that already.

2) Package your business components by "business domain"/functionality - NOT by class type!! This is very important. Most sample applications and books do it otherwise, and that's unfortunate. Packaging your classes by class type (e.g. "services", "persistence", "entities", etc.) automatically ties all your modules together and will force you to always load the whole project with all source modules - even if you only are working on one particular functional domain. Your complete project should be an assembly of multiple modules each of which should be available individually (as a separate project.) In other words, you should be able, if necessary, to create a smaller workspace with any single source module (or only a sub-set of modules), while all other dependencies are pulled as jars. I see horrendously packaged projects every day - sometimes with thousands of classes - that can't really be split into smaller manageable sub-projects. Don't make the same mistake - especially if you have an opportunity to start from a clean slate. Within each "functional domain" module you should consolidate all things related to this functional domain: entities/model, service API, utilities, etc. So your complete system might look like this:

common
-entities (entities reusable by all)
-service[-related stuff such as common interfaces, etc.]
-common utils (that may be used by any module)
- some common constants and exceptions (most should be specific to the related functionality though...)
user
- user model (entities)
- user service API, optional implementation (other implementations may reside in apps)
- user utils
- etc.
bookings
- bookings model (entities)
- bookings service API, optional implementation (other implementations may reside in apps)
- bookings utils
- etc.
awards
- similar to the above
etc.
webapp
-css
-js
-WEB-INF
- config/...
- views
- etc.
web.xml

Carefully define uni-directional dependencies so that each module has a dependency on "common" and the minimum other modules. Your webapp will have the dependencies on all the above. If you are using Maven, you can create a parent POM for your "business" components, so you can build them all separately from your webapp. You should be able to load each POM into your IDE separately/individually and work on that small project without having to load everything else.

You may still use a top "parent" POM where you define the versions and global properties. In that case, all you will need to load a small sub-project is the parent POM and the POM for that individual module.

3) Your webapp should define your Spring configuration for you business modules. I don't recommend putting any Spring config files into your business modules. Configuration is application specific. Although it is absolutely ok to have your business components to have Spring dependencies (nothing wrong with using Spring libs for various purposes, obviously), leave it up to your application to wire things together the way it needs it. Theoretically, another application may reuse the business/common components but provide different implementation of some of the collaborating beans, etc. For unit testing, I personally prefer not to use Spring contexts. Spring-wiring beans for unit tests makes it integration testing rather than unit testing. Wiring your Spring configuration (or stuff like log4j configuration) into the business modules makes them less reusable, and often just adds clutter and confusion. I often see projects where developers have a hard time telling which of the multiple configuration files is actually being used and when. They mistakenly modify the wrong files expecting results while the changes don't take effect... Keep things simple: consolidate your app configuration, e.g. under the webapp/WEB-INF/config/app-context directory, or something like that. I prefer to split configuration into multiple files - also based on the functionality and functional domains.

HTH,
Constantine

[Fatefree]

Thanks there is a lot of useful information in your post.

One thing I am guilty of is Package-By-Layer rather than Package-By-Feature. I know that feature is the winner, its just a hard transition for some reason (perhaps the OCD in me). However I will make the change for this project since I have a clean slate.

One thing I am still struggling with is the testing. For example, I keep the application context in the business project because I often need a Spring context to test a unit a code. For example, a Hibernate call using Criteria objects is something I most certainly need to test as a unit, but I can't do so without the dao which extends the hibernate template which needs a session factory and ultimately needs to pull in a lot from a spring configuration to run.
It seems that a unit test turns into an integration test.

And theres a side benefit of knowing my application context is wired up and running correctly since the web app is just referring to the one I am testing. However I can see the point that the client should be the one configuring, since it is indeed the application's responsibility.

So I'm still at a loss of the correct approach, because while I would love to have my small application modules isolated, testable, i'm not sure how to go about doing it.

[constv]

One thing I am still struggling with is the testing. For example, I keep the application context in the business project because I often need a Spring context to test a unit a code. For example, a Hibernate call using Criteria objects is something I most certainly need to test as a unit, but I can't do so without the dao which extends the hibernate template which needs a session factory and ultimately needs to pull in a lot from a spring configuration to run.
It seems that a unit test turns into an integration test.

And theres a side benefit of knowing my application context is wired up and running correctly since the web app is just referring to the one I am testing. However I can see the point that the client should be the one configuring, since it is indeed the application's responsibility.

So I'm still at a loss of the correct approach, because while I would love to have my small application modules isolated, testable, i'm not sure how to go about doing it.

Not a problem... Just put your test Spring config files into the Resources folder under the test tree, not main. I am assuming you are using the Maven standard directory structure with two distinct source trees: main and test. If not, use it. It will solve your problem. Put any config files you want under the test tree, in "resources". They will be invisible to your "main" source code.

[Fatefree]

Hmmmm. I had originally put the files in the src/main/resources, and was operating under the mindset that the config files should also be shared amongst the modules. I think thats whats causing my problems.

So in essence, to solve this, every module will have their own set of config files, hidden away in the src/test/resources folder and configured just small enough to get the tests running.

So now the modules are independent and testable, and ill switch to the new package structure to sweeten the deal.

Then the web application has a separate set of config files to wire up the app accordingly. I think I'm starting to see the big picture.

So as an almost unrelated topic, where might code go that is responsible for convenience methods that say, might clear your entire db, or put sample data for you app to use? Would you consider that test classes, or some sort of external support project?

[constv]

I think you got it!

So as an almost unrelated topic, where might code go that is responsible for convenience methods that say, might clear your entire db, or put sample data for you app to use? Would you consider that test classes, or some sort of external support project?

That sounds to me like stuff totally unrelated to you application. So, i would keep that in a separate "maintenance-utilities" module.

Also, you may want to have some utility-type classes in your test tree that would help you generate some test data. Although, as I said, personally, I am very much against doing any kind of integration stuff or messing with the database in JUnit tests. Perhaps, for integration tests, you could create a special module - that would mock your actual app but without the UI, etc. and have all the required dependencies on your business modules. You could add such maintenance classes/methods to such mock client, run the DB setup before running your integration tests, and then run the cleanup methods. But I would not mix such code with the application or business components themselves. For me, separation of concerns and minimizing complexity are the two most important principles that have never let me down. You can never go wrong with these.

[Fatefree]

One final question if you will. Do you consider the hibernate hbm files to be web configuration as well?

I can see it either way, where some clients may use different table names or perhaps have different columns required in their app than others and thus should define them itself.

And of course the other side of the argument is where it would be nice to have the framework components define them and test them once and for all. But perhaps that is the job of some theoretical 'integration' module. What are you thoughts?

[constv]

One final question if you will. Do you consider the hibernate hbm files to be web configuration as well? I can see it either way, where some clients may use different table names or perhaps have different columns required in their app than others and thus should define them itself.

Not "web" (obviously), but certainly application configuration. You basically answered your own question: different applications may not only [be allowed to] use different column mappings, but also work with different database schemas, different data sources, and/or different data access implementations altogether - depending on their needs, specifics, and deployment environments. Although it is a very common practice and convenient shortcut to wire in the persistence mappings into the potentially otherwise reusable business components, and even into the domain model (e.g. JPA annotations), I think that is substandard architecture that relies on the very lame thesis: we are not going to reuse anything anyway. Lame... Actually, more often than not, teams find themselves eventually having to implement another app, and another - within the same enterprise. So what do they do? They just duplicate their domain models and business components over and over - instead of reusing what could have been easily reused if they hadn't wired app-specific stuff into those components. I see this all the time.

And of course the other side of the argument is where it would be nice to have the framework components define them and test them once and for all. But perhaps that is the job of some theoretical 'integration' module. What are you thoughts?

I think you know what I think. I prefer independent components that have minimum dependencies. To me it is mind-boggling to suggest that an architectural decision could be made based on where some developer decided to place his/her test implementations. The only tests that belong with a module are unit tests. That is, the tests that test only the units implemented within the module - without their dependencies/collaborators. That's what mock objects are for. Integration tests should never affect your packaging or architecture. They should live outside and test the modules that know nothing about their existence. Most importantly, there is no such thing as integration testing "once and for all" since the modular design of your architecture and components implies that your components should be usable in various configurations/scenarios/deployments, not just one - hard-wired into the module itself. Hope this makes sense...

Good luck,
C

[Fatefree]

Thanks again constv, I completely agree with your points about reusable business components. I shy away from annotations for that very reason.

For me, the line always got blurry depending on which module I was working with. For example, a CommonFramework module may have reusable domain objects for any application, and as such I fully agree that it makes no sense to expose any configuration files, nor would I have the desire to run integration tests there.

But if I am building a specific web application, say a PetClinic application, I might have a PetClinicFramework, a PetClinicAdminFramework, and a PetClinicWeb. From there I have been tempted to both put the configuration/hbm/integration tests directly in PetClinicFramework, since the web application is really the only feasible application that would use them.

However I see the argument that if a PetClinicGui were to be written, I may have a different configuration, and in that sense it would be useful to reuse the framework with a new set of config files. I suppose thats where the voice in my head might say, will you really reuse the framework modules that were written specifically for a web application?

But I intend to do the correct thing architecturally, so I will move the configuration files to the PetClinicWeb and run the integration tests there. I hope this thread was useful for anyone else experiencing similar design decisions.