Spring Python and CherryPy 3

[gregturn]

The rest of Russ's inlined comments, and mention of fluent API
==================

> class PetClinicClientAndServer(object):
>     def __init__(self):
>         self.connectionFactory()
>
>     def connectionFactory(self):
>         connFactory = factory.MySQLConnectionFactory()
>         connFactory.username = "springpython"
>         connFactory.password = "springpython"
>         connFactory.hostname = "localhost"
>         connFactory.db = "petclinic"
>         return connFactory
>

>
> Russ> Hmm, yeah but this breaks all sorts of concerns. What @Component was
> really trying to do was flag a particular class as being a candidate for
> being a Spring-managed object. I think we might have lost that point here.
> The point was not really to call that factory method, more to let Spring
> know "Hey, here's an object I want you to instantiate!"

I think that's one point I'm not a fan of. It does make sense when loading
the component structure from XML but when you describe your component in a
class, why not calling each components as you see fit explicitly from your
constructor rather than doing introspection there?

Russ> In Spring, a component's lifecycle is managed by Spring (this is so
that it can do DI, AOP, transactional support, LOTS of stuff basically). So this
is in fact a very real and purposeful deviation. In a Spring application, Spring
creates and wires together the objects involved in an application. Each component
is NOT responsible for discovering its dependencies, that concern is deliberately
separated out.

Russ> In fact, this makes me wonder if we need an @Component decorator at all...
One of the things that are hot in SpringSource right now is creating fluent apis
that drive the configuration (creating and wiring together the different components).
I have an idea that this approach will not only render some of the @Spring annotation
approach less attractive but could well be the way forward... and in fact makes
even more sense when you're using a dynamic language like Python. I'll start
another email thread on that, but just a note here saying that it might be that
the @Component concern here goes away if we start to consider driving the configuration
of the application using a python-friendly DSL.

I do understand and even see the point of annotating a method for
readability purpose, much like CherryPy developers are used to the @expose
decorator. However asking that decorator to be more than that isn't as
understandable.

>
> Russ> I also find that libraries themselves are intrusive as you have to
> understand and couple your code to their specific usage, which can make it
> hard to move from one library implementation to the next of exactly the
> same functionality. If you're lucky, this results in small code changes,
> if you're unlucky, an entire tier of your enterprise app needs to
> change... and this was the case often before Spring. Because of this
> challenge people tended to create their own vendor-neutral abstractions
> around libraries, which was fine but again code that was plumbing rather
> than code that made them money. Spring took that headache away by acting
> as an abstraction around those common pieces of functionality. This is
> where Spring has some qualities of a library as it provides a facade
> around common concerns, such as data access, that mean you can plug-in
> different implementations, often through configuration only, that don't
> affect the rest of your application. The core library/api remains the
> same, and if you need to get at more of the specifics of an implementation
> then you can. This is where the template pattern is used, such as in the
> database package. The idea with Spring is not to provide a data access
> technology, but to make it simple, through the use of the template
> pattern, to work with a number of different data access technologies in a
> consistent way and for those advanced cases the template provides access
> to the underlying specifics of a data access technology. The idea is "all
> the convenience when you can, all of the power when you need it".
>
> You can think of Spring as somewhere between an intrusive framework (bad),
> and a an inherently intrusive library (equally as bad) - it tries to hit
> that sweet-spot.
>
> Libraries that are well designed usually don't have that issue. They
> offer a functionality but let the developer decide how he/she wants to
> apply it.
>
> Russ> True, but hopefully my point above shows why even that is not really
> enough.

It does. That's why I was saying "well designed libraries". It's
perhaps
one of the hardest task for a developer. Probably the fact I'm new to
Spring makes me want to bend it so that it fits my expectations rather
than trying to adapt myself a bit so that I can see its power. Time will
tell.

Russ> Even well-designed libraries differ from each other and make it tricky
to move from one, say, data access library to another. This headache is aleviated
somewhat, for common interaction with data access for example, by providing a consistent
facade around whatever library is being used underneath. This is the job of the
database package.

> Russ> I would agree here, but this is why Spring is seen as being
> different from the crowd. Spring is, right now, THE enterprise framework,
> and it is specifically for enterprise applications that it is targeted.
> But I definitely want to talk around this a bit more. My take is Spring is
> half way between a framework and a library, and THAT is the reason it has
> been so hugely popular. Rather than forcing people to work a certain way,
> Spring offers facilities that implement best practices and get people
> being more productive, while you get to pick and choose which of those
> facilities you want to apply. For example, it is totally feasible to not
> use Spring for dependency injection at all... but if you do then there are
> a bunch more facilities you can make use of.

That's good to hear

> Russ> Gonna have to disagree here, this package is very important for the
> reasons I outlined previously in terms of spring offering common
> abstractions around different libraries. We're not providing another ORM
> solution here, this is one of those abstractions that makes it a simple
> choice for a developer to use any of a number of different underlying
> technology solutions in a consistent manner. In fact, I'd argue that there
> might be a case for an ORM abstract not unlike the HibernateTemplate and
> NHibernateTemplates in Spring Java and Spring.NET respectively. But this
> is not what this package is doing, it is providing convenience and library
> agnostic methods to interact with data.

I understand the point yet I don't believe it's the right approach in
Python Spring. I shouldn't have used the term of ORM as it muddies the
discussion. I invite you guys to have a look at this project and see
whether or not that could fit the bill:

http://www.aminus.net/geniusql

I'm mentioning it because it has a solid history in that regards.

Russ> It looks like a solid implementation of ORM, and I have no worries about
suggesting that we might want to offer integration with Spring Python. But one library
is not THE only solution, and that's something that Spring recognises and another
place it differs from other frameworks. Spring doesn't typically have an opinion
as to which technologies you use to implement a given concern, ORM in this case.
What Spring does offer is consistent integration with whatever technology you want
to use. So what we are really building with Spring Python is the ability for developers
to pick the library that they want to use for a given concern and then integrate
with it in a consistent way regardless of the complexities and specifics of the
underlying library. For example, if I wanted to use Geniusql for ORM, then Spring
would provide a factory to generate the required connections and provide a template
to interact with the ORM, using common convenience functions to make my code as
simple as possible. If I then wanted to use another ORM solution, I would grab the
factory for that technologies connections and create a template based on that technology.
My ORM savvy code would not need to change, but my config would change to plug-in
the new technology.

Thanks Russ for the feedback. It's cool that you pushed back like that

Russ> No problems. I'm certainly passionate about Spring and believe Spring
Python can be a really amazing addition to the portfolio (taking things even beyond
being an extension) and it's these sorts of discussions that will drive us in
that direction. Heated discussion is good, no discussion at all is bad is my motto.

- Sylvain

[gregturn]

Hi gents,

Some of the discussions today about @Component got me thinking about some of the
directions that the Spring framework is looking at (one of the big benefits of making
Spring Python an extension is that I get to share some of the insider info )

Ok, so the idea is that people are getting rapidly more tired with driving their
application's configuration using XML. The annotation approach is a halfway
house, but it introduces problems when the language doesn't support the annotation
placement you want to use, nor does it offer a very clean solution as annotations,
or decorators, themselves can be seen as introducing dependencies in your objects
that shouldn't be there.

One way to get around this was spearheaded by the Java Config folks, http://www.springframework.org/javaconfig.
This approach, while typesafe, is seen by a lot of people as a little cludgy, even
more magic going on and ultimately was something developed quickly in response to
the immediate need to offer a solution that wasn't XML. That might be a little
unfair, as some people love it, but it's not a direction I would want to take
us in as already there are movements internally to jump to a better approach...

Enter fluent apis. The classic competitive example of this in the Java world is
Guice, http://code.google.com/p/google-guice/. Guice offers a typesafe way of configuring
your application's components, coding directly in Java. It provides a limited
solution as it only supports DI, but it does offer an example of a fluent api that
neatly shows configuration driven by actual code (no magic involved, ok Sylvain?
)

So, would such a fluent api be a better approach for Spring Python, especially given
our knowledge that Spring generally is going in this direction (esp the Spring .NET
folks, they already have a Spring Extension called SharpConfig that will be creating
a dsl for configuring their container)?

Some of the benefits of providing this approach are:

1) Code completion in an editor. You can configure the Spring and know whether you've
called the right methods, rather than the disconnected approach that XML can sometimes
offer (Type safety is another benefit here, but that's not exactly a concern
with dynamic languages)

2) You configure in python. This might keep python people more happy as there is
less magic going on.

3) Leading off of 2, the config becomes easier to read particularly if it is a well-designed
fluent api. The concepts are likely to be the same regardless of whether you're
configuring in Java or Python, so it offers arguable an even easier transition from
one environment to another while keeping the spring pieces consistent.

4) Python as a language would make it really easy for us to create a great fluent
api, i.e. no need for semi colons to cludge up the expression of the language

5) We get a chance to really trail-blaze this. Since Python is such an ideal environment
for this approach, imo, we get to put this in place and essentially lead the other
Spring projects by example. This would be a big win for the project internally.

Ok, that's my two cents, any thoughts anyone?

Cheers,
Russ

[gregturn]

>
> Ok, that's my two cents, any thoughts anyone?
>

I think that's a good idea. Coming back to CherryPy it's something that
CP3 has brought with its tools API. Config value may be provided either in
a config file, a config dict or per page handler (via decorators or method
attributes) (read component in the Spring context) with a unified API.
(You can in fact used the different forms all the the same time and they
are computed at run time.

In any case it's propbably a good idea to escape from XML here. Developers
tend to dislike having to describe their architecture that way and the
only framework that I know has been doing it is Zope 3 and they haven't
really made the Python folks applause for that decision.

- Sylvain

[gregturn]

Hey guys,

Sylvain, I appreciate the feedback you have provided me. Russ, I also like having
the perspective of what Spring is targeting. I think we definitely have the ingredients
for finding this balance I think we all desire.

1. Regarding @component, this was inspired by reading http://blog.springsource.com/main/20...on-for-spring/,
which was out before Spring actually had a solution. I opened Spring Python ticket
https://springpython.webfactional.com/ticket/18, where you can see the issues I
ran into trying to implement something similar in python, and the frustrations I
had dealing with the difference between decorators and annotations. In python, decorators
are function calls, not meta-data like in java. Python's decorators can only
be applied to methods, not classes. So replicating @Configuration was out. I was
concerned that python decorators and java annotations were too distant that this
wasn't possible.

However, one day I had a revelation (and a little help using someone's decorator
library), and thanks to python's speedy nature, implemented a solution. Since
python doesn't have "beans", and since the IoC container was already
using the term "component", I swapped out Spring's @Bean annotation
for Spring Python's @component decorator (and dropped the capital letter). Part
of my revelation was realizing the key thing I was needing was PROTOTYPE vs. SINGLETON
spring beans. Translation, I wanted the option to either call a factory-style method
from the IoC POPO every time (PROTOTYPE), or only once and cache the results in
a container-specific dictionary (SINGLETON). Unfortunately, my sample applications
don't really demonstrate this feature, so I could understand any developer,
pythonista or not, not seeing what I was really shooting for.

Sylvain is correct in observing that @component by itself, i.e. with no arguments,
doesn't seem to do anything you couldn't already do in the __init__ method.
But that is because I looked up Spring's default configuration status when Spring
beans have no arguments. The default is pre-building every Spring bean when the
container is created. Lazy initialization and prototype are lifecycle options that
allow you to override this default. I made @component do the same behavior and pre-instantiate
every @component item in the container when the container is created. Currently
I have one optional argument (https://springpython.webfactional.co...icationContext),
which allows you to spec whether or not you get a cached copy or new copy every
time. I haven't dug into python enough to even determine if this is feasible
to have multiple optional arguments in a function decorator. I hope so, because
this opens the door in the future having the other options Spring has such as lazy-init,
auto-wire-candidate, etc.

2. Regarding the security filters, I was just learning the ropes of WSGI, and the
only reference I had from spring security was their API based on servlet filters.
I guess you could say I replicated the servlet filter APIs in python, and shoe-horned
them into WSGI in order to plug-in my security filters. Since I believe the purpose
of WSGI is to allow people to define middleware filters, so I very much would like
to line them up to work better in that way. One thing I was noticing, is that the
chain of middleware filters is different based on the URL pattern being matched,
and thus becomes dynamic. Every example I could find on google that showed me how
to wire a stack of WSGI middleware was static in configuration. I was trying to
configure singleton filters that weren't locked into a particular chain. Probably
needs to be refactored to be cleaner, but I think they need to EASILY be able to
be plugged into a WSGI server.

3. Regarding ORM and database templates, in my day job, I work with a data-centric
application that has almost 300 custom queries written. When I was first evaluating
spring (3 years ago?), JdbcTemplate, and HibernateTemplate, I tried to create mappings,
etc. to work with the ORMs. Those queries had things like uniions, distincts, group-bys,
and joins across linked databases. Either I didn't know how to use hibernate,
or hibernate was not built for that level of SQL query complexity. Either way, it
was easier for me to utilize JdbcTemplate and the ANSI SQL I already knew, rather
than try to convert the queries to HQL. When I picked up another application, and
had to write more queries, I continued using JdbcTemplate, and from time to time,
would send my source code to a DBA, allowing him to work on indexies based on my
queries. Since I can do left-outer-joins in my sleep now and that is what our DBAs
use, it is actually easier for me to stick with SQL rather than learn either HQL
or EJB-QL or some other ORM-specific query language. DatabaseTemplate is my shortcut
to getting database calls working fast in python. Actually, whether or not this
will work with Google App Engine is my #1 concern regarding running PetClinic there.

I understand that other developers aren't in the same situation I am in. Perhaps
they don't know as much SQL. Or their app is simple, or their entities were
designed by them, and not by DBAs. However, I have yet to see the value in abstracting
SQLAlchemy, SLQObject, or the other one Sylvain mentioned. Users can use those tools
right now in their python apps, and when necessary, plug-in a databaseTemplate call
should the need arise. I opened https://springpython.webfactional.com/ticket/15
to track this, but found little value in writing an abstraction layer. I recognize
that Spring probably gave a little bit of competitive heat on the Hibernate crew
to make their APIs more use friendly, and I think it worked (http://blog.springsource.com/main/20...r-jpatemplate/).
Since no one has indicated they need any ORM templates yet for Spring Python, it
has been moved to the bottom of the list.

4. XML support? I have a grand vision that one day, someone can port their java-based
spring application to python, point at the same XML file, and either edit nothing
(or do some simple find/replace), and have their python equivalent working. THAT
would be a quick sell to migrate over to python, ehh? What if Spring IDE was tweaked
to interface with that and python code? But, back to reality. I won't kill myself
to duplicate the XML format. I have a format that works good enough for now. In
the mean time I have made all the sample apps and wiki documentation point at the
DecorateBasedApplicationContext as the first example, since that is the mode I prefer.
POPOs over XML.

5. This is all great material to publish in the forum, since I want anything exposed
to Spring Python to understand my perspective, as well as any other potential contributer
or user.

Thanks again,
Greg