Spring Python and CherryPy 3

[gregturn]

I recently had an email/IM chat between myself, Sylvain and Russ. This was before we had the forum setup. I got permission from them to capture this discussion on the forum. I have since marked up the code blocks and quotes, so it is easier to read this interchange of ideas.
================================================== ========
Hello Sylvain,

Glad to meet you. I appreciate getting any type of assistance regarding CherryPy.

One of my first things I did when creating Spring Python was to make a web app demo
like Spring's petclinic, and I decided to use CherryPy (https://springpython.webfactional.com/wiki/PetClinic).
I think cherrypy is pretty neat. Since my HTML was not very complicated, I don't
use any templating libraries.

The whole demo app is located at https://springpython.webfactional.co...ples/petclinic.

petclinic.py is my cherrypy runner
model.py contains all my business object, i.e. python POJOs
controller.py contains the business logic, in this case database calls to populate
the model objects
view.py contains the HTML, and is basically a set of cherrypy exposed methods
noxml.py contains three IoC containers, one demonstrating client-and-server, the
other two demonstraing client-only and server-only versions of the application (show
off remoting features of Spring Python). Right now, I am working on the client-and-server
version.

The version of petclinic.py I had working with CherryPy 2 is https://springpython.webfactional.co...nic.py?rev=109.

Last night, I downloaded CherryPy 3.1.0 (since it advertised out-of-the-box Google
App Engine support), and proceeded to adjust petclinic.py to work with it. You can
see what changes I made at https://springpython.webfactional.com/changeset/122.

I was able to view the petclinic app in my web browser, but my middleware security
filter isn't working. I wrote a WSGI-compliant security filter. It is meant
to intercept calls to "/", run them through the security filtering, and
if things succeed, pass through to the actual app I have mounted at root. My filter
code is located at https://springpython.webfactional.co...ecurity/web.py.

I would appreciate any help on how to migrate my application properly. If you have
questions, please feel free to email me, or IM me through skype me at "gregturn".

FYI: Spring Python is my "pet" project, not my day job, so I can't
always correspond in real time, but I will advise you about that.

Thanks,
Greg Turnquist

[gregturn]

Good morning folks,

Right, I've been navigating the Spring source code (I mean the Python
port here) and I've had some comments to make from a pythonista
perspective.

If I understand correctly, there are two ways to describe a Spring
components: either via a XML document (which I assume comes from the
Java world) or from the code itself.

In the latter case, a method is flagged as a Spring component through
the usage of the @component decorator (Russ, just in case, look at
this link for an introduction to decorators:
http://www.siafoo.net/article/68). Python developers, like any
developers, don't like digging to understand why something works the
way it works. Frameworks like Spring (I mean the framework in general,
not just the Python port) aim at finding the right balance between
providing added value while refraining from becoming a black box that
sooner or later forbid/prevent the developer from doing things in a
certain way. In my opinion, the @component decorator does very little
to justify its existence. In other words it breaks the aforementioned
balance.

Basically it does this:

1/ Inspects a class instance for its methods
2/ For each method that is decorated with @component, call the method
3/ If the method also has the following attribute:
"postProcessAfterInitialization", then it calls it.

One immediate issue to me is that it doesn't lazily calls the methods
and one might wonder the point of all this when the instance could
have done it in its __init__(). It looks a bit like voodoo magic

Take this:

class PetClinicClientAndServer(DecoratorBasedApplicationContext):
    def __init__(self):
        DecoratorBasedApplicationContext.__init__(self)

    @component
    def connectionFactory(self):
        connFactory = factory.MySQLConnectionFactory()
        connFactory.username = "springpython"
        connFactory.password = "springpython"
        connFactory.hostname = "localhost"
        connFactory.db = "petclinic"
        return connFactory

Since you want to define the application's components explicitly in
the code rather than in an XML, why not going all the way like this:

class PetClinicClientAndServer(object):
    def __init__(self):
        self.connectionFactory()

    def connectionFactory(self):
        connFactory = factory.MySQLConnectionFactory()
        connFactory.username = "springpython"
        connFactory.password = "springpython"
        connFactory.hostname = "localhost"
        connFactory.db = "petclinic"
        return connFactory

That has two advantages:

1/ It's explicit and self-contained while preserving functionality.
You don't need to lookup the declaration of
DecoratorBasedApplicationContext nor @component.
2/ By removing the DecoratorBasedApplicationContext you allow any
regular Python class (POPO) removing, in my opinion, unnecessary
inheritance.

If your method has a post initialization, make a decorator of it:

# Defined in spring somewhere
def postInit(func):
    def inner(obj):
        func(obj)
        print "Do some post proc here: Yeah! Rock on!"

    return inner

class PetClinicClientAndServer(object):
    def __init__(self):
        self.connectionFactory()

    @postInit
    def connectionFactory(self):
        connFactory = factory.MySQLConnectionFactory()
        connFactory.username = "springpython"
        connFactory.password = "springpython"
        connFactory.hostname = "localhost"
        connFactory.db = "petclinic"
        return connFactory

Still easy to grasp, no introspection here.


Why am i saying all this? I'm a huge believer in libraries rather than
frameworks. Libraries are tools that truly add value while, when well
designed, hardly coming into the developer's way. Frameworks define a
canvas with boundaries to more or less hint the developer in a given
way. The problem is that frameworks, no matter how smart they are,
usually hit a point where they cannot provide the flexibility required
to let developers with different background be efficient with them.
That's why you often see "Framework X sucks big time" but if you dig
a
biot the statement it's because the developer was used to framework Y
or that framework X came into the way at some stage.

Libraries that are well designed usually don't have that issue. They
offer a functionality but let the developer decide how he/she wants to
apply it.

I think, and it's not a mockery, that the Java world is rather a
framework type of environment which leads usually to have as many
frameworks as you have developers for the very reason I described
above. Python has also its share of frameworks but the language is
flexible enough to encourage libraries. I *strongly* believe that
Python Spring has both functionalities that could be attractive to
many developers while having the chance of doing so as a library
rather than a framework.

In fact Greg has done a great job in that regards with many parts of
Python Spring (I notably like the spring.aop and aop.remoting
packages) and I think it's not far off being a library rather than a
framework for instance with changes like I highlighted previously.

Would you lose Java developers on the way? I don't believe so because
the functionality and promises are still there. Only the syntax is
somehow different.
Would you gain Python developers? I believe so. It's easier to attract
people with a library because it's less intrusive than a whole new
framework. Less coupling is better.

A few more comments.

Drop the database sub-package. Python has excellent ORM like
SQLAlchemy and dejavu. The former is really well spread and there
isn't any reason to ask developers to learn a new way of setting up
their stack. Keeping the database package only adds up more work for
little added value. Let people use their own choice of package.
SQLAlchemy's syntax is rather close to the one describe as well.

I'm rather feeling the same way towards the web package but still not
yet sure about that one.

Focus on what makes Python Spring different: aop, remoting, IoC, security.


So there you go, sorry for the long email but I thought I needed to
make those points clear from the start. This is no reflection on your
great work Greg. It's just a point of view

Hopefully I didn't sound too harsh either. Russ knows I sometimes use
the wrong approach but always for making things better if i can


- Sylvain

[gregturn]

One more thing. I thought about why the PetClinic was causing so much
problem with CherryPy 3 and I believe it's because the Spring filters
are not WSGI middlewares per nature.

They could be applied from within a WSGI middleware or a CherryPy 3
tool but themselves are not WSGI middleware.

They are just functions to be applied at some stage of the request
processing but are not part of the HTTP stack.

Greg, we'll need to discuss this

- Sylvain

[gregturn]

Hi Sylvain,

Thanks for taking a look through the code, wow you've been busy See some
of my comments inline (demarcated with Russ>), Greg I'm sure will have more
when he's up and around later on today:

----- Original Message -----
From: "Sylvain Hellegouarch" <sh@defuze.org>

Good morning folks,

Right, I've been navigating the Spring source code (I mean the Python
port here) and I've had some comments to make from a pythonista
perspective.

If I understand correctly, there are two ways to describe a Spring
components: either via a XML document (which I assume comes from the
Java world) or from the code itself.

Russ> Absolutely right, altho all things Spring come from the Java world initially
so it's fair to just say that the XML document approach is a Spring thing -
most Spring developers get a warm fuzzy feeling when they see an XML config document
because it gives them a common ground to start understanding an application before
they delve into the specifics of the implementation language.

In the latter case, a method is flagged as a Spring component through
the usage of the @component decorator (Russ, just in case, look at
this link for an introduction to decorators:
http://www.siafoo.net/article/68). Python developers, like any
developers, don't like digging to understand why something works the
way it works. Frameworks like Spring (I mean the framework in general,
not just the Python port) aim at finding the right balance between
providing added value while refraining from becoming a black box that
sooner or later forbid/prevent the developer from doing things in a
certain way. In my opinion, the @component decorator does very little
to justify its existence. In other words it breaks the aforementioned
balance.

Russ> Hmm, not sure I agree here. @Component is meant to distinguish, at the
class level, that a particular class results in one Spring-managed object. To this
effect, I really like the fact that you just have to see the @component syntax and
know that this class results in something managed by Spring. However, decorators
are not supported at the class level, and that does feel like a disconnect and something
that might need some more thought around...

Basically it does this:

1/ Inspects a class instance for its methods
2/ For each method that is decorated with @component, call the method
3/ If the method also has the following attribute:
"postProcessAfterInitialization", then it calls it.

Russ> This is very much inline with what Spring does, and although it's not
exactly voodoo magic I think the current implementation does not give the full picture.
As I mentioned before, @Component is usually a class level annotation, and when
used that way it becomes more obvious (as there is no special code required in the
constructor) that this is a POPO that will be managed by Spring (or another container
if it understands that annotation) (I'm using annotation here on purpose, as
I'm not sure and will rely on your guys to tell me how python does this, if
at all). So if @component was at the class level, and the init function didn't
have to have any special calls to respect @component, then I don't think I would
have any issues with @postProcessAfterInitialization, although I'd be tempted
to bring it inline with what Spring developers are used to and maybe call is @PostConstruct?

One immediate issue to me is that it doesn't lazily calls the methods
and one might wonder the point of all this when the instance could
have done it in its __init__(). It looks a bit like voodoo magic

Russ> Not sure I understand the meaning of "... lazily calls" Can you
explain that a bit more?

Russ> I think it might look like voodoo magic purely because this is a Spring-ism.
However, if we are going to support decorator-based configuration then I think this
is a great implementation of it, the only concern I have is that there is some intrusiveness
if the constructor has to make a specific call, and that @Component cannot be applied
to the class itself...

Take this:

class PetClinicClientAndServer(DecoratorBasedApplicationContext):
    def __init__(self):
        DecoratorBasedApplicationContext.__init__(self)

    @component
    def connectionFactory(self):
        connFactory = factory.MySQLConnectionFactory()
        connFactory.username = "springpython"
        connFactory.password = "springpython"
        connFactory.hostname = "localhost"
        connFactory.db = "petclinic"
        return connFactory

Since you want to define the application's components explicitly in
the code rather than in an XML, why not going all the way like this:

class PetClinicClientAndServer(object):
    def __init__(self):
        self.connectionFactory()

    def connectionFactory(self):
        connFactory = factory.MySQLConnectionFactory()
        connFactory.username = "springpython"
        connFactory.password = "springpython"
        connFactory.hostname = "localhost"
        connFactory.db = "petclinic"
        return connFactory

That has two advantages:

1/ It's explicit and self-contained while preserving functionality.
You don't need to lookup the declaration of
DecoratorBasedApplicationContext nor @component.
2/ By removing the DecoratorBasedApplicationContext you allow any
regular Python class (POPO) removing, in my opinion, unnecessary
inheritance.

Russ> Hmm, yeah but this breaks all sorts of concerns. What @Component was really
trying to do was flag a particular class as being a candidate for being a Spring-managed
object. I think we might have lost that point here. The point was not really to
call that factory method, more to let Spring know "Hey, here's an object
I want you to instantiate!"

If your method has a post initialization, make a decorator of it:

# Defined in spring somewhere
def postInit(func):
    def inner(obj):
        func(obj)
        print "Do some post proc here: Yeah! Rock on!"

    return inner

class PetClinicClientAndServer(object):
    def __init__(self):
        self.connectionFactory()

    @postInit
    def connectionFactory(self):
        connFactory = factory.MySQLConnectionFactory()
        connFactory.username = "springpython"
        connFactory.password = "springpython"
        connFactory.hostname = "localhost"
        connFactory.db = "petclinic"
        return connFactory

Still easy to grasp, no introspection here.

Russ> This falls very much into what I'd expect to see, but we really are
talking about two separate things:
1) Letting Spring know to construct an object from our class without having to
explicitly mention it in the XML - that's what @Component is all about.
2) Hooking into the lifecycle of that object by telling Spring to invoke particular
methods on the object as it is instantiated (and destroyed). Actually that brings
me to another point, if we are going to support @PostConstruct, we might want to
consider @PreDestroy.

Why am i saying all this? I'm a huge believer in libraries rather than
frameworks. Libraries are tools that truly add value while, when well
designed, hardly coming into the developer's way. Frameworks define a
canvas with boundaries to more or less hint the developer in a given
way. The problem is that frameworks, no matter how smart they are,
usually hit a point where they cannot provide the flexibility required
to let developers with different background be efficient with them.
That's why you often see "Framework X sucks big time" but if you dig
a
biot the statement it's because the developer was used to framework Y
or that framework X came into the way at some stage.

Russ> While I can see the argument, one of the qualities of Spring is that it
is a framework, albeit as light and unintrusive a framework as possible. Spring
is also a proven framework for building enterprise applications, and because it
is unintrusive and very general purpose, not to mention easily extendable, there
are very few development tasks that it would ever prevent.

=== To be continued ====

[gregturn]

== The rest of Russ's reply ==

Russ> I also find that libraries themselves are intrusive as you have to understand
and couple your code to their specific usage, which can make it hard to move from
one library implementation to the next of exactly the same functionality. If you're
lucky, this results in small code changes, if you're unlucky, an entire tier
of your enterprise app needs to change... and this was the case often before Spring.
Because of this challenge people tended to create their own vendor-neutral abstractions
around libraries, which was fine but again code that was plumbing rather than code
that made them money. Spring took that headache away by acting as an abstraction
around those common pieces of functionality. This is where Spring has some qualities
of a library as it provides a facade around common concerns, such as data access,
that mean you can plug-in different implementations, often through configuration
only, that don't affect the rest of your application. The core library/api remains
the same, and if you need to get at more of the specifics of an implementation then
you can. This is where the template pattern is used, such as in the database package.
The idea with Spring is not to provide a data access technology, but to make it
simple, through the use of the template pattern, to work with a number of different
data access technologies in a consistent way and for those advanced cases the template
provides access to the underlying specifics of a data access technology. The idea
is "all the convenience when you can, all of the power when you need it".

You can think of Spring as somewhere between an intrusive framework (bad), and a
an inherently intrusive library (equally as bad) - it tries to hit that sweet-spot.

Libraries that are well designed usually don't have that issue. They
offer a functionality but let the developer decide how he/she wants to
apply it.

Russ> True, but hopefully my point above shows why even that is not really enough.

I think, and it's not a mockery, that the Java world is rather a
framework type of environment which leads usually to have as many
frameworks as you have developers for the very reason I described
above. Python has also its share of frameworks but the language is
flexible enough to encourage libraries. I *strongly* believe that
Python Spring has both functionalities that could be attractive to
many developers while having the chance of doing so as a library
rather than a framework.

Russ> I would agree here, but this is why Spring is seen as being different from
the crowd. Spring is, right now, THE enterprise framework, and it is specifically
for enterprise applications that it is targeted. But I definitely want to talk around
this a bit more. My take is Spring is half way between a framework and a library,
and THAT is the reason it has been so hugely popular. Rather than forcing people
to work a certain way, Spring offers facilities that implement best practices and
get people being more productive, while you get to pick and choose which of those
facilities you want to apply. For example, it is totally feasible to not use Spring
for dependency injection at all... but if you do then there are a bunch more facilities
you can make use of.

In fact Greg has done a great job in that regards with many parts of
Python Spring (I notably like the spring.aop and aop.remoting
packages) and I think it's not far off being a library rather than a
framework for instance with changes like I highlighted previously.

Russ> I echo that, but I think Greg probably has the right balance and there
are a couple of areas, like the application of @Component, that might need some
tweaking.

Would you lose Java developers on the way? I don't believe so because
the functionality and promises are still there. Only the syntax is
somehow different.
Would you gain Python developers? I believe so. It's easier to attract
people with a library because it's less intrusive than a whole new
framework. Less coupling is better.

Russ> Like I think I've said a couple of times now; I agree that strong frameworks
are bad, but I also think that pure libraries fall short too, especially when it
comes to enterprise apps. What I think we have with Spring Python is something that
sits squarely in the middle, offering framework capabilities where appropriate and
library-like qualities if that is what the developer prefers to pick from. The really
great thing about this discussion is that it's raised an area of the current
implementation that may need tweaking to make it more in the middle.

A few more comments.

Drop the database sub-package. Python has excellent ORM like
SQLAlchemy and dejavu. The former is really well spread and there
isn't any reason to ask developers to learn a new way of setting up
their stack. Keeping the database package only adds up more work for
little added value. Let people use their own choice of package.
SQLAlchemy's syntax is rather close to the one describe as well.

Russ> Gonna have to disagree here, this package is very important for the reasons
I outlined previously in terms of spring offering common abstractions around different
libraries. We're not providing another ORM solution here, this is one of those
abstractions that makes it a simple choice for a developer to use any of a number
of different underlying technology solutions in a consistent manner. In fact, I'd
argue that there might be a case for an ORM abstract not unlike the HibernateTemplate
and NHibernateTemplates in Spring Java and Spring.NET respectively. But this is
not what this package is doing, it is providing convenience and library agnostic
methods to interact with data.

I'm rather feeling the same way towards the web package but still not
yet sure about that one.

Russ> Well I'm certainly open for discussion there, but maybe now I've
explained the goals of Spring as the hybrid framework/library it might cast a different
perspective on this?

Focus on what makes Python Spring different: aop, remoting, IoC, security.


So there you go, sorry for the long email but I thought I needed to
make those points clear from the start. This is no reflection on your
great work Greg. It's just a point of view

Hopefully I didn't sound too harsh either. Russ knows I sometimes use
the wrong approach but always for making things better if i can

Russ> Not harsh at all! All good questions and for me this is a great learning
curve from a python perspective, as well as probably a learning curve for everyone
on some of the underlying reasons that Spring is the way it is

- Sylvain

Russ> Thanks for the feedback Sylvain! This is all great stuff and will help
us drive forward Spring Python. I think you just raised some good questions that
most python people would have coming to the framework.

Of course all of this is just my thoughts and I'm no pythonista as you can probably
tell. Greg, you're the project lead so ultimately your vote is the biggie -
what are your thoughts?

Cheers,
Russ

[gregturn]

A little background here too. Spring Security uses a filter chain to apply security
concerns, as described here:

http://static.springframework.org/sp...e.html#filters

This is what Greg has implemented. Is there a WSGI standard way to provide these
sorts of filters Sylvain?

By the way, I'd love for these conversations to be pushed to the public forums
that the Spring Python project will have really soon now. Does anyone have any issues
with taking the summaries of these conversations and pushing them to the forums
for all to see once I get that service in place?

Cheers,
Russ

----- Original Message -----
From: "Sylvain Hellegouarch" <sh@defuze.org>
To: gregturn@mindspring.com
Cc: "russell miles" <russell.miles@springsource.com>
Sent: Friday, August 8, 2008 9:16:13 AM GMT +00:00 GMT Britain, Ireland, Portugal
Subject: Re: Hello

One more thing. I thought about why the PetClinic was causing so much
problem with CherryPy 3 and I believe it's because the Spring filters
are not WSGI middlewares per nature.

They could be applied from within a WSGI middleware or a CherryPy 3
tool but themselves are not WSGI middleware.

They are just functions to be applied at some stage of the request
processing but are not part of the HTTP stack.

Greg, we'll need to discuss this

- Sylvain