spring-data-document: Support for mongodb group aggregation?

[ractive]

Hello everyone

Is there any support for group aggregation?
Is there maybe even support for mapreduce?


Best regards,
James

[trisberg]

Not yet. Do you have suggestions for how we should best support this?

[ractive]

Adding support for commands that use java script code will probably really be tricky, because JavaScript code comes into play here.

I made the experience with the group command that it is a real PITA to work with JavaScript snippets directly inside the java code. Building those JS functions with strings and passing it to the group command simply does not scale and makes the code almost not testable (not only in a unit-test sense) and very unreadable. What I ended up is creating *.js files in the resource folder, e.g. this file src/main/resource/mongodb/js/minMaxAvgReduce.js:

function (doc, out, varName) {
    var v = doc.values[varName];
    if(!isNaN(v)) {
        out.count++;
        out.total += v;
        out.min = Math.min(out.min, v);
        out.max = Math.max(out.max, v);
    }
} 

I then read this resource file in the bean init method with getClass().getResourceAsStream(fileName) and store this code as server-side function with a given name on the mongodb server with:

    private void saveJsFunction(String name, String js) {
        mongoTemplate.getCollection("system.js").save(new BasicDBObject("_id", name).append("value",
                new Code(js)));
    } 

I then can reference this server-side function (with the name REDUCE_FUNCTION_NAME here) in the reduce function that is actually passed to the group command:

    private static final String FINALIZE_FUNCTION = "function(out){out.avg = out.total / out.count}";

    Query query = new Query(where("fooId").is(123));
    String reduceFunction = String.format("function(doc, out) { %s(doc, out, '%s') }",
                REDUCE_FUNCTION_NAME, valueName);

    DBObject groupResult = coll.group(
            new BasicDBObject("fooId", true), // key
            query.getQueryObject(), // cond
            new BasicDBObject("count", 0).append("total", 0).append("min", Integer.MAX_VALUE)
                    .append("max", Integer.MIN_VALUE), // initial
            reduceFunction, // reduce
            FINALIZE_FUNCTION ); // finalize

    DBObject res = (DBObject) groupResult.get("0");
    Double min = (Double)res.get("min");
    .... 

I have no idea yet, if this approach is useful or not. But it allowed me to edit the javascript code in a separate file (with IDE support) instead of knocking it together with strings inside the java code. And I can reuse and test the uploaded server-side code directly as it is in the mongo shell without having to copy/paste this javascript code from the IDE into the shell.

Support I could imagine from Spring Data Document:

• helper functions to store code as server-side code/functions
• a simplified group function that
  • takes a pojo as the initial DBObject param
  • takes a pojo (or list or string or whatever) as the key param
  • a query as the condition

What experience have you made? Do you already have ideas how to support group and map/reduce functions in MongoDb?

[franktylerva]

I know this thread is a little old, but I figured I'd start it up again now that the new Aggregation Framework is just about ready. Are there any plans to add support for the Aggregation Framework?