mongodb marshalling/unmarshalling

[malahov]

Hi. I'am use spring data for mongodb. I have more documents in one collection. Time of select above 20 000 (30 000 - 40 000) records = 7+ seconds. Indexes are built. This is a problem in marshalling?

[Oliver Gierke]

Do you have a test case that we could look at? You might wanna try reading the documents into plain DBObjects and compare the results.

[malahov]

I have done test case.

#1

        DBCollection collection = mongoOperations.getCollection(table);
        BasicDBObject query2 = new BasicDBObject("login", login);
        current = System.currentTimeMillis();
        DBCursor cursor = collection.find(query2);

        count = cursor.count();

        logger.info("######## Count: {}, Time: {} ########", count, (System.currentTimeMillis() - current));

#2

        current = System.currentTimeMillis();
        List<Action> result =  mongoOperations.find(query, Action.class, table);
        logger.info("Time : {} ", (System.currentTimeMillis() - current));

Results:

#1

######## Count: 30847, Time: 35 ########

#2

Time : 7448

[Oliver Gierke]

Well, that's not a fair comparison by any means. In #1 you basically haven't read any data at all. So you would have to iterate the cursor and actually create Action objects from the DBObjects returned. Especially the first step will actually cause the data being read.

Beyond that I'd argue that it is not much different to reading 30k table rows of a relational database into domain objects VS. simply reading them into a ResultSet. It highly depends on what you actually want to do with the data and whether it really makes sense to read all objects at once.

Long story short, it would be cool if you could report back what times you get if you actually read and marshal data manually.

[malahov]

Yes, you are right.

How to calculate count of documents in collection and do not load their in memory ?

Query query = new Query();
query.addCriteria(Criteria.where("login").is(login));
long count = mongoOperations.count(query, table);

May be this ?