Neo4J Springdata: findAll()

[vincentvg]

Hello,

I'm using a neo4j db with springdata, and I want to find all nodes of a certain class type.
I have a Quiz class(@NodeEntity), and I need to find all the Quizzes in my database.

When I do template.findAll(Quiz.class), where template is a Neo4JTemplate, it takes more than 10 seconds to find 50 Quizzes (And it takes longer when the amount of quizzes increases).

Is there a way to run it faster?

I'm using neo4j 1.6, spring-data-neo4J 2.0.0.RELEASE

Thanks,

Vincent

[MichaelHunger]

Would it be possible for you to test this with a more recent version? Like 2.1.0.RC1 ? We're working on the release version of 2.1 so if that issue is still in there I'd like to have it fixed.

Thanks a lot

Michael

[vincentvg]

It still takes more than 10 seconds with the recent version (2.1.0.RC1)

Vincent

[MichaelHunger]

Do you execute this remotely to a server or embedded?

Can you share your db and code to see the cause for this? I've never seen that before.

Michael

[vincentvg]

We ran some tests and we found out that it's not the template.findAll() method after all.
This gives us a ClosableIterable in about 350ms. The delay occurs when we try to iterate over this iterable so we can add the items to a Set.

This problem occurs when we run it on the remote server, if we run it on the embedded server, there's no delay.


Here's a piece of code:

public Set<Quiz> findAllQuizzes() {

Set<Quiz> quizzes = new LinkedHashSet<Quiz>();
ClosableIterable<Quiz> findAll = template.findAll(Quiz.class);

for(Quiz q: findAll){
quizzes.add(q);
}

return quizzes;
}

[MichaelHunger]

Ok, what is your server latency for http requests to the server? 350ms for the initial request are already quite long.

And can you show me the spring setup code for the remote server?

It might be in your case that the time is too long for the default cache-time of rest entities.

Could you try to write a Unit Test that set's up the SpringRestGraphDatabase with the URL but also sets the property-cache time to a higher limit (default is one second).

restGraphdb.getRestAPI().setPropertyRefetchTimeInMillis(100*1000);

[vincentvg]

Here's the spring setup code for the remote server:

<bean id="graphDatabaseService"
		class="org.springframework.data.neo4j.rest.SpringRestGraphDatabase" >
		<constructor-arg index="0" value="http://127.0.0.1:7474/db/data" />
		
	</bean>

I tried the code in the test case, but it still takes 10 sec.


Vincent

[MichaelHunger]

Please share the test case, that was what I wanted to ask for

The test case should include the data generation so that I can run it on a blank SDN server.

And please answer this:

What is your server latency for http requests to the server? 350ms for the initial request are already quite long.

[vincentvg]

I 've set up a test project. Just run the testcase of that project. The first time it'll take longer because it will fill the database with some data.
The second time you run it will give the time it taked to load the quizzes and their relations.

Here's the link to the project:
https://github.com/vincentvg/testSpeedQuizzes

Do you mean the server latency from the database server? How can I measure/know it?


Thanks,

Vincent

[MichaelHunger]

With ping for instance on the shell/commandline.

ping hostname

and a simple curl http://server:port/db/data/ and see how much time it takes. Should be around 0-1 ms.