Lucene Integration

[javidnia]

There is an interesting search engine (jakarta lucene) which I need to integrate with spring to put my search results into http session.

Where should I start? Could anyone help?

[Alef Arendsen]

Could you elaborate a bit here? Specifically as to what architecture your application will have. Are you looking to use Lucene in combination with a database or files? Also, where in the process do you see Lucene coming into action?

If you're storing things in a database I could see an interceptor processing the object before (or after) they are being saved and add to the the Lucene indices. We've done this a long time ago and it worked pretty well (although this was done without Spring).

[Marcos Silva Pereira]

If you're storing things in a database I could see an interceptor processing the object before (or after) they are being saved and add to the the Lucene indices. We've done this a long time ago and it worked pretty well (although this was done without Spring).

Hi, I was searching for Lucene in forum and found this topic. So, I have implemented interceptors to deal with indexes in OpenNuke project:
https://opennuke.dev.java.net/source...ke/aop/lucene/

This works using a indexer class an process objects with processors classes that deal with domain objects. Later, full text search is used in daos to execute a query using a IN SQL clausule like made in fullTextSearch method in a basic dao class.

I really want to listen opinions about this.

Best Regards.

[Alarmnummer]

I have checked your sourcecode and have a few comments:

1) you don`t update your documents in batches. I have created a Indexer interface, and a BatchedIndexer implementation. THe last one processed new documents in batches, so you can delete a batch of documents..and write a batch of documents.

2) you have a writer open and a reader for deletes open at the same time. Doesn`t this give problems?

 public void updateDocument(Document doc) throws IOException {

        IndexWriter writer = makeWriter();

        deleteDocument(doc);

        writer.addDocument(doc);

        writer.optimize();
        writer.close();

    }

Should be:

 public void updateDocument(Document doc) throws IOException {

        deleteDocument(doc);

        IndexWriter writer = makeWriter();      
        writer.addDocument(doc);
        writer.optimize();
        writer.close();

    }

And if you often add documents, don`t optimize all the time.


I`m currently writing a small and light framework on top of lucene to let me work with typed documents (it wraps a lucene document). And offers a lot of ready to use services.

@topicstarter:
But in principle.. Spring has no influence on how you are going to use Lucene.

[Marcos Silva Pereira]

Hi, Alarmnummer, I appreciate your comments. Thanks for feedback.

1) you don`t update your documents in batches. I have created a Indexer interface, and a BatchedIndexer implementation. THe last one processed new documents in batches, so you can delete a batch of documents..and write a batch of documents.

Some code was changed but not commited and I have implemented a Indexer interface to provides abstraction over Lucene like was made with Searcher interface. BatchedIndexer will be implemented like batches inserts and new code send to cvs.

2) you have a writer open and a reader for deletes open at the same time. Doesn`t this give problems?

 public void updateDocument(Document doc) throws IOException {

        IndexWriter writer = makeWriter();

        deleteDocument(doc);

        writer.addDocument(doc);

        writer.optimize();
        writer.close();

    }

Should be:

 public void updateDocument(Document doc) throws IOException {

        deleteDocument(doc);

        IndexWriter writer = makeWriter();      
        writer.addDocument(doc);
        writer.optimize();
        writer.close();

    }

Thanks, it is really a bug. I will fix it.

And if you often add documents, don`t optimize all the time.

Hum, maybe a boolean flag can tell to method if it should or not make optimize? What you think about?

I`m currently writing a small and light framework on top of lucene to let me work with typed documents (it wraps a lucene document). And offers a lot of ready to use services.

It's a open source project? Did you plan use annotations. I will refactoring some code to create processors based on annotations in domain classes. Is it a good idea?

Best Regards.

[Alarmnummer]

Hum, maybe a boolean flag can tell to method if it should or not make optimize? What you think about?

I have an optimize service that optimizes my lucene index once and awhile. (I use Quartz for the scheduling)

It's a open source project?

Maybe in the future, but not at the moment.

Did you plan use annotations.

annotations for what?

I will refactoring some code to create processors based on annotations in domain classes. Is it a good idea?

Could you elaborate?

[Marcos Silva Pereira]

Hum... use a scheduling job sounds like a good idea. I will think about it.

Well, I plan use annotations in my pojos to create a processor that read this annotations and decides how (keyword, store, unstore and field name) to index each attribute in pojo. Actually I am using simple reflection and deals with all attributes at the same way. Seems that you are thinking in index documents but, I am concerned about index objects. Despite this, what is your opinion about use annotations to decide how each attribute will be indexed?

Best Regards...

[Alarmnummer]

Hum... use a scheduling job sounds like a good idea. I will think about it.

If you use the same scheduler thread for writing the index, and optimizing it.. you won`t have any problems.

Well, I plan use annotations in my pojos to create a processor that read this annotations and decides how (keyword, store, unstore and field name) to index each attribute in pojo. Actually I am using simple reflection and deals with all attributes at the same way. Seems that you are thinking in index documents but, I am concerned about index objects. Despite this, what is your opinion about use annotations to decide how each attribute will be indexed?

The thought has crossed my mind... But I want a typesafe document-object because the document is my main 'domain' object. I need typesafe methods on that document and therefor I have created a new document type (the basedoc) that wraps a luncene document and translates my typesafe methods to unsafe lucene methods.

Best Regards...[/quote]