Monday, November 24, 2008

Lucene 2.3.1 vs 2.4 benchmarks using LuSql

I have been doing some indexing performance tests with LuSql, and have some numbers comparing Lucene 2.3.1 with 2.4.

Despite some discussion about 2.4 having poorer indexing performance, my tests with LuSql 0.9 suggest otherwise:

Lucene 2.3.1

Number of records added= 2000000
Optimizing index
Closing index
Optimizing index time: 311 seconds
Closing JDBC: result set
Closing JDBC: statement
Closing JDBC: connection
*********** Elapsed time: 854 seconds
15m 18s

Lucene 2.4

Number of records added= 2000000
Optimizing index
Closing index
Optimizing index time: 322 seconds
Closing JDBC: result set
Closing JDBC: statement
Closing JDBC: connection
*********** Elapsed time: 759 seconds
12m 39s
Index size: 3.7GB.

It is interesting that the overall indexing time is significantly less, but the optimizing time is slightly higher.

Data, hardware and system configuration: as per my previous Lucene benchmarking.

Note that this is a simple benchmark, so YMWV. This benchmark was done with the LuSql default number of threads for the hardware in question, 20.
MySQL version used: v5.0.45 compiled from source, concurrency=8.

1 comment:

smithsan said...
This comment has been removed by a blog administrator.