we have one autonomy application which is used to synchronize the data between two databases and we try to load several thousand of objects into java heap(200000-300000)
and store in autonomy resultlist and we found It experiences bad performance . It takes about 8 hours for our autonomy application to find the object changes and if this number
of object changes get increase, the query gets slower and slower.
In the beginning we think It is GC issue, then I try https://github.com/OpenHFT/Chronicle-Map and store a lot of objects off heap , and after several times of testing , the autonomy
query is faster when objects are saved to chronicle map but the problem still exists when we load object from direct memory to java heap. then I try another library
Jhuge https://github.com/eobermuhlner/jhuge ,and It is the same thing. And for several days, we could not find what exactly caused the problem.
And with more testing with chronicle map and jhuge, I start to understand It is perhaps due to the autonomy resultlist performance ,so I decompile the autonomy api and found
It always check the document added to handle the duplicates and this degrades the performance, as if we put lots of objects in resultlist, each new added document will have
to compare with the existing collection objects(hashcode equal).
So when putting a lot of documents in autonomy ResultList,It is one problem. we can use java arraylist to store the ResultDocument and It can solve the problem.