Performance

[Paliscope]

Hi (Thank you for all support yesterday)

I am benchmarking against RocksDB and I dont get the speed bump I am expecting and hoping for, maybe there is a setting I am missing?

My Stats;
Zonetree,
Insert 10M records. Key is string and value is a byte()
7,8 seconds

RocksDB. Key is string and value is a byte()
4,5 seconds

When injecting data with RocksDB I am using a writecache to get the speed. First inject in writecache and then write the writecache
If I am NOT using writecache and write directly, Zonetree is 4 times faster than RocksDB

In fact the storage on disk is a little smaller with Rocks as well

Iterating over all records is more or less the same for both.
Randomly looking up 1000.000 values takes about 2 second for both Zonetree and RocksDB so it is no major difference there.'

Is there any settings I can apply? I am using the code you gave me yesterday.

Best regards

[koculu]

1. The code you demonstrated uses atomic upserts, which add overhead. You don't need them unless you are implementing a thread-safe counter. Just use a simple upsert or insert.
2. You will get the performance boost when you insert in parallel with multiple threads.
3. I am not familiar with the special tuning you have done unless you share the code.
4. Performance comparison of databases is a very hard topic. There are several other use cases and tuning possibilities for both libraries. For example, for primitive types, ZoneTree is much more faster because RocksDB does not allow key-value types other than byte arrays. In ZoneTree you can directly insert primitive types without any GC pressure.
5. ZoneTree is a managed library, and it is a clear advantage over a native library like RocksDb. If the performance is competitive, it is sufficient from my point of view.

Read here for more information: #103

[Paliscope]

Hi!

You are right, I changed for Upsert (TryAtomic was a lot slower). I agree that it is nice it is a native language, I was just hoping for even more perfornance. I think it is very impressive that you match a C++ library with managed code.

With Rocks it is very simple just to try

Dim db As RocksDb = RocksDbSharp.RocksDb.Open(options, path)

Dim temp As New RocksDbSharp.WriteBatch
Dim data As Byte()
Dim obj As New EntityValueObj

for i as integer=0 to 10000000
obj.RowIndex = Rnd() * 1000
obj.Category = Rnd() * 1000
obj.count = Rnd() * 1000
obj.hastags = True
obj.AppStoreResults = Rnd() * 1000
data = MessagePackSerializer.Serialize(Of EntityValueObj)(obj)
key = System.Text.Encoding.UTF8.GetBytes("kalle" & i)
temp.Put(key, data)

       End Sub)

db.Write(temp)

[koculu]

You are using bulk write feature of RocksDB. This means you bypass the Write Ahead Log writes for individual write requests. Comparing bulk writes versus individual write requests is not fair. Because the first one has constraints and is used for special cases.

Thanks for your input regarding performance.
I am not willing to invest more on performance side of ZoneTree.
For single machine performance both RocksDb and ZoneTree are fine.
I would recommend choosing the one that suits your needs.

[Paliscope]

Thanks for your great answers and support!

I will look into how I can use ZoneTree in some scenarios where I dont have the opportunity to work in batches.

All the best!