Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

How to boost performance #119

Copy link
Copy link
@pranasblk

Description

@pranasblk
Issue body actions

I'm testing Java driver with Agrona:0.9.1 and lmdbjava:0.6.3 and getting quite good lookup speed. Are there options to improve int => int map performance. My first attempt:

`

public class LMDBAgronaTest {
private static final String DB_NAME = "kv";
private static final long MAX_ENTRIES = 1_000_000;
private static final int ONE_K = 1_024;
private static final int MAX_ENTRY = 4 * ONE_K;

@rule
public final TemporaryFolder tmp = new TemporaryFolder();

// DB state
private Env env;
private Dbi db;
private MutableDirectBuffer key;
private MutableDirectBuffer val;

@before
public void setUp() throws IOException {
System.setProperty(Env.DISABLE_CHECKS_PROP, String.valueOf(true));
final File path = tmp.newFolder();
env = create(PROXY_DB)
.setMapSize(MAX_ENTRY * MAX_ENTRIES * 3)
.setMaxDbs(1)
.open(path, MDB_NOMETASYNC, MDB_NOSYNC);
db = env.openDbi(DB_NAME, MDB_CREATE);
key = new UnsafeBuffer(allocateDirect(4));
val = new UnsafeBuffer(allocateDirect(4));
}

@after
public void tearDown() {
out.println("Before close: " + new Date());
db.close();
env.close();
out.println("After close: " + new Date());
}

@test
public void nativeBuffersNoFlushOrderedKeys() {

// Write one by one
out.println("Begin insert: " + new Date());
for (int i = 0; i < MAX_ENTRIES; ++i) {
  val.putInt(0, i);
  key.putInt(0, i);
  db.put(key, val);
}

// Read one by one
out.println("Begin lookup: " + new Date());
final Txn<DirectBuffer> t = env.txnRead();
t.reset();
for (int i = 0; i < MAX_ENTRIES; ++i) {
  t.renew();
  key.putInt(0, i);
  final DirectBuffer result = db.get(t, key);
  assertNotNull(result);
  assertEquals(i, result.getInt(0));
  t.reset();
}

}
}

`

Begin insert: Fri Feb 15 12:26:40 AEDT 2019
Begin lookup: Fri Feb 15 12:27:05 AEDT 2019
Before close: Fri Feb 15 12:27:05 AEDT 2019
After close: Fri Feb 15 12:27:05 AEDT 2019

  • How do you estimate map size? Looks like 4k * MAX_ENTRIES is not sufficient ..
  • Is there best practice document for threading in java env.?
  • Has someone experience in serializing using Kryo and storing in LMLDB Java (e.g. type of buffer selection)?

I'm total newbie in LMDB and advices would be very welcome. I'm trying gradually extend value to store bloated java objects instead of currently int.

Thanks
Pranas

Reactions are currently unavailable

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions

      Morty Proxy This is a proxified and sanitized view of the page, visit original site.