Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

Write Performance #107

Copy link
Copy link
@scottazord

Description

@scottazord
Issue body actions

Hello all,

I fear that i'm not using the library correctly. Based on a benchmark review, i should be executing more writes / second.

I'm currently writing at 40,000 vals/sec.

I have tried bulking the data and using the Agrona DirectBuffers, however i see no performance increase.

Below is the code which i'm using for a performance test.

import org.lmdbjava.Cursor;
import org.lmdbjava.Dbi;
import org.lmdbjava.DbiFlags;
import org.lmdbjava.Env;
import org.lmdbjava.EnvFlags;
import org.lmdbjava.Txn;

import java.io.File;
import java.io.IOException;
import java.nio.ByteBuffer;

import static java.nio.charset.StandardCharsets.UTF_8;

public class EfficentTest {
    public static void main(String[] args) throws IOException
    {
        final File path = new File("/foo");

        if (!path.mkdirs() && !path.exists())
        {
            throw new IOException("Unable to create: " + path);
        }

        Env<ByteBuffer> env = Env.create()
                .setMapSize(1L << 31)
                .setMaxDbs(2)
                .open(path, EnvFlags.MDB_NOSYNC);

        final Dbi<ByteBuffer> names = env.openDbi("names", DbiFlags.MDB_CREATE);

        final ByteBuffer key = ByteBuffer.allocateDirect(4);
        final ByteBuffer val = ByteBuffer.allocateDirect(1024);

        final long t0 = System.currentTimeMillis();
        long tn = t0;
        for( int i = 0; i < 1000000 ; i++) {
            try (Txn<ByteBuffer> txn = env.txnWrite()) {
                final Cursor<ByteBuffer> c = names.openCursor(txn);
                key.putInt(0, i);
                val.put("Hello world".getBytes(UTF_8)).flip();
                c.put(key,val);
                txn.commit();
            }

            if (i % 1000 == 0)
            {
                long taken = System.currentTimeMillis() - tn;
                System.out.printf("Inserted: %d rows at %,2f vals/sec%n", i, (1000 * 1000D) / taken);
                tn = System.currentTimeMillis();
            }
        }
        final long t1 = System.currentTimeMillis();
        System.out.printf("Time to load db: %,dms%n", (t1 - t0));
    }
}

Any help would be much appreciated.

Reactions are currently unavailable

Metadata

Metadata

Assignees

No one assigned

    Labels

    cannot reproduceUsed for an issue we cannot currently reproduceUsed for an issue we cannot currently reproducequestionQueries on usage of the library, architectural patterns and alikeQueries on usage of the library, architectural patterns and alike

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions

      Morty Proxy This is a proxified and sanitized view of the page, visit original site.