I'm beginning to experiment with this project—and am likely doing plenty of things wrong—but encountered a cryptic malloc error and program crash after using this library successfully for several experiments. I'm posting what information I have in case this experience represents a stability bug in the code. If it only reflects me using the library incorrectly, my apologies along with appreciation for any tips on better usage.
While ingesting data into an lmdbjava system, it worked successfully for the first 340,000 items ingested, then hard crashed with the following error:
java(7179,0x70001ee7e000) malloc: *** error for object 0x70001ee7cf70: pointer being freed was not allocated
java(7179,0x70001e275000) malloc: *** error for object 0x70001ee7cf70: pointer being freed was not allocated
java(7179,0x70001ee7e000) malloc: *** set a breakpoint in malloc_error_break to debug
java(7179,0x70001e275000) malloc: *** set a breakpoint in malloc_error_break to debug
There was no other error message; no stack trace, etc. Before the error, everything seemed to be working fine. Running this system with a different key/value store (instead of lmdbjava) has worked successfully for a very long time.
The program terminated immediately and ungracefully after printing this message. Some of the data on disk was created during an earlier run of my application which ingested some data (~300,000 key/value pairs) before being shut down normally. Then I restarted and ran the same ingest. My relevant code (in Scala) is below:
val env = org.lmdbjava.Env
.create()
.setMapSize(3000485760L)
.setMaxDbs(32)
.setMaxReaders(1024)
.open(new File(path), MDB_WRITEMAP, MDB_NOSYNC, MDB_NOLOCK)
val db = env.openDbi("foo", org.lmdbjava.DbiFlags.MDB_CREATE)
def persistSnapshot(id: MyID, atTime: Milliseconds, state: Array[Byte]): Future[Unit] = {
val idBb = ByteBuffer.allocateDirect(id.array.length)
idBb.put(id.array).flip()
val stateBb = ByteBuffer.allocateDirect(state.length)
stateBb.put(state).flip()
Future(db.put(idBb, stateBb))
}
def getLatestSnapshot(id: MyID, upToTime: Option[Milliseconds]): Future[Option[(Milliseconds, Array[Byte])]] = Future{
val tx = env.txnRead()
val idBb: ByteBuffer = ByteBuffer.allocateDirect(id.array.length)
idBb.put(id.array).flip()
val result = Option(db.get(tx, idBb)).map(r => 0L -> r.array())
tx.close()
result
}
I'm beginning to experiment with this project—and am likely doing plenty of things wrong—but encountered a cryptic
mallocerror and program crash after using this library successfully for several experiments. I'm posting what information I have in case this experience represents a stability bug in the code. If it only reflects me using the library incorrectly, my apologies along with appreciation for any tips on better usage.While ingesting data into an lmdbjava system, it worked successfully for the first 340,000 items ingested, then hard crashed with the following error:
There was no other error message; no stack trace, etc. Before the error, everything seemed to be working fine. Running this system with a different key/value store (instead of lmdbjava) has worked successfully for a very long time.
The program terminated immediately and ungracefully after printing this message. Some of the data on disk was created during an earlier run of my application which ingested some data (~300,000 key/value pairs) before being shut down normally. Then I restarted and ran the same ingest. My relevant code (in Scala) is below: