For those of you looking at something like this, remember that JNI (no matter how you wrap it) is still incredibly slow when trying to pass larger data structures (>1KB).
On modern JVM we can use direct NIO buffers and let Java access native memory directly at full speed. There is (almost) no overhead at all anymore, so if we do it that way, it's pretty fast actually.