JAVA: ByteBuffer.Allocate vs AllocateDirect

Allocation on heap, GC involved.

the process used 2G RAM


the buffer is one the HEAP Old space,

If allocatedirect on RAM,

still 2G as a whole

but not on Heap,

so The heavily used buffer appropriately continues to use a non-heap, non-garbage-collected direct buffer.

Cassandra: InvalidRequestException(why:Key may not be empty)

When you get this exception when you try to submit some changes through the THRIFT gateway to the cluster,

the code are simple,
  

client.remove(KEYSPACE, key,cp, System.currentTimeMillis(), ConsistencyLevel.ALL);

and when you check the key==null, it’s not null for sure, but how comes still get the key empty error?

then check ThriftValidation class,

now you know, the key is not null, the remaning() is zeor.
typically this means the key itself has been read and the pointer is at the end, so before you read the key, try duplicate a new one.

use this utilit lib instead, http://svn.apache.org/repos/asf/cassandra/trunk/src/java/org/apache/cassandra/utils/ByteBufferUtil.java

How-to: Test the HBase Coprocessor tutorial

here is a quick tutorial to setup hbase instance and hook up with a basic coprocessor.

Download the tar file and unzip it from http://mirror.tcpdiag.net/apache/hbase/hbase-0.94.11/
   Change the Hbase-site.xml, point the hbase.rootdir to the local folder, by default Hbase will load the default one embedded in the hbase-x.jar, and default setting is local temp folder,

change it to a folder you fell comfortable,
  also change the hbase.temp.dir to a well know folder, you can see all the underlying folder it used,
My final change,

then you can start the hbase server ./bin/start-hbase.sh

  check the port it listened, should be a port called 60030
 

then you can go the http://ip:60030 to see the hbase console of the given region. and 60010 for the hbase master

then create a table and put a record

check the folder

Now let’s write a basic java program to read the data in hbase.

Create a new maven project by clicking the eclipse wizard,

Click the pom.xml, right click to add one dependency hbase.

Simple Query in another server, make sure the zookeeper ip and host name are accessible from client machine,

for the coprocessor, they are bascailly observers in 3 levels.
  Master level, you can hookup with all the DDL like create table,update column,etc (check BaseMasterObserver)


and DML level in the region servers,

And WAL level, all changes going to WAL, basically you can see all the changes. you can even reject the change

 
I will put the WALobserver as a example, just dump out any changes to the cluseter, we can send the changeset to another server do secondary indexing, or for logging /auditing purpose.

compile and package it to a jar file,then copy to the hbase lib folder, or any folder which is in the HBASE_CLASSPATH

then change hbase-site.xml, point to our WALobserver

then restart the instance, from the region server, you can see our WAL is loaded,

then make some changes to the data,


from the log, you should see our logging , data changes captured here

Please note the WALobserver runs in the same JVM of the hbase, make sure not fail safe and no huge extra performance hit.

How to: Sql Server encryption

 

If you want to export the key to another server, backup it and restore it.

Encryption by Cert

How to: Generate the same hash value in Sql Server, C# and Java.

Since SQL Server 2005, there is one built-in function called hashbytes

please by aware if you define the string type as char and varchar , the hash varies.

so , here is one example

to be safe, using Rtrim on nchar type to get the same result vs the varchar one


C# and Sql, If you want to get the same hash for a given string, be sure to use the same encoding logic for the string, like Ascii vs Unicode.


in Java, try the commons-codec,

if you use nvarchar or nchar, make sure use unicde in c# also.


Sql server unicode

something in Java,switched it to unicdoe, follow the same byte orders

How to: turn on the logging for System.Net in C# 4.0

If you checked the code in HttpWebRequest by reflector, you might found it has a lot logging support code

sometimes, you do want to check the request behavior on sockets, http level, you can simply turn on the logging. here is a quick how,
change your config to somehow like this,

when you run the app with the httprequest  request , you will see the detailed logging

different trace listeners in a nutshell: