My Note on Solutions.

Crystal report, A generic error occurred in GDI+

2012-02-29T15:01:00.001-08:00

one day, some app server has the error to render the asp.net Crystal report. It’s a server side problem, So run a debug and attach to the server side w3wp. turn on unhandled exception.

mdbg> ca ex
mdbg> a 2876
[p#:0, t#:75] mdbg> g
STOP: Exception thrown
Exception=System.Runtime.InteropServices.ExternalException
        _className=<null>
        _exceptionMethod=<null>
        _exceptionMethodString=<null>
        _message="A generic error occurred in GDI+."
        _data=<null>
        _innerException=<null>
        _helpURL=<null>
        _stackTrace=array [20]
        _watsonBuckets=<null>
        _stackTraceString=<null>
        _remoteStackTraceString=<null>
        _remoteStackIndex=0
        _dynamicMethods=<null>
        _HResult=-2147467259
        _source=<null>
        _xptrs=0
        _xcode=-532462766
        _ipForWatsonBuckets=2066954414
        _safeSerializationManager=System.Runtime.Serialization.SafeSerializatio
Manager
IP: 210 @ System.Drawing.Image.Save - MAPPING_APPROXIMATE
[p#:0, t#:75] mdbg> w
Thread [#:75]
*0. System.Drawing.Image.Save (source line information unavailable)
1. System.Drawing.Image.Save (source line information unavailable)
2. CrystalDecisions.Web.HtmlReportRender.ImageObjectRender.SaveImage (source l
ne information unavailable)

then keep going, you will find the foder there, it’s the cr engine which trys to save some file locally.

then Just check this temp folder to see whether the folder is accessible? whether the app has the access. for this case,
NObody can access this folder now. so that’s the root case.

Cordbg for .net 4.0

2012-02-27T11:40:00.001-08:00

I have been using the Cordbg for a while, it works pretty great. just zip those 2 files and drop it to the server, you can do the production debugging.

now, some app has been switched to 4.0 , I need to get the 4.0 cordbg, googled a lot, can’t find it. and it turns on cordbg for .net 4.0 is out, the alternative is MDBG.

it’s located in C:\Program Files (x86)\Microsoft SDKs\Windows\v7.0A\Bin\NETFX 4.0 Tools\x64
just 3 files beside the .net fx are required to run.

you can do the same as cordbg, the syntax is most same. Mdbg has more enhancement.
Mdbg has more conditional breakpoint settings,

More support for threading and concurrent app troubleshooting.

So no cordbg anymore, try mdbg instead.

Install JD GUI eclipse add-in for Eclipse 3.7 timeout

2012-02-24T11:33:00.001-08:00

Downloaded new eclipse 3.7 and try to install the new plugin JD-GUI,

get the following screen, and it get sucked there,

it keep trying different URl to download Content.jar,
the fix is cancel the installation, and uncheck the evil option “Contact all updates sites during install to find required software”

then the installation will go through quickly. that’s it.

How to: test C# Redis client sider , step by step

2012-02-22T15:27:00.001-08:00

After install and test Redis on centos 5, now let’s write a Simple C# application to do some basis data operation.
make sure Nuget is there, and right click your preject to add Sider to the reference.

,with 2 lines of Code,

you can also see the value from CLI

it also support Unicode,

you can pickup the encoding for the unicode in the RedisSetting option.

More syntax and API , check it out here, https://github.com/chakrit/sider

How to: install and test Redis on centos 5

2012-02-22T15:19:00.001-08:00

Redis is a lightweight key-value data store. in terms of the key-value support, it offers several popular data collection types. let’s test it quickly. or check it here for the real tutorial ,

Download the source code,

wget http://redis.googlecode.com/files/redis-2.4.8.tar.gz
tar xvf redis-2.4.8.tar.gz
cd redis-2.4.8
make

If you don’t have the C compiler, try yum install gcc to install the compiler. this may take a while, C has a lot libs/headers to download and update.

BY default, Redis pick up the port 6379, you can find it in the redis.conf setting,

then start the service ‘src/redis-server redis.conf’ which will run in forground by default.

you can change the settings in redis.conf by change demonize:yes, it will run in background.
you can tell the port 6379 is picked up the redis,

Now we can run the CLI client of redis, src/redis-cli

Here is the helloword test, set one key with one value as string,

We also can define a TTL for this key by put a expire keyword, it will disappear in 120 seconds

then some other collection data types, like Hash

Check whether a key exists

get keys and values,

then we can use a C# app to do the same thing,

PhoneAccentBrush in Code

2012-02-21T15:30:00.001-08:00

obj.BorderBrush = (Brush) App.Current.Resources["PhoneAccentBrush"];
Simple it is.

some controls of the phontext has the same way to pick up static resources

How to: Notepad Plus, turn off the setting of last opened files

2012-02-20T10:55:00.001-08:00

Notepad plus is really a plus tool for developers like me, sometimes, I don’t want the tool list all the last opened files. By default , it will remember the last opened files and leave it there next time you open it. like a append only file , keep increasing , never get shrinked

To turn off this setting, there is no obvious setting like “disable last opened files”, but there is one option pretty unobvious.

Under [Setting] –> [Preferences..]->[MISC]

Also, there is another option to keep the editor only open last N files, like a fixed FIFO queue.

Windows Phone 7, File System browser, Isolated storage

2012-02-06T17:24:00.001-08:00

If you want to do some cache validation, or viewing the sandbox file system of your app, here is one very great tool
http://wptools.codeplex.com/

you can use this tool to view the Filesystem on the emulator or real device. for real device, it has to connect through Zune.

And you can push or pull fill to and from you PC and Phone. like the Android DDM tools.

Also, you can install xap to the phone/emulator.

kill app, like the ps shell

A super great SSH shell on windows 7, far more than Putty. Free MobaXterm

2012-02-02T10:25:00.001-08:00

Just check this link, you will find how amazing this tool is.

http://mobaxterm.mobatek.net/

I have the features like this I used most on my windows.
just drag and drop this utility to the windows, you can do the SSH, FTP, VNC RDP to any linux server, it has the multi window support.

view the remote file system, drag and drop.
Plus it can run bunch of command on windows. like the sed,grep. Super!

HP QTP, Click LInk in specified cells in the data grid of siliverlight, SlvTable

2012-01-31T16:28:00.001-08:00

SlvTable has the click method, by the x, y has to be the position relative to the parent control (table or the grid here.)

so , get the top location of the table itself, then run some basic math cacaluation, you can click any cells with links,

Here is one snippet to click [3,3] links,

myx=browser("HomePage).Page("Module1").SlvWindow("RootVisualWrapper").SlvTable("Grid1").GetCellProperty(3,3,"x")
myy=browser("HomePage).Page("Module1").SlvWindow("RootVisualWrapper").SlvTable("Grid1").GetCellProperty(3,3,"y")

parentx=browser("HomePage).Page("Module1").SlvWindow("RootVisualWrapper").SlvTable("Grid1").GetROProperty("x")
parenty=browser("HomePage).Page("Module1").SlvWindow("RootVisualWrapper").SlvTable("Grid1").GetROProperty("y")

w=browser("HomePage).Page("Module1").SlvWindow("RootVisualWrapper").SlvTable("Grid1").GetCellProperty(3,3,"width")
h=browser("HomePage).Page("Module1").SlvWindow("RootVisualWrapper").SlvTable("Grid1").GetCellProperty(3,3,"height")

rx=myx + w/2-parentx
ry=myy + h/2-parenty

browser("HomePage).Page("Module1").SlvWindow("RootVisualWrapper").SlvTable("Grid1").Click rx, ry

Oracle Coherence Monitoring, Node max memory keep changing.

2012-01-20T15:47:00.001-08:00

If you turn on the coherence monitoring by setting the jvm property, -Dtangosol.coherence.management.remote=true, you can check the Coherence status via the JMX client like Jconsole. I.e you can click the Node of Coherence domain, and see all the Nodes information, like MaxMemory, FreeMemory.

always we set up the heap size constant. min and max to the same value, I set it up to 300M here.
But when I check the node attributes of max and free memory, the max one is not 300M, and sometimes, even keep changing.

then I check the code, to return the MemoryMaxMB, pretty simple, just call runtime to get the maxmemory.

I created the following bTrace script, and make sure it is calling this code. And I am right.

@OnMethod(clazz = "com.tangosol.coherence.component.net.management.model.localModel.ClusterNodeModel", method = "getMemoryMaxMB", location = @Location(value = Kind.RETURN))
public static void getMemoryMaxMB(@Return int i) {
BTraceUtils.println("get max");
BTraceUtils.println(i);

}

then the question is why the maxmemory here is 287 or some value instead of 300M ?
I did a another test, change the JVM to Jrockit, what do you expect? it shows 300M.

conclusion, the MemoryMaxMB reported here vary by JVM. because each JVM vendor has it’s own implementation of the JVM.

Cassandra OpsCenter Community features

2011-12-14T11:45:00.001-08:00

played with the Ops several hours, here are the major feature In my option.

Monitoring the whole cluster status, like req/s, disk capacity, IO

Node tool visualization and do maintenance job just a click away. Such as, drain one node, compact, or get one node out of cluster(decommission.), rebalance cluster is only available in enterprise version sadly.

Clicking the Node, you can do a lot stuff, such as view the multi-DC replications.

Data modeling and explor( create keyspace , and view data), update column family.

Exploring data,

Monitoring again,

Actions on node, (like the node tool)

Cassandra Opscenter, connected to the cluster, But No Node information get listed

2011-12-14T10:20:00.001-08:00

configration is pretty simple for opscenter, just change files under conf fodler, point to correct seed server, then I can query the data from Opscenter now.

How ever, the dashboard, there is no node information.

When I check the logs under opscenter , when it try to get node infrmation from the thift API, get error, that explains why, return no results.

After did a lot testing, I noticed this is because of the version mismatch, for Cassandra cluster, it runs on 1.0 version.

while the ops center version is 1.3.1, you can tell from the console on the left bottom

then I try update the Cassandra to the latest version 1.0.5, it works without any problem.

Cassandra OpsCenter,Failed to load application: libpython2.6.so.1.0

2011-12-14T09:44:00.001-08:00

Unzip the OpsCenter file, Instal the 2.7 Python, then run the opscenter, failed get the following error.

[root@e3 opscenter]# bin/opscenter
/usr/local/bin/python2.7
Traceback (most recent call last):
File "/usr/lib/cassandra/opscenter-1.3.1/lib/py-redhat/2.6/shared/amd64/twisted/application/app.py", line 631, in run
runApp(config)
File "/usr/lib/cassandra/opscenter-1.3.1/lib/py-redhat/2.6/shared/amd64/twisted/scripts/twistd.py", line 23, in runApp
_SomeApplicationRunner(config).run()
File "/usr/lib/cassandra/opscenter-1.3.1/lib/py-redhat/2.6/shared/amd64/twisted/application/app.py", line 374, in run
self.application = self.createOrGetApplication()
File "/usr/lib/cassandra/opscenter-1.3.1/lib/py-redhat/2.6/shared/amd64/twisted/application/app.py", line 439, in createOrGetApplication
application = getApplication(self.config, passphrase)
--- <exception caught here> ---
File "/usr/lib/cassandra/opscenter-1.3.1/lib/py-redhat/2.6/shared/amd64/twisted/application/app.py", line 450, in getApplication
application = service.loadApplication(filename, style, passphrase)
File "/usr/lib/cassandra/opscenter-1.3.1/lib/py-redhat/2.6/shared/amd64/twisted/application/service.py", line 400, in loadApplication
application = sob.loadValueFromFile(filename, 'application', passphrase)
File "/usr/lib/cassandra/opscenter-1.3.1/lib/py-redhat/2.6/shared/amd64/twisted/persisted/sob.py", line 210, in loadValueFromFile
exec fileObj in d, d
File "bin/start_opscenter.py", line 1, in <module>
from opscenterd import opscenterd_tap
File "build/lib/python2.7/site-packages/opscenterd/opscenterd_tap.py", line 16, in <module>

File "build/lib/python2.7/site-packages/opscenterd/Config.py", line 392, in init_config

File "build/lib/python2.7/site-packages/opscenterd/events/plugins/CassandraStore.py", line 12, in <module>

File "build/lib/python2.7/site-packages/opscenterd/CassandraService.py", line 17, in <module>

File "build/lib/python2.7/site-packages/opscenterd/Cluster.py", line 14, in <module>

File "build/lib/python2.7/site-packages/opscenterd/AgentServer.py", line 23, in <module>

File "build/lib/python2.7/site-packages/opscenterd/HttpUtils.py", line 10, in <module>

File "/usr/lib/cassandra/opscenter-1.3.1/lib/py-redhat/2.6/5/amd64/OpenSSL/__init__.py", line 11, in <module>
import rand, crypto, SSL, tsafe
exceptions.ImportError: libpython2.6.so.1.0: cannot open shared object file: No such file or directory

Failed to load application: libpython2.6.so.1.0: cannot open shared object file: No such file or directory

When I list the LibPython module under /usr/lib64/libpython* , It only has the 2.4 modue. So we need to install the 2.6 Module required by the opscenter.

1, first, Check my OS, it ‘s Centos 5

go to http://download.fedora.redhat.com/pub/epel/5/x86_64/, download the epel-release-5-4.noarch.rpm

2. Update the rpm, “rpm -Uvh epel-release*rpm”

3. yum install python26-libs

Once done, you can see the 2.6 modules under the lib folder,

Now we can run the opscenter now.

How to: write a key mapper or Key transformation utility in 10 minutes

2011-12-06T11:31:00.001-08:00

Question, need a utility to map or transform the key stoke, say when you press F6, replace the contents with a Mailing address, and Press F7 for the zip code of your city.

Answer, In C#, it’s easy and not easy. you have to wrap several Native APIs to hook the mapping point into the system, Easy one is it has one Great methods called Sendkeys.Send.

So eventually, we need one file to keep the mapping logic , here I just pickup the .config file, then the code. Code is here,you may just copy and save it as .cs file, then compile it.

using System;
using System.Diagnostics;
using System.Windows.Forms;
using System.Runtime.InteropServices;
using System.Text;
using System.Collections.Specialized;

public class KeyMapper
{
private const int WH_KEYBOARD_LL = 13;
private const int WM_KEYDOWN = 0x0100;

    private static LowLevelKeyboardProc _proc = HookCallback;
    private static IntPtr _hookID = IntPtr.Zero;
    private static NameValueCollection mappings;
    public static void Main()
    {
        //Check to see no duplicate running
        if(Process.GetProcessesByName("af").Length>1)
        {
            return;
        }
        //keep it small
        Console.SetWindowSize(1,2);

mappings = System.Configuration.ConfigurationManager.AppSettings;

        _hookID = SetHook(_proc);
        Application.Run();
        UnhookWindowsHookEx(_hookID);
    }

    private static IntPtr SetHook(LowLevelKeyboardProc proc)
    {
        using (Process curProcess = Process.GetCurrentProcess())
        using (ProcessModule curModule = curProcess.MainModule)
        {
            return SetWindowsHookEx(WH_KEYBOARD_LL, proc,
                GetModuleHandle(curModule.ModuleName), 0);
        }
    }

private delegate IntPtr LowLevelKeyboardProc(
int nCode, IntPtr wParam, IntPtr lParam);

    private static IntPtr HookCallback(
        int nCode, IntPtr wParam, IntPtr lParam)
    {
        if (nCode >= 0 && wParam == (IntPtr)WM_KEYDOWN)
        {
            int vkCode = Marshal.ReadInt32(lParam);
            String key = ((Keys)vkCode).ToString();

            if (mappings[key] != null)
            SendKeys.Send(mappings[key]);

        }
        return CallNextHookEx(_hookID, nCode, wParam, lParam);
    }

    [DllImport("user32.dll", CharSet = CharSet.Auto, SetLastError = true)]
    private static extern IntPtr SetWindowsHookEx(int idHook,
        LowLevelKeyboardProc lpfn, IntPtr hMod, uint dwThreadId);

    [DllImport("user32.dll", CharSet = CharSet.Auto, SetLastError = true)]
    [return: MarshalAs(UnmanagedType.Bool)]
    private static extern bool UnhookWindowsHookEx(IntPtr hhk);

    [DllImport("user32.dll", CharSet = CharSet.Auto, SetLastError = true)]
    private static extern IntPtr CallNextHookEx(IntPtr hhk, int nCode,
        IntPtr wParam, IntPtr lParam);

[DllImport("kernel32.dll", CharSet = CharSet.Auto, SetLastError = true)]
private static extern IntPtr GetModuleHandle(string lpModuleName);
}

Config file, like mine,

Then we you started the console application.
wherever you enter F6, it will put a sample address there.

If you can’t compile the code , leave a comment , I will email you the compiled bits.

Error: Vmplayer, NO Disk, There is no disk in the drive. Please insert a disk into drive \Device\Harddisk2\DR4.

2011-11-30T11:48:00.001-08:00

One Day I get this error, when I just open the Vmware player, Kept popping up this error when I click the VMs.

Then I run a diskpart from the cmd shell, then call “list disk” it will show all the hard disks and their sizes

the disk2 is a weird Disk, Size is 0B.

then checked all my hard disk, I plugged my phone into the USB port. unplug the cable, the disk 2 is gone.

the error of NO disk was fixed too.

How to: Install and Test apache mahout on hadoop

2011-11-16T10:52:00.001-08:00

Mahout and Hadoop are all java libraries basically, mahout use the Maven tool to build the source code and maintain the dependency.
So we need make sure we have the following bits ready.

JDK
Maven
Hadoop
Mahout

I will start from the fresh centos, then get all those stuff ready step by step.

install JDK.
GO to Oracle JDK download site, http://www.oracle.com/technetwork/java/javase/downloads/index.html, I still prefer the Java 6 instead of 7, pickup one the .bin link, download and run it directly. I will put the java under /usr/lib/jdk6 folder.

Export the bin directory to PATH, and jdk6 to JAVA_HOME environment variable.

Install MAVEN
Download the binary package from http://maven.apache.org/download.html, here I chose the 2.2.1 version which is more stable.

Extract the zip file, and link it to /usr/lib/maven, then.
Export /usr/lib/maven to the PATH. now, you can run mvn –version, to make sure it works at least we can get the version,

For settings like proxy, check it out here, http://maven.apache.org/download.html#Maven_Documentation

Install HADOOP.
you can check this out, if you want to install hadoop as a fully distributd mode, How to: install and config hadoop to run in fully distributed Mode, Centos.

here we just have one vm, so keep it easy for the mahout testing. I will use the Cloudera distribution,

Download the repo file for centos 5, http://archive.cloudera.com/redhat/cdh/cloudera-cdh3.repo and copy it to yum repo directory.
Now just search hadoop, you will see all the components,we will use the hadoop-0.20-conf-pseudo one.
yum install hadoop-0.20-conf-pseudo

once done, go to /usr/lib/hadoop/conf directory, change the java home to /usr/lib/jdk6 in hadoop-env.sh

Then run as hdfs, format the namenode,

then start those daemons like /etc/init.d/hadoop-*, run JPS, you should see all the java process there,

now we can run a simple test, go the /usr/lib/hadoop, run

we can just copy one file there,

open a browser, go to http://localhost:50070, you can see the file we just uploaded is there,

Now , HDFS is ready. we can run a mapreduce job to make sure hadoop is ready.

If no error, we are all set, hadoop is ready.

Install Mahout.

Download one source code, you can use svn to clone one trunk copy,

svn co http://svn.apache.org/repos/asf/mahout/trunk

and copy this code to /usr/lib/mahout

then run mvn install –DskipTests to compile the source, mvn will figure out the dependency and fetch those jars for you automatically,

it takes time to download all the jar, mileage depends.

Here is my MPG, take several minutes,

Now, export /usr/lib/mahout/bin to PATH , then we can run mahout from the shell.

If you cant exectute the mahout, give it one execute permission.

run mahout, will list all the options to go with different algorithms.

Then go the examples folder, run mvn compile

Now, you can run some example like the one to classify the news groups.

Here we didn’t sepecify the HADOOP_HOME, so it will run locally. the shell script will download data, prepare the data, then run the classifier.

when done, it will show the confusion matrix against the testing data.

How to: recover the Root password for Centos

2011-11-14T14:20:00.001-08:00

You may forget the Root password which is hard to believe, but it does happen everyday. here is the tips to recover the root password for Centos. bad day, no way to get root access.

Now, restart the VM and boot to the geekmode. by press any key, or CTL+X to the grub mode.

Now, Press ‘e’ to edit the command line. we need tell the kernal boot from graphics mode to single user mode
user arrow key up/down to select the kernal line,

Press e, to edit the command line, you will see the follow screen,
Now, it is on the grphics mode, rhgb=red hat graphical boot

Let’s change it to single user mode, by replacing the rhgb quiet.
Before,

after,

Press enter to switch the parent screen. then press b to boot the single mode kernel, you can tell linux single is there

Once Done, you are in single user mode.
remember the password this time, no typo please.

Coherence push replication, com.tangosol.net.DefaultConfigurableCacheFactory cannot be cast to com.oracle.coherence.environment.Environment

2011-10-25T09:41:00.001-07:00

Coherence has several incubator project, Push replication is one of them which enable you to turn on replication between several standalone clusters. i.e Cross WAN replication between data centers.

I just read their limited documentation, trying to setup one Master-Slave replication between two separate clusters in my pc. ON the master side, here is the cache configuration. basically load the default coherence config, reference the incubator pof file.

Here is just pickup the remote cluster publisher

using the distributed-scheme-with-publishing-cachestore scheme will enable the runtime to capture the entity change to a queue, then flushed to remote cluster using the remove invocation service.

All set, when I try to feed some data to the local cache which is Master, long stack trace appears.

Map (master): put a 2
2011-10-25 09:17:19.862/22.446 Oracle Coherence GE 3.7.1.0 <Error> (thread=main, member=2):
Portable(com.tangosol.util.WrapperException): (Wrapped: Failed request execution for DistributedCacheWithPublishingCacheStore service on Member(Id=1, Timestamp=
706, Address=192.168.137.1:8090, MachineId=63704, Location=site:E3,machine:androidyou-PC,process:6004, Role=CoherenceServer)) null
        at com.tangosol.util.Base.ensureRuntimeException(Base.java:288)
        at com.tangosol.coherence.component.util.daemon.queueProcessor.service.Grid.tagException(Grid.CDB:36)
        at com.tangosol.coherence.component.util.daemon.queueProcessor.service.grid.partitionedService.PartitionedCache.onPutRequest(PartitionedCache.CDB:50)
        at com.tangosol.coherence.component.util.daemon.queueProcessor.service.grid.partitionedService.PartitionedCache$PutRequest.run(PartitionedCache.CDB:1)
        at com.tangosol.coherence.component.util.DaemonPool$WrapperTask.run(DaemonPool.CDB:1)
        at com.tangosol.coherence.component.util.DaemonPool$WrapperTask.run(DaemonPool.CDB:32)
        at com.tangosol.coherence.component.util.DaemonPool$Daemon.onNotify(DaemonPool.CDB:63)
        at com.tangosol.coherence.component.util.Daemon.run(Daemon.CDB:42)
        at java.lang.Thread.run(Thread.java:619)
        at <process boundary>
        at com.tangosol.io.pof.ThrowablePofSerializer.deserialize(ThrowablePofSerializer.java:57)
        at com.tangosol.io.pof.PofBufferReader.readAsObject(PofBufferReader.java:3316)
        at com.tangosol.io.pof.PofBufferReader.readObject(PofBufferReader.java:2604)
        at com.tangosol.io.pof.ConfigurablePofContext.deserialize(ConfigurablePofContext.java:368)
        at com.tangosol.coherence.component.util.daemon.queueProcessor.Service.readObject(Service.CDB:1)
        at com.tangosol.coherence.component.net.Message.readObject(Message.CDB:1)
        at com.tangosol.coherence.component.net.message.SimpleResponse.read(SimpleResponse.CDB:6)
        at com.tangosol.coherence.component.util.daemon.queueProcessor.service.Grid.deserializeMessage(Grid.CDB:19)
        at com.tangosol.coherence.component.util.daemon.queueProcessor.service.Grid.onNotify(Grid.CDB:31)
        at com.tangosol.coherence.component.util.daemon.queueProcessor.service.grid.PartitionedService.onNotify(PartitionedService.CDB:3)
        at com.tangosol.coherence.component.util.daemon.queueProcessor.service.grid.partitionedService.PartitionedCache.onNotify(PartitionedCache.CDB:3)
        at com.tangosol.coherence.component.util.Daemon.run(Daemon.CDB:42)
        at java.lang.Thread.run(Thread.java:619)
Caused by: Portable(java.lang.UnsupportedOperationException)
        at java.util.AbstractMap.put(AbstractMap.java:186)
        at com.tangosol.util.WrapperObservableMap.put(WrapperObservableMap.java:151)
        at com.tangosol.coherence.component.util.daemon.queueProcessor.service.grid.partitionedService.PartitionedCache$Storage.postPut(PartitionedCache.CDB:70)
        at com.tangosol.coherence.component.util.daemon.queueProcessor.service.grid.partitionedService.PartitionedCache$Storage.put(PartitionedCache.CDB:17)
        at com.tangosol.coherence.component.util.daemon.queueProcessor.service.grid.partitionedService.PartitionedCache.onPutRequest(PartitionedCache.CDB:25)
        at com.tangosol.coherence.component.util.daemon.queueProcessor.service.grid.partitionedService.PartitionedCache$PutRequest.run(PartitionedCache.CDB:1)
        at com.tangosol.coherence.component.util.DaemonPool$WrapperTask.run(DaemonPool.CDB:1)
        at com.tangosol.coherence.component.util.DaemonPool$WrapperTask.run(DaemonPool.CDB:32)
        at com.tangosol.coherence.component.util.DaemonPool$Daemon.onNotify(DaemonPool.CDB:63)
        at com.tangosol.coherence.component.util.Daemon.run(Daemon.CDB:42)
        at java.lang.Thread.run(Thread.java:619)
        at <process boundary>
        at com.tangosol.io.pof.ThrowablePofSerializer.deserialize(ThrowablePofSerializer.java:57)

ON the storage Node, the error that may bring you here,

</class-scheme>) java.lang.reflect.InvocationTargetException
        at com.tangosol.util.Base.ensureRuntimeException(Base.java:288)
        at com.tangosol.run.xml.XmlHelper.createInstance(XmlHelper.java:2652)
        at com.tangosol.run.xml.XmlHelper.createInstance(XmlHelper.java:2536)
        at com.tangosol.net.DefaultConfigurableCacheFactory.instantiateAny(DefaultConfigurableCacheFactory.java:3476)
        at com.tangosol.net.DefaultConfigurableCacheFactory.instantiateCacheStore(DefaultConfigurableCacheFactory.java:3324)
        at com.tangosol.net.DefaultConfigurableCacheFactory.instantiateReadWriteBackingMap(DefaultConfigurableCacheFactory.java:1753)
        at com.tangosol.net.DefaultConfigurableCacheFactory.configureBackingMap(DefaultConfigurableCacheFactory.java:1500)
        at com.tangosol.net.DefaultConfigurableCacheFactory$Manager.instantiateBackingMap(DefaultConfigurableCacheFactory.java:4111)
        at com.tangosol.coherence.component.util.daemon.queueProcessor.service.grid.partitionedService.PartitionedCache$Storage.instantiateBackingMap(Partitione
        at com.tangosol.coherence.component.util.daemon.queueProcessor.service.grid.partitionedService.PartitionedCache$Storage.setCacheName(PartitionedCache.CD
        at com.tangosol.coherence.component.util.daemon.queueProcessor.service.grid.partitionedService.PartitionedCache$ServiceConfig$ConfigListener.entryInsert
CDB:17)
        at com.tangosol.util.MapEvent.dispatch(MapEvent.java:266)
        at com.tangosol.util.MapEvent.dispatch(MapEvent.java:226)
        at com.tangosol.util.MapListenerSupport.fireEvent(MapListenerSupport.java:567)
        at com.tangosol.util.ObservableHashMap.dispatchEvent(ObservableHashMap.java:229)
        at com.tangosol.util.ObservableHashMap$Entry.onAdd(ObservableHashMap.java:270)
        at com.tangosol.util.SafeHashMap.put(SafeHashMap.java:244)
        at com.tangosol.coherence.component.util.ServiceConfig$Map.put(ServiceConfig.CDB:43)
        at com.tangosol.coherence.component.util.daemon.queueProcessor.service.grid.partitionedService.PartitionedCache$StorageIdRequest.onReceived(PartitionedC
        at com.tangosol.coherence.component.util.daemon.queueProcessor.service.Grid.onMessage(Grid.CDB:34)
        at com.tangosol.coherence.component.util.daemon.queueProcessor.service.Grid.onNotify(Grid.CDB:33)
        at com.tangosol.coherence.component.util.daemon.queueProcessor.service.grid.PartitionedService.onNotify(PartitionedService.CDB:3)
        at com.tangosol.coherence.component.util.daemon.queueProcessor.service.grid.partitionedService.PartitionedCache.onNotify(PartitionedCache.CDB:3)
        at com.tangosol.coherence.component.util.Daemon.run(Daemon.CDB:42)
        at java.lang.Thread.run(Thread.java:619)
Caused by: java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
        at com.tangosol.util.ClassHelper.newInstance(ClassHelper.java:694)
        at com.tangosol.run.xml.XmlHelper.createInstance(XmlHelper.java:2611)
        ... 23 more
Caused by: java.lang.ClassCastException: com.tangosol.net.DefaultConfigurableCacheFactory cannot be cast to com.oracle.coherence.environment.Environment
        at com.oracle.coherence.patterns.pushreplication.PublishingCacheStore.<init>(PublishingCacheStore.java:179)
        ... 29 more
2011-10-25 09:17:16.120/18.685 Oracle Coherence GE 3.7.1.0 <D5> (thread=Cluster, member=1): Member 2 joined Service DistributedCacheWithPublishingCacheStore wit

trying to cast DefautCOnfigurableCacheFactory to Envionment? what is Environment class located, in standard coherence, or Incubator project? Iet me find it out in eclipse.
It is in the common lib used by push replication,

check the class hierarchy, it’s another cachefactory,

then change our cachefactory from the default DefautCOnfigurableCacheFactory to the incubator cache facotry, it will pick up the setting like sync namespace.

Old,
should be,

then all back to normal. Hope it helps

Android, Get the Process ID and Thread ID

2011-10-18T18:00:00.001-07:00

When you debug the Multi-Thread app in android, it will be better that you can tell the activity thread id. so I have one Simple Line of Code to dumpout the logcat message, time, processid, thread id and the message.

public static void MLog(String msg) {
Log.d("xxxnew", String.format("at %s Process ID %s, Thread ID %s %s ", new Date().toString(), Process.myPid(),Process.myTid(),msg));
}

HP Notebook, stuck on booting screen, Status: 0xc00000e9

2011-10-15T23:14:00.001-07:00

some bad day, My HP pavilion stuck on the booting screen, here is what it looks like,
Status: 0xc00000e9
Info: An unexpected I/0 error has occurred.

So basically, it’s an IO error, for sure, should be disk IO error. then I go to the bios, Diagnostics-> hard drive test,
Get an 303 error, looks like 404 for geeks,

here is the HP error codes, http://h20000.www2.hp.com/bizsupport/TechSupport/Document.jsp?objectID=c01443317

No magic, the hard drive died, replace a new one will fix the issue.

Hadoop Hbase, Test JAVA client and inspect the network flow

2011-10-14T13:31:00.001-07:00

I just setup three VMS, with the following roles

192.168.209.130 /HOME	NameNode, HbaseMaster(Standby server), Region Server, Zookeeper
192.168.209.132 /LA	DateNode, HbaseMaster (Active One), Region Server, Zookeeper
192.168.209.133 /NJ	Zookeeper

then I create one basic table, called ‘customer’ with Info as the only one column family.

then on the client machine which is win 7, I write a simple HBASE client to push some data to the HBASE. client runs on 192.168.209.1, Here is the Code.

on the client side, you need reference the HBASE jars , and put the conf folder into the classpath. in the client hbase-site.xml, just point to the zookeeper instances. Here I put three ZK quorums.

Then run the program, after done. you can tell from the hbase shell, 5 records are there.

from the client, you can tell from the console log. it will first talk to zookeeper, to get which master is active. then query the root region, then the meta region information.

after that, it will write data to the corresponding regions server directly.

here is the network flow layout.
First, setup TCP connection with Zookeeper, to locate the master server. 2181 is the zookeeper Listening Port.

Then talk to Root region server which is LA to get the region allocation.

once it has those information, it will cached in memory. then put the data to that regions server directly.

that’s all the client conversation. (client , zookeeper, region server.)

The general communication flow is that a new client contacts the ZooKeeper ensemble
(a separate cluster of ZooKeeper nodes) first when trying to access a particular row. It
does so by retrieving the server name (i.e., hostname) that hosts the -ROOT- region from
ZooKeeper. With this information it can query that region server to get the server name
that hosts the .META. table region containing the row key in question. Both of these
details are cached and only looked up once. Lastly, it can query the reported .META.
server and retrieve the server name that has the region containing the row key the client
is looking for.

Oops, Youtube support team are trained monkeys

2011-08-27T22:25:00.001-07:00

500 Internal Server Error

Sorry, something went wrong.
A team of highly trained monkeys has been dispatched to deal with this situation.

If you see them, show them this information:

WFlEO-pxslLQKQ8jn5sBkCwnuuZg1YYqX9bVND5_m5gRtSxuruNgKsTd17lg
uIm1sVMS98fpTtpzlNX96mH1WLq79xMZUvOLdhFvKX_WGYFfc38YlrkAm23q
kPFJ7mrbT12tQundh7bk1y0TEqJfe0SA336fLvzgVEm1tt8oIf8-ib7CH3QQ
n11bbMFCY1iJgCn1ezZIl1rXHkmuRU9xrlQ4d-XrG1JmJtaRRFY1kklSloUj
JECppOrM0wAaZXFODfOpE_ZQXT0iJk46_YrqBf0-rD7nXYmruWZqU4zcnvWN
tnXr70fXkkBoVohlx-yC7GiBtceyBSPMYhotqQD2gE7JMa_LtCXflVx15Hs3
yfKiDZIcjiaQ5U1L2YHgXwplA4SiOJ2SRM2HI1DO9_9bs9qMKWPvc-OUt3nV
PmZHNdlYW143KHOCRqZv0m19bQ2vxXhZjEHfc8GrLZu3Vz3_y1QvVAt4lj5J
2UPBp6mJvmB3Ouzak6VN1jXSa24n8gXE5VrGNcnWs1GKCpGNd6yRQWpq9shw
BF1eygVB8mZri2xeUBh4vLZ7pwEj06c-qmJr4RItLLXWT8CMqeM6KrYxWr6g

How To, Auto Suggesting with multi words/terms using solr ShingleFilterFactory

2011-08-22T17:34:00.001-07:00

Google has one amazing feature called auto sugesstion. basically, when you type one word, even just the prefix of the word, it will list the suggestion for you like this,

Even further, google has one unknown REST API that you can use to query the auto-completion word lists provided by Google.
http://google.com/complete/search?output=toolbar&q=%22intel+s%22

For Apache Solr, If you want to do the same thing as google does, here is one simple approach. Just using the ShingleFilter,

Create one text field that use this filter,

Then create one field to use that field type

After that, we can post a simple document to the Solr, I will put one wiki page, http://en.wikipedia.org/wiki/X25-M to the solr.

you can just create one doc and post it to solr using curl

Now you can go to the solr admin to check the description terms, you will see those multi-word terms

for application, we can run a facet query to get the keyword list and count.

http://localhost:8080/solrmaster/select/?q=description%3A*&version=2.2&start=0&rows=0&indent=on&facet=true&facet.field=description&facet.prefix=02g2

pretty simple to get started

How to, Hadoop , error to start Jobtracker of hadoop, FATAL org.apache.hadoop.mapred.JobTracker: java.lang.RuntimeException: Not a host:port pair: local

2011-08-17T21:44:00.001-07:00

One day, I just created a fresh centos and download/install the hadoop, by default it runs on local mode. so I just change the config to run in distributed mode. When I run bin/start-all.sh. get the follow errors in the tasktrackr log.

2011-08-16 21:00:12,136 INFO org.apache.hadoop.mapred.JobTracker: STARTUP_MSG:
/************************************************************
STARTUP_MSG: Starting JobTracker
STARTUP_MSG:   host = hadoop/192.168.209.131
STARTUP_MSG:   args = []
STARTUP_MSG:   version = 0.20.2
STARTUP_MSG:   build = https://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.20 -r 911707; compiled by 'chrisdo' on Fri Feb 19 08:07:34 UTC 2010
************************************************************/
2011-08-16 21:00:12,280 WARN org.apache.hadoop.conf.Configuration: bad conf file: element not <property>
2011-08-16 21:00:12,280 WARN org.apache.hadoop.conf.Configuration: bad conf file: element not <property>
2011-08-16 21:00:12,280 WARN org.apache.hadoop.conf.Configuration: bad conf file: element not <property>
2011-08-16 21:00:12,299 INFO org.apache.hadoop.mapred.JobTracker: Scheduler configured with (memSizeForMapSlotOnJT, memSizeForReduceSlotOnJT, limitMaxMemForMapTasks, limitMaxMemForReduceTasks) (-1, -1, -1, -1)
2011-08-16 21:00:12,344 FATAL org.apache.hadoop.mapred.JobTracker: java.lang.RuntimeException: Not a host:port pair: local
        at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:136)
        at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:123)
        at org.apache.hadoop.mapred.JobTracker.getAddress(JobTracker.java:1807)
        at org.apache.hadoop.mapred.JobTracker.<init>(JobTracker.java:1579)
        at org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:183)
        at org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:175)
        at org.apache.hadoop.mapred.JobTracker.main(JobTracker.java:3702)

it looks like the runtime still pick up the default mapreduce configuration located in the hadoop-x.core.jar, you can get this file by unzipping hadoopcore.jar,

So the error means it pick up this default setting of Local. Then I read careflly about the mapred-site.xml, here it is.

It looks like perfect, right, cause I just copied the setting from the default xml, while, I just keep the weird error.
well, here is final xml that works great, I highlighted the difference, I forgot to put the property tag. so the runtime will ignore this xml and pick up the default settings.

Summary, when you get weird error, check the log first, then make sure those conf xml are in right format that comforts the scheme
tricky, hah?

Download Bing.com Daily Wallpaper App for Android

2011-08-02T10:45:00.001-07:00

https://market.android.com/details?id=com.androidyou.bing&feature=search_result
Features,

Save images to use them as your Android wallpaper
Access and download Bing's daily wallpapers
Read comments on wallpaper images
Share images via e-mail or your favorite social networking site
Recall and view your download history
set as wallpaper automatically
background run

AD Free Edition,

Exchange 2010 Provisioning using ILM 2007

2011-08-01T17:45:00.001-07:00

If you just upgraded Exchange to 2010, and still using the ILM 2007. you need install the SP1 and featured package, the only document you can follow is http://technet.microsoft.com/en-us/magazine/ff472471.aspx

Basically, for the provisioning to exchange 2010 mailbox. you have call the utility to create a mailbox in your Povision DLL.something like,

void IMVSynchronization.Provision (MVEntry mventry)
{
ConnectedMA adMA;
CSEntry csentry;
String nickName, mailboxMDB;
ReferenceValue dn;

try
{
adMA = mventry.ConnectedMAs["Fabrikam AD MA"];

nickName = mventry["mailNickname"].Value;
mailboxMDB = mventry["homeMDB"].Value;

// Construct the distinguished name
dn = adMA.EscapeDNComponent("CN=" + mventry["cn"].Value).Concat("ou=mailboxes,dc=fabrikam,dc=com");

if(0 == adMA.Connectors.Count)
{
csentry = ExchangeUtils.CreateMailbox(adMA, dn, nickName, mailboxMDB);
}
}

// Handle any exceptions
catch(Exception ex)
{
//...
}
}

For the Magic code in CreateMailBox, you can see the source code. it basically add several attributes there,

After that, also you should config the AD ma to run the extension Code , you need to specify the Powershell remote address, and Extension of 2010 or 2007 ,

After those two major change been applied, the ILM will call the provision code to create the mailbox, and Run the Extension Code for every object listed in the pending export stage of the Connected Space. there are two Extension DLLs in the ILM Extension Folder.
Extension2007 and Extension2010.dll

for the code in Exch20xxExtension.dll, it basically call the powershell command remotly on the server you specified in the RPS URI.

basically, call the PowerShell command remotely.

Update-Recipient -Identity "<ExportedDN>"-Credential <PSCredential> -DomainController <Fqdn>

Here is the pain you may have.
ILM will call this command for every Object which has some pending export in the MA. even just a title change, it will call the Update-receipent , which may cause the default mailpolicy to run. (in some case, this is not what you want, i.e you have different mail policy. )

however, we have the Extension point.
you can get the source code of the extension dll, put you logic there, and sign the new dll. which will be picked up by ILM.
you can do some loggin there, or check wheter the object is required to run the update-recepient command. or even just call you powershell command.

How to Install ADB Driver for HTC Inspire 4G

2011-07-27T09:56:00.001-07:00

following ADB Driver for HTC wildfire and T-Mobile G2, add the entries to the android_winusb.inf,

; HTC Inspire
%SingleAdbInterface% = USB_Install, USB\VID_0BB4&PID_0CA2
%CompositeAdbInterface% = USB_Install, USB\VID_0BB4&PID_0CA2&MI_01

then go to device manager, refresh the device to install the driver. that’s it.

It’s there in the device manager,

more about ADB drivers,
ADB Driver for HTC wildfire and T-Mobile G2
My Note on Solutions.: How to: install ADB driver for Motorola Xoom
Install ADB Driver for Samsung galaxy tablet 10.1 from google io /Samsung galaxy SGH-T959

Android Developers Blog: Multiple APK Support in Android Market

2011-07-22T23:16:00.000-07:00

Android Developers Blog: Multiple APK Support in Android Market

What a great idea, Android rocks!

How to : Set Android WebView HTTP Referrer programmatically

2011-06-16T14:14:00.001-07:00

In Android, you may use the webview.LoadData to show html. using the following code,

public class WebViewTestActivity extends Activity {

WebView wv;

@Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.mainbrownser);
wv = (WebView) findViewById(R.id.wv);
wv.getSettings().setJavaScriptEnabled(true);
wv.loadData(
"<img src='http://images.google.com/intl/en_ALL/images/logos/images_logo_lg.gif'>",
"text/html", "UTF-8");
}

}

When you run the app, you can see the google logo there,

however, if you try to access some image whose server will check the referrer, you will be in trouble to viewing the image.
by default, the webview doesn’t setup the referrer when fetching html resouces,

I searched a lot, there is no explicit way to setup the referrer . after checking the webview source code, here is the trick to setup the referrer, using the loaddatawithbaseurl, the baseurl will be the referrer picked by webview

Code,

wv.loadDataWithBaseURL("http://wwwgoogle.com/myreferrer",
"<img src='http://images.google.com/intl/en_ALL/images/logos/images_logo_lg.gif?a=1'>",
"text/html", "UTF-8","");

Hope it helps,

No provisioned iOS devices are available. Connect an iOS device or choose an iOS simulator as the destination.

2011-06-13T11:23:00.001-07:00

One day, I want to write a Iphone app for my son, then Just download the latest Xcode, and plug my Iphone to the Mac, supposedly,XCode should run the profile provisioning, I just click to Run the app. However, I get an error,

here, it’s a versioning issue, First, Click Window->Organizer, check Iphone OS

So for the Phone, It runs on 4.2.1,then make sure project building setting, using the same base level of SDK

by default, Xcode use the lastest sdk , that’s why you see the error.
Switch it to 4.2, then you will see you device in the target list,

How to: Test if an object is of a specified type. instanceof JAVA, C# and Objective C

2011-06-08T16:23:00.001-07:00

In Java, just using the instanceof keyword.

In C#, it has the similar syntax, Just called IS

In Objective-C, using keyword iskingofclass,

How to get the SOS.DLL for silverlight 4.0.60310.0 for windbg

2011-06-02T11:30:00.001-07:00

If you want to do the Silverlight debugging using windbg, the first thing is you have to get the sos extension for Silverlight.
When I try to run the command ‘.loadby sos coreclr’

get the error , it can’t find the sos.dll, in the silverlight folder,

I try to google “how to get the sos.dll for silverglight4.’, nothing is helpful, then I spend a little time to figure out how to get the sos.dll. So here is the answer,

1. Install visual studio 2010, if you are a developer, you probably have already installed the IDE
2. this is important, Download and install the Microsoft Silverlight 4.0 SDK

that’s IT.

#1 is very important, if you skip it and install the SDK only, it wont install the sos.dll for you.

IronRuby Tutorial, Ruby For C# programmers,Part 1

2011-05-31T15:59:00.001-07:00

IronRuby is one of the DLR extension on Microsoft .net platform that enable you programming against .net using the Dynamic Language. I will put a very short tutorial, here.

Download the assemblies or just the installer file from http://ironruby.codeplex.com/releases/view/49097#DownloadId=159561
for the installation, it basically puts several .net assemblies to the GAC folder.

Then you can run the IronRuby console, which is the C# version of Irb

when you run process explorer, you will see the loaded Assemblies,

Ir.exe is a standard .net application,

Now, let’s write a basic HelloWorld Ruby application,
First in Ruby Style,

then In C#, we just call System.Console.WriteLine(“Message”). namespace will be mapped to module in Ruby,

Now, you can call the standard .net library, like Datetime

In Ruby, if you want to query the methods of the Object, you can use .methods, we can do this too.

if we want to change the Console foreground color, which we cant do in ruby easily,just call system.foreground_color
here, we can use ruby style naming like foreground_color, or C# style ForegroundColor, the dynamic method dispatcher will delver the call to the right stub,

To using the Base64 encoding that I used pretty often,

in Ruby, it’s easy to subclass to add more methods, say thing error here, I will add one method called writeerror

How to: Fix IE9 rendering issues with Google Gmail

2011-05-25T15:42:00.001-07:00

One day, When I open Gmail, everything is messed up, the rendering is totally wrong, the screenshot looks like this

When I click the about, I use the latest IE9 version.

To locate it why, I just click F12 to open the developer tools,
it picked up the IE9 Compact Browse mode somehow.

Switch it back to regular IE9 Mode will fix the rending issue.

ActivityManager: Error: Activity class {com.androidyou.asia/com.androidyou.asia.taballActivity} does not exist.

2011-05-24T22:15:00.001-07:00

I got this weird error when I try to run a basic android application, the error console tells nothing but one activity doesn't exist

when I check the logcat, nothing related there.

after a lot try, seems put a . in front of the lanuch activity will be the trick,

Install ADB Driver for Samsung galaxy tablet 10.1 from google io /Samsung galaxy SGH-T959

2011-05-13T10:06:00.001-07:00

If you want to install the ADB Driver for the samsung tablet you get from google IO, you may find it’s hard to get the correct driver from samsung download site.

I found another trick to do the installation, Download and Install the Samsung Kies which will install the driver for the tablet,
Download Link, http://www.samsungapps.com/about/onPc.as?LOCALE=hi_IN

After installation of the kies, you will see adb driver is there in the device manager,

and you can run adb devices, to see your samsung device,

For the galaxy phone , please go to the download site, http://www.samsung.com/us/downloads

Chose the cell-phone and your model, click the software tab, download the zipped usb driver (has the ADB one too)

More Adb Driver installation Blogs,

Seconds since the Unix epoch in C#/java, the same output of Time() in PHP

2011-05-03T16:43:00.001-07:00

in Php, there is one Time() function, Returns the current time measured in the number of seconds since the Unix Epoch (January 1 1970 00:00:00 GMT).
in C#, there is no built-in function to return seconds since UNix epoch, here is the code snippet to return the same thing

int t = (int)DateTime.Now.ToUniversalTime().Subtract(new DateTime(1970, 1, 1, 0, 0, 0)).TotalSeconds;

In Java, there is one standard function, System.currentTimeMillis(),which measured in milliseconds, between the current time and midnight, January 1, 1970 UTC.

System.currentTimeMillis()/1000

Adobe Content Server, Error getting license License server communication problem: E_LIC_WRONG_OPERATOR_KEY

2011-05-03T16:42:00.001-07:00

When I try to finish the fulfillment of one ebook using the digital edition, It returns this error.

However, when I test the http://server:8080/fulfillment/statuscheck, all passed.

wrong operater key always means that the p12 file we setup in the c:\config\fulfillment-conf.txt is incorrect. it doesn’t match the url that do the fulfillment.

So there could be stupid typo mistake, check the fulfillment-conf make sure the com.adobe.adept.serviceURL is matching the cert url.
if you want to check the cert url, just click and import it to the certification store and open the certificate mmc

also it looks like the statuscheck never check the url match the operator certificate which we think it will do.

adobe content server, Error getting license Server communication problem: E_ADEPT_REQUEST_EXPIRED

2011-05-03T16:40:00.000-07:00

I just wrote a simple C# Bookstore by copying the logic in the sample Php bookstore application, all looks pretty straightforward. when I click download to fulfill the book, the Adobe digital edition returns error.

I recalled in the php program, the client will pass a dateval variable to the fulfillment server, it just pick up the time() function which returns seconds since Unix epoch. so I just create my own edition like

int dataval = (int)DateTime.Now.Subtract(new DateTime(1970, 1, 1, 0, 0, 0)).TotalSeconds;

then when the server get the fulfillment request , it think the time has been passed. Why?

here is the trick, the time function that the php sample program uses Returns the current time measured in the number of seconds since the Unix Epoch (January 1 1970 00:00:00 GMT).
I am in the PST timezone, so there is 8 hours difference,

fixing is easy,change it as,
(int)DateTime.Now.ToUniversalTime().Subtract(new DateTime(1970, 1, 1, 0, 0, 0)).TotalSeconds;

Android IntentService, how to write a background service keep polling data and notify Activity for Change.

2011-04-17T09:59:00.000-07:00

Let me put a simple test application, One activity and One Service. for the service, it will do the background job like networking, computation. and notify the Acidity to do some UI side rendering or notification to User.

first , Create one Simple Android application with just one Activity,

public class UIActivity extends Activity {
    /** Called when the activity is first created. */

    TextView tv;
    @Override
    public void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        tv=new TextView(this);
        tv.setText("hello,World");
        setContentView(tv);
    }


}

then Add one Handler in the activity which will be called by the bakcground service when there are any change get triggered,

Handler handler=new Handler()
    {
        @Override
        public void handleMessage(Message msg) {
            //get data from msg

            String result=msg.getData().getString("result");
            tv.setText(result);
            Log.d("xxxxx", "get data" + result);
            super.handleMessage(msg);
        }
    };

then Create one Service inherited from IntentService, here we defined a timer which will do the periodic checking and send back data change to activity,

public class BackService extends IntentService {

    public BackService()
    {
        super("myintentservice");
    }


    Messenger messenger;
    Timer t=new Timer();


    @Override
    protected void onHandleIntent(Intent intent) {
        messenger=(Messenger) intent.getExtras().get("handler");

        t.schedule(new TimerTask() {

            @Override
            public void run() {
                // just call the handler every 3 Seconds

                Message msg=Message.obtain();
                Bundle data=new Bundle();
                data.putString("k", "value" + System.currentTimeMillis() );
                msg.setData(data);

                try {
                    messenger.send(msg);
                } catch (RemoteException e) {
                    // TODO Auto-generated catch block
                    e.printStackTrace();
                }
            }
        }, 100,3000);

    }

}

for Activity, need to specify the handler through putextra and start the service,

public void onCreate(Bundle savedInstanceState) {
     super.onCreate(savedInstanceState);
     tv=new TextView(this);
     tv.setText("hello,World");
     setContentView(tv);

     Intent i=new Intent(this, BackService.class);
     i.putExtra("handler", new Messenger(this.handler));
     this.startService(i);

}

remember to register the backservice into the manifest.xml,
when you run the app, you will see the textview will change every 3 seconds,

run adb logcat XXXXX:D *:S, even the activity is brought to background , the Activity still get the change from service,

How to : QT Hello world localization and I18N

2011-04-12T17:20:00.001-07:00

When you installed QT SDK, it came with several useful tools to do the I18N and localization, I’ll put a very basic QT program and show the basic steps to localize the app to support different local /languages.

Create a Basic QT hello world application, make sure all user-visible strings are used as translatable strings (using tr function)
Create a empty QT Project using QT creator,

then add a new source file,

#include <QtGui>

int main(int argc, char * argv[])
{
    QApplication app(argc,argv);
    QPushButton * button=new QPushButton(QObject::tr("Hello,World!"));
    button->show();
    app.exec();
}

then run the app by pressing F5, you will see the hello world button,

Now we are going to add the Chinese and Japanese Support to the app.

>>> go the the project folder, Run a command called Lupdate(include in the sdk ), to extract all translatable strings to a file,

here default.ts is just a xml file as the content bellows,

>>>now use the linguist tool to translate it into Japanese, run “linguist” In the qt shell or click the tool from start-menu
open the default.ts, and chose Japanese as the target language,

put the translation for hello world in Japanese, and click to mark done, then Save it as as jp.ts

Now quite the translation tool and start over again to translate it to Chinese, and export to cn.ts
>>>compile the cn.ts and jp.ts to binary format using lrelease

>>> now in the app, load those two qm files,

QApplication app(argc,argv);

QTranslator translator ;

QLocale curent;
if(curent==QLocale::Chinese)
{
translator.load("F:\\helloworld\\cn.qm");
app.installTranslator(&translator);
}
if(curent==QLocale::Japanese)
{
translator.load("F:\\helloworld\\jp.qm");
app.installTranslator(&translator);
}

QPushButton * button=new QPushButton(QObject::tr("Hello,World"));
button->show();
app.exec();

Now, if you setup your current locale to Japanese, you will see the JP version of helloword

or you can change the default local in code,

QLocale::setDefault(QLocale::Japanese);

How to: install ADB driver for Motorola Xoom

2011-04-08T10:32:00.001-07:00

Download and install the Driver from Motorola,
http://developer.motorola.com/docstools/USB_Drivers/

in your xoom device, enable the USB debugging under application section settings,

then in windows, it will find new device and apply the ADB driver.

Now you can run adb devices, to see the device is there

for all other non-google device, here is the list of their drivers,
http://developer.android.com/sdk/oem-usb.html

Hadoop: How to install and test PIG

2011-04-06T16:21:00.001-07:00

like install and test Hive on hadoop cluster, Pig is another end-user facing analytical tools for Hadoop. unlike SQL style query for Hive, Pig use the scripting language which will be more familiar to Sys admins.

Download Pig From Yahoo, http://www.apache.org/dyn/closer.cgi/pig

Unzip the files and go the bin folder, only change is we need to add the hadoop conf folder to the class path. then pig knows how to deal with hadoop cluster,

# cygwin path translation
if $cygwin; then
    CLASSPATH=`cygpath -p -w "$CLASSPATH"`
    PIG_HOME=`cygpath -d "$PIG_HOME"`
    PIG_LOG_DIR=`cygpath -d "$PIG_LOG_DIR"`
fi

#add hadoop conf folder to the classpath

export CLASSPATH=$CLASSPATH:/usr/lib/hadoop/conf

After that, you can run the ./bin/pig to start the shell of Pig,

Here are the Pig syntax.
http://pig.apache.org/docs/r0.8.0/cookbook.html

Hadoop, java.lang.ClassCastException: org.apache.hadoop.io.LongWritable cannot be cast to org.apache.hadoop.io.Text

2011-04-06T16:12:00.001-07:00

If you work on a Mapper which expects the file format of keyvalue, and forget to specify the inputformat to jobconf, you will get the following errors.

java.lang.ClassCastException: org.apache.hadoop.io.LongWritable cannot be cast to org.apache.hadoop.io.Text
at MR.map(MR.java:1)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
at org.apache.hadoop.mapred.Child.main(Child.java:170)

here is why,

for the jobconf, the default inputformat is TextInputFormat,

you will get Key/value type of longwritable/text in your Map function.
if you put something like Text as the key, you will get the classcast exception

public void map(Text key, Text value,
OutputCollector<Text, Text> collector,
Reporter paramReporter) throws IOException {

So always remember to specify the InputFormat.

How to :Test IIS Media service Smooth streaming tutorial

2011-03-24T17:55:00.001-07:00

Http for streaming, for IIS, there is the media service, let’s download and install it using the platform installer

http://www.iis.net/media

using the web platform installer 3.0 (like the Yum for Centos), pick up the media service package and click to install.

also install the smooth string client,

when done, you will noticed that in install several HttpHandlers to IIS,

and check the handler mapping features,

Path Handler

*.ism Smoothhandler

*.isml LiveStreamingHandler

*.isx PlayListHandler

Also you will see several media tools,

Now we have the infurscture and components to Host and support the smooth streaming, we need import those assets to the virtual Directory just like other aspx files.

encoder , there is a encoder to encode one raw video file to tens of other formats with different qualities. (used for smooth streaming, when the network bandwidth is great enough, throw the 1080P one to the client. )
Download encoder full version (Expression Studio 4 Premium (x86) - DVD (English))from your MSDN subscription. note, the free downloadable encoder 3 doesn’t support the smooth streaming encoding. No free lunch,

>>so download Expresion 4, and install the Microsoft Expression Encoder 4 with Service Pack 1 (SP1)

Create a new project, import the raw video. search and select all streaming presets and click apply. then click encode to go,

this process is a CPU intensive work, try to run on more powerful server with lots cpu cores.
the output folder looks like this, one ism file and several ismv files based on your presets selectons

Now copy this folder to the iis virtual folder you created,
then you can see the file listed in the iis add-on , smooth streaming,

host/runtime is ready, content is there, now need to write a Silverlight client to view the content,

let’s build using the Microsoft Silverlight Media Framework 2.4, which comes with the built-in streaming player.

create one smooth streaming application( a Silverlight project which has been customized to reference the extensions dll.)

replace the ism file with ours,

 <Grid x:Name="LayoutRoot"  Loaded="LayoutRoot_Loaded" Background="White">
        <smf:SMFPlayer x:Name="s" AutoPlay="True">
            <smf:SMFPlayer.Playlist>
                <media:PlaylistItem DeliveryMethod="AdaptiveStreaming"  MediaSource="http://localhost/hellomedia/Wildlife.ism/Manifest"/>
            </smf:SMFPlayer.Playlist>
        </smf:SMFPlayer>
    </Grid>

put a clientaccesspolicy.xml to the iis root folder,

<?xml version="1.0" encoding="utf-8"?>

<access-policy>

<cross-domain-access>

    <policy>

      <allow-from http-request-headers="*">

        <domain uri="*"/>

        </allow-from>

      <grant-to>

        <resource path="/" include-subpaths="true"/>

      </grant-to>

    </policy>

</cross-domain-access>

</access-policy>

then click F5 to run, that’s it.

in case you can’t see the video, check this out ,why? Silverlight smf player never play.

why? Silverlight smf player never play.

2011-03-24T17:49:00.001-07:00

Just installed the SMF sdk and point the ism to my IIS steaming server, click f5 to run the app, it’s a black screen. it never check crossdomain.xml, it never sends out the request

Click the play, nothing happened, then I run the debug and turn on all exceptions,

run again, aha, one dll is missing,

So you need to install streaming client 1.5, http://www.iis.net/download/SmoothClient
just serach streaming in the web platform installer ,click and install

after that, reference the dll located in [C:\Program Files (x86)\Microsoft SDKs\IIS Smooth Streaming Client\v1.5\Silverlight] to your project, then rebuild, all set, now the horses can run now, lol

Also, make sure crossdomain.xml is there.

in case you can’t see the video, check this out ,why? Silverlight smf player never play.

How to: install and test Hive on hadoop cluster

2011-03-21T16:03:00.001-07:00

After we install and config hadoop to run in fully distributed Mode, Centos, then we will move forward and test Hive.

Download one stable release from apache site, and decompress it.

wget http://mirror.metrocast.net/apache//hive/stable/hive-0.6.0.tar.gz

tar -xvf hive-0.6.0.tar.gz
cp –rf hive-0.6.0 /usr/lib/hive
export PATH=$PATH:/usr/lib/hive/bin
#need to setup the hadoop home variable
export HADOOP_HOME=/usr/lib/hadoop

then you can run hive command shell,

the hive here is just a standard java application, here is all the arguments to start hive shell,

/usr/lib/jvm/java/bin/java -Xmx4096m -Dhadoop.log.dir=/usr/lib/hadoop/logs -Dhadoop.log.file=hadoop.log -Dhadoop.home.dir=/usr/lib/hadoop -Dhadoop.id.str= -Dhadoop.root.logger=INFO,console -Djava.library.path=/usr/lib/hadoop/lib/native/Linux-amd64-64 -Dhadoop.policy.file=hadoop-policy.xml -classpath "all the jars" org.apache.hadoop.util.RunJar /usr/lib/hive/lib/hive-cli-0.6.0.jar org.apache.hadoop.hive.cli.CliDriver

the clidriver is a standard java application to process users commands

By default , hive use the java derby as the store to keep meta data.

given the raw input like this

"01","35004","AL","ACMAR",86.51557,33.584132,6055,0.001499

we will create a table called rawtable first just with one column to hold all the data, using the syntax of creating table http://wiki.apache.org/hadoop/Hive/HiveQL

Create table rawtable(raw string);
load data inpath ‘/user/hadpoop/input’ into table rawtable;

when you run a simple query, (select count ( *) from rawtable, you can tell the mr job is invoked to get the results.

if you check the hdfs file systems, every table you just created will be stored under /user/hive/warehouse/[tablename]

After that, we can write quite complex query like join, subquery, check it out here, http://wiki.apache.org/hadoop/Hive/HiveQL

Besides, CLI , you can run ‘hive –service hwi’ to expose a simple web interface for data accessing, http://home:9999/hwi

Also, we can start a hive server

remote user can use the thrift api or JDBC to connect this DB powered by Hive.

ADB Driver for HTC wildfire and T-Mobile G2

2011-03-17T12:03:00.001-07:00

when you see the blog, you may have the problem to install driver for those two Phones, why?

the download driver might be old and doesn’t contain the information about new released Phones.

you can open the USB driver folder, there is one file [android_winusb.inf], here is mine by default. two sections are highlighted

[Google.NTx86]

; HTC Dream
%SingleAdbInterface%        = USB_Install, USB\VID_0BB4&PID_0C01
%CompositeAdbInterface%     = USB_Install, USB\VID_0BB4&PID_0C02&MI_01
%SingleBootLoaderInterface% = USB_Install, USB\VID_0BB4&PID_0FFF
; HTC Magic
%CompositeAdbInterface%     = USB_Install, USB\VID_0BB4&PID_0C03&MI_01
;
;Moto Sholes
%SingleAdbInterface%        = USB_Install, USB\VID_22B8&PID_41DB
%CompositeAdbInterface%     = USB_Install, USB\VID_22B8&PID_41DB&MI_01
;
;Google NexusOne
%SingleAdbInterface%        = USB_Install, USB\VID_18D1&PID_0D02
%CompositeAdbInterface%     = USB_Install, USB\VID_18D1&PID_0D02&MI_01
%SingleAdbInterface%        = USB_Install, USB\VID_18D1&PID_4E11
%CompositeAdbInterface%     = USB_Install, USB\VID_18D1&PID_4E12&MI_01

[Google.NTamd64]
; HTC Dream
%SingleAdbInterface%        = USB_Install, USB\VID_0BB4&PID_0C01
%CompositeAdbInterface%     = USB_Install, USB\VID_0BB4&PID_0C02&MI_01
%SingleBootLoaderInterface% = USB_Install, USB\VID_0BB4&PID_0FFF
; HTC Magic
%CompositeAdbInterface%     = USB_Install, USB\VID_0BB4&PID_0C03&MI_01
;
;Moto Sholes
%SingleAdbInterface%        = USB_Install, USB\VID_22B8&PID_41DB
%CompositeAdbInterface%     = USB_Install, USB\VID_22B8&PID_41DB&MI_01
;
;Google NexusOne
%SingleAdbInterface%        = USB_Install, USB\VID_18D1&PID_0D02
%CompositeAdbInterface%     = USB_Install, USB\VID_18D1&PID_0D02&MI_01
%SingleAdbInterface%        = USB_Install, USB\VID_18D1&PID_4E11
%CompositeAdbInterface%     = USB_Install, USB\VID_18D1&PID_4E12&MI_01

All you need to do is manually add the Adbinterface information to both seconds, then install the driver again.

; HTC Wildfire
%SingleAdbInterface% = USB_Install, USB\VID_0BB4&PID_0C8B
%CompositeAdbInterface% = USB_Install, USB\VID_0BB4&PID_0C8B&MI_01

;T-Mobile G2
%SingleAdbInterface% = USB_Install, USB\VID_0BB4&PID_0C91
%CompositeAdbInterface% = USB_Install, USB\VID_0BB4&PID_0C91&MI_01

Android Debugging, Start the Gallery App programmatically, decode/decompress AndroidManifest.xml

2011-03-15T18:02:00.001-07:00

I just create one code snippet to save one Image from internet to SD card using the follow code, after that, I want to open the gallery app to view this Image.

URL url = new URL(address);
Object content = url.getContent();
InputStream is = (InputStream)content;

BufferedInputStream bis = new BufferedInputStream(is);
Bitmap bm = BitmapFactory.decodeStream(bis);
bis.close();
is.close();
iv.setImageBitmap(bm); //bind to UI

String sdurl = Media.insertImage(getContentResolver(), bm, "pic1", null);
//url will be, content://media/external/images/media/29

How to Start the Gallery app and show this Image?

Intent intent = new Intent(Intent.ACTION_VIEW);
intent.setData( Uri.parse(temp));

this.startActivity(intent);

If you don’t know how to start a system activity, here is the trick, using the aapktool
basically, pull the apk from the Phone using the DDMS file browser plug-in, here I am pulling out the gallery apk located in /syste/app/*.apk

Download the extract the appktool. then run aapt.
here is the trick, all activity has the AndroidManifest.xml, in that xml, you can find all the declared intent filters to start this activity. By default , this file is compressed. so you need the appt tool to decompress it. here is how.

aapt dump xmltree Gallery3DGoogle.apk AndroidManifest.xml >a.txt

then in the a.txt, you can see all the declared intent. I just pickup the view one.

How to : install and configure DRBD on two Centos Vms tutorial

2011-03-08T15:07:00.001-08:00

DRBD is a Linux Kernel module that constitutes a distributed storage system. You can
use DRBD to share block devices between Linux servers and, in turn, share file systems and data.
in a nutshell, Data change on Master Node will be synced to Slave node in the real time mostly. changes on master will be captured on the kernel level, then push the change to slave node and get network ack when nodes are connected. If network is broken, the data change will be queued locally and flush to remote node when it get online.

Here , I will install and configure a two nodes cluster using two centos vm. we need make sure both VM are network-reachable. then just install, make and config steps.

Let’s say I have two Nodes, they are Home and LA. I will start the setup from Home

On Both Nodes, expand the Disk to add more space and create a new partition hda4 with the same size (not required.)

expand the disk space , here i will expend 1G more space.

then create the partition hda4

Also make sure each node has two NIC card for failover.

Now, we will use a GUI tool to install and config the drbd cluster, it’s called DRBD-MC which is a standard java application. It will call yum to install the drbd and push the change the those conf files when user click the gui.

Download the jar file from http://oss.linbit.com/drbd-mc/, drbc-mc.jar and run the jar file on master node.

java –jar ~/Desktop/drmc-*.jar

the main screen ,

Click the Host wizard and add one master node called Home and enter the root password.
here is the sweet set of the tool, it prerequisites component is missing, just click install to bring it in.

here is the generated script to install pacemaker

wget -N -nd -P /etc/yum.repos.d/ http://www.clusterlabs.org/rpm/epel-5/clusterlabs.repo && rpm -Uvh http://download.fedora.redhat.com/pub/epel/5/i386/epel-release-5-4.noarch.rpm ; yum -y -x resource-agents-3.* -x openais-1.* -x openais-O.9* -x heartbeat-2.1.* install pacemaker.x86_64 heartbeat.x86_64 && /sbin/chkconfig --add heartbeat

after install drbd and necessary dependency component like pacemaker, click Cluster wizard to create a cluster. I will name it helloworld

And select two available hosts

chose the default communication protocol.

Click to add two interfaces

Create the create config button to generate the conf file

after done, click the storge manu, chose the hda4 partition, click the action manu to select the LA mirror device

give it a resource name hda4

click next then you need to create a metadata, here we will keep the meta data into the device itself

then chose Home as the primary node, and create a ext3 filesystem

Now the device will be synced from home node to la node.

you can also run service drbd status. or just cat /porc/drbd

from the secondary node, you can tell its role is secondary.

then wait for the sync to be done.

Now, we can mount the /dev/drbd0 device to a file system.
mkdir /mnt/drbd/hda4
mount /dev/drbd0 /mnt/drbd/hda4

then every change to /mnt/drbd/hda4 folder will be synced to secondary node.
Please note here the secondary is read-only mode, you can’t mount the system unless you switch the role to primary node.

let’s create some file on master first.
touch helloworld.master

then on the primary node, bring down it’s interface to simulate a network error.
ifdown eth0
ifdown eth1
and on the slave node, change its role to primary.

the cluster is in disconnected state, let’s switch secondary node to take over the primary role

drbdadm primary hda4, now secondary has the primary role.

mount /dev/drbd0 /mnt/drbd/hda4
then we can mount the device and see the helloworld.master is there.

As I say, drbd is one low level service to enable block level replication. we can use this mechanism to replicate the DB binary log of mysql or wal log of postgres , then we can get some level of High availability.

If we want to switch back the role ( now LA is the primary node, Home is unknow. )

one LA Node. (invalidate the primary role.)

On Home Node , just turn on the network service, ifup eth0, ifup eth1. and run “drbdadm connect hda4”
all set, role switched back to initial setting. Home is Primary, La is secondary.

Hope it helps.

How To : using Postgresql C# Client Npgsql to access postgresql database

2011-03-07T16:46:00.001-08:00

After How to : install and test PostgreSql on Centos, we need to change several things to enable C# Client Access to the remote DB.

Create a User (login) with a password
Change the pg_hba.conf to turn on the user access from the giving IP range. change postgres.conf to change the listening ip address to * instead of just localhost by default.
install Npgsql .net assembly on the client machine and write C# code.

To Create a user demo with password demo

\h Create User # \h will list the help tips for every command
Create User demo superuser login password ’demo’;
after that, you can query pg_shdow system table to list all users and their hashed password

PG_HBA.conf is located in the pgdata directory, here it will /usr/local/pgsql/data
add one line in the top , to enable demo user from host 192.168.209.1 to access all db with the md5 credential.

host all demo 192.168.209.1/32 md5

for postgresql.conf, change the listen address to * include all ips

#listen_addresses = 'localhost'
listen_addresses = '*'

the restart the service. “ pg_ctl -D /usr/local/pgsql/data/ restart”

Download the NPGsql .net library, create a C# Console application and reference the download Assembly Npgsql.dll

all the NPGXXX class are simply the implantation of ado.net standard interface.

Here is 10 lines of code to connect to remote postgresql db hellodb and do a basic query.

string connString = "Server=192.168.209.130;port=5432;user id=demo;password=demo;database=hellodb";

using (NpgsqlConnection conn = new NpgsqlConnection(connString))
{
conn.Open();

NpgsqlCommand cmd = new NpgsqlCommand("SELECT name, countrycode from City limit 10", conn);
NpgsqlDataReader reader = cmd.ExecuteReader();
while (reader.Read())
{
Console.WriteLine("City " + reader.GetString(0) + "Code " + reader.GetString(1));
}
}

hit f5 to run

to my suprice, it use its own data protocol.

How to : install and test PostgreSql on Centos

2011-03-07T15:14:00.001-08:00

basically, just download and config , then make && make install.

Download the latest and stable source from Postgresql Site

wget http://wwwmaster.postgresql.org/redir/407/h/source/v9.0.3/postgresql-9.0.3.tar.gz

Unzip the tarball file.

tar -zxf postgresql-9.0.3.tar.gz

run the config, make sure prerequisites are there

yum install openssl-devel
yum install zlib-devel
yum install gnu-crypto
yum install readline-devel
yum install gcc-objc++

./configure –help # list all options to run the compilation, you can override the installation folder using the –-pre-fix options.

Fine tuning of the installation directories:
--bindir=DIR            user executables [EPREFIX/bin]
--sbindir=DIR           system admin executables [EPREFIX/sbin]
--libexecdir=DIR        program executables [EPREFIX/libexec]
--sysconfdir=DIR        read-only single-machine data [PREFIX/etc]
--sharedstatedir=DIR    modifiable architecture-independent data [PREFIX/com]
--localstatedir=DIR     modifiable single-machine data [PREFIX/var]
--libdir=DIR            object code libraries [EPREFIX/lib]
--includedir=DIR        C header files [PREFIX/include]
--oldincludedir=DIR     C header files for non-gcc [/usr/include]
--datarootdir=DIR       read-only arch.-independent data root [PREFIX/share]
--datadir=DIR           read-only architecture-independent data [DATAROOTDIR]
--infodir=DIR           info documentation [DATAROOTDIR/info]
--localedir=DIR         locale-dependent data [DATAROOTDIR/locale]
--mandir=DIR            man documentation [DATAROOTDIR/man]
--docdir=DIR            documentation root [DATAROOTDIR/doc/postgresql]
--htmldir=DIR           html documentation [DOCDIR]
--dvidir=DIR            dvi documentation [DOCDIR]
--pdfdir=DIR            pdf documentation [DOCDIR]
--psdir=DIR             ps documentation [DOCDIR]

you can also just run ./configure , it will pick up all default options.

make and make install

make && make install

make sure it’s installed without error, add the bin folder to profile , then we can initialize the db.

In /etc/profile or ~/.profile
PATH="/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/pgsql/bin"
export PATH

Create postgres user and init the db

useradd postgres
mkdir /usr/local/psql/data
chown postgres /usr/local/psql/data

[root@home pgsql]# su - postgres
[postgres@home ~]$ initdb --pgdata=/usr/local/pgsql/data

start the service and client

pg_ctl -D /usr/local/pgsql/data start

Now, DB server is ready, we can use the postgres which is a superuser to create some database.

Su – postgres
psql
\l will list all the database.

Create database HelloDB;
Create database helloDB;
#connect helloDb
\c helloDB;

Here,let’s download some sample data from http://pgfoundry.org/projects/dbsamples/
world-1.0.tar.gz
upzip and use \I to install the db from the sql .

table city, will list all rows in city table.

I will write a C# application using the standard ado.net library to read the city table.

How To : using Postgresql C# Client Npgsql to access postgresql database

How to: create a hadoop map/reduce job tutorial

2011-02-21T17:39:00.001-08:00

After we install and config hadoop to run in fully distributed Mode, Centos, now it’s time to write a hello world program to run on top of the hadoop cluster in distributed mode.

I use the 1.8M text file downloaded from http://www.census.gov/tiger/tms/gazetteer/zips.txt, here is a file layout.

"01","35004","AL","ACMAR",86.51557,33.584132,6055,0.001499
"01","35005","AL","ADAMSVILLE",86.959727,33.588437,10616,0.002627
"01","35006","AL","ADGER",87.167455,33.434277,3205,0.000793
"01","35007","AL","KEYSTONE",86.812861,33.236868,14218,0.003519
"01","35010","AL","NEW SITE",85.951086,32.941445,19942,0.004935
"01","35014","AL","ALPINE",86.208934,33.331165,3062,0.000758
"01","35016","AL","ARAB",86.489638,34.328339,13650,0.003378
"01","35019","AL","BAILEYTON",86.621299,34.268298,1781,0.000441
"01","35020","AL","BESSEMER",86.947547,33.409002,40549,0.010035
"01","35023","AL","HUEYTOWN",86.999607,33.414625,39677,0.00982
"01","35031","AL","BLOUNTSVILLE",86.568628,34.092937,9058,0.002242

we will write a map reduce job to do a basic summary to list the number of citys in each State.

some configuration change,

Since the file is only 1.8M and we have two datanodes(task nodes), we will override the default block size to 500K, so that two task nodes can run at the same time.
>>>hdfs-site.xml

<property>
<name>dfs.block.size</name>
<value>512000</value>
<description>The default block size for new files.</description>
</property>

then we use the hadoop command line to push the file to the cluster.

hadoop fs –mkdir raw
hadoop fs –copyFromLocal zip.txt raw

then from the name web interface, we can tell the zip.txt has 4 Blocks

Now time to write a simple map/reduce java program. use your favorite IDE eclipse , create a java project and add the lib of those hadoo*.jar to the project builder lib. and here is my basic program. I put some comments, here

public class statecount extends Configured implements Tool {

@Override
public int run(String[] args) throws Exception {

    if (args != null && args.length != 2) {
        System.out.println("statecount   <input> <output>");
        ToolRunner.printGenericCommandUsage(System.out);
        return -1;
    }

    Configuration conf = this.getConf();
    JobConf jobconf = new JobConf(conf, statecount.class);
    Path in = new Path(args[0]);
    Path out = new Path(args[1]);

    jobconf.setJobName("Hello world/City count by State");
    jobconf.setMapperClass(SMapClass.class);
    jobconf.setReducerClass(SMapClass.class);

FileInputFormat.setInputPaths(jobconf, in);
FileOutputFormat.setOutputPath(jobconf, out);

jobconf.setInputFormat(TextInputFormat.class);
jobconf.setOutputFormat(TextOutputFormat.class);

jobconf.setMapOutputKeyClass(Text.class);
jobconf.setMapOutputValueClass(LongWritable.class);

jobconf.setOutputKeyClass(Text.class);
jobconf.setOutputValueClass(Text.class);

JobClient.runJob(jobconf);

return 0;
}

public static void main(String[] args) throws Exception {
int res = ToolRunner.run(new Configuration(), new statecount(), args);
System.exit(res);
}

public static class SMapClass extends MapReduceBase implements
Mapper<LongWritable, Text, Text, LongWritable>,
Reducer<Text, LongWritable, Text, Text> {

@Override
public void map(LongWritable key, Text value,
        OutputCollector<Text, LongWritable> output,
        Reporter paramReporter) throws IOException {
    // "01","35004","AL","ACMAR",86.51557,33.584132,6055,0.001499

    String s = value.toString();
    output.collect(new Text(s.split(",")[2]), new LongWritable(1));
    //Collect AL, 1, here
    //get the third value which will be the State
}

@Override
public void reduce(Text key, Iterator<LongWritable> values,
OutputCollector<Text, Text> output, Reporter paramReporter)
throws IOException {

    long i = 0;
    while (values.hasNext()) {
        i += 1;
        Object o = values.next(); //I forget this line, there will be deadloop, reduce never over:(

    }
    output.collect(key, new Text("#number of citys" + i));

}}}

then click the export menu to export classes to a jar file , then copy to the name server.
run “hadoop jar helloworld.jar statecount raw/zip.txt ouputraw” on name server(job track server.)

from 50030, we can see two task nodes are running

whole statistics,

From the console you sumit job, you can see the job output.

after done, go to filesystem browser from :50070 web portal. we can see the final result.

if you want to make sure that reduce task is distributed, you can put a hostname into the reduce job and run again.

output.collect(key, new Text("#number of citys" + i + "on host" + InetAddress.getLocalHost().getHostName() ));

and assign the reduce tasks

jobconf.setJobName("Hello world/City count by State");
    jobconf.setMapperClass(SMapClass.class);
    jobconf.setReducerClass(SMapClass.class);
   jobconf.setNumReduceTasks(4);

in the output hdfs folder, you will see four files.

click one file, this reduce job is executed on host Home

Another one is one Host LA

More Hadoop Blogs

how to :eclipse win64 , java.lang.UnsatisfiedLinkError: Can't load library: /C:/eclipse/eclipse64/eclipse/configuration/org.eclipse.osgi/bundles/292/1/.cp/lib/linux/x86/libjd-eclipse.so

2011-02-21T16:24:00.001-08:00

I found it’s a very tricky issue when you are using jdeclipse add-in, and switching between win 32 and win 64.

one day, I get this error, as attached,
eclipse win64 , java.lang.UnsatisfiedLinkError: Can't load library: /C:/eclipse/eclipse64/eclipse/configuration/org.eclipse.osgi/bundles/292/1/.cp/lib/linux/x86/libjd-eclipse.so

How to fix that.

Make sure you download the Right version of Eclipse, on Windows 64 , please download the 64 bit version eclipse and using the 64 bit JVM.
( in the eclipse.int, put a –vm pathyour64bitjre/bin/java.exe”)
After you finish the Eclipse download, Download the jdeclipse http://java.decompiler.free.fr/jd-eclipse/update/jdeclipse_update_site.zip and unzip the file to eclipse folder.
When you open a Workspace which is generated by 32 bit eclipse, Please chose a new workspace, or manually edit the pref file [workspace]\.metadata\.plugins\org.eclipse.core.runtime\.settings, replace x86 with x86_64.
otherwise, you will be in a situation that 64 bit host to load 32 bit dll, ( eclipse figure out a wrong system architecture., cached by previous editor)

Mac OSX stacks on Windows 7

2011-02-20T10:15:00.000-08:00

A stack is a Dock item that gives you fast access to a folder. When you click a Stack, the files within spring from the Dock in a fan or a grid, depending on the number of items (or the preference you set). Snow Leopard starts you off with three default Stacks: one for Applications, one for downloads, and the other for documents.

here is a application stack,

If you need the same feature in windows 7, there is one free add on called 7Stack. you can get some effect like this

Steps, Download the 7Stack

create one stack, basically Chose one folder, and assign the options, then click generate , it will create one shortcut. you can drag and drop the shortcut to the taskbar.

How to: install and config hadoop to run in fully distributed Mode, Centos

2011-02-17T15:20:00.001-08:00

in order to run the hadoop in fully distributed mode, at least we need two VMs. I will config a hadoop cluster with two Data Nodes , one namenode and secondary namenode.

MachineName	Role
Home (192.168.209.130)	Primary + Secondary Namenode, Slave(data Node), Job Tracker Node
LA (192.168.209.132)	Slave Node (Data Node)

there are some prerequisites to run hadoop,

Create Hadoop user and group on each Server.
on Home Node, generate the ssh key and export the pub key to both servers. hadoop user on home node can access each server (login without password)
install java 1.6
configuration change. (hosts file, hadoop conf directory.)

for the configuration, basically 3 xml located in the hadoop/conf directory.

HDFS-Site.xml, the essential configration to HDFS, since we only have two data nodes, change the replciation factor to 2. and assign the path of the name server, you ‘d better keep the file into the hadoop directory, or somewhere else make sure hadoop can update that folder.

<configuration>
<property>
    <name>dfs.replication</name>
    <value>2</value>
</property>
<property>
     <name>dfs.permissions</name>
     <value>false</value>
</property>
<property>
     
     <name>dfs.name.dir</name>
    <value>/usr/lib/hadoop/cache/hadoop/dfs/name</value>
</property>
</configuration>

Core-site.xml, assign the cluster name, I use name server Home here

<configuration>
<property>
<name>fs.default.name</name>
<value>hdfs://Home:8020</value>
</property>

<property>
<name>hadoop.tmp.dir</name>
<value>/usr/lib/hadoop/cache/${user.name}</value>
</property>
</configuration>
~

mapred-site.xml

<configuration>
<property>
<name>mapred.job.tracker</name>
<value>home:8021</value>
</property>
</configuration>
~

then How can we tell the cluster what are those members? the magic is the masters and slavers file under conf directory

masters
just put the secondary name server, I will put Home here

slavers,
both servers, home and LA

After that, copy the hadoop directory with permission to LA server.

then on the name server , here will be home, run “bin/hadoop nameserver –format”
once done, make sure no error.

time to start up the Clusters.
How to do that, just run “bin/start-all.sh”, the script will check all those server list for their roles, and ssh to that server to start the JVM.

Basically, as I mentationed in the beginning, in Home server just 5 role with each as a jvm process.
here is the 5 jvm argument.

NameNode	java -Dhadoop.log.dir=/usr/lib/hadoop/bin/../logs -Dhadoop.log.file=hadoop-hadoop-namenode-home.log -Dhadoop.home.dir=/usr/lib/hadoop/bin/.. -Dhadoop.id.str=hadoop -Dhadoop.root.logger=INFO,DRFA -Djava.library.path=/usr/lib/hadoop/bin/../lib/native/Linux-amd64-64 -Dhadoop.policy.file=hadoop-policy.xml -classpath org.apache.hadoop.hdfs.server.namenode.NameNode
data node	java -Dhadoop.log.dir=/usr/lib/hadoop/bin/../logs -Dhadoop.log.file=hadoop-hadoop-datanode-home.log -Dhadoop.home.dir=/usr/lib/hadoop/bin/.. -Dhadoop.id.str=hadoop -Dhadoop.root.logger=INFO,DRFA -Djava.library.path=/usr/lib/hadoop/bin/../lib/native/Linux-amd64-64 -Dhadoop.policy.file=hadoop-policy.xml -classpath org.apache.hadoop.hdfs.server.datanode.DataNode
secondary name node	java -Dhadoop.log.dir=/usr/lib/hadoop/bin/../logs -Dhadoop.log.file=hadoop-hadoop-secondarynamenode-home.log -Dhadoop.home.dir=/usr/lib/hadoop/bin/.. -Dhadoop.id.str=hadoop -Dhadoop.root.logger=INFO,DRFA -Djava.library.path=/usr/lib/hadoop/bin/../lib/native/Linux-amd64-64 -Dhadoop.policy.file=hadoop-policy.xml -classpath org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode
jobtrackr	java -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote -Dhadoop.log.dir=/usr/lib/hadoop/bin/../logs -Dhadoop.log.file=hadoop-hadoop-jobtracker-home.log -Dhadoop.home.dir=/usr/lib/hadoop/bin/.. -Dhadoop.id.str=hadoop -Dhadoop.root.logger=INFO,DRFA -Djava.library.path=/usr/lib/hadoop/bin/../lib/native/Linux-amd64-64 -Dhadoop.policy.file=hadoop-policy.xml -classpath org.apache.hadoop.mapred.JobTracker
tasktrackr	java -Xmx1000m -Dhadoop.log.dir=/usr/lib/hadoop/bin/../logs -Dhadoop.log.file=hadoop-hadoop-tasktracker-home.log -Dhadoop.home.dir=/usr/lib/hadoop/bin/.. -Dhadoop.id.str=hadoop -Dhadoop.root.logger=INFO,DRFA -Djava.library.path=/usr/lib/hadoop/bin/../lib/native/Linux-amd64-64 -Dhadoop.policy.file=hadoop-policy.xml -classpath org.apache.hadoop.mapred.TaskTracker

once done, you can access http://home:50070, to see the cluster is running with two nodes.

More Hadoop Blogs

How to : setup SSH authentication over keys, hadoop installation

2011-02-16T15:55:00.001-08:00

When you install hadoop, you will required to create one user hadoop with group hadoop. and setup the SSH keys that it can login to the server without enter a password.

#Create one group hadoop
groupadd hadoop
#create one user hadoop, create home directory
useradd –g hadoop –-create-home hadoop
#login as hadoop
su – hadoop
#generate ssh public/private keys
ssh-keygen –t rsa –P ‘’
# cp the public key to authorized_keys
cat .ssh/id_rsa.pub >> .ssh/authorized_keys
if you want to enable hadopp to access another server without login using ssh,
cat the .put key to the server authorized_keys files
#remember to setup the permission on the authorized_keys , that only owner can read/write, 600 permission, if you missed this step, you will be asked for password, even you setup the pub key to be trusted.
chmod 600 .ssh/authorized_keys
Now, you can login to localhost without password

whole steps as captured.

More Hadoop Blogs

Decode Oracle Coherence cache extend protocol, Tangosol.Net.RequestTimeoutException

2011-02-16T11:55:00.001-08:00

on day, C# applications which use coherence cache extend get an Tangosol.Net.RequestTimeoutException. I spent a lot time to figure out what happened? what’s the potential reason of the requestimeout. whole stack trace

Tangosol.Net.RequestTimeoutException: request timed out after 30000 millis at Tangosol.Net.Messaging.Impl.Request.RequestStatus.get_Response()

at Tangosol.Net.Messaging.Impl.Request.RequestStatus.WaitForResponse(Int64 millis)

at Tangosol.Net.Messaging.Impl.Channel.Request(IRequest request, Int64 millis)

at Tangosol.Net.Messaging.Impl.Channel.Request(IRequest request)

at Tangosol.Net.Impl.RemoteNamedCache.BinaryNamedCache.get_Item(Object key)

at Tangosol.Util.ConverterCollections.ConverterDictionary.get_Item(Object key)

at Tangosol.Net.Impl.RemoteNamedCache.get_Item(Object key)

at Tangosol.Net.Impl.SafeNamedCache.get_Item(Object key)

at CacheClient.CustomerProfile.button12_Click(Object sende

Question 1, is it a network problem? C# application will talk to proxy node (JVM on another machine), then proxy node talk to storage node to get the data back.
then I capture the network traffic between C# application and the Proxy jvm node. you can use tcpview to figure out which proxy node has the TCP connection established with the client.

the traffic looks good, request out, and response immediately get back. (so no firewall blackout, no package drop)

here, 10.1.111.22 is the client running C# application. It send out the request to Proxy Node (10.1.25.151)
then after 0.024 seconds, it get the response back.(client send out ack in package 7)

all looks great on network level. Then it comes to my second question, are those data valid in the return.

in this special case, the server returns 38 bytes array here it will be

25:90:d0:92:cb:05:00:02:00:6b:03:4c:18:15:a4:37:01:00:4e:04:74:65:73:74:01:42:cf:ff:ce:91:b8:f7:b1:b2:ee:01:40:40

what the encoding mechanism, I can only figure out 1st byte 25 which is 37 means that package data length is 38-1=37, how about the rest?

then I use the .net reflector to read the Coherence.dll. and figure out the raw format.

[datalength][chennelid][typeid][versionid][objectitself]

it used packed int32 format here.
I write a Simple c# program. here to decode the channel id and tyepid/versionid

String s =”25:90:d0:92:cb:05:00:02:00:6b:03:4c:18:15:a4:37:01:00:4e:04:74:65:73:74:01:42:cf:ff:ce:91:b8:f7:b1:b2:ee:01:40:40”;
byte[] data = new byte[(s.Length + 1) / 3];
for (int i = 0; i < data.Length; i++)
{
data[i] = (byte)
( "0123456789ABCDEF".IndexOf(s[i * 3]) * 16 + "0123456789ABCDEF".IndexOf(s[i * 3 + 1]));
}

MemoryStream ms = new MemoryStream(data);

DataReader dr = new DataReader(ms);
string msg = string.Format("Package Length {0} \n Channel ID {1} \n TypeID {2}\n Version id {3} ",
dr.ReadPackedInt32(), dr.ReadPackedInt32(), dr.ReadPackedInt32(), dr.ReadPackedInt32());
MessageBox.Show(msg);

When run the code, you will see the decoded value.

it you try decode the output package sending to proxy, it has the same naming format.

Client send a package with the Channel ID and receive the response with channelID, if the channel state is maintained incorrectly in the client side, you will get the exception>
here the reflector code for client decode message.

if the channel =null or is closed, even server returns back the response. it will cause the requestimeoutexception.
stupid code;(

you can also do the inspecting on the runtime level.
Define one filter.

public class MyFilterDump : IWrapperStreamFactory
    {
        public MyFilterDump()
        {

        }
        public System.IO.Stream GetInputStream(System.IO.Stream stream)
        {
            System.Diagnostics.Debug.WriteLine("Get Response " + stream.Length);

            DataReader dr = new DataReader(stream);
            System.Diagnostics.Debug.WriteLine("Chennel ID " + dr.ReadPackedInt32());

            stream.Seek(0, SeekOrigin.Begin);
            return stream;
        }

        public System.IO.Stream GetOutputStream(System.IO.Stream stream)
        {
            System.Diagnostics.Debug.WriteLine("Send Request " + stream.Length);
            return stream;
            //did the same thing on input
        }
    }

then put it into client-coherence.xml and client-cache-control.xml

<coherence xmlns="http://schemas.tangosol.com/coherence">
<cluster-config>
<filters>
<filter>
<filter-name>debugfilter</filter-name>
<filter-class>ExtendLib.MyFilterDump, ExtendLib</filter-class>
</filter>
</filters>

….

client-cache-control.xml

<remote-cache-scheme>
      <scheme-name>extend-dist</scheme-name>
      <service-name>ExtendTcpCacheService</service-name>
      <initiator-config>
        <tcp-initiator>
          <remote-addresses>
            <socket-address>
              <address>localhost</address>
              <port>9999</port>
            </socket-address>

          </remote-addresses>
        </tcp-initiator>

        <outgoing-message-handler>
        <request-timeout>30s</request-timeout>
      </outgoing-message-handler>
    <use-filters>
        <filter-name>debugfilter</filter-name>
        </use-filters>

Have fun@

How to: resize expand partition on Centos hosted by Vmplayer

2011-02-14T16:26:00.001-08:00

when you limited size warrning, run “df-h”, 80% is used on /, we will span the /dev/mapper/volgroup00-logvol00 here.

display LVM , “lvdisplay”, there are one volum group volGroup00 with two logic volumes 00 and 01

just one disk with 10G, shut down the VM, and click the utility to expand the disk to 13G

run “fdisk /dev/had” to add a lvm partition to the new expended disk.
p –>print current partition
n-> add one new partition with primary partation
t-> toggle the partition type to 8e which is lvm

press W to save the partition

after done, Run Pvcreate to create a new LVM.

if you get error like disk us not found , just reboot the vm

Extend the VolGroup00 , add /dev/hda3 to the group, and extend the /dev/VolGroup00/LogVol00 to the new lvm hda3

run resize2fs , refresh the change. run df –h again, you will see free space is increased.

how to install and configure Nginx + PHP FPM

2011-02-10T11:49:00.001-08:00

Building PHP with FPM Support,

# ./configure --enable-fpm --prefix=/opt/phpfpm

Make
Make Install

then you can run the php-fpm by run “php-fpm ”, php-fpm is located in the sbin dir
if you get error like unable to find the conf file like following ,

just rename /opt/phpfmp/etc/php-fpm.conf.default as php-fpm.conf , change the min/max/startserver value

Also copy the /tmp/php-x/php.ini-production to /opt/phpfpm/php.ini
run netstat –an |grep 9000

Now install nginx following install and config Nginx/PhP fastcgi on Centos
change the nginx.conf to include fpm for php backend proxy processing. just uncomment the default conf settings,

location ~ \.php$ {
        root           html;
        fastcgi_pass   127.0.0.1:9000;
        fastcgi_index index.php;
        fastcgi_param SCRIPT_FILENAME /opt/nginx/html/$fastcgi_script_name;
        include        fastcgi_params;
    }

restart nginx –s restart

then create one info.php and put it under /opt/ngnix/html

<?php phpinfo();?>

Now, you can access http://localhost/info.php, you will get php info page.

Here is the raw traffic between Nginx and PhpFPM

If you get 404 error no matter how, check the nginx.conf point to the absolute path of the script

fastcgi_param SCRIPT_FILENAME /opt/nginx/html/$fastcgi_script_name;

How to : install and config Nginx/PhP fastcgi on Centos

2011-02-09T10:51:00.001-08:00

To Install Nginx, setup the prerequisites as following.

yum install GCC #C compiler
yum install pcre* # perl regular expression
yum install zlib zlib-devel
yum install openssl openssl-devel

Download Nginx From http://nginx.org/en/download.html, and extract to a source folder, I will use /tmp here

wget http://nginx.org/download/nginx-0.8.54.tar.gz
tar -zxvf nginx-0.8.54.tar.gz
cd /tmp/nginx-0.8.54

./configure --prefix=/opt/nginx
make && make install

once done, you can run ./nginx –? to show all available options ,

for the default loading, it read the conf file which is conf/nginx.conf

to start the nginx, just run ./nginx, you will get the helloworld page.

here, I changed ngingx.conf a little with a New port 88, and basic proxy forward to Apache,PHP (check installation instructions, Install PHP to Apache web server)

server {
listen 88;
server_name localhost;

#charset koi8-r;

#access_log logs/host.access.log main;

        location / {
            root   html;
            index index.html index.htm;
        }

#error_page 404 /404.html;

        # redirect server error pages to the static page /50x.html
        #
        error_page   500 502 503 504 /50x.html;
        location = /50x.html {
            root   html;
        }

        # proxy the PHP scripts to Apache listening on 127.0.0.1:80
        #
       location ~ \.php$ {
            proxy_pass   http://127.0.0.1;
        }

if I access http://localhost:88/info.php, nginx will forward the request to http://localhost/info.php

But the server header is nginx Now. Also the ngnix has enabled the GZIP support now. ( a great reverse proxy, offloading those cut-off features)

more how-tos:

How to: install apache 2.2 on centos

How To: Install PHP to Apache web server

2011-02-08T17:30:00.001-08:00

After install apache 2.2 on centos , there is no PHP Module built-in, You have to do the same thing like installing apache, Download the source file , compile and put the modules to the folder that will be picked up by apache. Here is the rough steps,

Download and extract the tarball from php.net, http://us.php.net/get/php-5.3.5.tar.gz/from/this/mirror

tar –zxvf php-5.3.5.tar.gz

go into the uncompressed php folder and run the config

./configure –with-apxs2=/opt/apache/bin/apxs –-prefix=/opt/php
if needed, yum install libevent-devel libxml2-devel

Compile and Test

make && make test

Install the module to the modules folder

make install
this will automatically upate apache’s conf/httpd.conf with
LoadModule php5_module modules/libphp5.so
Add the handler of php manully.
AddType application/x-httpd-php .php

restart the apache daemon

TO test, create a single line file to htdocs/test.php

<?php phpinfo();?>

when access that url , you will get the information of php settings, now you are good to go.

How to: Test the apache syslog with the free kiwi syslog server

2011-02-08T10:44:00.001-08:00

Let’s say you have two OS, one in windows which will host the syslog server, and one Centos OS as the container of Apache , here is the tutorial to test the syslog logging. both on server side (view the logs) and and client side (turn on logging to syslog)

Download and install one free version of kiwi syslog

when you run, you will get the GUI to show and filter logs.

when the log server runs, it will listen on the UDP port 514.

on the Centos, OS level, to Point lone log facility to our kiwi log server

vi /etc/syslog.conf
#Add one line

localcal6.* @192.168.209.1

#restart syslog server
service syslog restart # or killall –HUP syslogd

Now, you can test the syslog setting now, by write several logs to syslog

logger –p local6.info Hello,errorlog

from the kiwi server, you will see the log

For the Apache, change the httpd.conf, to point the errorlog and accesslog to our syslog facility local6

#ErrorLog "logs/error_log"
ErrorLog syslog:local6

#access Log
CustomLog "|/usr/bin/logger -p local6.info" combined

Restart the httpd , all logs will be dispatched to our syslog server. (here both error log and access log)

How to: install apache 2.2 on centos

2011-02-07T13:47:00.001-08:00

It’s pretty straightforward to install apache 2.2 on centos. download the source file and compile it, then make install.

cd /tmp
wget http://download.nextag.com/apache//httpd/httpd-2.2.17.tar.gz
tar zxvf httpd-2.2.17.tar.gz
then switch to the decompressed folder
./configure --prefix=/opt/apache
make
make install
#Done

After that, start the httpd, there is one apachectl utility to control the start/stop operation of httpd

/opt/apache/bin/apachectl start
#start the httpd daemon

then browse to localhost:80, you will get the It works page. (httpd is working and listens on port 80)

you can change the index.html located in the /opt/apache/htdocs/ to something else like hello world.

run “ps -aux|grep httpd”, you will see the processes.

To add the httpd to service.

ln -s /opt/apache/bin/apachectl /etc/init.d/apache2
add two lines to the /etc/init.d/apache2
#chkconfig:3 80 80
#description:my apache web server

add it to the chkconfig
chkconfig –add apache2

now you can query chkconfig –list apache2

linux putty SSH arrow key not working macbook ubantu, ^[[A

2011-02-06T09:53:00.001-08:00

Unlike One Simple Windows, there are a lot ad-hoc on Linux. I get a strang behavior with ubantu this Moring, the arrow key to show the last command history never work any more. it shows the escaped encoding,

Always, when I press the left arrow, it returns the last command history,. But not, it returns ^[[A

First make sure which shell you are using, by run “echo $0”

I get a stupid shell –su now. now switch to the bash by run “/bin/bash”

all works now

is it tricky?

also we can add it to the default profile or .bashrc

Copy MacBook MAC and Windows Partition from one Disk to another one

2011-01-22T22:03:00.001-08:00

I have one 60G SSD HD with the mac OS and bootcamp windows 7 64. at one day, I get a new 160G SSD, so need to find one way to copy all the bits in MAC and Windows to the new SSD HD.

first, copy the MAC Os from old disk to the new one. this is much easy compared with the windows partition. Just connect the second HD using the USB adapter, or external HD enclosure. boot the os to mac, and run the Disk Utility.
format the New HD to MAC journal format.

restore the existing MAC partition to the new HD. (drag and drop) That’s it.

For the windows partition, Need back up it to a external HD using the winclone, then restore.
Download and install the winclone(it’s free.)

Click preference, to disable the compressing.

Then click Image button to dump the windows partition to a DMG file.

It may take a while depending on your IO and CPU.

once done, Boot with the New HD to mac, create a windows partition using the bootcamp assistant.

then run the winclone to restore the windows partition to the new HD partition.

after that, done. for the first windows run, It may prompt you to run a diskcheck, you may skip it if you believe your HD is pretty reliable. Now you can boot the MAC/Win with the new HD.

Hello world EJB 3.1 with JBOSS

2011-01-12T16:21:00.001-08:00

I’ve been in the Microsoft world for a while, there are a lot distributed technology in the Microsoft stack. DCOM, COM+,Remoting, ASMX/WCF. What’s the equivalent term for EJB? I would say that’s dot net remoting.

for a typical remoting project, we need define the MBR Object, Host it ( single call, singleton or Client control activation.) using different options, (IIS or in-process hosting. the runtime will take care the proxy/sub stuff.)

here I will put a hello world EJB. Jboss will be the role of IIS.

like remoting,First define a Interface (shared as a contract between server and client. )

public interface ICalculatorBusinessObjects {
public int Add(int ... args);
}

then Implement a MBR like remoting.

@Stateless
@Remote
public class CalculatorBeanBase implements ICalculatorBusinessObjects {

    @Override
    public int Add(int... args) {
        // TODO Auto-generated method stub
        int i=0;
        System.out.println("------------------" + "get HIt on Server side, My hashcode" + this.hashCode() );
        for (int j : args) {
            i+=j;
        }
        return i;
    }

}

like the Microsoft attribute, those @xx are annotations. the runtime will check thos annotation and enforce some ad-hoc logic. like provide different hosting control, transaction control,etc.

Once Done, need deployed it to a container or server . for .net , it’s just a assembly or a dll. for J2ee, a zip file called EJB jar. you can click eclipse and export the build to a jar file firstEjb.jar

then deploy the jar to Jboss. drag and drop the jar to c:\jboss6\server\default\deploy folder. ( for standard configuration, c:\jboss6\server\all\deploy if you run with –c all option.)

the server discovers new bits and host it, also register a JDNI entry. like the remoting .rem stuff.

the ejb is hosted and ready for client connection now.

for client code, just find the proxy by querying the JNDI server and call the biz logic.

import javax.naming.Context;
import javax.naming.InitialContext;
import javax.naming.NamingException;

public class main {

    public static void main(String[] args) throws NamingException {
        // TODO Auto-generated method stub
        Context context=new InitialContext();
        ICalculatorBusinessObjects o=(ICalculatorBusinessObjects)context.lookup("java:CalculatorBeanBase/remote");
        System.out.println(o.getClass().getCanonicalName());
        System.out.println(o.Add(1,2,3,4,5));
    }

}

when you run the jboss, Jboss is also one JNDI server listening one a specific port , 1099 by default.

so you need tell the client, which jndi server we should talk to by put some environment variables like the following.

-Djava.naming.factory.initial=org.jnp.interfaces.NamingContextFactory
-Djava.naming.provider.url=jnp://localhost:1099
-Djava.naming.factory.url.pkgs=org.jnp.interfaces

-Djnp.timeout=0
-Djnp.sotimeout=0

When you run the client, you will get the result as you expected, a simple calculator. but all logic runs on server, the Jboss jvm.

Jboss log.

here concludes the basic tutorial. have fun.

How to : test hibernate data access with SQL Server 2008

2011-01-12T14:23:00.001-08:00

there are some prerequisites to test the hibernate data access with SQL server.

Hibernate Lib. http://www.hibernate.org/downloads
SQL Server JDBC Driver 3.0 http://msdn.microsoft.com/en-us/sqlserver/aa937724
some XML configurations ( hibernate.cfg.xml, POJO.hbm.xml)

I will provide some snippets as following,

Create a java project, add the jars to the builder path. 101 with Hibernate, first create one POJO ( a basic entity bean with “attributes”), I will create a basic Class called TODO with two attributes, ID and Title.

package Domain;

public class TODO {

private int ID;

private String Title;

    public void setID(int iD) {
        ID = iD;
    }

    public int getID() {
        return ID;
    }

    public void setTitle(String title) {
        Title = title;
    }

    public String getTitle() {
        return Title;
    }
}

Then Add one XML mapping for this Class. (the file will used by hibernate to interpret the field-column mapping, ID generation rules.) , always using the naming rules as [ENtity].HBM.XML which is optional.
for the entitye TODO, will be mapped to a table called TODOs, the ID generation rule is implemented by SQL server( identity column)

<?xml version="1.0"?>
<!DOCTYPE hibernate-mapping PUBLIC "-//Hibernate/Hibernate Mapping DTD 3.0//EN" "http://hibernate.sourceforge.net/hibernate-mapping-3.0.dtd">
<hibernate-mapping package="Domain">

</class>
</hibernate-mapping>

then create one file HIbernate.cfg.xml ( the driver used to communicate with sql server. )

<?xml version='1.0' encoding='utf-8'?><!DOCTYPE hibernate-configuration PUBLIC        "-//Hibernate/Hibernate Configuration DTD 3.0//EN"        "http://hibernate.sourceforge.net/hibernate-configuration-3.0.dtd">
<hibernate-configuration>
    <session-factory>        
        <property name="connection.driver_class">com.microsoft.sqlserver.jdbc.SQLServerDriver</property>
        <property name="connection.url">jdbc:sqlserver://localhost;databaseName=test;integratedSecurity=true; </property>
        <property name="connection.username">not required</property>
        <property name="connection.password"></property>        
        <property name="connection.pool_size">1</property>        
        <property name="dialect">org.hibernate.dialect.SQLServer2008Dialect</property>        
        <property name="current_session_context_class">thread</property>        
        <property name="cache.provider_class">org.hibernate.cache.NoCacheProvider</property>        
        <property name="show_sql">true</property>        
        <property name="hbm2ddl.auto">create-drop</property>
        <mapping resource="test/TODO.hbm.xml" />
    </session-factory>
</hibernate-configuration>

Then, read /write to and from DB using the hibernate API.

package test;

import org.hibernate.Session;
import org.hibernate.SessionFactory;
import org.hibernate.Transaction;
import org.hibernate.cfg.Configuration;

public class Main {

    /**
     * @param args
     * @throws ClassNotFoundException
     */
    public static void main(String[] args) {

        SessionFactory factory = new Configuration().configure(
                "test/hibernate.cfg.xml").buildSessionFactory();
        Session session = factory.getCurrentSession();
        Transaction trans = session.beginTransaction();

        Domain.TODO newobj = new Domain.TODO();
        newobj.setTitle("FirstTODO");
        session.save(newobj);

        Domain.TODO newobj2 = new Domain.TODO();
        newobj2.setTitle("SecondTODO");
        session.save(newobj2);
        // list all objects

        java.util.List lists = session.createQuery("from TODO").list();
        System.out.println(lists.size());
        for (int i = 0; i < lists.size(); i++) {
            System.out.println(((Domain.TODO) lists.get(i)).getTitle());
        }

trans.commit();

}

When you run the program, it will create two records and save it to DB. here is the sql used captured by SQL profiler.

If you get the error when you run the app,

Jan 12, 2011 2:19:19 PM com.microsoft.sqlserver.jdbc.AuthenticationJNI <clinit>
WARNING: Failed to load the sqljdbc_auth.dll cause :- no sqljdbc_auth in java.library.path

that means the JDBC driver need loaded some DLL in order to work with SQL server. just copy this dll [located in the driver auth folder] to the class path folder. or just c:\windows folder.
this only applies if we use the Windows authentication against sql server instead of sql authentication.

<property name="connection.url">jdbc:sqlserver://localhost;databaseName=test;integratedSecurity=false; </property>

Varnish Cache , Blocked Proxy access to your Server / IP Address

2011-01-10T15:48:00.000-08:00

Varnish is one great reverse-proxy to cache and accelerate your web application. If you deployed one varnish instance which is facing internet. you may expose some kind of security threats.
i.e somebody will try to use your instance as a regular proxy. or send junk url to the backend.

you can simply disable those junk request.

sub vcl_recv {
//only allow IP address for Global DNS based Loadbalancing Or your hostname
if(!req.http.host ~ "yourwebsite" && !req.http.host ~"([0-9]{1,3})\.([0-9]{1,3})\.([0-9]{1,3})\.([0-9]{1,3})" )
{
set req.http.host = “yourwebsite";
set req.url="/" ; //redirect to home page. instead of forward to the backend
}

Path	Handler
*.ism	Smoothhandler
*.isml	LiveStreamingHandler
*.isx	PlayListHandler