Saturday, August 27, 2011

Oops, Youtube support team are trained monkeys


500 Internal Server Error

Sorry, something went wrong.
A team of highly trained monkeys has been dispatched to deal with this situation.

If you see them, show them this information:


Monday, August 22, 2011

How To, Auto Suggesting with multi words/terms using solr ShingleFilterFactory

Google has one amazing feature called auto sugesstion. basically, when you type one word, even just the prefix of the word, it will list the suggestion for you like this,
Even further, google has one unknown REST API that you can use to query the auto-completion word lists provided by Google.


For Apache Solr, If you want to do the same thing as google does, here is one simple approach. Just using the ShingleFilter,

Create one text field that use this filter,

<fieldType name="text_shingle" class="solr.TextField" positionIncrementGap="100">
                <tokenizer class="solr.StandardTokenizerFactory"/>
        <filter class="solr.LowerCaseFilterFactory"/>
         <filter class="solr.ShingleFilterFactory" maxShingleSize="4" outputUnigrams="true"/>

Then create one field to use that field type

After that, we can post a simple document to the Solr, I will put one wiki page, to the solr.

you can just create one doc and post it to solr using curl



Now you can go to the solr admin to check the description terms, you will see those multi-word terms


for application, we can run a facet query to get the keyword list and count.


pretty simple to get startedSmile

Wednesday, August 17, 2011

How to, Hadoop , error to start Jobtracker of hadoop, FATAL org.apache.hadoop.mapred.JobTracker: java.lang.RuntimeException: Not a host:port pair: local

One day, I just created a fresh centos and download/install the hadoop, by default it runs on local mode. so I just change the config to run in distributed mode. When I run bin/ get the follow errors in the tasktrackr log.

2011-08-16 21:00:12,136 INFO org.apache.hadoop.mapred.JobTracker: STARTUP_MSG:
STARTUP_MSG: Starting JobTracker
STARTUP_MSG:   host = hadoop/
STARTUP_MSG:   args = []
STARTUP_MSG:   version = 0.20.2
STARTUP_MSG:   build = -r 911707; compiled by 'chrisdo' on Fri Feb 19 08:07:34 UTC 2010
2011-08-16 21:00:12,280 WARN org.apache.hadoop.conf.Configuration: bad conf file: element not <property>
2011-08-16 21:00:12,280 WARN org.apache.hadoop.conf.Configuration: bad conf file: element not <property>
2011-08-16 21:00:12,280 WARN org.apache.hadoop.conf.Configuration: bad conf file: element not <property>
2011-08-16 21:00:12,299 INFO org.apache.hadoop.mapred.JobTracker: Scheduler configured with (memSizeForMapSlotOnJT, memSizeForReduceSlotOnJT, limitMaxMemForMapTasks, limitMaxMemForReduceTasks) (-1, -1, -1, -1)
2011-08-16 21:00:12,344 FATAL org.apache.hadoop.mapred.JobTracker: java.lang.RuntimeException: Not a host:port pair: local
        at org.apache.hadoop.mapred.JobTracker.getAddress(
        at org.apache.hadoop.mapred.JobTracker.<init>(
        at org.apache.hadoop.mapred.JobTracker.startTracker(
        at org.apache.hadoop.mapred.JobTracker.startTracker(
        at org.apache.hadoop.mapred.JobTracker.main(

it looks like the runtime still pick up the default mapreduce configuration located in the hadoop-x.core.jar, you can get this file by unzipping hadoopcore.jar,


So the error means it pick up this default setting of Local. Then I read careflly about the mapred-site.xml, here it is.

It looks like perfect, right, cause I just copied the setting from the default xml, while, I just keep the weird error.
well, here is final xml that works great, I highlighted the difference, I forgot to put the property tag. so the runtime will ignore this xml and pick up the default settings.

Summary, when you get weird error, check the log first, then make sure those conf xml are in right format that comforts the scheme
tricky, hah?

Tuesday, August 2, 2011

Download Daily Wallpaper App for Android


  • Save images to use them as your Android wallpaper
  • Access and download Bing's daily wallpapers
  • Read comments on wallpaper images
  • Share images via e-mail or your favorite social networking site
  • Recall and view your download history
  • set as wallpaper automatically
  • background run


AD Free Edition,

Monday, August 1, 2011

Exchange 2010 Provisioning using ILM 2007

If you just upgraded Exchange to 2010, and still using the ILM 2007. you need install the SP1 and featured package, the only document you can follow is

Basically, for the provisioning to exchange 2010 mailbox. you have call the utility to create a mailbox in your Povision DLL.something like,

void IMVSynchronization.Provision (MVEntry mventry)
ConnectedMA adMA;
CSEntry csentry;
String nickName, mailboxMDB;
ReferenceValue dn;

adMA = mventry.ConnectedMAs["Fabrikam AD MA"];

nickName = mventry["mailNickname"].Value;
mailboxMDB = mventry["homeMDB"].Value;

// Construct the distinguished name
dn = adMA.EscapeDNComponent("CN=" + mventry["cn"].Value).Concat("ou=mailboxes,dc=fabrikam,dc=com");

if(0 == adMA.Connectors.Count)
 csentry = ExchangeUtils.CreateMailbox(adMA, dn, nickName, mailboxMDB);

// Handle any exceptions
catch(Exception ex)

For the Magic code in CreateMailBox, you can see the source code. it basically add several attributes there,

After that, also you should config the AD ma to run the extension Code , you need to specify the Powershell remote address, and Extension of 2010 or 2007 ,

After those two major change been applied, the ILM will call the provision code to create the mailbox, and Run the Extension Code for every object listed in the pending export stage of the Connected Space. there are two Extension DLLs in the ILM Extension Folder.
Extension2007 and Extension2010.dll

for the code in Exch20xxExtension.dll, it basically call the powershell command remotly on the server you specified in the RPS URI.

basically, call the PowerShell command remotely.

Update-Recipient -Identity "<ExportedDN>"-Credential <PSCredential> -DomainController <Fqdn>

Here is the pain you may have.
ILM will call this command for every Object which has some pending export in the MA. even just a title change, it will call the Update-receipent , which may cause the default mailpolicy to run. (in some case, this is not what you want, i.e you have different mail policy. )

however, we have the Extension point.
you can get the source code of the extension dll, put you logic there, and sign the new dll. which will be picked up by ILM. 
you can do some loggin there, or check wheter the object is required to run the update-recepient command. or even just call you powershell command.

Locations of visitors to this page