Saturday, August 27, 2011

Oops, Youtube support team are trained monkeys

image

500 Internal Server Error

Sorry, something went wrong.
A team of highly trained monkeys has been dispatched to deal with this situation.

If you see them, show them this information:

WFlEO-pxslLQKQ8jn5sBkCwnuuZg1YYqX9bVND5_m5gRtSxuruNgKsTd17lg
uIm1sVMS98fpTtpzlNX96mH1WLq79xMZUvOLdhFvKX_WGYFfc38YlrkAm23q
kPFJ7mrbT12tQundh7bk1y0TEqJfe0SA336fLvzgVEm1tt8oIf8-ib7CH3QQ
n11bbMFCY1iJgCn1ezZIl1rXHkmuRU9xrlQ4d-XrG1JmJtaRRFY1kklSloUj
JECppOrM0wAaZXFODfOpE_ZQXT0iJk46_YrqBf0-rD7nXYmruWZqU4zcnvWN
tnXr70fXkkBoVohlx-yC7GiBtceyBSPMYhotqQD2gE7JMa_LtCXflVx15Hs3
yfKiDZIcjiaQ5U1L2YHgXwplA4SiOJ2SRM2HI1DO9_9bs9qMKWPvc-OUt3nV
PmZHNdlYW143KHOCRqZv0m19bQ2vxXhZjEHfc8GrLZu3Vz3_y1QvVAt4lj5J
2UPBp6mJvmB3Ouzak6VN1jXSa24n8gXE5VrGNcnWs1GKCpGNd6yRQWpq9shw
BF1eygVB8mZri2xeUBh4vLZ7pwEj06c-qmJr4RItLLXWT8CMqeM6KrYxWr6g

Monday, August 22, 2011

How To, Auto Suggesting with multi words/terms using solr ShingleFilterFactory

Google has one amazing feature called auto sugesstion. basically, when you type one word, even just the prefix of the word, it will list the suggestion for you like this,
image
Even further, google has one unknown REST API that you can use to query the auto-completion word lists provided by Google.
http://google.com/complete/search?output=toolbar&q=%22intel+s%22

 image

For Apache Solr, If you want to do the same thing as google does, here is one simple approach. Just using the ShingleFilter,

Create one text field that use this filter,

<fieldType name="text_shingle" class="solr.TextField" positionIncrementGap="100">
            <analyzer>
                <tokenizer class="solr.StandardTokenizerFactory"/>
        <filter class="solr.LowerCaseFilterFactory"/>
         <filter class="solr.ShingleFilterFactory" maxShingleSize="4" outputUnigrams="true"/>
            </analyzer>
        </fieldType>

Then create one field to use that field type
image

After that, we can post a simple document to the Solr, I will put one wiki page, http://en.wikipedia.org/wiki/X25-M to the solr.

you can just create one doc and post it to solr using curl

image


image

Now you can go to the solr admin to check the description terms, you will see those multi-word terms

image

for application, we can run a facet query to get the keyword list and count.
image

http://localhost:8080/solrmaster/select/?q=description%3A*&version=2.2&start=0&rows=0&indent=on&facet=true&facet.field=description&facet.prefix=02g2

pretty simple to get startedSmile

Wednesday, August 17, 2011

How to, Hadoop , error to start Jobtracker of hadoop, FATAL org.apache.hadoop.mapred.JobTracker: java.lang.RuntimeException: Not a host:port pair: local

One day, I just created a fresh centos and download/install the hadoop, by default it runs on local mode. so I just change the config to run in distributed mode. When I run bin/start-all.sh. get the follow errors in the tasktrackr log.

2011-08-16 21:00:12,136 INFO org.apache.hadoop.mapred.JobTracker: STARTUP_MSG:
/************************************************************
STARTUP_MSG: Starting JobTracker
STARTUP_MSG:   host = hadoop/192.168.209.131
STARTUP_MSG:   args = []
STARTUP_MSG:   version = 0.20.2
STARTUP_MSG:   build = https://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.20 -r 911707; compiled by 'chrisdo' on Fri Feb 19 08:07:34 UTC 2010
************************************************************/
2011-08-16 21:00:12,280 WARN org.apache.hadoop.conf.Configuration: bad conf file: element not <property>
2011-08-16 21:00:12,280 WARN org.apache.hadoop.conf.Configuration: bad conf file: element not <property>
2011-08-16 21:00:12,280 WARN org.apache.hadoop.conf.Configuration: bad conf file: element not <property>
2011-08-16 21:00:12,299 INFO org.apache.hadoop.mapred.JobTracker: Scheduler configured with (memSizeForMapSlotOnJT, memSizeForReduceSlotOnJT, limitMaxMemForMapTasks, limitMaxMemForReduceTasks) (-1, -1, -1, -1)
2011-08-16 21:00:12,344 FATAL org.apache.hadoop.mapred.JobTracker: java.lang.RuntimeException: Not a host:port pair: local
        at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:136)
        at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:123)
        at org.apache.hadoop.mapred.JobTracker.getAddress(JobTracker.java:1807)
        at org.apache.hadoop.mapred.JobTracker.<init>(JobTracker.java:1579)
        at org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:183)
        at org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:175)
        at org.apache.hadoop.mapred.JobTracker.main(JobTracker.java:3702)


it looks like the runtime still pick up the default mapreduce configuration located in the hadoop-x.core.jar, you can get this file by unzipping hadoopcore.jar,
image

image

So the error means it pick up this default setting of Local. Then I read careflly about the mapred-site.xml, here it is.
image

It looks like perfect, right, cause I just copied the setting from the default xml, while, I just keep the weird error.
well, here is final xml that works great, I highlighted the difference, I forgot to put the property tag. so the runtime will ignore this xml and pick up the default settings.
image

Summary, when you get weird error, check the log first, then make sure those conf xml are in right format that comforts the scheme
tricky, hah?

Tuesday, August 2, 2011

Download Bing.com Daily Wallpaper App for Android

bing

https://market.android.com/details?id=com.androidyou.bing&feature=search_result
Features,

  • Save images to use them as your Android wallpaper
  • Access and download Bing's daily wallpapers
  • Read comments on wallpaper images
  • Share images via e-mail or your favorite social networking site
  • Recall and view your download history
  • set as wallpaper automatically
  • background run

 

AD Free Edition,
bingdx

Monday, August 1, 2011

Exchange 2010 Provisioning using ILM 2007

If you just upgraded Exchange to 2010, and still using the ILM 2007. you need install the SP1 and featured package, the only document you can follow is http://technet.microsoft.com/en-us/magazine/ff472471.aspx

Basically, for the provisioning to exchange 2010 mailbox. you have call the utility to create a mailbox in your Povision DLL.something like,

void IMVSynchronization.Provision (MVEntry mventry)
{
ConnectedMA adMA;
CSEntry csentry;
String nickName, mailboxMDB;
ReferenceValue dn;

try
{
adMA = mventry.ConnectedMAs["Fabrikam AD MA"];

nickName = mventry["mailNickname"].Value;
mailboxMDB = mventry["homeMDB"].Value;

// Construct the distinguished name
dn = adMA.EscapeDNComponent("CN=" + mventry["cn"].Value).Concat("ou=mailboxes,dc=fabrikam,dc=com");

if(0 == adMA.Connectors.Count)
{
 csentry = ExchangeUtils.CreateMailbox(adMA, dn, nickName, mailboxMDB);
}
}

// Handle any exceptions
catch(Exception ex)
{
//...
}
}

For the Magic code in CreateMailBox, you can see the source code. it basically add several attributes there,
image

After that, also you should config the AD ma to run the extension Code , you need to specify the Powershell remote address, and Extension of 2010 or 2007 ,
image

After those two major change been applied, the ILM will call the provision code to create the mailbox, and Run the Extension Code for every object listed in the pending export stage of the Connected Space. there are two Extension DLLs in the ILM Extension Folder.
Extension2007 and Extension2010.dll
image

for the code in Exch20xxExtension.dll, it basically call the powershell command remotly on the server you specified in the RPS URI.
image

basically, call the PowerShell command remotely.

Update-Recipient -Identity "<ExportedDN>"-Credential <PSCredential> -DomainController <Fqdn>


Here is the pain you may have.
ILM will call this command for every Object which has some pending export in the MA. even just a title change, it will call the Update-receipent , which may cause the default mailpolicy to run. (in some case, this is not what you want, i.e you have different mail policy. )

however, we have the Extension point.
you can get the source code of the extension dll, put you logic there, and sign the new dll. which will be picked up by ILM. 
you can do some loggin there, or check wheter the object is required to run the update-recepient command. or even just call you powershell command.

 
Locations of visitors to this page