tag:blogger.com,1999:blog-46530704517614566682024-03-17T20:03:48.345-07:00My Note on Solutions.Ryanhttp://www.blogger.com/profile/09245836008728584469noreply@blogger.comBlogger276125tag:blogger.com,1999:blog-4653070451761456668.post-22981523693851238462016-10-24T11:58:00.000-07:002016-10-24T11:58:00.142-07:00How to test pacemaker to create a Virtual IP for LB purpose for 3 nodessometimes, you need have more than one LBs for HA purpose. and we always want to have one floatip virual IP cross all LB nodes. and we can use pacepaker/corosyncd to do this very easily. for the testing purpose, I will use Vagrant to create 3 nodes (node1, node2, node3). once booted up , all 3 machine are well figured for hostname.<br />
<br />
nodeone= 'node1'<br />nodetwo= 'node2'<br />nodethree= 'node3'<br />
<br />Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|<br />
#### node 1###########<br />
config.vm.define :"node1" do |node1|<br /> node1.vm.box = BOX<br /> node1.vm.box_url = BOX_URL<br /> node1.vm.network :private_network, ip: "192.168.2.101"<br /> node1.vm.hostname = nodeone<br /> node1.vm.synced_folder ".", "/vagrant", disabled: true<br /> node1.ssh.shell = "bash -c 'BASH_ENV=/etc/profile exec bash'"<br /> node1.vm.provision "shell", path: "post-deploy.sh" ,run: "always"<br /> node1.vm.provider "virtualbox" do |v|<br /> v.customize ["modifyvm", :id, "--memory", "1300"]<br /> v.name = nodeone<br /> v.gui = true<br /> end<br />
end<br />
<br />
and post-deploy.sh<br />
#!/bin/bash<br />value=$( grep -ic "entry" /etc/hosts )<br />if [ $value -eq 0 ]<br />then<br />echo "<br />################ hosts entry ############<br />
192.168.2.101 node1<br />192.168.2.102 node2<br />192.168.2.103 node3<br />
######################################################<br />" >> /etc/hosts<br />fi<br />
<br />
once done, you can call vagrant up node1 node2 node3 to have 3 VMs ready. <br />
<br />
now vagran ssh all nodes to install the dependencies. <br />
<br />
yum install -y pcsd pacemaker corosync<br />
<br />
after done, enable all daemon to start automatically. <br />
systemctl enable pcsd<br />
systemctl enable pacemaker<br />
systemctl enable corosync<br />
//disable firewalld<br />
systemctl disable firewalld<br />
<br />
Now time to configure the corosync for nodes topology. <br />
<br />
you can just copy the /etc/corosync/corosync.conf.example to /etc/corosync/corosync.conf<br />
just change your ip and quorum settings (by dfault it use mcast to maintain the cluster membership)<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhXE3hZyoA8C1F5KcFjjPzurIgtoN0wfC_o8bhuZ41IKK3tnv5Yc45-0bTAMZU5sO4ra0Sw_L7pFTRy5oWMrgucPgK6T06BpXV7_En9ETXaXzFaDI5CBvjO9H6jqXRViaZKbVKx5JlHk-w/s1600/Capture.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhXE3hZyoA8C1F5KcFjjPzurIgtoN0wfC_o8bhuZ41IKK3tnv5Yc45-0bTAMZU5sO4ra0Sw_L7pFTRy5oWMrgucPgK6T06BpXV7_En9ETXaXzFaDI5CBvjO9H6jqXRViaZKbVKx5JlHk-w/s1600/Capture.PNG" /></a></div>
<br />
then create a user named haclusteruser among 3 nodes<br />
<br />
echo you_special_compliceted_password|passwd --stdin haclusteruser<br />
<br />
once done, you can call pcs cluster auth to authorize eacy host using the shared credential<br />
pcs cluster auth node1 <br />
<br />
after this. start the corosync and pacemaker on all nodes. <br />
<br />
run crm_mon to see all the nodes<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi1vgtdfGb8m42aPkxDxfjMyIy52cfsNqoU7IQqqdbEmSi4QIa4CnjyayFgerHEKgikZXdlXGoIKI_6Loc7OtFORzmCVoUdC8wSe10ErIXZediO0-U0xQkRxk9427FydsG8iF8ldrPZhfM/s1600/Capture+-+Copy.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi1vgtdfGb8m42aPkxDxfjMyIy52cfsNqoU7IQqqdbEmSi4QIa4CnjyayFgerHEKgikZXdlXGoIKI_6Loc7OtFORzmCVoUdC8wSe10ErIXZediO0-U0xQkRxk9427FydsG8iF8ldrPZhfM/s1600/Capture+-+Copy.PNG" /></a></div>
<br />
now we can create a floating vip to HA purpose.<br />
pcs resource create VIP2 IPaddr2 ip=192.168.2.100 cidr_netmask=32 nic=enp0s8 op monitor interval=30s<br />
<br />
after that, you might see the status is always stopped. we need to disable the stonith <br />
pcs property set stonith-enabled=false<br />
<br />
try the check again for the status, the VIP is bound to one node (node1 here) <br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgikFm-1DJqdvtdDpOxY3woYOHPtHwM-eJe7pa0N1m7MFgOOJoiy08kXvjNaSdc0pDhFLWeH_GrKSxsqbzkQ4eXN-qUeFO_xBpUdrlrLmCUaA_EMLBsGe0fUpRcV056L5K8A1oDA5j_AfA/s1600/CCC.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgikFm-1DJqdvtdDpOxY3woYOHPtHwM-eJe7pa0N1m7MFgOOJoiy08kXvjNaSdc0pDhFLWeH_GrKSxsqbzkQ4eXN-qUeFO_xBpUdrlrLmCUaA_EMLBsGe0fUpRcV056L5K8A1oDA5j_AfA/s1600/CCC.PNG" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhQWaq-uY5mn6p7GYwQM9U91t39OCqc3EVTYYXQDKh9KIplt4BI2JgNcB5VZEHX4AFszj3wInXN9YhSUWecdhBDgKfwwW0N0GQj5kL-Tsjc-7hA-3a242Xszgk2fqEZoJyXUI5JwWV51yQ/s1600/AAA.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhQWaq-uY5mn6p7GYwQM9U91t39OCqc3EVTYYXQDKh9KIplt4BI2JgNcB5VZEHX4AFszj3wInXN9YhSUWecdhBDgKfwwW0N0GQj5kL-Tsjc-7hA-3a242Xszgk2fqEZoJyXUI5JwWV51yQ/s1600/AAA.PNG" /></a></div>
<br />
<br />
to test the HA, shutdown node1. and run crm_mon again, the VIP was assigned on node2<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiP88qzHaqfaPIq0ieV5wrGmv7E6AHef7485oV8KDFIPdxX9k-mRN-_XodlrA9_OrdO_tnL5eMcaJUlLq-hLzRTxwhy2Fe1FvSUHHeYjiZJG6R8G2SiL_TANDu9MM7u6JoYBT_FVdKfk3A/s1600/ASSS.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiP88qzHaqfaPIq0ieV5wrGmv7E6AHef7485oV8KDFIPdxX9k-mRN-_XodlrA9_OrdO_tnL5eMcaJUlLq-hLzRTxwhy2Fe1FvSUHHeYjiZJG6R8G2SiL_TANDu9MM7u6JoYBT_FVdKfk3A/s1600/ASSS.PNG" /></a></div>
<br />
Ryanhttp://www.blogger.com/profile/09245836008728584469noreply@blogger.com10tag:blogger.com,1999:blog-4653070451761456668.post-62218088854065799692016-08-10T11:47:00.001-07:002016-08-10T11:47:26.399-07:00How to locate Magento credit card leakgeJust got a chance to help locate the leaking issues of a customer who got complaints from customer about credit card leak after made purchase.<br />
<br />
and it turns out to be a very smart hacker. here is the steps I try to locate the issue.<br />
<br />
I use tcpdump to <b><u>capture all traffic for a couple hours</u></b> and do a quick analysis to see whether there are some special. like sending out credit card using SMTP with the port 25, or just use stand http post, if it https. you can sort the TLS certificate to see any special certificate which represents some evil 3rd party hosts.<br />
<br />
Nothing found special for me. most just traffic to this site and some 3rd party API call like shipping rate calculation, Fedex integration.<br /><br />then <b>do a analysis about the request log t</b>o see any url which has huge hit from the same IP.<br />
you can use some handy linux command to do the aggregation.<br />
<br />awk '{print $8$2}' requestlog|sort|uniq -c|sort -r -n|more<br /><br />nothing special, most hits are from crawlers ip with Google.<br />
<br />
feel a little big frustrated now, this must be a smart hacker. then <b>I watch for file change within 5 minutes</b>. if they hacker intercept the request and store somewhere, I will definitly find out where did he/she store the file.<br />
<br />
go the root directly of the magento, I spect all files change within 5 mintues (I placed a order using a dummy credit card"<br /><br /> find . -mmin -5 -ls<br />
<br />
there we go, one file called db-tab-footer_bg.gif was changed minutes ago. there should not be any chagne for gif files. this turns out to be a complicated image file. since it's all stored with encrypted data ( it must be credit card there),<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgzP_91kqz0sBw9VIB_fTt3F7Hkj8Kk3VhqNyF_8BbPenuCjQJ7RPlWIpZEVabrOHSo_h8oxscpwjk1nsjnbvoNOEcibfHq6tMgZJY4c-uT4-A3O3AcIa897T5TS8g32C1k46e0ps9KTwE/s1600/Screen+Shot+2016-08-10+at+11.43.09+AM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="120" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgzP_91kqz0sBw9VIB_fTt3F7Hkj8Kk3VhqNyF_8BbPenuCjQJ7RPlWIpZEVabrOHSo_h8oxscpwjk1nsjnbvoNOEcibfHq6tMgZJY4c-uT4-A3O3AcIa897T5TS8g32C1k46e0ps9KTwE/s400/Screen+Shot+2016-08-10+at+11.43.09+AM.png" width="400" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
if I check the access log, not a lot request to this file, only once a week coming from the hacker's IP.<br />
<br />
now, time to <b>locate how can they capture the data and dum</b>p to this file.<br />
just search all php files containing db-tab-footer_bg.gif.<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiV-g482xZxQcA6s6XAy9RqkKIyXSW_yY_X88h5oaFcKCnqiFWQyFvYIwbt1FN194oL5teEL0G3EejcE84SUtrwUGo_1cPbNY0mUyIHxb2jkRxpzBJKD2mSMPKrLTdtI3zKOM00eiee2eU/s1600/Screen+Shot+2016-08-10+at+11.47.25+AM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="147" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiV-g482xZxQcA6s6XAy9RqkKIyXSW_yY_X88h5oaFcKCnqiFWQyFvYIwbt1FN194oL5teEL0G3EejcE84SUtrwUGo_1cPbNY0mUyIHxb2jkRxpzBJKD2mSMPKrLTdtI3zKOM00eiee2eU/s400/Screen+Shot+2016-08-10+at+11.47.25+AM.png" width="400" /></a></div>
it's in the global config file, here is the content. essentially they intercept all post request and encrypt using his/her RSA public key and put to the gif file. nobody can decrypt it.<br />
<br />
now time to get ride of the backdoor and do the housekeeping to locate how did they inject the backdoor on the server<br /><br />
<br />Ryanhttp://www.blogger.com/profile/09245836008728584469noreply@blogger.com8tag:blogger.com,1999:blog-4653070451761456668.post-89337904997463279912016-07-31T14:26:00.000-07:002016-07-31T14:26:03.081-07:00Group All established connections by target IPhere is a very common case that you want to check whether you have even connection for a distributed system on the server side. or whether the client send the requests cross all server endpoints.<br />
<br />
Basic idea. netstat -an|grep 'EST' will show you all the established connection.<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhPbxbP0hPBUTtQ7BdlMhtRPQ35FdmSO2iwOja6WiMTIZyP2pdyvznf0vYycI-54LEdfnOVFP9YCnWZx9yhRh4Za-2HUGo2u3VT1nVWm5St-uCBgA0cfaoaYToDLqUC3R-wQb_D1sGyEmc/s1600/Screen+Shot+2016-07-31+at+2.18.52+PM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="163" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhPbxbP0hPBUTtQ7BdlMhtRPQ35FdmSO2iwOja6WiMTIZyP2pdyvznf0vYycI-54LEdfnOVFP9YCnWZx9yhRh4Za-2HUGo2u3VT1nVWm5St-uCBgA0cfaoaYToDLqUC3R-wQb_D1sGyEmc/s640/Screen+Shot+2016-07-31+at+2.18.52+PM.png" width="640" /></a></div>
<br />so the 4th column is the client endpoints, and 5th column is the target one.<br />
<br />
if we want to see how many connection for each Target as a IP. you can do the following command.<br />
netstat -an -p tcp|grep "EST"|awk '{print $5}'|awk -F "." '{print $1"."$2"."$3"."$4 }'|sort |uniq -c |sort -n -r<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEioGskZb6TNH47EselFMNSJsk10YN0_D_aC_5YA2IfQWGBZSHZqtsPhvdAlHbFoTaTMp_O9sx_15L9pWp0B7jnn45mrARfXZNt5lrg7fvHmcbrxodDpECrmaJu2nZ5UJi6b15utEP9I7Lo/s1600/Screen+Shot+2016-07-31+at+2.23.12+PM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="348" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEioGskZb6TNH47EselFMNSJsk10YN0_D_aC_5YA2IfQWGBZSHZqtsPhvdAlHbFoTaTMp_O9sx_15L9pWp0B7jnn45mrARfXZNt5lrg7fvHmcbrxodDpECrmaJu2nZ5UJi6b15utEP9I7Lo/s400/Screen+Shot+2016-07-31+at+2.23.12+PM.png" width="400" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
and if we only want to see the top 5. we can put rows index for the awk hint. </div>
<div class="separator" style="clear: both; text-align: left;">
netstat -an -p tcp|grep "EST"|awk '{print $5}'|awk -F "." 'FNR<6 {print $1"."$2"."$3"."$4 }'|sort |uniq -c |sort -n -r</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<br />
<br />Ryanhttp://www.blogger.com/profile/09245836008728584469noreply@blogger.com1tag:blogger.com,1999:blog-4653070451761456668.post-13078561251504225482016-05-17T17:18:00.001-07:002016-05-17T17:19:05.856-07:00Spark window function, failure: ``union'' expected but `(' foundthis a very weird error when I try to run a simple window ranking, all looks good from the syntax perspective.<br />
<pre style="background-color: white; font-family: Menlo; font-size: 12pt;">team=[(<span style="color: green; font-weight: bold;">"Lakers"</span>,<span style="color: green; font-weight: bold;">"WEST"</span>,<span style="color: blue;">29 </span>),(<span style="color: green; font-weight: bold;">"Golden State"</span>,<span style="color: green; font-weight: bold;">"WEST"</span>,<span style="color: blue;">89 </span>),
(<span style="color: green; font-weight: bold;">"MIA HEAT"</span>,<span style="color: green; font-weight: bold;">"EAST"</span>,<span style="color: blue;">79 </span>),(<span style="color: green; font-weight: bold;">"SAS"</span>,<span style="color: green; font-weight: bold;">"WEST"</span>,<span style="color: blue;">9 </span>),
(<span style="color: green; font-weight: bold;">"RAPTORS"</span>,<span style="color: green; font-weight: bold;">"EAST"</span>,<span style="color: blue;">29 </span>) ]
sql.createDataFrame(
sc.parallelize(
team).map(<span style="color: navy; font-weight: bold;">lambda </span>x: Row(<span style="color: #660099;">Team</span>=x[<span style="color: blue;">0</span>],<span style="color: #660099;">Division</span>=x[<span style="color: blue;">1</span>], <span style="color: #660099;">Score</span>=x[<span style="color: blue;">2</span>])))\
.registerAsTable(<span style="color: green; font-weight: bold;">"team"</span>)
<span style="color: navy; font-weight: bold;">print </span>sql.sql(<span style="color: green; font-weight: bold;">"SELECT team, division, score, rank() </span><span style="color: green; font-weight: bold;">OVER (PARTITION BY division ORDER BY score desc)"</span><span style="color: green; font-weight: bold;"> " as rank FROM team"</span>).take(<span style="color: blue;">10</span>)</pre>
<pre style="background-color: white; font-family: Menlo; font-size: 12pt;"></pre>
<pre style="background-color: white; font-family: Menlo; font-size: 12pt;">And I got this errors complaining the syntax</pre>
<pre style="background-color: white;"><span style="font-family: "menlo";"><span style="font-size: 16px;">4j.protocol.Py4JJavaError: An error occurred while calling o36.sql.
: java.lang.RuntimeException: [1.43] failure: ``union'' expected but `(' found
SELECT team, division, score, rank() OVER (PARTITION BY division ORDER BY score desc) as rank FROM team
^
at scala.sys.package$.error(package.scala:27)
at org.apache.spark.sql.catalyst.AbstractSparkSQLParser.parse(AbstractSparkSQLParser.scala:36)
at org.apache.spark.sql.catalyst.DefaultParserDialect.parse(ParserDialect.scala:67)
at org.apache.spark.sql.SQLContext$$anonfun$2.apply(SQLContext.scala:211)
at org.apache.spark.sql.SQLContext$$anonfun$2.apply(SQLContext.scala:211)</span></span></pre>
<pre style="background-color: white;"><span style="font-family: "menlo";"><span style="font-size: 16px;">
</span></span></pre>
<pre style="background-color: white;"><span style="font-family: "menlo";"><span style="font-size: 16px;">To Fix this, please make sure you are using HiveContext instead of SqlContext</span></span></pre>
X.Y.Zhttp://www.blogger.com/profile/13381244514393182781noreply@blogger.com6tag:blogger.com,1999:blog-4653070451761456668.post-6075723359119276712016-05-17T17:11:00.001-07:002016-05-17T17:12:42.510-07:00How to Run Spark testing application in your fav Python IDEHere is a quick step to run and test your spark application using python IDE, essentially, we need load the dependency module. setup the environment and load the context.<br />
<br />
1. copy and grab pyspark folder under the standard spark distribution to your project folder<br />
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjHNO-q5_HsgN627815IZ-BxbuInuQiMLTq4FKknUkDRWT1a2b7ljComfccmTQgBr3rPsnEq7hpt1DtpTC_hKwUYOISxoqghb9A9vVmroxwZG-yLJ0_6k0SGs4Ga5xnUVcCsHgI97haZis/s1600/Screen+Shot+2016-05-17+at+5.06.37+PM.png" imageanchor="1"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjHNO-q5_HsgN627815IZ-BxbuInuQiMLTq4FKknUkDRWT1a2b7ljComfccmTQgBr3rPsnEq7hpt1DtpTC_hKwUYOISxoqghb9A9vVmroxwZG-yLJ0_6k0SGs4Ga5xnUVcCsHgI97haZis/s1600/Screen+Shot+2016-05-17+at+5.06.37+PM.png" /></a><br />
2. setup some bootstrap to take care the environments using the following code , I use 1.6.1 as an example. and you may create this as a module.<br />
<pre style="background-color: white; font-family: Menlo; font-size: 12pt;"><span style="color: navy; font-weight: bold;">class </span>Setup(<span style="color: navy;">object</span>):
<span style="color: navy; font-weight: bold;">def </span>setupSpark(<span style="color: #94558d;">self</span>):
os.environ[<span style="color: green; font-weight: bold;">"SPARK_HOME"</span>] = <span style="color: green; font-weight: bold;">"/Users/and/Development/spark/spark-1.6.1-bin-hadoop2.6/"</span><span style="color: green; font-weight: bold;"> </span>os.environ[<span style="color: green; font-weight: bold;">"PYSPARK_SUBMIT_ARGS"</span>]=<span style="color: green; font-weight: bold;">"--master local[2]"</span>;
spark_home = os.environ.get(<span style="color: green; font-weight: bold;">"SPARK_HOME"</span>)
spark_release_file = spark_home + <span style="color: green; font-weight: bold;">"/RELEASE"</span><span style="color: green; font-weight: bold;"> </span><span style="color: navy; font-weight: bold;">if </span>os.path.exists(spark_release_file) <span style="color: navy; font-weight: bold;">and </span><span style="color: green; font-weight: bold;">"Spark 1.6.1" </span><span style="color: navy; font-weight: bold;">in </span><span style="color: navy;">open</span>(spark_release_file).read():
pyspark_submit_args = os.environ.get(<span style="color: green; font-weight: bold;">"PYSPARK_SUBMIT_ARGS"</span>, <span style="color: green; font-weight: bold;">""</span>)
<span style="color: navy; font-weight: bold;">if not </span><span style="color: green; font-weight: bold;">"pyspark-shell" </span><span style="color: navy; font-weight: bold;">in </span>pyspark_submit_args: pyspark_submit_args += <span style="color: green; font-weight: bold;">" pyspark-shell"</span><span style="color: green; font-weight: bold;"> </span>os.environ[<span style="color: green; font-weight: bold;">"PYSPARK_SUBMIT_ARGS"</span>] = pyspark_submit_args
sys.path.insert(<span style="color: blue;">0</span>, spark_home + <span style="color: green; font-weight: bold;">"/python"</span>)
sys.path.insert(<span style="color: blue;">0</span>, os.path.join(spark_home, <span style="color: green; font-weight: bold;">"python/lib/py4j-0.9-src.zip"</span>))
<span style="color: navy; font-weight: bold;">return </span>pyspark.SparkContext()</pre>
<pre style="background-color: white; font-family: Menlo; font-size: 12pt;">3. you are good to go</pre>
<pre style="background-color: white; font-family: Menlo; font-size: 12pt;"><pre style="font-family: Menlo; font-size: 12pt;"><span style="color: navy; font-weight: bold;">from </span>lib.setup <span style="color: navy; font-weight: bold;">import </span>Setup
sc=Setup().setupSpark()
<span style="color: navy; font-weight: bold;">print </span>sc.parallelize(<span style="color: navy;">range</span>(<span style="color: blue;">1</span>,<span style="color: blue;">10</span>)).count()</pre>
</pre>
X.Y.Zhttp://www.blogger.com/profile/13381244514393182781noreply@blogger.com5tag:blogger.com,1999:blog-4653070451761456668.post-29791021952561981672015-12-15T11:07:00.000-08:002015-12-15T11:07:13.783-08:00How to: test ElasticSearch geoLocation supportHere is a quick tutorial to setup ES 2.1 and index some earthquake data by
using the REST api. and then do by query by assign the POI within couple miles
and return those earthquakes count aggregated by Range. <br />
the query looks like this. basically query all the earthquake happened near
San Diego within 300KMs.<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiELRdsCw484Oy4v5Jypul2OWwGyjJDfYw6zT49lukWAsMVASh0PuwHjFYKVdSVI8jp1wzRhxa3IL2C9zAr3_DPY7H6BCoBC0wAO_nR3I6q4hKR8cmIJFiDyvykU2sfCqdjr_jzTgKAXak/s1600/A.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em; text-align: left;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiELRdsCw484Oy4v5Jypul2OWwGyjJDfYw6zT49lukWAsMVASh0PuwHjFYKVdSVI8jp1wzRhxa3IL2C9zAr3_DPY7H6BCoBC0wAO_nR3I6q4hKR8cmIJFiDyvykU2sfCqdjr_jzTgKAXak/s1600/A.png" /></a></div>
<br />
and the results showing the individual earthquakes and the buckets aggregated. <br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh2X8dhRqyfZ4i7FbjSkeilfUTjwFZsIWmQlv7frLua02caTPkX0a6fX8pvk4aWRwm7_6krsjVvLqVph-F-ZT-C2swmBg3eC4bExTLi2udFv6nDjxMW2JzTKv7tvodwf1WcPXhWGWr5zsk/s1600/A.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh2X8dhRqyfZ4i7FbjSkeilfUTjwFZsIWmQlv7frLua02caTPkX0a6fX8pvk4aWRwm7_6krsjVvLqVph-F-ZT-C2swmBg3eC4bExTLi2udFv6nDjxMW2JzTKv7tvodwf1WcPXhWGWr5zsk/s1600/A.png" /></a></div>
you can grad the data source from USGS , I closed the <a href="http://earthquake.usgs.gov/earthquakes/feed/v1.0/csv.php">CSV file
here</a> <br />to do the Indexing, first we define the mapping , I used Postman to
do the REST call. My mapping looks like this,<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiU8Xe5Kz2znp8BJ4Z6nKt2D7dKyL7DdWWF1FAXP-8LeJT3H-dZruhAj3AMxo5Uc7X-3jlSwGHp2tUgdNuQSjHstunNEb-UOZuJfMK63HfXcSAHL3dFzMmndNNZ_o6A8idzHUq-rBLQEMM/s1600/A.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiU8Xe5Kz2znp8BJ4Z6nKt2D7dKyL7DdWWF1FAXP-8LeJT3H-dZruhAj3AMxo5Uc7X-3jlSwGHp2tUgdNuQSjHstunNEb-UOZuJfMK63HfXcSAHL3dFzMmndNNZ_o6A8idzHUq-rBLQEMM/s1600/A.png" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div style="text-align: left;">
once we have the mapping and Index, we can feed the data to ES using the <a href="https://chrome.google.com/webstore/detail/sense-beta/lhjgkmllcaadmopgmanpapmpjgmfcfig?hl=en">Sense
tool</a>. I put a small script to parse the CSV data to JSON following the Bulk
format. </div>
like this<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjM2JED8NF10yNyONt_ym3vXc0vyvaB9zg76a_GEFCx2qfil1xX8Vvqeqs_CnHShOorEiTtMTWfu7Nwp9X1A6kZN0M3Ufvscg-kAq4w8sU-knPwWrCKFxuEFSkFozeI1bFCyDhHLr8rFQU/s1600/A.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjM2JED8NF10yNyONt_ym3vXc0vyvaB9zg76a_GEFCx2qfil1xX8Vvqeqs_CnHShOorEiTtMTWfu7Nwp9X1A6kZN0M3Ufvscg-kAq4w8sU-knPwWrCKFxuEFSkFozeI1bFCyDhHLr8rFQU/s1600/A.png" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
And I found Sense is the best client when you do the Bulk indexing, all other
REST plugin in Chome give me the encoding error. <br />then copy and paste the
data we convered, basically tell the importer we want to create a new document
called eq, then the doc itself<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg5Zqrrayirg42ovFIhAv0hznAH9WPjsZUoDQ4LzsU2LwrlfaYHp85Ag5FiJLc64psWN1JcavSCZg4Tz0EP0eR7UJ_VnnLH3d2foRSFUxs5dpON9KNFnUUIJdEx1rjgXwoQKUkhhyphenhyphencdQmo/s1600/A.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg5Zqrrayirg42ovFIhAv0hznAH9WPjsZUoDQ4LzsU2LwrlfaYHp85Ag5FiJLc64psWN1JcavSCZg4Tz0EP0eR7UJ_VnnLH3d2foRSFUxs5dpON9KNFnUUIJdEx1rjgXwoQKUkhhyphenhyphencdQmo/s1600/A.png" /></a></div>
Ryanhttp://www.blogger.com/profile/09245836008728584469noreply@blogger.com0tag:blogger.com,1999:blog-4653070451761456668.post-35541151216583954212015-05-21T12:28:00.001-07:002015-05-21T12:28:10.455-07:00Http 1.1 Chunked transfer.<p>If you inspect the http response with some zipped resources, you may find 2 things. I assure we are on HTTP 1.1. there is no content-length header ins the response, also the transfer-encoding is chunked. </p> <p><a href="http://lh3.googleusercontent.com/-QVwD-_xvvy4/VV4xqeSkJKI/AAAAAAAACyU/gY6yzGmxr1E/s1600-h/image%25255B3%25255D.png"><img title="image" style="border-top: 0px; border-right: 0px; background-image: none; border-bottom: 0px; padding-top: 0px; padding-left: 0px; border-left: 0px; display: inline; padding-right: 0px" border="0" alt="image" src="http://lh3.googleusercontent.com/-sr2TG7H7buk/VV4xq3byNGI/AAAAAAAACyY/nsXvvD1E2UQ/image_thumb%25255B1%25255D.png?imgmax=800" width="824" height="252" /></a></p> <p>and Http 1.1 have a full spec about the chunked encoding, check it here. <a title="http://www.w3.org/Protocols/rfc2616/rfc2616-sec3.html#sec3.6" href="http://www.w3.org/Protocols/rfc2616/rfc2616-sec3.html#sec3.6">http://www.w3.org/Protocols/rfc2616/rfc2616-sec3.html#sec3.6</a></p> <p>if you use node.js , we can write a simple test to try out the chunked transfer. here is the source code</p> <p><a href="http://lh3.googleusercontent.com/-gmV1MrDQ4F8/VV4xrbebbFI/AAAAAAAACyg/J8VNDH98PAw/s1600-h/image%25255B7%25255D.png"><img title="image" style="border-top: 0px; border-right: 0px; background-image: none; border-bottom: 0px; padding-top: 0px; padding-left: 0px; border-left: 0px; display: inline; padding-right: 0px" border="0" alt="image" src="http://lh3.googleusercontent.com/-ijL0wrEqRiQ/VV4xrjHdjvI/AAAAAAAACyo/d8WTDk7jnBM/image_thumb%25255B3%25255D.png?imgmax=800" width="840" height="509" /></a></p> <p>and try pull the response using wget. you will notice the 3 seconds delay between Okay3 and 4.</p> <p><a href="http://lh3.googleusercontent.com/-14w9ztSSv7E/VV4xsCgk6mI/AAAAAAAACyw/KweLe6RQaXw/s1600-h/image%25255B12%25255D.png"><img title="image" style="border-top: 0px; border-right: 0px; background-image: none; border-bottom: 0px; padding-top: 0px; padding-left: 0px; border-left: 0px; display: inline; padding-right: 0px" border="0" alt="image" src="http://lh3.googleusercontent.com/-2_1qUkG3Fco/VV4xskGYjjI/AAAAAAAACy4/PbGRgo0XHgw/image_thumb%25255B6%25255D.png?imgmax=800" width="174" height="97" /></a></p> <p>and inspect the traffic in between, you can see the wirelevel bits</p> <p><a href="http://lh3.googleusercontent.com/-WmUZkAjXHNY/VV4xs-6H1XI/AAAAAAAACzA/O40PwUx16zs/s1600-h/image%25255B16%25255D.png"><img title="image" style="border-top: 0px; border-right: 0px; background-image: none; border-bottom: 0px; padding-top: 0px; padding-left: 0px; border-left: 0px; display: inline; padding-right: 0px" border="0" alt="image" src="http://lh3.googleusercontent.com/-mfuFxdkzV-8/VV4xtUkeohI/AAAAAAAACzI/b5tqYOsjyCg/image_thumb%25255B8%25255D.png?imgmax=800" width="1323" height="183" /></a></p> <p>1st response returns Okay1</p> <p><a href="http://lh3.googleusercontent.com/-QQcLMgrPXLg/VV4xt3vsCqI/AAAAAAAACzU/cwqv8FbbU4o/s1600-h/image%25255B20%25255D.png"><img title="image" style="border-top: 0px; border-right: 0px; background-image: none; border-bottom: 0px; padding-top: 0px; padding-left: 0px; border-left: 0px; display: inline; padding-right: 0px" border="0" alt="image" src="http://lh3.googleusercontent.com/-pU4d8mzyjI8/VV4xuWtFLcI/AAAAAAAACzY/Bf2mNIb1bwU/image_thumb%25255B10%25255D.png?imgmax=800" width="728" height="167" /></a></p> <p>then the 2nd returns2 and 3</p> <p><a href="http://lh3.googleusercontent.com/-4gPy4OjCenk/VV4xu0Q8eiI/AAAAAAAACzg/J_oK0yPD3rE/s1600-h/image%25255B24%25255D.png"><img title="image" style="border-top: 0px; border-right: 0px; background-image: none; border-bottom: 0px; padding-top: 0px; padding-left: 0px; border-left: 0px; display: inline; padding-right: 0px" border="0" alt="image" src="http://lh3.googleusercontent.com/-v4zq8ScpH6I/VV4xveKpvwI/AAAAAAAACzs/xDpVDdMmE_Q/image_thumb%25255B12%25255D.png?imgmax=800" width="579" height="120" /></a></p> <p>after 3 seconds delay</p> <p>we get okay4</p> <p><a href="http://lh3.googleusercontent.com/-P-lMyyYbyRI/VV4xvyzS6XI/AAAAAAAACzw/xNRhE6vVIgc/s1600-h/image%25255B28%25255D.png"><img title="image" style="border-top: 0px; border-right: 0px; background-image: none; border-bottom: 0px; padding-top: 0px; padding-left: 0px; border-left: 0px; display: inline; padding-right: 0px" border="0" alt="image" src="http://lh3.googleusercontent.com/-DN57SHjuoF0/VV4xwUjXPYI/AAAAAAAACz4/QEePDwTLz8E/image_thumb%25255B14%25255D.png?imgmax=800" width="622" height="121" /></a></p> <p><a href="http://lh3.googleusercontent.com/-NISRSBwCrpI/VV4xwm5ckMI/AAAAAAAAC0A/d8448BX-Yr4/s1600-h/image%25255B32%25255D.png"><img title="image" style="border-top: 0px; border-right: 0px; background-image: none; border-bottom: 0px; padding-top: 0px; padding-left: 0px; border-left: 0px; display: inline; padding-right: 0px" border="0" alt="image" src="http://lh3.googleusercontent.com/-ejLHCErywWo/VV4xxGZSlyI/AAAAAAAAC0I/yHy0kzjgtCY/image_thumb%25255B16%25255D.png?imgmax=800" width="409" height="66" /></a></p> <p>once we call the end , you can see the end trailer. </p> <p>wireshark are smart enough, to tell you 4 frames invloved in this http req/res</p> <p><a href="http://lh3.googleusercontent.com/-QsaCikmDQGM/VV4xxikWuiI/AAAAAAAAC0U/xFg5tHm2Fso/s1600-h/image%25255B36%25255D.png"><img title="image" style="border-top: 0px; border-right: 0px; background-image: none; border-bottom: 0px; padding-top: 0px; padding-left: 0px; border-left: 0px; display: inline; padding-right: 0px" border="0" alt="image" src="http://lh3.googleusercontent.com/-Y2apwBV-YKc/VV4xyLnmxAI/AAAAAAAAC0Y/MLmADIrY8uE/image_thumb%25255B18%25255D.png?imgmax=800" width="700" height="314" /></a></p> Ryanhttp://www.blogger.com/profile/09245836008728584469noreply@blogger.com21tag:blogger.com,1999:blog-4653070451761456668.post-5117614247180425492015-05-18T15:46:00.001-07:002015-05-18T15:56:19.634-07:00Power-shell , Filter and Projection<p>To getstart with any command in PS, run help. </p> <p><a href="http://lh3.googleusercontent.com/-kSFeOvsJv1M/VVprkN8u7lI/AAAAAAAACvk/4wwbN10o44k/s1600-h/image%25255B3%25255D.png"><img title="image" style="border-top: 0px; border-right: 0px; background-image: none; border-bottom: 0px; padding-top: 0px; padding-left: 0px; border-left: 0px; display: inline; padding-right: 0px" border="0" alt="image" src="http://lh3.googleusercontent.com/-Uzv0pBCSHOI/VVprki7kH7I/AAAAAAAACvs/hksr0rxuW78/image_thumb%25255B1%25255D.png?imgmax=800" width="849" height="211" /></a></p> <p><a href="http://lh3.googleusercontent.com/-vzZGkwl36ng/VVprlHeHuGI/AAAAAAAACvw/pFRa6m34V1o/s1600-h/image%25255B7%25255D.png"><img title="image" style="border-top: 0px; border-right: 0px; background-image: none; border-bottom: 0px; padding-top: 0px; padding-left: 0px; border-left: 0px; display: inline; padding-right: 0px" border="0" alt="image" src="http://lh3.googleusercontent.com/-p8al-mp7pbg/VVprllfujQI/AAAAAAAACv8/XMvjyEumvgk/image_thumb%25255B3%25255D.png?imgmax=800" width="324" height="112" /></a></p> <p>To filter it, use the where-object or use ? directly. </p> <p><a href="http://lh3.googleusercontent.com/-mz-1f6Wtd_0/VVprmTrnUnI/AAAAAAAACwE/xiF8QSzSP-w/s1600-h/image%25255B11%25255D.png"><img title="image" style="border-top: 0px; border-right: 0px; background-image: none; border-bottom: 0px; padding-top: 0px; padding-left: 0px; border-left: 0px; display: inline; padding-right: 0px" border="0" alt="image" src="http://lh3.googleusercontent.com/-XonTWGqESUc/VVprm6ku6kI/AAAAAAAACwI/IomNGR8cV2k/image_thumb%25255B5%25255D.png?imgmax=800" width="854" height="225" /></a></p> <p>get running service</p> <p><a href="http://lh3.googleusercontent.com/-gu3d1lvgIJU/VVprncGoNsI/AAAAAAAACwU/LFycIrhD7_4/s1600-h/image%25255B16%25255D.png"><img title="image" style="border-top: 0px; border-right: 0px; background-image: none; border-bottom: 0px; padding-top: 0px; padding-left: 0px; border-left: 0px; display: inline; padding-right: 0px" border="0" alt="image" src="http://lh3.googleusercontent.com/-nueX9Ferlks/VVproLBMeQI/AAAAAAAACwY/Vr3H2LMi7oo/image_thumb%25255B8%25255D.png?imgmax=800" width="583" height="240" /></a></p> <p>Or just using ? instead.</p> <p><a href="http://lh3.googleusercontent.com/-edamRFVxRmI/VVpro461hDI/AAAAAAAACwg/P2o2WCH2bns/s1600-h/image%25255B20%25255D.png"><img title="image" style="border-top: 0px; border-right: 0px; background-image: none; border-bottom: 0px; padding-top: 0px; padding-left: 0px; border-left: 0px; display: inline; padding-right: 0px" border="0" alt="image" src="http://lh3.googleusercontent.com/-2Pt2e8PWSVo/VVprpR2KGPI/AAAAAAAACwo/pkUfbgvKNd8/image_thumb%25255B10%25255D.png?imgmax=800" width="480" height="161" /></a></p> <p>Using Select to run a projection, select name, status only</p> <p><a href="http://lh3.googleusercontent.com/-i9OQUKZzk5g/VVprp9gM5nI/AAAAAAAACww/faF98i05nyY/s1600-h/image%25255B24%25255D.png"><img title="image" style="border-top: 0px; border-right: 0px; background-image: none; border-bottom: 0px; padding-top: 0px; padding-left: 0px; border-left: 0px; display: inline; padding-right: 0px" border="0" alt="image" src="http://lh3.googleusercontent.com/-1Gh83OGYbkc/VVprq87d4_I/AAAAAAAACw8/Gi93K3DclMU/image_thumb%25255B12%25255D.png?imgmax=800" width="854" height="120" /></a></p> <p>also, you can skip and tail the results</p> <p><a href="http://lh3.googleusercontent.com/-Pvh8_ilzY8M/VVprrT37qjI/AAAAAAAACxE/IXuWKQQXxfE/s1600-h/image%25255B28%25255D.png"><img title="image" style="border-top: 0px; border-right: 0px; background-image: none; border-bottom: 0px; padding-top: 0px; padding-left: 0px; border-left: 0px; display: inline; padding-right: 0px" border="0" alt="image" src="http://lh3.googleusercontent.com/-qMwMULGjVFI/VVprsJcQMeI/AAAAAAAACxI/FuRFgWIcoYQ/image_thumb%25255B14%25255D.png?imgmax=800" width="771" height="249" /></a></p> <p>sellect last 5 only</p> <p><a href="http://lh3.googleusercontent.com/-s8ZBUXkf1Ko/VVprsu6W1pI/AAAAAAAACxQ/yg_29Gx21-A/s1600-h/image%25255B32%25255D.png"><img title="image" style="border-top: 0px; border-right: 0px; background-image: none; border-bottom: 0px; padding-top: 0px; padding-left: 0px; border-left: 0px; display: inline; padding-right: 0px" border="0" alt="image" src="http://lh3.googleusercontent.com/-YbeDzcNrzt8/VVprtANAgRI/AAAAAAAACxY/PFPnGfsRBDM/image_thumb%25255B16%25255D.png?imgmax=800" width="863" height="161" /></a></p> <p>convert the result to a Html page? using the convert*</p> <p><a href="http://lh3.googleusercontent.com/-_uD10SOFqwk/VVprt16tslI/AAAAAAAACxg/_B5CSUr4jHY/s1600-h/image%25255B36%25255D.png"><img title="image" style="border-top: 0px; border-right: 0px; background-image: none; border-bottom: 0px; padding-top: 0px; padding-left: 0px; border-left: 0px; display: inline; padding-right: 0px" border="0" alt="image" src="http://lh3.googleusercontent.com/-7gCNNEGVs9g/VVprudZUa9I/AAAAAAAACxo/jhSSDpHdYfM/image_thumb%25255B18%25255D.png?imgmax=800" width="841" height="225" /></a></p> <p>to show it in a gridview</p> <p><a href="http://lh3.googleusercontent.com/-BHUn0eLEbXE/VVpuDvyjXQI/AAAAAAAACx4/gZHeT2eoeoc/s1600-h/image%25255B40%25255D.png"><img title="image" style="border-top: 0px; border-right: 0px; background-image: none; border-bottom: 0px; padding-top: 0px; padding-left: 0px; border-left: 0px; display: inline; padding-right: 0px" border="0" alt="image" src="http://lh3.googleusercontent.com/-UwFA9J2R7pY/VVpuEF-hw6I/AAAAAAAACyA/6c7xQQ25Ceg/image_thumb%25255B20%25255D.png?imgmax=800" width="644" height="255" /></a></p> Ryanhttp://www.blogger.com/profile/09245836008728584469noreply@blogger.com0tag:blogger.com,1999:blog-4653070451761456668.post-31984611580417729362015-05-01T20:00:00.003-07:002015-05-01T20:00:32.891-07:00What's new in asp.net 5<br />
<br />
updates for asp.net 4.6, web forms mvc5<br />
<br />
roslyn support. .net compiler<br />
<br />
roslen c# languae fature<br />
<span class="Apple-tab-span" style="white-space: pre;"> </span>var name="ss"<br />
<span class="Apple-tab-span" style="white-space: pre;"> </span>var messgage=$"this is a amessag {name}"<br />
<br />
Features<br />
<br />
<span class="Apple-tab-span" style="white-space: pre;"> </span>totally modular<br />
<span class="Apple-tab-span" style="white-space: pre;"> </span>>old days, all features on. now , you can turn on off , faster<br />
<span class="Apple-tab-span" style="white-space: pre;"> </span>>cloud support<br />
<span class="Apple-tab-span" style="white-space: pre;"> </span>>faster dev cycle. 2 seconds to compiler<br />
<span class="Apple-tab-span" style="white-space: pre;"> </span>>cross platform. run on mac, linux, windows<br />
<span class="Apple-tab-span" style="white-space: pre;"> </span>>faster. less memory<br />
<br />
<br />
<br />
docs.asp.net<br />
powerd by readthedocs, markdown syntax<br />
<br />
readmedocs.org<br />
1. turn on featues on and off, moduleer. can run on IoT<br />
<br />
<span class="Apple-tab-span" style="white-space: pre;"> </span>//like the nodejs<br />
<span class="Apple-tab-span" style="white-space: pre;"> </span>public void configure(IapplicationBuild app)<br />
<span class="Apple-tab-span" style="white-space: pre;"> </span>{<br />
<br />
<span class="Apple-tab-span" style="white-space: pre;"> </span>app.run(async (context)=><br />
<span class="Apple-tab-span" style="white-space: pre;"> </span>{<br />
<span class="Apple-tab-span" style="white-space: pre;"> </span>await context.Response.WriteAsync("Hello World")<br />
<span class="Apple-tab-span" style="white-space: pre;"> </span>})<br />
<span class="Apple-tab-span" style="white-space: pre;"> </span>}<br />
<br />
<br />
<span class="Apple-tab-span" style="white-space: pre;"> </span>//add dependency .microsfot.aspnet.diagnostis<br />
<br />
<span class="Apple-tab-span" style="white-space: pre;"> </span>app.userErrorPage();<br />
<br />
<span class="Apple-tab-span" style="white-space: pre;"> </span>dnx . web //start the app<br />
<br />
<span class="Apple-tab-span" style="white-space: pre;"> </span>//not static handler by default. need add dependency<br />
<span class="Apple-tab-span" style="white-space: pre;"> </span>//add dependency<br />
<span class="Apple-tab-span" style="white-space: pre;"> </span>microsoft.aspnet.staticfiles<br />
<br />
<span class="Apple-tab-span" style="white-space: pre;"> </span>app.useStatiffiles (); // extension methods<br />
<br />
2. roslyn engine<br />
<span class="Apple-tab-span" style="white-space: pre;"> </span>instead of compile cs to dll, then load it.<br />
<span class="Apple-tab-span" style="white-space: pre;"> </span>now load it in memory<br />
<br />
<span class="Apple-tab-span" style="white-space: pre;"> </span>DNX (Dontne Execution Environment)<br />
<span class="Apple-tab-span" style="white-space: pre;"> </span>4.5.1<br />
<span class="Apple-tab-span" style="white-space: pre;"> </span>5.0 core<br />
<br />
<span class="Apple-tab-span" style="white-space: pre;"> </span>old file have the csproj, inclue all files. cause merge issues.<br />
<span class="Apple-tab-span" style="white-space: pre;"> </span>now all fiels in the project<br />
<br />
<span class="Apple-tab-span" style="white-space: pre;"> </span>commands in package.json<br />
<span class="Apple-tab-span" style="white-space: pre;"> </span>like alias in npm<br />
<br />
<span class="Apple-tab-span" style="white-space: pre;"> </span>dnx . web //run web command in the folder.<br />
<br />
<span class="Apple-tab-span" style="white-space: pre;"> </span>target framework<br />
<span class="Apple-tab-span" style="white-space: pre;"> </span>frameworks:{<br />
<span class="Apple-tab-span" style="white-space: pre;"> </span>dnx451:{}<br />
<span class="Apple-tab-span" style="white-space: pre;"> </span>dnxcore50:{}<br />
<span class="Apple-tab-span" style="white-space: pre;"> </span>}<br />
<br />
<span class="Apple-tab-span" style="white-space: pre;"> </span>bower support<br />
<span class="Apple-tab-span" style="white-space: pre;"> </span>>>>nuget not versioned<br />
<br />
<span class="Apple-tab-span" style="white-space: pre;"> </span>gulp support<br />
<span class="Apple-tab-span" style="white-space: pre;"> </span>task runner support (show gupp task)<br />
<br />
<span class="Apple-tab-span" style="white-space: pre;"> </span>>before build<br />
<span class="Apple-tab-span" style="white-space: pre;"> </span>>after build<br />
<br />
<br />
<span class="Apple-tab-span" style="white-space: pre;"> </span>publish the app to a disk and can run it directly. by run the web.command<br />
<br />
<br />
3.Environment<br />
<br />
<span class="Apple-tab-span" style="white-space: pre;"> </span><Environment names="Dev"><br />
<br />
<span class="Apple-tab-span" style="white-space: pre;"> </span><link rel="stylesheet" href="cdnlink" asp-fall-back-ref="otherlinks"><br />
<span class="Apple-tab-span" style="white-space: pre;"> </span></Environment><br />
<br />
<span class="Apple-tab-span" style="white-space: pre;"> </span>model injection.<br />
<span class="Apple-tab-span" style="white-space: pre;"> </span>more html<br />
<span class="Apple-tab-span" style="white-space: pre;"> </span><input asp-for="email" class="form-control"/><br />
<br />
<span class="Apple-tab-span" style="white-space: pre;"> </span>used to be html.Textbox(m->m.Email, new {"classs=formcontrol"})<br />
<br />
4. configuration.<br />
<br />
<span class="Apple-tab-span" style="white-space: pre;"> </span>new Configuration().addInifile() or addjson()<br />
<br />
<br />
<br />
<span class="Apple-tab-span" style="white-space: pre;"> </span>app.addUserSecretcs(), no connectionstring in web.config.<br />
<span class="Apple-tab-span" style="white-space: pre;"> </span>also for hosted on cloud, IT ops will asign those value<br />
<br />
<span class="Apple-tab-span" style="white-space: pre;"> </span>user-script //<br />
<span class="Apple-tab-span" style="white-space: pre;"> </span>user-script set AppSettings:SiteTitle "sec title"<br />
<br />
5. controller<br />
<br />
<span class="Apple-tab-span" style="white-space: pre;"> </span>new ScrottController () , no need for base calss<br />
<br />
<span class="Apple-tab-span" style="white-space: pre;"> </span>public xxxController()<br />
<span class="Apple-tab-span" style="white-space: pre;"> </span>{<br />
<br />
<span class="Apple-tab-span" style="white-space: pre;"> </span>public string Index()<br />
<span class="Apple-tab-span" style="white-space: pre;"> </span>{<br />
<span class="Apple-tab-span" style="white-space: pre;"> </span>return "hello, index"<br />
<span class="Apple-tab-span" style="white-space: pre;"> </span>}<br />
<span class="Apple-tab-span" style="white-space: pre;"> </span>}<br />
<br />
6.dotnet version manager<br />
<span class="Apple-tab-span" style="white-space: pre;"> </span>dnvm list<br />
<span class="Apple-tab-span" style="white-space: pre;"> </span>c:\users\currentuser\.dnx<br />
<br />
7. mac<br />
<br />
<span class="Apple-tab-span" style="white-space: pre;"> </span>yo asp.net<br />
<span class="Apple-tab-span" style="white-space: pre;"> </span>dnu restore<br />
<br />
<span class="Apple-tab-span" style="white-space: pre;"> </span>dnx run kestrol<br />
<span class="Apple-tab-span" style="white-space: pre;"> </span>Mcirosfot.aspnet.serverhosting -server kestrol<br />
<br />
<br />
8. run on rsp pi<br />
kestrel<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />Ryanhttp://www.blogger.com/profile/09245836008728584469noreply@blogger.com0tag:blogger.com,1999:blog-4653070451761456668.post-78783569698023442332015-05-01T19:59:00.001-07:002015-05-01T19:59:09.242-07:00What's new in C# 6<br />
<br />
Roslyn open source<br />
<br />
IDE Features (CTL+.)<br />
. lightball to remove unused namespace<br />
.. fix the scope , remove all unused cross the project / solutions<br />
. refactor<br />
rename, introduce local variable, show conflict<br />
add this automaticaly if their is conflict that could be resolved by IDE<br />
. Array vs ImmutableArray<br />
<br />
var c=new ImmutableArray(); call c.length will trigger null exception.<br />
//you can build code analyzer<br />
//tell the ide you shoudl you ImmutableArray<int>.empty<br />
<br />
language new features<br />
instead of big change, little things added.<br />
<br />
1.using static System.Console. //simar typescript import {WriteLine} from systemcnosole<br />
<br />
then you can call WriteLine methods.<br />
2.Immutable, auto property.<br />
public class Point<br />
{<br />
public int x {get;}<br />
public int y {get;}=default10;<br />
}<br />
3 lambda for methods<br />
public void string ToString()=>String.format("tostring {0}", x);<br />
<br />
4 $String,<br />
public void String toString()=>$({X}{Y})<br />
5. nameof(variable)<br />
Log.d(nameof(variable))<br />
6.?.<br />
if p!=null && p.name=="xxx"<br />
will be if p?.name=="xx"<br />
if json!=null && json['x']!=null && json['x']=="mon"<br />
will be<br />
if(json?['x']?=="mon")<br />
7.initlize elements<br />
public JObject tojson()=>return new jsonobject(){['x']=x, ['Y']=y }<br />
8. awit in catch block<br />
<br />
try<br />
{<br />
var result=await repo.DosomethingAsync();<br />
}<br />
catch(Excepton ex)<br />
{<br />
await repo.LogException(ex) //doable now<br />
}<br />
<br />
9.catch(Exception ex) when (ex.Occurences>3)<br />
<br />
<br />
Debugging features<br />
1. you can edit code, even add new class , and do initialization code when debugging the app<br />
run linq query<br />
<br />
also in watch window<br />
people.find(p=>p.age>20)<br />
<br />
C# extensions toolkit in the extnsion gallery<br />
2. C# interactive window<br />
<br />
#r "System"<br />
#r "System.Core"<br />
using System.Diagnostics;<br />
using System.linq;<br />
var memoryPigs =from p in Process.getProcesses() where p.workset64 >64*1024*1024 select new<br />
{p.ProcessName, p.WorkingSet}<br />
foreach(var r in memopigs)<br />
{<br />
Console.writeline(r)7y<br />
}<br />
<div>
<br /></div>
Ryanhttp://www.blogger.com/profile/09245836008728584469noreply@blogger.com0tag:blogger.com,1999:blog-4653070451761456668.post-20942569721116465572015-04-24T14:09:00.001-07:002015-04-24T14:10:41.991-07:00wse 2 run windows server 2008 R2<p>For some legacy issues, you app stack might use WSE to do the message level authentication. and if you have to run wse 2 in your asmx hosted on windows server 2008 R2. here could be some issues and solutions.</p> <p>1. If you bind multi Binary kerberos token to the request and send to the server, you might be rejected even both of them are valid. like you put host to abc and abs.fullqualifiedname.com</p> <p>solution: manually downgrade the wse 2 from 2.0.3 to 2.0.1. I found since 2.0.3, the request filter does not allow 1+ tokens </p> <p>2. unable to extract username from the binary token passed from client.</p> <p>solution: config the application pool to run on 32 bit mode. twick the code a little bit, to remove the session key extraction. </p> <p><a href="http://lh3.googleusercontent.com/-5FywKp-sFiY/VTqxHsCAQdI/AAAAAAAACvE/xfqwj_8INK0/s1600-h/image%25255B3%25255D.png"><img title="image" style="border-top: 0px; border-right: 0px; background-image: none; border-bottom: 0px; padding-top: 0px; padding-left: 0px; border-left: 0px; display: inline; padding-right: 0px" border="0" alt="image" src="http://lh3.googleusercontent.com/-eEYI_wTRKZ8/VTqxIVfiuOI/AAAAAAAACvI/fo05k1hNg7w/image_thumb%25255B1%25255D.png?imgmax=800" width="450" height="544" /></a></p> <p>if always shows me the memroy access exception when it try to do the session key marshaling. </p> <p>check the code in Microsoft.Web.Services2.Security.Tokens.Kerberos.LsaServerContext.LogonUser(byte[] inToken), you might just decompile the code ,chagne it and sign it back using your own key. and update the reference to your version of wse.dll </p> Ryanhttp://www.blogger.com/profile/09245836008728584469noreply@blogger.com0tag:blogger.com,1999:blog-4653070451761456668.post-84512462724128929282015-04-03T17:19:00.001-07:002015-04-03T17:19:17.198-07:00JWT token, signed in C#, and Decode in JS<p>JWT is a widly used token system that we can share token info between APIs or between Apps.</p> <p>i.e we can sign the token by using a shared key in C#, then verify the token in javascript by using the same sharedkey.</p> <p>in C#, you can search nuget for the jwt , there are several libraries. <br /><a href="http://lh4.ggpht.com/-jpB1rP9tT6Q/VR8t-LdguzI/AAAAAAAACt4/rQLe2AdM4qg/s1600-h/image%25255B3%25255D.png"><img title="image" style="border-top: 0px; border-right: 0px; background-image: none; border-bottom: 0px; padding-top: 0px; padding-left: 0px; border-left: 0px; display: inline; padding-right: 0px" border="0" alt="image" src="http://lh5.ggpht.com/-yKSehhpvj1g/VR8t-9WCQCI/AAAAAAAACt8/yVDTzUwCLVg/image_thumb%25255B1%25255D.png?imgmax=800" width="685" height="498" /></a></p> <p>let’s I want to sign a simple clamin using the simple password. <br /><a href="http://lh5.ggpht.com/-szdzcVFQHV0/VR8t_dDksVI/AAAAAAAACuE/MC9HGF8LUzg/s1600-h/image%25255B7%25255D.png"><img title="image" style="border-top: 0px; border-right: 0px; background-image: none; border-bottom: 0px; padding-top: 0px; padding-left: 0px; border-left: 0px; display: inline; padding-right: 0px" border="0" alt="image" src="http://lh3.ggpht.com/-PPorfvUnbvQ/VR8t_gcR6PI/AAAAAAAACuM/dmMrgRgCfXU/image_thumb%25255B3%25255D.png?imgmax=800" width="525" height="321" /></a></p> <p>we get token eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzUxMiJ9.eyJuYW1lIjoiUGV0ZXIiLCJSb2xlIjoiQWRtaW5zLFJlYWRlciJ9.Qh2J93epa7ls0y6DAn_8YvIRoRHq28hLT9N-93fStcajCd90nLzLG6Vit-Qdfl1TsPtL56qh4jiKDOzpfs1OyA <br /> <br />then follow the library for different languages from jwt.io to do the decoding</p> <p>Past the token into the validation box, and enter pass in the shared key, you will see token get decoded and verified</p> <p><a href="http://lh3.ggpht.com/-YEuKAwrrEZU/VR8uAM2X6UI/AAAAAAAACuU/ioVMmShoxyM/s1600-h/image%25255B11%25255D.png"><img title="image" style="border-top: 0px; border-right: 0px; background-image: none; border-bottom: 0px; padding-top: 0px; padding-left: 0px; border-left: 0px; display: inline; padding-right: 0px" border="0" alt="image" src="http://lh3.ggpht.com/-uv-HLM6VMQk/VR8uAtSxXqI/AAAAAAAACuc/-uAh_tsI_S4/image_thumb%25255B5%25255D.png?imgmax=800" width="1051" height="595" /></a></p> Ryanhttp://www.blogger.com/profile/09245836008728584469noreply@blogger.com4tag:blogger.com,1999:blog-4653070451761456668.post-52717918565417369102015-03-27T17:23:00.001-07:002015-03-30T10:58:49.543-07:00C# TPL some basic code<p>Fast win pattern, you might query several urls for the same info, which one return first , that one will be the winner. <br /><a href="http://lh5.ggpht.com/-hkJNHjPR3dU/VRX0hBS3-4I/AAAAAAAACsc/1Z0b_BOOLz8/s1600-h/image%25255B4%25255D.png"><img title="image" style="border-top: 0px; border-right: 0px; background-image: none; border-bottom: 0px; padding-top: 0px; padding-left: 0px; border-left: 0px; display: inline; padding-right: 0px" border="0" alt="image" src="http://lh4.ggpht.com/-eDEDHaAeKEA/VRX0hqnYgXI/AAAAAAAACsg/R76m5i3NNhM/image_thumb%25255B2%25255D.png?imgmax=800" width="768" height="373" /></a> <br /> <br />Send in parallel and join together. <br /><a href="http://lh3.ggpht.com/-7eTvVlXskhY/VRX0iKJ12lI/AAAAAAAACso/Mh-Ih7ZTN4I/s1600-h/image%25255B9%25255D.png"><img title="image" style="border-top: 0px; border-right: 0px; background-image: none; border-bottom: 0px; padding-top: 0px; padding-left: 0px; border-left: 0px; display: inline; padding-right: 0px" border="0" alt="image" src="http://lh5.ggpht.com/-9ygNCVqz4FU/VRX0igMJ6fI/AAAAAAAACs0/_QgH9NPIeqs/image_thumb%25255B5%25255D.png?imgmax=800" width="485" height="343" /></a> <br />long running and cancellable task, if get canceld, try compensate it</p> <p><a href="http://lh4.ggpht.com/-U83zziwy9b8/VRX0jBBWypI/AAAAAAAACs8/Vu0LUC1H8dM/s1600-h/image%25255B13%25255D.png"><img title="image" style="border-top: 0px; border-right: 0px; background-image: none; border-bottom: 0px; padding-top: 0px; padding-left: 0px; border-left: 0px; display: inline; padding-right: 0px" border="0" alt="image" src="http://lh4.ggpht.com/-fzrLNF6qxVg/VRX0jvT8M3I/AAAAAAAACtA/roKBU3E5-4U/image_thumb%25255B7%25255D.png?imgmax=800" width="817" height="637" /></a></p> <p>“mapreduce”</p> <p><a href="http://lh6.ggpht.com/-fo-7oE-2vEM/VRmO1hFaG8I/AAAAAAAACtY/HagXCe4dBJU/s1600-h/image%25255B18%25255D.png"><img title="image" style="border-top: 0px; border-right: 0px; background-image: none; border-bottom: 0px; padding-top: 0px; padding-left: 0px; border-left: 0px; display: inline; padding-right: 0px" border="0" alt="image" src="http://lh6.ggpht.com/-XufM60pMSws/VRmO2GTEdzI/AAAAAAAACtc/4baems7DPuA/image_thumb%25255B10%25255D.png?imgmax=800" width="976" height="390" /></a></p> Ryanhttp://www.blogger.com/profile/09245836008728584469noreply@blogger.com0tag:blogger.com,1999:blog-4653070451761456668.post-52974553674602681712015-03-12T10:56:00.001-07:002015-03-12T10:56:58.840-07:00Vagrant, Proxy issue, centos local repo and ad-hoc testing<p>Vagrant is a good VM automation tool for both developers and devops, here are some basic tips that I found useful </p> <p><font size="4" face="Bell MT">Proxy</font> <br />   if you are stand behind a proxy, you can install one vagran proxy module. the module will setup the proxy on guest machines, like yum.conf, http_proxy variable etc. <br />here is the plugin <a title="https://github.com/tmatilai/vagrant-proxyconf" href="https://github.com/tmatilai/vagrant-proxyconf">https://github.com/tmatilai/vagrant-proxyconf</a></p> <p><a href="http://lh4.ggpht.com/-cZ1xwNpcXMA/VQHTUWAR_TI/AAAAAAAACqo/nXQCdeZzrUM/s1600-h/image%25255B3%25255D.png"><img title="image" style="border-top: 0px; border-right: 0px; background-image: none; border-bottom: 0px; padding-top: 0px; padding-left: 0px; border-left: 0px; display: inline; padding-right: 0px" border="0" alt="image" src="http://lh4.ggpht.com/-zGSu0DsiCSI/VQHTU4a3_FI/AAAAAAAACqw/o1JZ2Up3lr4/image_thumb%25255B1%25255D.png?imgmax=800" width="677" height="125" /></a></p> <p> </p> <p>when you boot up the vm, you can see the proxy setting applied depends on your OS type</p> <p><a href="http://lh6.ggpht.com/-FSeNXWqxUiw/VQHTVhMO_WI/AAAAAAAACq0/AKzvwjPbFVo/s1600-h/image%25255B7%25255D.png"><img title="image" style="border-top: 0px; border-right: 0px; background-image: none; border-bottom: 0px; padding-top: 0px; padding-left: 0px; border-left: 0px; display: inline; padding-right: 0px" border="0" alt="image" src="http://lh3.ggpht.com/--RafrnJwtLU/VQHTV8wC3gI/AAAAAAAACq8/N2zAc1bTDuc/image_thumb%25255B3%25255D.png?imgmax=800" width="692" height="112" /></a></p> <p><font size="4" face="Bell MT">Yum Repo. </font> <br />   you might have one local repo in the company, you can either build all boxes internally which already assign the repo url to local. or just copy one repo to override the system one by using the provisioning scripts. </p> <p><a href="http://lh4.ggpht.com/-lQv31eZoAIc/VQHTWYutpMI/AAAAAAAACrE/TGHbwCV7eLY/s1600-h/image%25255B11%25255D.png"><img title="image" style="border-top: 0px; border-right: 0px; background-image: none; border-bottom: 0px; padding-top: 0px; padding-left: 0px; border-left: 0px; display: inline; padding-right: 0px" border="0" alt="image" src="http://lh3.ggpht.com/-ntcVDvpk9f8/VQHTW-5i3KI/AAAAAAAACrM/Q7jnbn3SZCE/image_thumb%25255B5%25255D.png?imgmax=800" width="504" height="64" /></a></p> <p><a href="http://lh6.ggpht.com/-Uce0UUVmXic/VQHTXRXvz-I/AAAAAAAACrU/geZ5NdDeiv0/s1600-h/image%25255B15%25255D.png"><img title="image" style="border-top: 0px; border-right: 0px; background-image: none; border-bottom: 0px; padding-top: 0px; padding-left: 0px; border-left: 0px; display: inline; padding-right: 0px" border="0" alt="image" src="http://lh4.ggpht.com/-LRxVfCYi4bc/VQHTXnmE4aI/AAAAAAAACrc/9eavwIx3fQ0/image_thumb%25255B7%25255D.png?imgmax=800" width="676" height="63" /></a></p> <p>and in your local file, change the ip to internal one </p> <p><a href="http://lh5.ggpht.com/-a0ghXyw7xM8/VQHTYNovgfI/AAAAAAAACro/SqDSTsx3yRY/s1600-h/image%25255B19%25255D.png"><img title="image" style="border-top: 0px; border-right: 0px; background-image: none; border-bottom: 0px; padding-top: 0px; padding-left: 0px; border-left: 0px; display: inline; padding-right: 0px" border="0" alt="image" src="http://lh6.ggpht.com/-XL9549Cg9JA/VQHTY5ZxxrI/AAAAAAAACrw/avx0K5Wk4Nc/image_thumb%25255B9%25255D.png?imgmax=800" width="809" height="163" /></a></p> <p>Provision scripts to check whether package installed or not. using rpm to query package or use command to determine whether a command exists</p> <p><a href="http://lh4.ggpht.com/-SFidXcrlOrQ/VQHTZXy95MI/AAAAAAAACr0/T_ZDTAESN6A/s1600-h/image%25255B23%25255D.png"><img title="image" style="border-top: 0px; border-right: 0px; background-image: none; border-bottom: 0px; padding-top: 0px; padding-left: 0px; border-left: 0px; display: inline; padding-right: 0px" border="0" alt="image" src="http://lh4.ggpht.com/-ubxXCeLH74s/VQHTZ78veJI/AAAAAAAACsA/AQRVx-Vt9wc/image_thumb%25255B11%25255D.png?imgmax=800" width="440" height="311" /></a></p> Ryanhttp://www.blogger.com/profile/09245836008728584469noreply@blogger.com1tag:blogger.com,1999:blog-4653070451761456668.post-12053324717757597752015-02-19T15:26:00.001-08:002015-02-19T15:26:26.923-08:00Opencart fix, show the shipping method in orders admin page and filter by shippingmethod<p>it turns that this should be a common feature for opencart admins, basically the operation team would like to see orders with priority shipping method and process those orders first. on 1.5, no way you can see this at once. basically the orders page looks like this by default</p> <p><a href="http://lh3.ggpht.com/-QGhljZYXP90/VOZxAYYzFXI/AAAAAAAACpc/dUE4A6WvGmU/s1600-h/image%25255B5%25255D.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="http://lh4.ggpht.com/-0If9_a7B_KA/VOZxA5UmGgI/AAAAAAAACpg/p27vqUdWZs4/image_thumb%25255B3%25255D.png?imgmax=800" width="1026" height="631" /></a></p> <p>they actually want to see the UI like this, with shipping method on the overview</p> <p><a href="http://lh5.ggpht.com/-1cu1AnswLgE/VOZxBWy19nI/AAAAAAAACpo/tmM5_jYmoQI/s1600-h/image%25255B9%25255D.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="http://lh3.ggpht.com/--tJp-_JOkCs/VOZxBx1hVQI/AAAAAAAACpw/-7O_reQEhSg/image_thumb%25255B5%25255D.png?imgmax=800" width="1028" height="535" /></a></p> <p>even more, they want a filter to see all orders with priority shipping selected. </p> <p><a href="http://lh3.ggpht.com/-Fe7Z2m70ago/VOZxCS5HuyI/AAAAAAAACp4/Id3n_BbxeWs/s1600-h/image%25255B13%25255D.png"><img title="image" style="border-left-width: 0px; border-right-width: 0px; background-image: none; border-bottom-width: 0px; padding-top: 0px; padding-left: 0px; display: inline; padding-right: 0px; border-top-width: 0px" border="0" alt="image" src="http://lh6.ggpht.com/-vJY0vQ4HPzo/VOZxDHarv_I/AAAAAAAACqA/vXn3wbHiYTE/image_thumb%25255B7%25255D.png?imgmax=800" width="1028" height="483" /></a></p> <p>To fix this, pretty straight forward. change the orders module/controller/template file, to include shippingmethod in the data query back. and add a filter logic to narrow down records.</p> <p>I have a patch file, email me if you are interested. click about me on the page to get my email</p> Ryanhttp://www.blogger.com/profile/09245836008728584469noreply@blogger.com1tag:blogger.com,1999:blog-4653070451761456668.post-77227545228138647832015-02-11T15:58:00.001-08:002015-02-11T15:58:47.829-08:00C# merge sorting vs insert sorting<p>Quick test for a 50K random double array with random order, to soft it using the system libarry  and my own insert sorting/ merge sorting, the time takes to do the soring is quite different. </p> <p>given a array with size 50K of  double values, </p> <p><a href="http://lh4.ggpht.com/-Xy5trktb-hk/VNvsrVf7WfI/AAAAAAAACo0/TxAliHqSeg0/s1600-h/image%25255B3%25255D.png"><img title="image" style="border-top: 0px; border-right: 0px; background-image: none; border-bottom: 0px; padding-top: 0px; padding-left: 0px; border-left: 0px; display: inline; padding-right: 0px" border="0" alt="image" src="http://lh3.ggpht.com/-wzt-CU80y20/VNvssPaP8KI/AAAAAAAACo8/qf_aFKrB4f8/image_thumb%25255B1%25255D.png?imgmax=800" width="394" height="522" /></a></p> <p>if increase to 100K, more time for insertion sorting</p> <p><a href="http://lh3.ggpht.com/-F1H9jn4gBjw/VNvsskiFHZI/AAAAAAAACpA/Rf8QoX90YG8/s1600-h/image%25255B7%25255D.png"><img title="image" style="border-top: 0px; border-right: 0px; background-image: none; border-bottom: 0px; padding-top: 0px; padding-left: 0px; border-left: 0px; display: inline; padding-right: 0px" border="0" alt="image" src="http://lh6.ggpht.com/-naxnx0saaxc/VNvstH1a7MI/AAAAAAAACpI/F3vGflNPGmU/image_thumb%25255B3%25255D.png?imgmax=800" width="390" height="553" /></a></p> <p> </p> <p><a title="https://github.com/ryandh/CSharpSortingInsertedMerged/blob/master/Program.cs" href="https://github.com/ryandh/CSharpSortingInsertedMerged/blob/master/Program.cs">https://github.com/ryandh/CSharpSortingInsertedMerged/blob/master/Program.cs</a></p> Ryanhttp://www.blogger.com/profile/09245836008728584469noreply@blogger.com1tag:blogger.com,1999:blog-4653070451761456668.post-57864804296130956252015-02-06T10:54:00.001-08:002015-02-06T10:59:15.679-08:00Jquery , Simple plugin with default options<p>check this sample fiddler, <a title="http://jsfiddle.net/androidyou/dox5x8pp/" href="http://jsfiddle.net/androidyou/dox5x8pp/">http://jsfiddle.net/androidyou/dox5x8pp/</a></p> <p>basically, we can use the $.extend to assign a default value if options are not assigned. </p> <p><a href="http://lh5.ggpht.com/-aNVTEzHcNtM/VNUN272Mp5I/AAAAAAAACn8/bEAUsqCW9kc/s1600-h/image%25255B8%25255D.png"><img title="image" style="border-top: 0px; border-right: 0px; background-image: none; border-bottom: 0px; padding-top: 0px; padding-left: 0px; border-left: 0px; display: inline; padding-right: 0px" border="0" alt="image" src="http://lh6.ggpht.com/-YQWQoyykYeg/VNUN3SjDjZI/AAAAAAAACoA/SrVTmklIdjE/image_thumb%25255B4%25255D.png?imgmax=800" width="552" height="151" /></a></p> <p>if you put default option,</p> <p><a href="http://lh5.ggpht.com/-P4sI34xQQWg/VNUN32xsADI/AAAAAAAACoI/fF-iiEVXoXI/s1600-h/image%25255B12%25255D.png"><img title="image" style="border-top: 0px; border-right: 0px; background-image: none; border-bottom: 0px; padding-top: 0px; padding-left: 0px; border-left: 0px; display: inline; padding-right: 0px" border="0" alt="image" src="http://lh5.ggpht.com/-Er7lk2n8RwQ/VNUN4DyDKVI/AAAAAAAACoU/NlW4eokMz54/image_thumb%25255B6%25255D.png?imgmax=800" width="297" height="110" /></a></p> <p><a href="http://lh4.ggpht.com/-ZtZmPfvhqfU/VNUN4q-wybI/AAAAAAAACoc/m_SR_w5yqas/s1600-h/image%25255B16%25255D.png"><img title="image" style="border-top: 0px; border-right: 0px; background-image: none; border-bottom: 0px; padding-top: 0px; padding-left: 0px; border-left: 0px; display: inline; padding-right: 0px" border="0" alt="image" src="http://lh5.ggpht.com/-_221rog66s0/VNUN5WNGRjI/AAAAAAAACog/DmtkNxxAXsw/image_thumb%25255B8%25255D.png?imgmax=800" width="301" height="115" /></a></p> Ryanhttp://www.blogger.com/profile/09245836008728584469noreply@blogger.com0tag:blogger.com,1999:blog-4653070451761456668.post-44468618073310663162015-01-07T14:47:00.001-08:002015-01-07T14:50:49.991-08:00DNS troubleshooting tools and tips - nslookp<p>if you are on Windows, dig is not there, you can do most DNS query through nslookup. </p> <p>to see the DNS cache on your local server,  run “ipconfig /displayDNS”</p> <p><a href="http://lh6.ggpht.com/-ReQmM-A-N_M/VK23ZwtNnvI/AAAAAAAACl0/AYjeZlLIEvg/s1600-h/image%25255B9%25255D.png"><img style="border-bottom: 0px; border-left: 0px; display: inline; border-top: 0px; border-right: 0px" title="image" border="0" alt="image" src="http://lh5.ggpht.com/-lp0d-ytDh1Q/VK23aVCOrLI/AAAAAAAACl4/YOgPn7eABIw/image_thumb%25255B5%25255D.png?imgmax=800" width="948" height="840" /></a> </p> <p>same thing, ipconfig /flushDNS to purge the cache. </p> <p>to see name server of a giving domain</p> <p><a href="http://lh3.ggpht.com/-NhT0494dJL4/VK23ar2z-6I/AAAAAAAACmA/PUvz7RuQPcY/s1600-h/image%25255B8%25255D.png"><img style="border-bottom: 0px; border-left: 0px; display: inline; border-top: 0px; border-right: 0px" title="image" border="0" alt="image" src="http://lh4.ggpht.com/-8lsjTCrEaLs/VK23bJV6pMI/AAAAAAAACmI/LDZgiL4frR4/image_thumb%25255B4%25255D.png?imgmax=800" width="800" height="440" /></a> </p> <p></p> <p>to see all infomration like root server, try nslookup , set all</p> <p><a href="http://lh5.ggpht.com/-rqOrRfYr27w/VK23biUd5TI/AAAAAAAACmQ/UDmEfyUA2B8/s1600-h/image%25255B13%25255D.png"><img style="border-bottom: 0px; border-left: 0px; display: inline; border-top: 0px; border-right: 0px" title="image" border="0" alt="image" src="http://lh5.ggpht.com/-seUQ6gQonFk/VK23cCVzdNI/AAAAAAAACmc/l1yPhk7KJ8U/image_thumb%25255B7%25255D.png?imgmax=800" width="648" height="698" /></a> </p> <p>to see mx record of amazon.</p> <p><a href="http://lh6.ggpht.com/-nl-qLuV7yKA/VK23ckmemPI/AAAAAAAACmg/-PpSGak8JxE/s1600-h/image%25255B23%25255D.png"><img style="border-bottom: 0px; border-left: 0px; display: inline; border-top: 0px; border-right: 0px" title="image" border="0" alt="image" src="http://lh5.ggpht.com/-TlBUzpDgoec/VK23dNp_BrI/AAAAAAAACmo/DtFuICKluqE/image_thumb%25255B13%25255D.png?imgmax=800" width="644" height="105" /></a> </p> <p>to see the spf record?</p> <p><a href="http://lh3.ggpht.com/-SJ_HKeq6qqg/VK23dX04nwI/AAAAAAAACmw/5y9H8NyyKX4/s1600-h/image%25255B22%25255D.png"><img style="border-bottom: 0px; border-left: 0px; display: inline; border-top: 0px; border-right: 0px" title="image" border="0" alt="image" src="http://lh3.ggpht.com/-NsNYhYzk8_w/VK23d3gSppI/AAAAAAAACm4/pMONX-H8FKY/image_thumb%25255B12%25255D.png?imgmax=800" width="644" height="196" /></a> </p> <p>what about debug information</p> <p><a href="http://lh6.ggpht.com/-fQBsBdz7rOQ/VK23eITAkVI/AAAAAAAACnA/pYt18SFAQ1k/s1600-h/image%25255B27%25255D.png"><img style="border-bottom: 0px; border-left: 0px; display: inline; border-top: 0px; border-right: 0px" title="image" border="0" alt="image" src="http://lh3.ggpht.com/-kkvNJPGt6T0/VK23eae4fRI/AAAAAAAACnM/qgSTScqhN7c/image_thumb%25255B15%25255D.png?imgmax=800" width="644" height="291" /></a> </p> <p>to see all information </p> <p><a href="http://lh4.ggpht.com/-59nxlJBOqTo/VK23fHK6B4I/AAAAAAAACnU/5hh6DUrtWlg/s1600-h/image%25255B32%25255D.png"><img style="border-bottom: 0px; border-left: 0px; display: inline; border-top: 0px; border-right: 0px" title="image" border="0" alt="image" src="http://lh4.ggpht.com/-lLUX1rFTiYc/VK23fsp8yhI/AAAAAAAACnY/Xu_Xtas6Au4/image_thumb%25255B18%25255D.png?imgmax=800" width="644" height="354" /></a> </p> <p>to see all the ips of a domain or even this history. </p> <p><a title="https://www.virustotal.com/en/domain/dl.dropbox.com/information/" href="https://www.virustotal.com/en/domain/dl.dropbox.com/information/">https://www.virustotal.com/en/domain/dl.dropbox.com/information/</a></p> <p><a href="http://lh3.ggpht.com/-0OuWR2IHrhw/VK24RxrKfKI/AAAAAAAACnk/eEDizV27BEQ/s1600-h/image%25255B38%25255D.png"><img style="border-bottom: 0px; border-left: 0px; display: inline; border-top: 0px; border-right: 0px" title="image" border="0" alt="image" src="http://lh4.ggpht.com/-fqAAVk4ArK8/VK24SJmKz1I/AAAAAAAACno/oB952WyhLHA/image_thumb%25255B22%25255D.png?imgmax=800" width="644" height="294" /></a></p> Ryanhttp://www.blogger.com/profile/09245836008728584469noreply@blogger.com0tag:blogger.com,1999:blog-4653070451761456668.post-72156441500440955712015-01-07T14:37:00.001-08:002015-01-07T14:37:34.754-08:00DNS troubleshooting tools and tips - dig<p>To see top level Name Servers,  dig –t ns com</p> <p><a href="http://lh4.ggpht.com/-H6ypJ2DqKa0/VK21EVfhH2I/AAAAAAAACkQ/ahHIHZR22mg/s1600-h/image%25255B4%25255D.png"><img style="border-bottom: 0px; border-left: 0px; display: inline; border-top: 0px; border-right: 0px" title="image" border="0" alt="image" src="http://lh6.ggpht.com/-fz_OffQBEtg/VK21EyylTKI/AAAAAAAACkY/HXjlYmNeXYE/image_thumb%25255B2%25255D.png?imgmax=800" width="976" height="772" /></a> </p> <p>If we want to see what’s are the name server of Google by asking any top level com servers. </p> <p><a href="http://lh6.ggpht.com/-k75bgmwt0ts/VK21FbRH8cI/AAAAAAAACkc/y00WykzkNco/s1600-h/image%25255B8%25255D.png"><img style="border-bottom: 0px; border-left: 0px; display: inline; border-top: 0px; border-right: 0px" title="image" border="0" alt="image" src="http://lh6.ggpht.com/-Q1NvW8RZ22A/VK21GMpUgQI/AAAAAAAACko/giKV8UFm2VM/image_thumb%25255B4%25255D.png?imgmax=800" width="873" height="772" /></a> </p> <p>now we can ask any nameserver of google, what’s are the ip of <a href="http://www.google.com">www.google.com</a></p> <p><a href="http://lh4.ggpht.com/-Xx5VJa2ccns/VK21GmzUxWI/AAAAAAAACks/FJkO3AmHNaU/s1600-h/image%25255B12%25255D.png"><img style="border-bottom: 0px; border-left: 0px; display: inline; border-top: 0px; border-right: 0px" title="image" border="0" alt="image" src="http://lh4.ggpht.com/-nswsk4bW-Wk/VK21HGk6hFI/AAAAAAAACk4/vh5MHBHytog/image_thumb%25255B6%25255D.png?imgmax=800" width="1028" height="650" /></a> </p> <p>we can also using host command to get the ip or alias information</p> <p><a href="http://lh4.ggpht.com/-dhw3KFbiSrs/VK21H8KHA0I/AAAAAAAACk8/Mf-c-kEdfa4/s1600-h/image%25255B18%25255D.png"><img style="border-bottom: 0px; border-left: 0px; display: inline; border-top: 0px; border-right: 0px" title="image" border="0" alt="image" src="http://lh3.ggpht.com/-IGF-wOIYWYU/VK21IWsI1tI/AAAAAAAAClI/MhNLRfyoHWo/image_thumb%25255B10%25255D.png?imgmax=800" width="1028" height="379" /></a> </p> <p>we can see the alias gmail.google.com </p> <p><a href="http://lh4.ggpht.com/-r0hYqzIfkw0/VK21IzB_MlI/AAAAAAAAClQ/DmhjR3uo1ZY/s1600-h/image%25255B22%25255D.png"><img style="border-bottom: 0px; border-left: 0px; display: inline; border-top: 0px; border-right: 0px" title="image" border="0" alt="image" src="http://lh3.ggpht.com/-fJH4q9rIuwU/VK21JyUOIkI/AAAAAAAAClY/ma_OucKlla0/image_thumb%25255B12%25255D.png?imgmax=800" width="1028" height="584" /></a> </p> <p>if we want to see some debug information. we can put debug there</p> <table border="0" cellspacing="0" cellpadding="2" width="445"><tbody> <tr> <td valign="top" width="443"> <p>[~]dig +trace  -t mx  dropbox.com </p> <p>; <<>> DiG 9.8.3-P1 <<>> +trace -t mx dropbox.com <br />;; global options: +cmd <br />.            15634    IN    NS    a.root-servers.net. <br />.            15634    IN    NS    b.root-servers.net. <br />.            15634    IN    NS    c.root-servers.net. <br />.            15634    IN    NS    d.root-servers.net. <br />.            15634    IN    NS    e.root-servers.net. <br />.            15634    IN    NS    f.root-servers.net. <br />.            15634    IN    NS    g.root-servers.net. <br />.            15634    IN    NS    h.root-servers.net. <br />.            15634    IN    NS    i.root-servers.net. <br />.            15634    IN    NS    j.root-servers.net. <br />.            15634    IN    NS    k.root-servers.net. <br />.            15634    IN    NS    l.root-servers.net. <br />.            15634    IN    NS    m.root-servers.net. <br />;; Received 228 bytes from 8.8.8.8#53(8.8.8.8) in 29 ms </p> <p>com.            172800    IN    NS    a.gtld-servers.net. <br />com.            172800    IN    NS    b.gtld-servers.net. <br />com.            172800    IN    NS    c.gtld-servers.net. <br />com.            172800    IN    NS    d.gtld-servers.net. <br />com.            172800    IN    NS    e.gtld-servers.net. <br />com.            172800    IN    NS    f.gtld-servers.net. <br />com.            172800    IN    NS    g.gtld-servers.net. <br />com.            172800    IN    NS    h.gtld-servers.net. <br />com.            172800    IN    NS    i.gtld-servers.net. <br />com.            172800    IN    NS    j.gtld-servers.net. <br />com.            172800    IN    NS    k.gtld-servers.net. <br />com.            172800    IN    NS    l.gtld-servers.net. <br />com.            172800    IN    NS    m.gtld-servers.net. <br />;; Received 489 bytes from 128.63.2.53#53(128.63.2.53) in 144 ms </p> <p>dropbox.com.        172800    IN    NS    ns-564.awsdns-06.net. <br />dropbox.com.        172800    IN    NS    ns-315.awsdns-39.com. <br />dropbox.com.        172800    IN    NS    ns-1162.awsdns-17.org. <br />dropbox.com.        172800    IN    NS    ns-1949.awsdns-51.co.uk. <br />;; Received 198 bytes from 192.52.178.30#53(192.52.178.30) in 167 ms </p> <p>dropbox.com.        3600    IN    MX    1 aspmx.l.google.com. <br />dropbox.com.        3600    IN    MX    10 aspmx2.googlemail.com. <br />dropbox.com.        3600    IN    MX    10 aspmx3.googlemail.com. <br />dropbox.com.        3600    IN    MX    5 alt1.aspmx.l.google.com. <br />dropbox.com.        3600    IN    MX    5 alt2.aspmx.l.google.com. <br />dropbox.com.        172800    IN    NS    ns-1162.awsdns-17.org. <br />dropbox.com.        172800    IN    NS    ns-1949.awsdns-51.co.uk. <br />dropbox.com.        172800    IN    NS    ns-315.awsdns-39.com. <br />dropbox.com.        172800    IN    NS    ns-564.awsdns-06.net. <br />;; Received 296 bytes from 205.251.194.52#53(205.251.194.52) in 21 ms</p> </td> </tr> </tbody></table> <a href="http://lh3.ggpht.com/-FXyXUnJLm_A/VK21Kvv_8LI/AAAAAAAAClc/ZUjQkeRWans/s1600-h/image%25255B27%25255D.png"><img style="border-bottom: 0px; border-left: 0px; display: inline; border-top: 0px; border-right: 0px" title="image" border="0" alt="image" src="http://lh4.ggpht.com/-1G2sqbqbPsM/VK21LYD14ZI/AAAAAAAAClo/A46AGiBnxoY/image_thumb%25255B15%25255D.png?imgmax=800" width="816" height="772" /></a> Ryanhttp://www.blogger.com/profile/09245836008728584469noreply@blogger.com0tag:blogger.com,1999:blog-4653070451761456668.post-79259565134563428622014-12-31T11:11:00.001-08:002014-12-31T11:16:52.590-08:00Opencart security, a must-have checklist for webmasters<p>Just played with the opencart 1.5.6.4, which is a very popular e-commerce solution for SMBs . and If you are tech-savy webmaster, you may know that 1.5.6.4 is pretty safe in terms of code security. before this version, you may have all kinds of issues like XSS, SQL Injection. but 1.5.6.4 is pretty safe though.  </p> <p>However, code safety doesn’t means operation safety. I will show you what does this mean in 4 examples, you may have those issues already which means your customer’s credit card or sensitive information is leaking now. </p> <h4>Turn off Debugging for both usps and USA epay from the admin console. this is very important. (this is rule number 1)</h4> <p>if you turn on the debugging for usa epay in the following screen, gosh, you need pay attention .</p> <p><a href="http://lh4.ggpht.com/-7Ay1ADImmN0/VKRKRt55xaI/AAAAAAAACjU/G0HdmHpzCz0/s1600-h/image%25255B3%25255D.png"><img title="image" style="border-top: 0px; border-right: 0px; background-image: none; border-bottom: 0px; padding-top: 0px; padding-left: 0px; border-left: 0px; display: inline; padding-right: 0px" border="0" alt="image" src="http://lh3.ggpht.com/-zI3qAjMGtjE/VKRKS_GLbmI/AAAAAAAACjc/hJfnXj_MD9s/image_thumb%25255B1%25255D.png?imgmax=800" width="1079" height="919" /></a></p> <p>if you turn on debug logging, check the system/logs folder, a file called <strong><font size="4">usaepay_server_debug.txt</font></strong> will have all the Live credit card information and customer information. </p> <p>to be worst, the hacker might just access </p> <h5><font size="3"><font style="background-color: #ffff00">http://www.yoursite.com/system/logs/<strong>usaepay_server_debug.txt</strong></font> to see all your raw credit card information. </font></h5> <p><strong><font size="4">usaepay_server_debug.txt</font></strong> </p> <p><font size="3"><font style="background-color: #ffff00"><a href="http://www.yoursite.com/system/logs/error.txt">http://www.yoursite.com/system/logs/<strong>error.txt</strong> to see all debug information.</a></font></font></p> <p><font size="3"><font style="background-color: #ffff00"><strong>Here is one real example that one hacker is pulling this file</strong></font></font></p> <p><a href="http://lh3.ggpht.com/-9Deshic4bGY/VKRKUyEzgaI/AAAAAAAACjk/LqB5H8Nkxxw/s1600-h/image%25255B7%25255D.png"><img title="image" style="border-top: 0px; border-right: 0px; background-image: none; border-bottom: 0px; padding-top: 0px; padding-left: 0px; border-left: 0px; display: inline; padding-right: 0px" border="0" alt="image" src="http://lh3.ggpht.com/-nAuhZ49zbzg/VKRKV9gPwhI/AAAAAAAACjs/K8TAqZds9GQ/image_thumb%25255B3%25255D.png?imgmax=800" width="1157" height="818" /></a> </p> <p>risky? though I put a fake information there. if you are not lucky, your customers’s credit card is gone, cross fingers.</p> <p> </p> <h4>Rule number 2, put a .htaccess file to block /system/logs access. </h4> <p>this is very obvious, what every issue you have , you may put sensitive information into the logs folder, like stack trace? customer information, error to running a sql statement?</p> <p>put a .htaccess under system/logs with content “Deny from all”, locked down the access from public access.</p> <p>this is a easy fix, definitly you should do it. </p> <h4></h4> <p><strong><font size="4">Rule number 3, check your access logs for POST requests. since most user should be get Only, some hacker might found site voluability and inject some evile scripts like webadmin.php <a title="http://cker.name/webadmin/" href="http://cker.name/webadmin/">http://cker.name/webadmin/</a>, get is limited by the url length, definitly the hacker will send a post request to inject the blackdoor. </font></strong></p> <p><font size="4">you can write a cron job with python script to do a daily access, and email you daily to double check those special posts, if you see special urls with POST, pay attention. </font></p> <p><font size="4"></font></p> <p><font size="4"><strong>Rule Number 4, suPHP, if you run suphp as the PHP handler. double check the execution logs</strong></font></p> <p><font size="4">suphp is fast, but it runs with a high privileged user might shoot the gun to yourself. so check the suphp logs to see which php file get executed daily. </font></p> <p><font size="4">here is one pythong script to dump out the files list daily. typicall you should only see index.php and admin/index.php, no other evil php like webadmin.php? images/index.php</font></p> <p><a href="http://lh4.ggpht.com/-HjbDrbA3D0c/VKRKWlD5D0I/AAAAAAAACjw/f5HiaGIWzuM/s1600-h/image%25255B12%25255D.png"><img title="image" style="border-top: 0px; border-right: 0px; background-image: none; border-bottom: 0px; padding-top: 0px; padding-left: 0px; border-left: 0px; display: inline; padding-right: 0px" border="0" alt="image" src="http://lh6.ggpht.com/-GBACLmePwnA/VKRKXaWAG8I/AAAAAAAACj8/XRzKnRNeS3Y/image_thumb%25255B6%25255D.png?imgmax=800" width="936" height="660" /></a></p> <p>once you have those 4 rules ready, you should be feel much safer about your website operation security. any more questions, email me and I will get back to your for more details or even do a consultation for you . </p> <p><font size="4"></font></p> <p><font size="4"> </font></p> Ryanhttp://www.blogger.com/profile/09245836008728584469noreply@blogger.com0tag:blogger.com,1999:blog-4653070451761456668.post-63758068856424080822014-12-25T16:03:00.001-08:002014-12-31T11:17:16.712-08:00Iptables rules , Drop or Reject?<p>By Default, Pinging is fine to both yahoo.com and Google.com </p> <p><a href="http://lh4.ggpht.com/-KaCdV4XCLtI/VJylsrzZyBI/AAAAAAAACiM/0lKgxxIT95c/s1600-h/image%25255B3%25255D.png"><img title="image" style="border-top: 0px; border-right: 0px; border-bottom: 0px; border-left: 0px; display: inline" border="0" alt="image" src="http://lh6.ggpht.com/-OmxhLTdlglQ/VJyltBCp7NI/AAAAAAAACiQ/ZtCQ8NcTLp8/image_thumb%25255B1%25255D.png?imgmax=800" width="682" height="321" /></a> </p> <p>If we turn on the firewall, to Drop icmp to google, and Reject UDP to yahoo.com</p> <p><a href="http://lh4.ggpht.com/-r1P8R3mNYaY/VJyltZZyPfI/AAAAAAAACiY/KNhwQ8OuOiw/s1600-h/image%25255B15%25255D.png"><img title="image" style="border-top: 0px; border-right: 0px; border-bottom: 0px; border-left: 0px; display: inline" border="0" alt="image" src="http://lh6.ggpht.com/-0lGSVQvJajY/VJyltwq6UVI/AAAAAAAACig/7_KgYoO-mWw/image_thumb%25255B7%25255D.png?imgmax=800" width="754" height="293" /></a> </p> <p>then ping google by ip which is rejected by firewall policy</p> <p><a href="http://lh5.ggpht.com/-de3zXqY18IA/VJyluJQECiI/AAAAAAAACio/TVqKb1T_3-Y/s1600-h/image%25255B14%25255D.png"><img title="image" style="border-top: 0px; border-right: 0px; border-bottom: 0px; border-left: 0px; display: inline" border="0" alt="image" src="http://lh6.ggpht.com/-qIwQTrEcfds/VJylurcjtLI/AAAAAAAACiw/5fnClZDH75s/image_thumb%25255B6%25255D.png?imgmax=800" width="575" height="340" /></a> </p> <p></p> <p>What about TCP</p> <p><a href="http://lh6.ggpht.com/-ajQsx8rc1Aw/VJymUyHq8PI/AAAAAAAACi8/dAAimnYqII4/s1600-h/image%25255B19%25255D.png"><img title="image" style="border-top: 0px; border-right: 0px; border-bottom: 0px; border-left: 0px; display: inline" border="0" alt="image" src="http://lh5.ggpht.com/-A475y38saCM/VJymVC-UjPI/AAAAAAAACjA/-JjXlRfjiiQ/image_thumb%25255B9%25255D.png?imgmax=800" width="673" height="177" /></a> </p> <p>SO, Drop means pretend the port is not open, which is like a implicit denial .</p> <p>Reject means explicit denial. </p> Ryanhttp://www.blogger.com/profile/09245836008728584469noreply@blogger.com0tag:blogger.com,1999:blog-4653070451761456668.post-66188952303557210872014-12-19T11:38:00.001-08:002014-12-31T11:17:25.812-08:00log4net legacy configuration<p>just got one legacy app with log4net early version bundled, something wrong and there is no way to see the log . here is the trick</p> <p>if you use refelctor, you can see the assermbly attribute. [assembly: <a href="http://www.aisto.com/roeder/dotnet/Default.aspx?Target=code://log4net:1.2.0.30714:b32731d11ce58905/log4net.Config.DOMConfiguratorAttribute/.ctor()">DOMConfigurator</a>(<a href="http://www.aisto.com/roeder/dotnet/Default.aspx?Target=code://log4net:1.2.0.30714:b32731d11ce58905/log4net.Config.DOMConfiguratorAttribute/property:ConfigFileExtension:String">ConfigFileExtension</a>="log4net", <a href="http://www.aisto.com/roeder/dotnet/Default.aspx?Target=code://log4net:1.2.0.30714:b32731d11ce58905/log4net.Config.DOMConfiguratorAttribute/property:Watch:Boolean">Watch</a>=true)] </p> <p><a href="http://lh3.ggpht.com/-_tICFDpqfCI/VJR-r9mzFDI/AAAAAAAAChk/Xf52AqHETgA/s1600-h/image%25255B3%25255D.png"><img title="image" style="border-top: 0px; border-right: 0px; background-image: none; border-bottom: 0px; padding-top: 0px; padding-left: 0px; border-left: 0px; display: inline; padding-right: 0px" border="0" alt="image" src="http://lh3.ggpht.com/-Jdx25gA6SNY/VJR-sfYpH8I/AAAAAAAACho/BtpgpFlDM-E/image_thumb%25255B1%25255D.png?imgmax=800" width="455" height="244"></a></p> <p>then for the config file, should you yourexe.exe.log4net, you can tell from the logic</p> <p><a href="http://lh6.ggpht.com/-u2Knh_c1x6U/VJR-s_sWprI/AAAAAAAACh0/sqXAZaJjIPw/s1600-h/image%25255B7%25255D.png"><img title="image" style="border-top: 0px; border-right: 0px; background-image: none; border-bottom: 0px; padding-top: 0px; padding-left: 0px; border-left: 0px; display: inline; padding-right: 0px" border="0" alt="image" src="http://lh3.ggpht.com/-lThPUL2EMp0/VJR-tRLxdWI/AAAAAAAACh4/t0nmVkL92uk/image_thumb%25255B3%25255D.png?imgmax=800" width="856" height="556"></a></p> <p>one sample config</p> <p><?xml version="1.0" encoding="utf-8"?><br><configuration><br> <configSections><br> <section name="log4net" type="log4net.Config.Log4NetConfigurationSectionHandler, log4net" /><br> </configSections><br> <log4net><br> <appender name="FileAppender" type="log4net.Appender.FileAppender"><br> <file value="logfile.txt" /><br> <appendToFile value="true" /><br> <layout type="log4net.Layout.PatternLayout"><br> <conversionPattern value="%date: %-5level – %message%newline" /><br> </layout><br> </appender><br> <root><br> <level value="INFO" /><br> <appender-ref ref="FileAppender" /><br> </root><br> </log4net><br></configuration></p><pre> </pre> Ryanhttp://www.blogger.com/profile/09245836008728584469noreply@blogger.com0tag:blogger.com,1999:blog-4653070451761456668.post-69205704604806308912014-12-16T11:03:00.001-08:002014-12-31T11:17:44.638-08:00Windows azure Mobile service, delete expired data automatically.<p>It turns out Azure mobile service is way more powerful than you think. I setup a simple mobile service table to hold some incoming logs. </p> <p>on the table itself, I created one node.js API for insert, basically insert a timestamp in long named dt. (I found this very useful for you to delete the old data , like logs week ago)</p> <p><a href="http://lh3.ggpht.com/-9Ox36H_1Gns/VJCB3v9M6mI/AAAAAAAACgc/RoXDxgv_NIU/s1600-h/image%25255B3%25255D.png"><img title="image" style="border-top: 0px; border-right: 0px; background-image: none; border-bottom: 0px; padding-top: 0px; padding-left: 0px; border-left: 0px; display: inline; padding-right: 0px" border="0" alt="image" src="http://lh4.ggpht.com/-Hw6_2XIaKxs/VJCB4WliZfI/AAAAAAAACgk/2gq1e5letn8/image_thumb%25255B1%25255D.png?imgmax=800" width="339" height="151" /></a></p> <p>then every incoming logs will be applied with the timestamp. now create another scheduler to run  daily and check to delete old logs </p> <p><a href="http://lh6.ggpht.com/-0hlnh7Stp1Q/VJCB49h2MGI/AAAAAAAACgo/0j7O3J93o4A/s1600-h/image%25255B7%25255D.png"><img title="image" style="border-top: 0px; border-right: 0px; background-image: none; border-bottom: 0px; padding-top: 0px; padding-left: 0px; border-left: 0px; display: inline; padding-right: 0px" border="0" alt="image" src="http://lh4.ggpht.com/-eC4lhrtb2GQ/VJCB5WcyruI/AAAAAAAACgw/R5JksSf7m1A/image_thumb%25255B3%25255D.png?imgmax=800" width="737" height="458" /></a></p> <p>and config it to run daily</p> <p><a href="http://lh6.ggpht.com/-NgJ1jnAOrHg/VJCB5gFLjYI/AAAAAAAACg4/L_JpXA5pIjs/s1600-h/image%25255B11%25255D.png"><img title="image" style="border-top: 0px; border-right: 0px; background-image: none; border-bottom: 0px; padding-top: 0px; padding-left: 0px; border-left: 0px; display: inline; padding-right: 0px" border="0" alt="image" src="http://lh5.ggpht.com/-8firgScSl08/VJCB6BDwg6I/AAAAAAAAChA/SAeyEayCGKA/image_thumb%25255B5%25255D.png?imgmax=800" width="560" height="455" /></a></p> <p>then in the log you will see the daily run</p> <p><a href="http://lh6.ggpht.com/-TqXFKfJp6pc/VJCB6nOPVsI/AAAAAAAAChI/URHyZWdEY5U/s1600-h/image%25255B15%25255D.png"><img title="image" style="border-top: 0px; border-right: 0px; background-image: none; border-bottom: 0px; padding-top: 0px; padding-left: 0px; border-left: 0px; display: inline; padding-right: 0px" border="0" alt="image" src="http://lh4.ggpht.com/-tVqgJuN4qJA/VJCB7BZf-FI/AAAAAAAAChU/j-c-PbSbWD4/image_thumb%25255B7%25255D.png?imgmax=800" width="859" height="151" /></a></p> Ryanhttp://www.blogger.com/profile/09245836008728584469noreply@blogger.com0tag:blogger.com,1999:blog-4653070451761456668.post-68822855293603178042014-11-04T11:23:00.001-08:002014-12-31T11:18:03.136-08:00WordPress 4.0 on IIS 7.X, Permalinks dead loop for hybird unicoed urls<p>Just helped my friend did the migration from LAMP to IIS7, with fastcgi module. all smoothly, and based on the tutorial. on the platform of IIS, you don’t have the .htaccess rule, install a url rewrite2 module instead.</p> <p><a title="http://www.microsoft.com/web/gallery/install.aspx?appid=urlrewrite2" href="http://www.microsoft.com/web/gallery/install.aspx?appid=urlrewrite2">http://www.microsoft.com/web/gallery/install.aspx?appid=urlrewrite2</a> <br /> <br /> on the root of the web app, put the rewrite rules there , which will intercept all permallinks or friendly url which is neither a folder or file , and redirect them to the index.php to handle. so the code looks like this. </p> <p><a href="http://lh3.ggpht.com/-dP1dGqs6rnQ/VFkn4iWjAuI/AAAAAAAACf8/uhqF6dyYbpo/s1600-h/image%25255B23%25255D.png"><img title="image" style="border-top: 0px; border-right: 0px; background-image: none; border-bottom: 0px; padding-top: 0px; padding-left: 0px; border-left: 0px; display: inline; padding-right: 0px" border="0" alt="image" src="http://lh6.ggpht.com/-zDo0MHAZtr0/VFkn5IRMuCI/AAAAAAAACgE/CdIfumSwBk4/image_thumb%25255B11%25255D.png?imgmax=800" width="1124" height="211" /></a></p> <p>after this, one the wp settings page, you can turn on the friendly url . <br /><a href="http://lh5.ggpht.com/-acHhc1GkQB4/VFknpFs7u7I/AAAAAAAACfI/lILPUlrnHhw/s1600-h/image%25255B7%25255D.png"><img title="image" style="border-top: 0px; border-right: 0px; background-image: none; border-bottom: 0px; padding-top: 0px; padding-left: 0px; border-left: 0px; display: inline; padding-right: 0px" border="0" alt="image" src="http://lh4.ggpht.com/-4lP7zm4TDt4/VFknpgUMXiI/AAAAAAAACfQ/TrwgK0wwHUo/image_thumb%25255B3%25255D.png?imgmax=800" width="600" height="413" /></a></p> <p>now, when you go to some page with url like page-你好 , or post-ヤフオク, basically english and unicode in url combined, you will see server no response issue. </p> <p><a href="http://lh5.ggpht.com/-kgvgiobKcp4/VFknp2nM55I/AAAAAAAACfY/y-Qdtbexp10/s1600-h/image%25255B11%25255D.png"><img title="image" style="border-top: 0px; border-right: 0px; background-image: none; border-bottom: 0px; padding-top: 0px; padding-left: 0px; border-left: 0px; display: inline; padding-right: 0px" border="0" alt="image" src="http://lh5.ggpht.com/-eQxGFWTW1DA/VFknqb-6BwI/AAAAAAAACfg/j1jh4nciaNw/image_thumb%25255B5%25255D.png?imgmax=800" width="417" height="210" /></a></p> <p>and when you put some page like hello-你好.html, on iis foder, it works. so IIS is fine, something wrong with the PHP to handle the unicode urls. </p> <p>fix is here, basically , send the correct unicode url to php by the url-rewrite module. replease the request_uri with unicodeurl <br /><a title="http://www.iis.net/learn/extensions/url-rewrite-module/url-rewrite-module-configuration-reference" href="http://www.iis.net/learn/extensions/url-rewrite-module/url-rewrite-module-configuration-reference">http://www.iis.net/learn/extensions/url-rewrite-module/url-rewrite-module-configuration-reference</a></p> <p>here is the updated web.config for the rules. <br /><a href="http://lh3.ggpht.com/-UoeUBfcSbhU/VFknq5v6W2I/AAAAAAAACfo/c5Pw_kQ03ac/s1600-h/image%25255B19%25255D.png"><img title="image" style="border-top: 0px; border-right: 0px; background-image: none; border-bottom: 0px; padding-top: 0px; padding-left: 0px; border-left: 0px; display: inline; padding-right: 0px" border="0" alt="image" src="http://lh4.ggpht.com/-liSky-lRjW0/VFknrUrVcDI/AAAAAAAACfw/-JheZfOu-pc/image_thumb%25255B9%25255D.png?imgmax=800" width="1225" height="326" /></a></p> <p>after the change, all back to normal. </p> Ryanhttp://www.blogger.com/profile/09245836008728584469noreply@blogger.com0tag:blogger.com,1999:blog-4653070451761456668.post-92023579388442315322014-10-16T16:29:00.001-07:002014-12-31T11:18:22.810-08:00Keyboard / trackpad non-functional after winclone restore for macbook pro 2014 mid retina<p>Problem first, I restored one windows 7 using winclone, when I boot back to the new restored windows 7 partition. Gosh, neither keyboard/Trackpad works, tried several external USB keyboard/mouse no lock at all. </p> <p>then I Googled, found the official faq from winclone, tried the bless tips. No luck. then after 2 hours struggling. Here is the step to make it work. </p> <p>Boot into Mac, remove the windows partition through the disk utility. <font style="background-color: #ffff00">Don’t Create a new windows one using the disk utility. </font></p> <p><font style="background-color: #ffff00"><font style="background-color: #ffffff">open the Bootcamp Assistant utility, check the all the options. and continue, then it will load the ISO and download the drivers, after that, you get chance to resize the bootcamp partition. Now it’s the time to create the bootcamp partition .</font></font></p> <p>once done, it will reboot and enter the setup process.<font style="background-color: #ffff00"> Stop here</font>, reboot and press option then select the mac partition.</p> <p>Now it’s safe to recover the bootcamp partition using winclone. after done, here is what I did. not sure which step is important, but this works. </p> <ul> <li>go to utility, terminal</li> <ul> <li>sudo /usr/sbin/bless –-mount /Volumes/BOOTCAMP –nextonly –legacy –legacydrivehint /dev/disk0 </li> </ul> <li>in the windows boot time, press any key to <font style="background-color: #ffff00">cancel</font> the disk check. </li> <li>that’s it, it works now. both keyboard and touchpad. download and install the new driver for windows now  </li> <ul> <li>check this table and select the right one/ good luck</li> </ul> <li><a title="http://support.apple.com/kb/HT5634" href="http://support.apple.com/kb/HT5634">http://support.apple.com/kb/HT5634</a></li> <li><a href="http://lh4.ggpht.com/-aPWsLR0AUUE/VEBUu8Ko1rI/AAAAAAAACdo/hYv11pWGtsU/s1600-h/image%25255B3%25255D.png"><img title="image" style="border-top: 0px; border-right: 0px; background-image: none; border-bottom: 0px; padding-top: 0px; padding-left: 0px; border-left: 0px; display: inline; padding-right: 0px" border="0" alt="image" src="http://lh4.ggpht.com/-CHa7mAoQzaQ/VEBUvaLOmRI/AAAAAAAACdw/FRkD0UKRRQ0/image_thumb%25255B1%25255D.png?imgmax=800" width="644" height="404" /></a></li> </ul> Ryanhttp://www.blogger.com/profile/09245836008728584469noreply@blogger.com1