Thursday, November 3, 2016

one way to pivot table

trade table:

stock side amount
----------------------
`ibm b 100
`ibm s 200
`appl b 300

-- sql server syntax
select stock, sum(iif(side='b', amount, 0)) as b, sum(iif(side='s', amount, 0)) as s
 from trade
 group by stock;

stock b s
------------------
`ibm 100 200
`aapl 300 0

Tuesday, November 1, 2016

Column names as parameters to functions

http://code.kx.com/wiki/Cookbook/ProgrammingIdioms#Column_names_as_parameters_to_functions

Wednesday, October 12, 2016

How many times do u expect to roll a die b4 getting two consecutive sixes?

No, it's not 36!

http://www.wikihow.com/Calculate-an-Expected-Value#Calculating_an_Expected_Number_of_Coin_Flips_for_a_Specific_Result_sub

Tuesday, September 27, 2016

Conditional Probability - a visualization



No need to remember the formula.  Instead, understand it and u can derive it.

Looking at the diagram above, 

  • Omega is the entire probability space.  [rectangle]
  • Event A is a subset of Omega with probability of happening P(A). [left circle]
  • Event B is a subset of Omega with probability of happening P(B).  [right circle]
  • A intersect B is when A and B both happening. [middle ellipsis]


P(B|A) is 

  • conditional prob of B given A
  • ie knowing that A has happened, it is the chance of B happening


What does it really mean?

  1. knowing that A has happened -> the prob space has shrunk from Omega [rectangle] to P(A) [red circle]
  2. the chance of B happening left is A intersect B [middle ellipsis]
  3. note that we don't look at the rest of P(B) [outside of red circle] anymore, since it's outside of the realm of possibility knowing that A has happened.


Picturing the diagram in your mind, you will figure out that

  • Given A has happened, chance of B happening is [middle ellipsis] divided by [red circle]
  • ie P(B|A) = P(A intersect B) / P(A)



source:
https://courses.edx.org/courses/course-v1:MITx+6.008.1x+3T2016/courseware/1__Probability_and_Inference/conditioning_on_events/

Sunday, August 28, 2016

kdb group by weekday, hour

select count i by weekday:-1+`int$date mod 7, "n"$0D01:00:00.0 xbar date+time from table

Note: 

  1. `int$date cast date to the number of days since year 2000
  2. by hour can be done with time.hh, but the above will actually return a time column instead of int column

Thursday, July 28, 2016

one-line if then else

C++
<condition> ? <if true return> : <else return>

Python
<if true return> if <condition> else <else return>

example:
rsl = f[len(prefix):] if f.startswith(prefix) else f

https://mail.python.org/pipermail/python-list/2010-July/581933.html

Wednesday, June 15, 2016

Friday, February 12, 2016

putty ssh X11 forwarding not working?

have you installed xorg-x11-xauth at the server?  (Thanks to http://praveen.kumar.in/2006/11/15/xorg-x11-xauth-needed-for-ssh-x-forwarding-to-work/)

how to allow remote server to access my local resource via ssh tunnel?

Sunday, January 31, 2016

How to install additional kernel to ipython / jupyter?

Assuming you already have python 3 and ipython installed with anaconda,

Create python 2.7 env
$ conda create -n py27 python=2.7

Activate the env
$ source activate py27

Install ipykernel
$ conda install notebook ipykernel

Write kernel spec
$ ipython kernelspec install-self --user
which should write to ~/.local/share/jupyter/kernels/python2

Deactivate 2.7 env
$ source deactivate

link the kernel spec to the location where ipython will read
$ cd ~/.ipython/
$ ln -s ~/.local/share/jupyter/kernels/

Confirm now that we have additional kernel
$ ipython kernelspec list

Start ipython notebook and you should now have both kernels
$ ipython notebook --no-browser