Viewing posts by kerem
I don't know how many times I have tried to keep semi-technical blogs, even if only for the sake of remembering stuff I have done. I have always failed to keep writing. Either I was too interested in solving the problem to take notes, or was too pressed for time. Or some other excuse if you can think of one.
But now that I actually have a personal website that I have to take care of, perhaps I will finally find the discipline to write down all those little things that I have to remember Google queries for. I also moved all the posts from my previous (and again, very sparsely populated) research blog here in time.
These are the most likely themes: Python, R, Machine learning, Django, Mezzanine, Linux, OS/X.
Let me also put a picture of Einstein being blue here because I can. I didn't sift through all that Django stuff just to post text.
So, initially when I was using CTW the problem was with VQ, in particular, independence of the quantization from the inference in the Markovian model. I solved that using HMMs so that the quantization is learned simultaneously with the structure.
Can we use left-to-right HMMs (probably with emissions from a mixture of Gaussians), and still use the number of states as a complexity measure? Or perhaps a combination of \(N_{states}\) and \(N_{mixtures}\)? This might function as an indirect way of representing temporal structure.
I just realized that I can meaningfully compare many HMMs if I sort the states. If I put in initial emission distribution means so that for a univariate case the first state is the one with the lowest mean whereas the last one is the one with the highest mean, then I can ensure that I get the same state labeling for all trained models. This naturally extends to the multivariate case.