Viewing posts by kerem

Things that went into this site

This is more of a personal reminder than anything.

Blogging Episode IV: A New Hope

I don't know how many times I have tried to keep semi-technical blogs, even if only for the sake of remembering stuff I have done. I have always failed to keep writing. Either I was too interested in solving the problem to take notes, or was too pressed for time. Or some other excuse if you can think of one. 

But now that I actually have a personal website that I have to take care of, perhaps I will finally find the discipline to write down all those little things that I have to remember Google queries for. I also moved all the posts from my previous (and again, very sparsely populated) research blog here in time. 

These are the most likely themes: Python, R, Machine learning, Django, Mezzanine, Linux, OS/X. 

Let me also put a picture of Einstein being blue here because I can. I didn't sift through all that Django stuff just to post text. 

Idea - Using compression to extract building blocks

So, initially when I was using CTW the problem was with VQ, in particular, independence of the quantization from the inference in the Markovian model. I solved that using HMMs so that the quantization is learned simultaneously with the structure.

Idea - Can we use left-to-right HMMs?

Can we use left-to-right HMMs (probably with emissions from a mixture of Gaussians), and still use the number of states as a complexity measure? Or perhaps a combination of \(N_{states}\) and \(N_{mixtures}\)? This might function as an indirect way of representing temporal structure.

Comparing HMMs state-by-state and Choosing Initial Values for Emission Means

I just realized that I can meaningfully compare many HMMs if I sort the states. If I put in initial emission distribution means so that for a univariate case the first state is the one with the lowest mean whereas the last one is the one with the highest mean, then I can ensure that I get the same state labeling for all trained models. This naturally extends to the multivariate case.