Viewing posts by kerem

## Things that went into this site

This is more of a personal reminder than anything.

## Blogging Episode IV: A New Hope

I don't know how many times I have tried to keep semi-technical blogs, even if only for the sake of remembering stuff I have done. I have always failed to keep writing. Either I was too interested in solving the problem to take notes, or was too pressed for time. Or some other excuse if you can think of one.

But now that I actually have a personal website that I have to take care of, perhaps I will finally find the discipline to write down all those little things that I have to remember Google queries for. I also moved all the posts from my previous (and again, very sparsely populated) research blog here in time.

These are the most likely themes: Python, R, Machine learning, Django, Mezzanine, Linux, OS/X.

Let me also put a picture of Einstein being blue here because I can. I didn't sift through all that Django stuff just to post text.

## Idea - Using compression to extract building blocks

So, initially when I was using CTW the problem was with VQ, in particular, independence of the quantization from the inference in the Markovian model. I solved that using HMMs so that the quantization is learned simultaneously with the structure.

## Idea - Can we use left-to-right HMMs?

Can we use left-to-right HMMs (probably with emissions from a mixture of Gaussians), and still use the number of states as a complexity measure? Or perhaps a combination of $$N_{states}$$ and $$N_{mixtures}$$? This might function as an indirect way of representing temporal structure.

## Comparing HMMs state-by-state and Choosing Initial Values for Emission Means

I just realized that I can meaningfully compare many HMMs if I sort the states. If I put in initial emission distribution means so that for a univariate case the first state is the one with the lowest mean whereas the last one is the one with the highest mean, then I can ensure that I get the same state labeling for all trained models. This naturally extends to the multivariate case.